547 Open Source Distributed Systems Software Projects

😮 互联网 Java 工程师进阶知识完全扫盲：涵盖高并发、分布式、高可用、微服务、海量数据处理等领域知识，后端同学必看，前端同学也可学习

Distributed reliable key-value store for the most critical data of a distributed system

The Patterns of Scalable, Reliable, and Performant Large-Scale Systems

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Flink and DataFlow

A realtime distributed messaging platform

A Go microservices development framework

Machine Learning、Deep Learning、PostgreSQL、Distributed System、Node.Js、Golang

Build, distribute and collaborate on components.

Build highly concurrent, distributed, and resilient message-driven applications on the JVM

SeaweedFS is a distributed object store and file system to store and serve billions of files fast! Object store has O(1) disk seek, transparent cloud integration. Filer supports cross-cluster active-active replication, Kubernetes, POSIX, S3 API, encryption, Erasure Coding for warm storage, FUSE mount, Hadoop, WebDAV.

Micro is a platform for cloud native development

High-Performance server for NATS, the cloud native messaging system.

A fault tolerant, protocol-agnostic RPC system

Orleans is a cross-platform framework for building distributed applications with .NET

The lightweight, distributed relational database built on SQLite.

🐼 Chinese translations for classic IT resources

A curated list to learn about distributed systems

《Designing Data-Intensive Application》DDIA中文翻译

open source training courses about distributed database and distributed systemes

Python Stream Processing

Distributed, offline-first bug tracker embedded in git, with bridges

Reproducible Data Science at Scale!

Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way.

Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)

:rocket: Progressive microservices framework for Node.js

⟁ Tendermint Core (BFT Consensus) in Go

A feature complete and high performance multi-group Raft library in Go.

A curated list of awesome System Design (A.K.A. Distributed Systems) resources.

A distributed and coördination-free log management system

Service Fabric is a distributed systems platform for packaging, deploying, and managing stateless and stateful distributed applications and containers at large scale.

Gluster Filesystem : Build your storage in minutes

Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.

Reactive Microservices for the JVM

Dkron - Distributed, fault tolerant job scheduling system https://dkron.io

Next generation distributed, event-driven, parallel config management!

Modern applications with built-in automation

MIT课程《Distributed Systems 》学习和翻译

A reactive Java framework for building fault-tolerant distributed systems

Lightweight, fault-tolerant message streams.

Cloud-Native distributed block storage built on and for Kubernetes

The most popular service bus for .NET

a microservice framework for rapid development of micro services in Go with rich eco-system

An enhanced HTTP client for Go

The Streaming SQL Database powered by Timely Dataflow

Sandglass is a distributed, horizontally scalable, persistent, time sorted message queue.

Highly-available Distributed Fault-tolerant Runtime

A list of useful resources to learn Data Engineering from scratch

Orbit - Virtual actor framework for building distributed systems

Awesome list of distributed systems resources

Distributed Big Data Orchestration Service

Lightweight, facility, high performance golang based game server framework

Raft distributed consensus algorithm implemented in Rust.

Home of the community managed version of Presto, the distributed SQL query engine for big data, under the auspices of the Presto Software Foundation.

Jupiter是一款性能非常不错的, 轻量级的分布式服务框架

Pluggable foundation blocks for building distributed apps.

Keybase Filesystem (KBFS)

GNES is Generic Neural Elastic Search, a cloud-native semantic search system based on deep neural network.

Microdot: An open source .NET microservices framework

A distributed operating system

Ongoing Storj v3 development. Decentralized cloud object storage that is affordable, easy to use, private, and secure.

Curated list of resources on testing distributed systems

Industrial IoT Messaging and Device Management Platform

Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin

A curated list of research in machine learning system. I also summarize some papers if I think they are really interesting.

:books: 深入浅出分布式基础架构，Linux 与操作系统篇 | 分布式系统篇 | 分布式计算篇 | 数据库篇 | 网络篇 | 虚拟化与编排篇 | 大数据与云计算篇

List of Elixir books

A powerful caching library for Elixir with support for transactions, fallbacks and expirations

Prototype implementation of Lasp in Erlang.

Docker-based utility for testing network failures and partitions in distributed applications

Distributed realtime messaging framework based on workerman.

A distributed MySQL binlog storage system built on Raft

LizardFS is an Open Source Distributed File System licensed under GPLv3.

Py4J enables Python programs to dynamically access arbitrary Java objects

nact ⇒ node.js + actors ⇒ your services have never been so µ

🔬 Writing reliable & fault-tolerant microservices in Node.js https://hemerajs.github.io/hemera/

This guide should help fellow researchers and hobbyists to easily automate and accelerate there deep leaning training with their own Kubernetes GPU cluster.

A distributed system for embedding-based retrieval

高性能golang网络库，游戏开发脚手架

An open source serverless computing platform

A reference use of Hashicorp's Raft implementation

A library for event sourcing in Python.

Mysterium Network Node - official implementation of distributed VPN network (dVPN) protocol

Cloud Haskell core library

Pyro 4.x - Python remote objects

A long list of academic papers on the topic of distributed consensus

High-performance, high-scalability distributed computing with Erlang and Elixir.

A high-performance framework for eventually consistent distributed event traceability based on the Virtual Actor model.

Lightweight and Scalable framework that combines mainstream algorithms of Click-Through-Rate prediction based computational DAG, philosophy of Parameter Server and Ring-AllReduce collective communication.

An extensible distributed system for reliable nearline data streaming at scale

Meaningful control of data in distributed systems.

Golimit is Uber ringpop based distributed and decentralized rate limiter

Simple + Powerful interface to the Mnesia Distributed Database 💾

Simple. Scalable. Multi-cloud monitoring.

Kubernetes-native Deep Learning Framework

A novel implementation of the Raft consensus algorithm

The Reactive Manifesto

Privacy-Preserving Bitcoin Light Client

A cluster consistency platform

Minecraft server based on Orleans

Metaparticle/Package: Language Fluent Containerization and Deployment in Java, .NET and Javascript (and more coming soon)