gardnervickers/local-meetups — explained in plain English
Analysis updated 2026-07-09 · repo last pushed 2016-02-04
Prototype a distributed data pipeline in Clojure with local files before deploying to production.
Build a cluster-based data processing system that reads from Kafka and writes to MySQL.
Use as a reference template for structuring Onyx jobs with separate development and production configurations.
Learn tested patterns for Onyx job definition, peer management, and environment lifecycle handling.
| gardnervickers/local-meetups | gardnervickers/untangled-components | replikativ/katzen | |
|---|---|---|---|
| Stars | — | — | 16 |
| Language | Clojure | Clojure | Clojure |
| Last pushed | 2016-02-04 | 2016-10-19 | — |
| Maintenance | Dormant | Dormant | — |
| Setup difficulty | hard | moderate | easy |
| Complexity | 4/5 | 2/5 | 3/5 |
| Audience | developer | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Requires ZooKeeper running for coordination and, in production mode, Kafka and MySQL infrastructure alongside cluster configuration.
Local-meetups is a starter template for building distributed data processing applications using a framework called Onyx. Rather than starting from scratch, you get a sample job and a set of helper tools already wired up, so you can focus on building your actual workflow instead of figuring out how to set up the infrastructure. At its core, the project helps you move data through a pipeline of tasks that can run across multiple machines. During development, you work with static files and simple input/output channels on your local machine. A built-in macro handles starting and stopping the processing environment, including cleanup if something fails or you interrupt it. When you're ready for production, you flip a switch that swaps those simple file-based inputs and outputs for real data sources like a Kafka message queue and a MySQL database. This project is aimed at Clojure developers who want to build systems that process data across a cluster of computers. The name suggests it could be used to organize or process data related to local meetups, but the README doesn't go into detail about a specific end product. Instead, it's positioned as a general starting point for anyone building Onyx-based data pipelines who wants tested patterns for development and deployment. One notable design choice is how the project separates development from production. The same job definition works in both contexts, with only a keyword change determining whether it reads from a local file or pulls from a production database. In production, you can run multiple processing peers on a single machine, with a recommendation of one peer per CPU core. The system relies on ZooKeeper to coordinate between machines, which is standard for distributed systems but adds operational complexity to deployment.
A starter template for building distributed data processing pipelines in Clojure using the Onyx framework, with pre-configured development and production setups so you can focus on your workflow logic.
Mainly Clojure. The stack also includes Clojure, Onyx, ZooKeeper.
Dormant — no commits in 2+ years (last push 2016-02-04).
No license information is provided in this repository, so you should contact the author before using it.
Setup difficulty is rated hard, with roughly 1h+ to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Verify against the repo before relying on details.