whatisgithub

What is bigtable-follower-testing?

doitintl/bigtable-follower-testing — explained in plain English

Analysis updated 2026-08-08 · repo last pushed 2021-09-20

PythonAudience · developerComplexity · 4/5DormantSetup · moderate

In one sentence

An experimental testing project that benchmarks different ways to query Google BigTable for a user's followed streamers who are currently live, useful for large-scale social or streaming platforms.

Mindmap

mindmap
  root((repo))
    What it does
      Tests BigTable query strategies
      Finds online followed users
      Toggles streamer online status
    Tech stack
      Python
      Google BigTable
      HappyBase
    Use cases
      Live streaming platforms
      Social presence systems
      Large scale follower lists
    Audience
      Backend engineers
      Platform builders
    Key findings
      Regex filtering performed poorly
      Client multiprocessing not recommended
      Row-key lookup preferred

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

What do people build with it?

USE CASE 1

Build a live-streaming following list that shows only currently online streamers.

USE CASE 2

Benchmark BigTable query strategies for massive social relationship data.

USE CASE 3

Test row-key design patterns for real-time presence filtering at scale.

USE CASE 4

Prototype a Twitch-like presence system using Google BigTable.

What is it built with?

PythonGoogle BigTableHappyBase

How does it compare?

doitintl/bigtable-follower-testing0verflowme/learnings0verflowme/r2ai
LanguagePythonPythonPython
Last pushed2021-09-202022-06-182025-11-19
MaintenanceDormantDormantQuiet
Setup difficultymoderateeasymoderate
Complexity4/51/53/5
Audiencedeveloperresearcherdeveloper

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · moderate Time to first run · 30min

Requires a Google Cloud BigTable instance and associated credentials to run the query experiments.

No license information is provided in this repository, so permission to use, modify, or distribute the code is unclear.

So what is it?

This project is a testing playground for figuring out how to efficiently look up who a user is following, but only show the ones who are currently online. Think of a Twitch-like app where you follow several streamers: when you open your following list, you want to see which of those streamers are live right now, without having to check each one individually. The data is stored in Google BigTable, a database designed for very large-scale applications. Each user's ID is converted into a hashed string for even data distribution. A record's key looks like "user_id:following:streamer_id," creating a clear link between a viewer and each person they follow. The project explores several approaches to querying this data, ranging from filtering on the database side to pulling records and filtering on the application side. The most complete script also demonstrates toggling a streamer's status between online and offline across all of their followers' records. This would be useful for engineers building social platforms or live-streaming services where relationship data is massive and needs to be filtered by real-time presence. For example, a platform with millions of users following thousands of streamers needs a way to quickly serve a viewer's personalized list of who is currently live. The project tests which query strategies perform well enough to handle that scale. What stands out is the honest, experimental nature of the repo. The author openly notes which approaches performed poorly, such as regex-based filtering on the database side and multiprocessing on the client side, neither of which are recommended. The project compares using a direct BigTable client versus a compatibility layer called HappyBase, and ultimately leans toward a specific row-key lookup approach that matches an existing production architecture. It reads like an engineer's working notebook for benchmarking real tradeoffs.

Copy-paste prompts

Prompt 1
I'm building a live-streaming app where users follow many streamers. Help me design a BigTable row-key schema so I can quickly look up which followed streamers are currently online, inspired by the bigtable-follower-testing project.
Prompt 2
Write a Python script using the Google BigTable client that queries all streamers a user follows and filters for those currently live, using a row-key prefix lookup approach.
Prompt 3
Compare the performance tradeoffs of filtering BigTable rows on the database side using regex versus fetching rows and filtering in the application layer. Which approach is better for a Twitch-like following list at scale?
Prompt 4
I need to toggle a streamer's online status across all of their followers' records in BigTable. Write a Python function that updates all relevant rows when a streamer goes live or offline.

Frequently asked questions

What is bigtable-follower-testing?

An experimental testing project that benchmarks different ways to query Google BigTable for a user's followed streamers who are currently live, useful for large-scale social or streaming platforms.

What language is bigtable-follower-testing written in?

Mainly Python. The stack also includes Python, Google BigTable, HappyBase.

Is bigtable-follower-testing actively maintained?

Dormant — no commits in 2+ years (last push 2021-09-20).

What license does bigtable-follower-testing use?

No license information is provided in this repository, so permission to use, modify, or distribute the code is unclear.

How hard is bigtable-follower-testing to set up?

Setup difficulty is rated moderate, with roughly 30min to a first successful run.

Who is bigtable-follower-testing for?

Mainly developer.

Open on GitHub → Ask about another repo

This repo across BitVibe Labs

Verify against the repo before relying on details.