Github Designing Data-intensive Applications Page

This is where gh-ost (GitHub Online Schema Tool) shines. Traditional ALTER TABLE locks the table, blocking writes for minutes or hours. gh-ost instead creates a shadow table with the new schema, copies data in small chunks, and replays the binary log of writes from the original table onto the shadow table—all while the application continues running. At the final moment, it performs a near-instantaneous atomic swap of table names. This is a direct implementation of Kleppmann’s discussion of and eventual consistency . The system is in a temporary, inconsistent state (rows exist in both tables), but the application logic hides this complexity. The maintainability payoff is immense: GitHub can deploy schema changes hundreds of times per day, a velocity unthinkable in a system that required scheduled maintenance windows. Conclusion: The Eternal Trade-Offs GitHub is not a perfect system. It has suffered outages, data inconsistencies, and scaling pains. But its evolution from a single MySQL database to a global, polyglot data platform exemplifies every major idea in Designing Data-Intensive Applications . It teaches us that there is no “one true way.” Reliable systems use replication, but fight lag. Scalable systems use sharding, but lose distributed transactions. Maintainable systems evolve online, but pay the complexity of dual-writes and temporary inconsistency.

GitHub’s architecture reflects this through and reconciliation . Consider the git push operation. Network requests can time out, and clients will retry. If GitHub processes the same push twice, it must not duplicate commits or corrupt the repository. By leveraging Git’s own immutable, content-addressed nature (where the same data yields the same hash), pushes are naturally idempotent. However, metadata operations are harder. When a webhook delivers a “push” event to an integration, the integration might fail. GitHub therefore implements an outbox pattern : the event is written to a persistent queue (like Kafka or their internal Resque system) before being sent. If delivery fails, the queue retries with exponential backoff, guaranteeing at-least-once delivery. The consumer, in turn, must be written to handle duplicates gracefully. github designing data-intensive applications

GitHub’s response was a masterclass in the two primary scaling techniques: and sharding . This is where gh-ost (GitHub Online Schema Tool) shines