šŸ§˜ā€ā™‚ļø
Jin's Muse on Engineering
  • About Me
  • Leadership
    • About Work
      • Work Like Watching a Movie
      • The day you leave …
      • The inner game of work
      • Routinization - how great institutions become mediocre
      • Have You Lost Your Stories
      • I wish I knew when I was facing layoffs
      • How to get out of this boring job
      • Why Setbacks Can Be Good for Your Career
      • What do we get from returning to office?
      • What if this is the last year you work here
      • Learning, Balancing, Falling - Don’t Stop
      • Who is the boss?
      • Under the Emperor’s New Clothes
      • Are you getting the right opportunities?
      • The elements of a healthy organization
      • Do What You Love vs. Love What You Do
    • About Life
      • Accept the things we cannot change ...
      • Live backwards
      • Life, one breath away, one breath a time
      • Will to Power
      • Learn from stand-up comedian
      • Connect the dots - submit to life’s entropy
    • Leadership Pinciples
      • The thing about Amazon’s leadership principles
      • Vision, Mission, Roadmap - But First, Eat What You Cook
      • Calibration - how we compare
      • But We are on the same side!
      • Leaders need two plans
      • Whose results are you delivering?
      • Coaching through open ended questions
      • This is the new year, and who will you delight?
      • Anti-fragility and Over-thinking
      • So ... what’s your decision
      • No Super Hero in Team
      • Congrats, you are the owner!
    • Communication
      • What, So What, Now What
        • Just tell me what’s next!
      • The Power of Repetition
      • Passing the Story Along
      • Lost in Translation
      • Chopstick War
      • How you speak about the most impactful project
      • Are we really listening
    • Problem Solving
      • How we resolve conflicts
      • Problem Solving: Leader Define Process and Better Leader Define StandardPage
      • If Kant and Nietzsche read Annie Duke's "How to Decide"
  • Engineering
    • Best Practices
      • Accuracy Ain't Always the Truth
      • No excuse for bad engineering
      • Secret of good engineering: constraint, not more time, or resource
      • The Art of Picking the Right Problems
      • Choosing the Right Battle: The Path to Productivity and Satisfaction
      • The Power Balance in Software Development
      • Three Ways of Invent and Simplify
      • Work backwards from demo-able sprint goals
      • That last 1%: from good to great
      • When it is too hard ... don't do it!
      • What Engineering can learn from Sports: Time it!
      • Struggling with Sprint Goals? Try Working Backward
      • Why working on a ā€œlegacy serviceā€ is a good thing
    • Operational Excellence
      • On dashboard
        • Dashboard Quality Vs. Product Quality
      • How we turn operation problems into big data innovation
      • Don’t automate 100%
        • 99% Automated - deleted an on-call rotation!
      • How we got rid of log diving
      • No perfect runbook - map is not territory
      • Someone have to be oncall, why they can’t be you?
        • Why being oncall is a good thing
        • If Sherlock is On-Call ...
        • Inner Peace During Oncall
      • Monitoring Your Service Health Like Human
    • Software Architecture
      • The Curse of the "Next Generation Project"
      • Absurdity in Decoupling
        • Tension of Decoupling and Cohesion
      • The ideal software form
        • Composition Over Inheritance
      • Security and Safety
      • Architecture Properties for Security Services
      • Learn system design from Git - Multilayered Architecture
        • Learn system design from Git - Immutable content-addressable datastore
        • Learn system design from Git - Event Sourcing
        • Learn system design from Git - Evolutionary Architecture
      • Quality starts with local stack
      • Intention-Revealing Interfaces with Examples
      • Design Principles
    • Distributed Computing
      • The beauty of multi-process applications
      • Cook Concurrent, Parallel, Async, and Non-blocking Together (1)
      • Cook Concurrent, Parallel, Async and Non-blocking Together (2)
      • Cook Concurrent, Parallel, Async and Non-Blocking Together (3) - The Multitasking Myth
      • The Local and Global Optimization Trade-Off in System Design
      • Throttling, Traffic Shaping, Traffic Shedding and Circuit Breakers in System Design
      • Differences Between Service Discovery and Load Balancing
        • Load balancer
      • Power of Two Choices
      • Using High Percentile Latency to Detect Resource Constraints
      • Queuing's Impact on High Percentile Latency
    • Machine Learning
      • Are you underfitting or overfitting?
      • How to lead an underfitting team
      • How to lead an overfitting team
      • Project Management and Neural Networks
      • Cryptography and Machine Learning
    • Programming
      • Can you create a language interpreter?
    • Cryptography
      • Explore KMS with CodeWhisperer (and a Dash of Cryptography) - AES-GCM
      • Exploring KMS with a Dash of Philosophy - HASH
      • Explore KMS' RSA encryption: a tale of two keys
      • Explore KMS - the monkey business of Elliptic Curve Cryptography (ECC)Page
  • Project Management
    • Setting goals
      • The Words of Goals
      • Why deadline is a good thing
      • Can you ship it?
      • One Thing Worth Remembering Per Quarter
    • The art of project planning
      • Are you still playing poker at scrum planning
      • Plan and Planning - How we start a project
      • A project’s journey - from inception to reflection
      • Component Team, Feature Team and Tiger Team
      • Sprint demo is a stage
      • Milestone and date
      • Onboarding project
      • Have You Sent Your Status Reports
      • When do you know a project smells
      • Hackathon is coming
    • How to prioritize
      • How to prioritize your 700 ticket backlog
      • Kitchen Sink and Tech Debt
  • Product Management
    • Find Problem
      • Dare to be 10x better
      • It is better to do the right problem the wrong way …
      • The Art of Goal setting: reactive goals, proactive goals, and inspirational goals
      • The things we don’t do …
      • No, I don’t want a platform
  • SDE Career Development
    • Promotion
      • The thing about promotion
        • Write your own promo doc
          • How to write your promo doc
        • The Reason Not To Promote
        • The myth of promotion project
      • The thing about leveling: are you SDE I, II or III?
      • What a L5 SDE is expected in Amazon
      • Customer Obsession - Of Your Own Promotion
      • Still waiting for that perfect manager?
      • You and Your Manager
    • Being SDE
      • What does a KMS SDE do?
      • From Knowing to Mastery: SDE Levels and Guild Ranks
      • How do SDE paint a Tiger from a Cat
      • The need to code
        • So ... where is your code
        • Quantity and Quality - How many line of code did you write?
      • Learn the Machine First
      • Is college major important?
      • Become an industry expert
      • How, What and Why in Problem Solving
      • Integrate both Breath and Depth in Life Long Learning
      • Do one thing well
      • CS students should know computer science history
      • Why Micromanagement is good for you
      • The power of imprinting
    • The Different Levels of Diving Deep
    • Becoming a tech lead
      • The traits of top engineers
      • Road to L6 SDE
      • There is no such thing called L6 task
      • Become a SME
      • Owning a Project vs. Owning a Problem Space
      • Leading, coaching and instructing
  • SDM Career Development
    • What does a SDM do
      • The thing about SDM - organization leader
      • The thing about SDM - manager of managers
      • The thing about SDM - line manager
      • The satisfaction of being a manager
      • Coach first, manager second, learner always - how I learn to coach
      • Gardener vs. SDM
      • SDM wearing three hats
      • What is a SDM doing in a design review?
    • How to be a SDM
      • How not to Micro-Manage - Stopping Being A Task Master
      • How SDM Build Momentum And Connect the Dots
      • How a SDM shoots at a moving target
      • The Danger of Over-Abstraction for Managers
        • How AWS SDM Dive Deep (1) - Dashboard
        • How AWS SDM Dive Deep (2) - Datastore
        • How AWS SDM Dive Deep (3) - Infrastructure
        • How AWS SDM Dive Deep (4) - Deployment
        • How AWS SDM Dive Deep (5) - Correction of Error (COE)
        • How AWS SDM Dive Deep (6) - OPS Review
        • How AWS SDM Dive Deep (7) - Bar Raiser
      • You are a manager, but are you a coach?
      • What an SDM can learn from swimming coach
      • Building a Team like a Waterpolo Coach
      • What a SDM can learn from a new programming language?
      • Should a SDM be able to code like a SDE?
      • Manager, Leader and Ruler
      • Performance Evaluation - Scope, Complexity and Impact
      • A Manager’s Vision
      • A Manager's Judgment
      • Build team with a soul
      • Build a learning organization - tech talk
      • Should You Become a Manager?
      • Becoming a SDM
    • Lead Business Application Team
      • Platform Team and Business Application Team
      • Page
    • What should you expect from your manager?
    • Build a Team like a Human Learning System
    • A New Manager's Tribal Survival Guide
    • SDM should Advocate for Intention Revealing Interface
    • A 30 minutes daily standup? You are doing it wrong!
    • Project Status Meeting that Takes an Hour? You are doing it wrong!
    • How to Ask Questions as a New Manager
    • Constructive Feedback: A Managerial Dialogue
    • Distinguishing "Improvement" from "Development" Feedback
  • Interview Tips
    • Level of Your LinkedIn Profile?
    • Make it personal
    • Tell me about yourself in 3 minutes
  • Import
Powered by GitBook
On this page

Was this helpful?

  1. Engineering
  2. Software Architecture
  3. Learn system design from Git - Multilayered Architecture

Learn system design from Git - Evolutionary Architecture

PreviousLearn system design from Git - Event SourcingNextQuality starts with local stack

Last updated 1 year ago

Was this helpful?

Like most people I heard the super hero story about how Linus Torvalds wrote Git in four days. We all like creation stories, don’t we?

The story was definitely incomplete. It seemed Linus did write the core Git functions in a few days but that was not the Git we are familiar with. The early version of Git was notoriously hard to learn and hard to use. It was designed for a few specific use cases. The so called user interface was just some shell scripts on top of the core primitives.

But what amazed me was how Git evolved from a specialized version control tool for Linux kernel community, into the best distributed version control system we know it today. Its evolutionary architecture allowed it to iterate rapidly through guided, incremental change across multiple dimensions.

Looking back, we can summarize some important architectural decisions that made Git possible.

  1. Clear, narrowly focused requirements. Git was developed out of a crisis: Linux kernel group needed a version control system and none of the existing systems would work for their use cases. Linus designed Git essentially for himself and the kernel community. He knew exactly what they wanted. There was no product management needed. The customer and developer were the same people. This was Customer Obsession at its extreme.

  2. Clear tenets to guide the design decisions and architecture evolution. Linus called out in his presentation he needed a version control system that is Reliable (with built in content integrity validation), Distributed (optimized for offline mode), High Performance (branch merge needs to be fast), and Tracking Content of the entire project, not individual files. The user interface was not his top concern but he knew user interface can evolve for the better quickly if he got the underlying architecture right.

  3. Agile development process. Linus shipped the Minimum Viable Product of Git in days and within a month Linux kernel was running on Git. A lot of the rich tools and performance optimizations like pack were done much later. He created a fast feedback loops from customers to guide where Git should go.

  4. Understand constraints. Linus knew clearly what he didn’t want - his constraints. He didn’t want Git to be like CVS. He knew Git needed to limit the number of files it could manage inside one repository. Linux kernel at the same time had 20k files and he tested 100k files. But Linus clearly said Git was not designed to manage 1 million files in one repository. The moral: knowing what you don’t want to do is as important as knowing what you want to do, maybe more important in many cases.

The success of Git is an evolution story.

ā€œSome call it evolution, And others call it God.ā€ - WILLIAM HERBERT CARRUTH

https://youtu.be/4XpnKHJAok8