🧘‍♂️
Jin's Muse on Engineering
  • About Me
  • Leadership
    • About Work
      • Work Like Watching a Movie
      • The day you leave …
      • The inner game of work
      • Routinization - how great institutions become mediocre
      • Have You Lost Your Stories
      • I wish I knew when I was facing layoffs
      • How to get out of this boring job
      • Why Setbacks Can Be Good for Your Career
      • What do we get from returning to office?
      • What if this is the last year you work here
      • Learning, Balancing, Falling - Don’t Stop
      • Who is the boss?
      • Under the Emperor’s New Clothes
      • Are you getting the right opportunities?
      • The elements of a healthy organization
      • Do What You Love vs. Love What You Do
    • About Life
      • Accept the things we cannot change ...
      • Live backwards
      • Life, one breath away, one breath a time
      • Will to Power
      • Learn from stand-up comedian
      • Connect the dots - submit to life’s entropy
    • Leadership Pinciples
      • The thing about Amazon’s leadership principles
      • Vision, Mission, Roadmap - But First, Eat What You Cook
      • Calibration - how we compare
      • But We are on the same side!
      • Leaders need two plans
      • Whose results are you delivering?
      • Coaching through open ended questions
      • This is the new year, and who will you delight?
      • Anti-fragility and Over-thinking
      • So ... what’s your decision
      • No Super Hero in Team
      • Congrats, you are the owner!
    • Communication
      • What, So What, Now What
        • Just tell me what’s next!
      • The Power of Repetition
      • Passing the Story Along
      • Lost in Translation
      • Chopstick War
      • How you speak about the most impactful project
      • Are we really listening
    • Problem Solving
      • How we resolve conflicts
      • Problem Solving: Leader Define Process and Better Leader Define StandardPage
      • If Kant and Nietzsche read Annie Duke's "How to Decide"
  • Engineering
    • Best Practices
      • Accuracy Ain't Always the Truth
      • No excuse for bad engineering
      • Secret of good engineering: constraint, not more time, or resource
      • The Art of Picking the Right Problems
      • Choosing the Right Battle: The Path to Productivity and Satisfaction
      • The Power Balance in Software Development
      • Three Ways of Invent and Simplify
      • Work backwards from demo-able sprint goals
      • That last 1%: from good to great
      • When it is too hard ... don't do it!
      • What Engineering can learn from Sports: Time it!
      • Struggling with Sprint Goals? Try Working Backward
      • Why working on a “legacy service” is a good thing
    • Operational Excellence
      • On dashboard
        • Dashboard Quality Vs. Product Quality
      • How we turn operation problems into big data innovation
      • Don’t automate 100%
        • 99% Automated - deleted an on-call rotation!
      • How we got rid of log diving
      • No perfect runbook - map is not territory
      • Someone have to be oncall, why they can’t be you?
        • Why being oncall is a good thing
        • If Sherlock is On-Call ...
        • Inner Peace During Oncall
      • Monitoring Your Service Health Like Human
    • Software Architecture
      • The Curse of the "Next Generation Project"
      • Absurdity in Decoupling
        • Tension of Decoupling and Cohesion
      • The ideal software form
        • Composition Over Inheritance
      • Security and Safety
      • Architecture Properties for Security Services
      • Learn system design from Git - Multilayered Architecture
        • Learn system design from Git - Immutable content-addressable datastore
        • Learn system design from Git - Event Sourcing
        • Learn system design from Git - Evolutionary Architecture
      • Quality starts with local stack
      • Intention-Revealing Interfaces with Examples
      • Design Principles
    • Distributed Computing
      • The beauty of multi-process applications
      • Cook Concurrent, Parallel, Async, and Non-blocking Together (1)
      • Cook Concurrent, Parallel, Async and Non-blocking Together (2)
      • Cook Concurrent, Parallel, Async and Non-Blocking Together (3) - The Multitasking Myth
      • The Local and Global Optimization Trade-Off in System Design
      • Throttling, Traffic Shaping, Traffic Shedding and Circuit Breakers in System Design
      • Differences Between Service Discovery and Load Balancing
        • Load balancer
      • Power of Two Choices
      • Using High Percentile Latency to Detect Resource Constraints
      • Queuing's Impact on High Percentile Latency
    • Machine Learning
      • Are you underfitting or overfitting?
      • How to lead an underfitting team
      • How to lead an overfitting team
      • Project Management and Neural Networks
      • Cryptography and Machine Learning
    • Programming
      • Can you create a language interpreter?
    • Cryptography
      • Explore KMS with CodeWhisperer (and a Dash of Cryptography) - AES-GCM
      • Exploring KMS with a Dash of Philosophy - HASH
      • Explore KMS' RSA encryption: a tale of two keys
      • Explore KMS - the monkey business of Elliptic Curve Cryptography (ECC)Page
  • Project Management
    • Setting goals
      • The Words of Goals
      • Why deadline is a good thing
      • Can you ship it?
      • One Thing Worth Remembering Per Quarter
    • The art of project planning
      • Are you still playing poker at scrum planning
      • Plan and Planning - How we start a project
      • A project’s journey - from inception to reflection
      • Component Team, Feature Team and Tiger Team
      • Sprint demo is a stage
      • Milestone and date
      • Onboarding project
      • Have You Sent Your Status Reports
      • When do you know a project smells
      • Hackathon is coming
    • How to prioritize
      • How to prioritize your 700 ticket backlog
      • Kitchen Sink and Tech Debt
  • Product Management
    • Find Problem
      • Dare to be 10x better
      • It is better to do the right problem the wrong way …
      • The Art of Goal setting: reactive goals, proactive goals, and inspirational goals
      • The things we don’t do …
      • No, I don’t want a platform
  • SDE Career Development
    • Promotion
      • The thing about promotion
        • Write your own promo doc
          • How to write your promo doc
        • The Reason Not To Promote
        • The myth of promotion project
      • The thing about leveling: are you SDE I, II or III?
      • What a L5 SDE is expected in Amazon
      • Customer Obsession - Of Your Own Promotion
      • Still waiting for that perfect manager?
      • You and Your Manager
    • Being SDE
      • What does a KMS SDE do?
      • From Knowing to Mastery: SDE Levels and Guild Ranks
      • How do SDE paint a Tiger from a Cat
      • The need to code
        • So ... where is your code
        • Quantity and Quality - How many line of code did you write?
      • Learn the Machine First
      • Is college major important?
      • Become an industry expert
      • How, What and Why in Problem Solving
      • Integrate both Breath and Depth in Life Long Learning
      • Do one thing well
      • CS students should know computer science history
      • Why Micromanagement is good for you
      • The power of imprinting
    • The Different Levels of Diving Deep
    • Becoming a tech lead
      • The traits of top engineers
      • Road to L6 SDE
      • There is no such thing called L6 task
      • Become a SME
      • Owning a Project vs. Owning a Problem Space
      • Leading, coaching and instructing
  • SDM Career Development
    • What does a SDM do
      • The thing about SDM - organization leader
      • The thing about SDM - manager of managers
      • The thing about SDM - line manager
      • The satisfaction of being a manager
      • Coach first, manager second, learner always - how I learn to coach
      • Gardener vs. SDM
      • SDM wearing three hats
      • What is a SDM doing in a design review?
    • How to be a SDM
      • How not to Micro-Manage - Stopping Being A Task Master
      • How SDM Build Momentum And Connect the Dots
      • How a SDM shoots at a moving target
      • The Danger of Over-Abstraction for Managers
        • How AWS SDM Dive Deep (1) - Dashboard
        • How AWS SDM Dive Deep (2) - Datastore
        • How AWS SDM Dive Deep (3) - Infrastructure
        • How AWS SDM Dive Deep (4) - Deployment
        • How AWS SDM Dive Deep (5) - Correction of Error (COE)
        • How AWS SDM Dive Deep (6) - OPS Review
        • How AWS SDM Dive Deep (7) - Bar Raiser
      • You are a manager, but are you a coach?
      • What an SDM can learn from swimming coach
      • Building a Team like a Waterpolo Coach
      • What a SDM can learn from a new programming language?
      • Should a SDM be able to code like a SDE?
      • Manager, Leader and Ruler
      • Performance Evaluation - Scope, Complexity and Impact
      • A Manager’s Vision
      • A Manager's Judgment
      • Build team with a soul
      • Build a learning organization - tech talk
      • Should You Become a Manager?
      • Becoming a SDM
    • Lead Business Application Team
      • Platform Team and Business Application Team
      • Page
    • What should you expect from your manager?
    • Build a Team like a Human Learning System
    • A New Manager's Tribal Survival Guide
    • SDM should Advocate for Intention Revealing Interface
    • A 30 minutes daily standup? You are doing it wrong!
    • Project Status Meeting that Takes an Hour? You are doing it wrong!
    • How to Ask Questions as a New Manager
    • Constructive Feedback: A Managerial Dialogue
    • Distinguishing "Improvement" from "Development" Feedback
  • Interview Tips
    • Level of Your LinkedIn Profile?
    • Make it personal
    • Tell me about yourself in 3 minutes
  • Import
Powered by GitBook
On this page
  • L4 Load Balancer (Transport Layer Load Balancer) - AWS NLB
  • L7 Load Balancer (Application Layer Load Balancer) - AWS ALB
  • DNS-based Load Balancer - AWS Route53/CloudMap
  • Client side load balancing

Was this helpful?

  1. Engineering
  2. Distributed Computing
  3. Differences Between Service Discovery and Load Balancing

Load balancer

L4 load balancer works on connections; L7 load balancer works on requests; Client load balancer is more flexible but it needs to handle the complexity of making distributed load balancing decisions.

PreviousDifferences Between Service Discovery and Load BalancingNextPower of Two Choices

Last updated 1 year ago

Was this helpful?

L4 Load Balancer (Transport Layer Load Balancer) - AWS NLB

An L4 load balancer operates at the transport layer (Layer 4) of the OSI model. It balances client traffic based on transport layer information such as the source and destination IP addresses and ports.

When a client initiates a TCP connection, the L4 load balancer selects a backend server based on its load balancing algorithm (like Round Robin, Least Connections, etc.) and establishes a TCP connection with it. The load balancer then forwards packets from the client to the selected backend server without inspecting the content of the packets.

The load balancer maintains a table to track the client's IP and port with the corresponding backend server's IP and port. This allows the load balancer to route future packets in the same TCP connection to the same backend server, enabling connection persistence.

Common pitfalls of L4 load balancing include the inability to make routing decisions based on application-level data (like HTTP headers or cookies), and the potential for uneven load distribution if the load balancer's algorithm doesn't account for the actual load on the backend servers.

L7 Load Balancer (Application Layer Load Balancer) - AWS ALB

An L7 load balancer operates at the application layer (Layer 7) of the OSI model. It can inspect the content of the client's request and make routing decisions based on it. This allows for more sophisticated load balancing strategies, such as routing based on HTTP headers, cookies, or the URL path.

When a client initiates a TCP connection, the L7 load balancer establishes a TCP connection with the client. It then waits for the client to send an HTTP request. Once the load balancer receives the request, it selects a backend server based on its load balancing algorithm and the content of the request. It then establishes a separate TCP connection with the backend server and forwards the client's request.

Because the L7 load balancer maintains separate TCP connections with the client and the backend server, it can route different requests from the same client TCP connection to different backend servers. This allows for session persistence based on cookies or other application-level data.

Common pitfalls of L7 load balancing include increased latency due to the overhead of inspecting each request and maintaining separate TCP connections, and the potential for resource exhaustion if the load balancer has to maintain a large number of TCP connections.

DNS-based Load Balancer - AWS Route53/CloudMap

A DNS-based load balancer uses the Domain Name System (DNS) to distribute client requests across multiple backend servers. When a client makes a DNS query for the load balancer's domain name, the load balancer responds with the IP address of one of the backend servers. The client then establishes a TCP connection directly with the backend server.

The load balancer can use various strategies to select a backend server, such as Round Robin, geographic location of the client, or the load on the backend servers. However, once the load balancer has responded to the DNS query, it has no control over the client's TCP connection with the backend server.

Common pitfalls of DNS-based load balancing include the inability to perform session persistence (since the load balancer is not involved in the TCP connection), and the potential for uneven load distribution due to DNS caching (since clients may continue to use a cached IP address even if the load balancer would have selected a different backend server).

Client side load balancing

NGINX and the "Power of Two Choices" Load-Balancing Algorithm - NGINXNGINX
Logo
Rethinking Netflix’s Edge Load BalancingMedium
d
Logo