Absurdity in Decoupling

“Hey Tommy, your team is launching a new fleet (a cluster of hosts running the same software across multi availability zones in AWS), what is it for?”

“Ah, we have a light load background workflow to run and we can’t use server-less stuff like Lambda or Elastic Container Service.” Tommy answered.

“Why can’t you run the workflow on the existing fleet?” I inquired further.

“Isn’t that coupling the workflow with the other services running on the host?” Tommy was confused, “Aren’t we supposed to decouple and go MicroService?”

“Yeah we should decouple. But that does not mean we have to run each tiny function on its own fleet!” I sighed, “Decoupling means we can change and extend modules, deploy and run them without impacting others. Sometimes running them inside the same monolithic process makes more engineering sense.”

The essence of 'coupling' is the dependency between two entities that necessitates simultaneous changes. Yet, the industry has observed an increasing tendency to “decouple” everything into micro services by default, regardless of its value. As a result, numerous hosts run single processes at meager utilization rates, all in the name of decoupling and adherence to the microservices paradigm. This prevalent anti-pattern squanders resources, and often complicates changes.

The process of deploying new features can become significantly harder, requiring numerous deployments across a host of different fleets, a situation further exacerbating host-level coupling complexity.

To optimize resources and workflow, it is crucial to reconsider the decoupling dogma:

1. Source Code Level Modularization: Code-level modularization allows for efficient organization and management. By confining changes to the smallest possible scope, it reduces the risk of unrelated parts of the system being affected.

2. Careful Consideration for Process Separation: The decision to split runtime into multiple processes or maintain a monolithic process should be based on careful study. Container often offers a good abstraction to allow processes to live in harmony in one host and scale to separate hosts seamlessly.

3. Economic and Engineering Prudence for Host Separation:” Splitting processes into different hosts or fleets should only be pursued when it makes economic and engineering sense. The cost of additional hosts, potential latency issues, network reliability, and operational overhead are among the factors that need careful assessment.

4. Availability Zones Deployment based on Necessity: Not all services require deployments across all AZs. Some services merely need to be present in more than one AZ to meet the Service Level Agreement (SLA) on high availability.

We can’t build a shed using the blueprint of a mansion, keep in mind Amazon’s Frugality leadership principle: “Accomplish more with less. Constraints breed resourcefulness, self-sufficiency, and invention”