3 min readJust now

–

Press enter or click to view image in full size

Image Generated with AI

“Changes in one part of a system should not affect other unrelated parts.” This statement captures one of the biggest benefits of micro-services when done well.

📦 Isolating Change:

I’ve mentioned this before, but I see a system as a living, breathing thing constantly evolving.

With a micro-services design, you can evolve a system by adding new services with new functionality without redesigning the whole system. This capability is one of the design’s biggest benefits.

However, evolution is not always about adding new services but also replacing them. You can also replace services with a micro-service design without disrupting the whole platform.

📚 Breaking down an Example:

Let’s say we have a micro-service-based platform with five services. All of the services depend on one service, service B.

Service B was not originally designed to be in such a pivotal role, but as time progressed, it found itself here. And, of course, it has some scaling issues.

To fix Service B’s scaling issues, we need to change the underlying database, implement additional caching, and rewrite it in a language that better suits the performance and concurrency requirements. Service B needs a significant overhaul.

With a micro-services-based design, as long as the new instance of Service B retains the interface contract and behaviors of the original, replacement doesn’t mean a major overhaul for the rest of our services.

In fact, outside of traffic management, the other services don’t even need to change. The new Service B can be a drop-in replacement.

At least, a drop-in replacement is how it’s supposed to work.

💥 Where it All Goes Wrong:

The approach falls apart when micro-services are not executed well.

🙅 Sharing the Database:

Changing Service B’s database should not inherently require the other services to change their database.

While you want to manage technology sprawl, if there is a good reason Service B should be different, that’s okay; let it be different.

If a change to Service B’s database must be coordinated with the other services, you find yourself in a design anti-pattern.

😵‍💫 Undocumented Contracts:

Another common problem with the drop-in replacement approach is when the contract for Service B is either undocumented, under-documented, where documentation exists but doesn’t fully capture the contract, or worse, just wrong.

While you can still build a drop-in replacement without clear contracts, a smooth transition requires a lot more testing and reverse engineering.

This is one of the reasons I’m a fan of gRPC and Protobuf; if versioned and released correctly, changes to the contract are very visible and documented to at least some degree.

😠 Shared Code:

Another challenge can be shared code across micro-services.

As a disclaimer, I advocate creating packages that can be shared across services. However, those packages should be built assuming the code will be used across multiple services.

When the package implementation assumes only one service will leverage it, it will likely have dependencies unique to that service. This tight coupling makes it very hard to make changes that only affect one service.

📰 A Conflicting Reality:

While avoiding the above, in theory, allows you to drop in and replace Service B, the reality is sometimes different.

Even with the best-documented contracts, there are sometimes hidden behaviors.

Exercise caution while making changes to your micro-services. There are many ways to accomplish this, but I’ve found great success with Canary deployments.

Where only a portion of traffic is sent to the new instance/release. By sending only a portion of traffic to the new instance or release, you can validate that there are no issues while minimizing impact.

Have you had to replace a micro-service? Did it go well? Do you have any lessons learned that you want to share?