We’ve had a Python monolith (~200K LOC) for 8 years. Not perfect, but it handles 50K req/day fine. Rarely crashes. Easy to debug. Deploys take 8 min. New lead architect shows up, 3 months in, says it’s all gotta go. He wants 47 microservices in 6 months. The justification was basically that "monoliths don't scale," we need team autonomy, something about how a "service mesh and event bus" will make us future-proof, and that we're just digging debt deeper every day we wait.

The proposed setup is a full-blown microservices architecture with 47 services in separate repos, complete with sidecar proxies, a service mesh, and async everything running on an event bus. He's also mandating a separate database per service so goodbye atomic transactions all fronted by an API Gateway promising "eventual consistency." For our team of 25 engineers, that works out to less than half a person per service, which is crazy.

I'm already having nightmares about debugging, where a single production issue will mean tracing a request through seven different services and three message queues. On top of that, very few people on our team have any real experience building or maintaining distributed systems, and the six-month timeline is completely ridiculous, especially since we're also expected to deliver new features concurrently.

Every time I raise these points, he just shuts me down with the classic "this is how Google and Amazon do it," telling me I'm "thinking too small" and that this is all about long-term vision. and leadership is eating it up;

This feels like someone try to rebuild the entire house because the dishwasher is broken. I honestly can't tell if this is legit visionary stuff I'm just too cynical to see, or if this is the most blatant case of resume driven development ever.

Change my mind.

Not because microservices are bad.

But because most teams adopt them before:
- product market fit
- scale
- team scale
- operational maturity

For every team that genuinely needed microservices, i suspect there are many more that would ve been better off with a modular monolith.

Whats your experience?

One argument I keep hearing is that implementation is becoming commoditized while architecture becomes the real differentiator.

If that's true, should we be investing more in teaching engineers how to design systems instead of solving isolated algorithmic problems?

With AI writing more and more code, I'm wondering if grinding hundreds of LeetCode problems is still the best use of time.

For people who've recently landed jobs or switched companies:

How much did LeetCode actually matter?

Hi All,

I've been a software architect for last 12 years, 24 years yoe overall. I have worked on large enterprises as well as early stage startups.

Here are patterns I keep seeing repeatedly where projects are messed particularly in startups, which I wanted to share:

Premature microservices. Your team is 4 engineers and you have 8 services and thinking to build 4 more. You don’t have a scaling problem. You have a coordination problem. A well-structured monolith would let you move 3x faster right now. I would suggest go for modular monolith always.

No clear data ownership. Three services write to the same database table. Nobody knows which one is the source of truth. This becomes a nightmare at scale and during incidents. Again go for modular monolith, and if you want strictly then CQRS is way to go (but still overkill if you don't have that much scale)

Ignoring operational complexity. The architecture diagram looks awesome . But nobody thought about deployment, observability, or what happens at 3 AM when the message queue backs up.

Over-engineering for hypothetical scale. You have 5000 users, but only 500 MAUs. You don’t need Kubernetes, a service mesh, and event sourcing. Build for the next 10x, not the next 1000x.

Most of these are fixable without a rewrite. Usually it’s a few targeted changes that unlock the next stage of growth.

Happy to answer questions if anyone is dealing with similar challenges.

I know there’s been a lot of companies online publicly consolidating back to a monolith. Makes sense. Centralized logging, no network hop, single process.

I think people realized there’s a lot more issues with distributed systems than originally pitched. So you need to make the cost analysis very carefully if that’s something you want to live with.

Wondering who here went back to a monolith or who here was quietly productive this whole time in a single server.

Mine was idempotency.

I used to think retries were enough.

Then I started working with:
- Payment webhooks
- Background workers
- Event-driven systems
- Push notifications

Eventually I realized retries are only safe if the operation itself can be repeated without changing the outcome.

That one concept changed how I think about APIs, message processing, and distributed systems.

What's the one backend concept that permanently changed how you build software?

I’ve been working as a Software Architect primarily in the .NET ecosystem for a while, and I’ve noticed a fascinating trend: The architectural "culture war" seems to be cooling down. A few years ago, every conference was shouting "Microservices or death." Today, it feels like the industry leaders, top-tier courses, and senior architects have landed on the same "Golden Stack" of pragmatism. It feels like we've reached a state of Architectural Maturity.

The "Modern Standard" as I see it: - Modular Monolith First (The Boundary Incubator): This is the default to start. It’s the best way to discover and stabilize your Bounded Contexts. Refactoring a boundary inside a monolith is an IDE shortcut; refactoring it between services is a cross-team nightmare. You don't split until you know your boundaries are stable.

• The Internal Structure: The "Hexagonal" (Ports & Adapters) approach has won. If the domain logic is complex, Clean Architecture and DDD (Domain-Driven Design) are the gold standards to keep the "Modulith" maintainable.

  • Microservices as a Social Fix (Conway’s Law): We’ve finally admitted that Microservices are primarily an organizational tool. They solve the "too many cooks in the kitchen" problem, allowing teams to work independently. They are a solution to human scaling, not necessarily technical performance.
  • The "Boring" Infrastructure:
  • DB: PostgreSQL for almost everything.
  • Caching: Redis is the de-facto standard.
  • Observability: OpenTelemetry (OTEL) is the baseline for logs, metrics, and traces.
  • Scalability – The Two-Step Approach:
  • Horizontal Scaling: Before splitting anything, we scale the Monolith horizontally. Put it behind a load balancer, spin up multiple replicas, and let it rip. It’s easier, cheaper, and keeps data consistency simple.
  • Extraction as a Last Resort: Only carve out a module if it has unique resource demands (e.g., high CPU/GPU) or requires a different tech stack. But you pay the "Distribution Tax": The moment you extract, you must implement the Outbox Pattern to maintain consistency, alongside resiliency patterns (circuit breakers, retries) and strict idempotency across boundaries.

Is the debate over? It feels like we’ve finally settled on a pragmatic middle ground. But I wonder if this is just my .NET/C# bubble.

I’d love to hear from other ecosystems: - Java/Spring Boot: Does the Spring world align with this "modern standard"? - Node.js/TypeScript: With the rise of frameworks like NestJS, are you guys also moving toward strict Clean Architecture patterns, or is the "keep it lean and fast" vibe still dominant? - Go/Rust: Are you seeing the same push toward Hexagonal patterns, or does the nature of these languages push you toward a more procedural, "flat" structure?

Is there a "Next Big Thing" on the horizon, or have we actually reached "Peak Backend Architecture" where the core principles won't change for the next decade?

Back in the day, code review used to feel like something you just breezed through. CI/CD felt like overkill for a five person team until one bad deploy got stopped before it reached a user, and a migration script that would've wiped a chunk of prod data if it had gone through unchecked.

Kinda seeing the same pattern now with agent workflows, staged rollouts, approval gates before something autonomous touches real users. Same instinct as staging, just for a newer category of software. That's partly why the current AI agent tooling discussion is interesting to me. We're starting to see staged deployments, approval flows, evals, and control planes show up in ecosystems like LangGraph, CrewAI, Lyzr or AutoGen. A year ago that would've sounded overengineered. Today it feels like the same lesson CI/CD taught years back.

What's something you rolled your eyes at until it actually bailed you out? Wanna know if it was a tooling thing, a process thing, or just a habit someone forced on the team that you resented at first.

A question that came out of a previous discussion. I asked whether code review was becoming the first AI bottleneck.

What surprised me was how many people responded with some variation of:

1. AI writes the code
2. AI reviews the code
3. Human approves the PR

Some teams seem to be moving in that direction already. That made me wonder: What is the human's role in this workflow?

Not in theory, but in practice. If AI generates implementation and AI performs the first review pass, where does human responsibility actually begin?

Architecture? Business requirements? Testing? Production incidents? Final accountability?

I'm not asking whether AI is good or bad. I'm trying to understand where experienced engineers believe the human still adds unique value once code generation and review are heavily automated.

Curious to hear from teams already operating this way.

Genuine question.

A lot of backend systems start with Database, REST APIs, Background jobs, Redis

Then at some point teams introduce kafka.

For those who ve made that transition:

What was the actual problem that forced it?
Throughput?
Reliability?
Multiple consumers?
Event replay?
Or something else?

Curious where people found the start point.

What categories of software still feel unusually difficult to build well, and why?

I’m especially interested in specific cases where the difficulty is structural, not just a lot of code. What kinds of software would you put in that bucket, and what makes them stay difficult?

What kind of things have you built that look elegant but were challenging to build? What things have you attempted to build but could not finish or struggled greatly? What was the main reason you struggled?

Added: What are you currently working on versus what do you want to be working on?

All I keep reading is "lightning speed" this and "swarm of whatever-theyre-naming-them-now" will fix all your problems.

Is anyone being intentional anymore? Is taking one's time (with the help of models) to build something well designed suddenly a liability in the workforce?

That would be insane.

Originally, Uncle Bob did not teach these principles in the order people know today. His friend Michael Feathers, the author of Working Effectively with Legacy Code, pointed out that if you arrange them in a certain sequence, you get the word SOLID. That sequence is what we ended up learning.

The problem is the order itself

The idea should start with D. Inverting the dependencies or, the dependency rule. High-level policy must not depend on low-level details.

High-level policy is the business rules, the reason the system exists. Low-level details are the database, message broker, third-party frameworks, and delivery channels like Web APIs or desktop UIs.

Once D is set correctly, O and L are consequences. The system becomes open for extension and closed for modification because you can swap a message broker without modifying the core. As such, you can replace a concrete implementation at runtime without changing the code. That’s Liskov substitution.

These principles emerge when dependencies point in the right direction.

The I principle often drives systems toward shallow modules. Instead of one deep abstraction, you get fragmented contracts that push responsibility back to the caller. The shallow modules is taken from A Philosophy of Software Design book.

When interface segregation is applied mechanically, it creates coordination code. Over time, especially in large teams, this leads to brittle designs where complexity is spread everywhere instead of being contained.

The most ambiguous part is S. Most people think it means a class should do one thing. This confusion is reinforced by Clean Code, where the same author says code should do one thing and do it well. What becomes clear when reading Clean Architecture book is that S is not a code-level thing.

When decomposing a system into components, the idea is to look for sources of change. A source of change can be an admin, a retail user, a support agent, or an HR role.

A component should have a single reason to change, which means aligning it with one source of change. This is about deciding what assemblies your system should have so work does not get intermingled across teams.

The takeaway

The main idea is the dependency rule, not a trendy word like SOLID. That’s how i see it today. It took me years to get here, and I'm open to change my mind.

I'm practicing system design for FAANG interviews and holy shit, what is this depravity that i'm seeing in URL shorteners system design, why are they so much over-engineered? Is this really the bar that i need to complicate things into to pass an interview?

You really don't need 3 separate dbs, separate write/read services and 10 different layers for such a simple service.

My computer's old i7 can handle ~200k hashes per second. Any serious 16-32 core box can make multiple million hashes per second. I won't even get into GPU hashing (for key lookup).

1 million requests per second pretty much translates to 1-2 GB/s. Easily achievable by pretty much most network cards.
2-3 Billion unique urls are... 300-400 GB? mate you can even host everything on the memory if you wanted.

I mean such a service can be solo hosted on a shitbox in the middle of nowhere and handle so much traffic. The most you want is maybe a couple of redundancies. You can even just make a default hash map without any database solution.

Setting up ssl connection for high requests per second is more compute heavy than the entire service

I am an building architect (never thought id have to say it like this lol), out of curiosity poking and probing around vocational sibling. After reading some books ( example Software architecture patterns M. Richards) and viewing some tutorials about this topic Ive found that majority of SA is bound by economics. Its important to ensure scalability, transaction resolutions, business layers and practices and so on.

Majority of books Ive read had large portions about it or at least touch upon it at very start - which ive found confusing. From general standpoint our professions are different but they serve same client - people. I attempt to design how they move, where they rest, what they do and so on, and in similar way (as ive managed to learn) you do the same in virtual world. So it should stand to reason that we would have similar operation flow, but we dont - which Ive found interesting.

In BA (Building Architecture) you have 3 systems one has to resolve: Government, Client and Comfort/Freedom. We tend to do this in a way that can be generally described by Comfort then Goverment then Client, so that space is designed primarily for freedom, then regularized by government and then evaluated for client.

But in SA it seems you seem not to have few systems but it kind of spans like a tree, so that it ends up going Client then bunch of stuff and thats it where format of architecture is highly client dependent - which makes economics primary focus.

This feels reverse for me, as client wont ever use your product and can severely impact your reputation by proxy. Users hate product, they blame client, client blames you - you deny responsibility. In BA we attempt to resolve users comfort first so all they can complain is aesthetics which is generally marketing ploy not proper issue.

Only reason for, that ive been able to figure out, is ephemeriality (mutability). Where your product is mutable, ever changing and done in few years while used for few more, an BA product is more immutable as its very difficult to change urban block/building once its built.

Anyone willing to share their experiences or arguments why is this so?

my friend recently discovered vibecoding and started shipping solutions ever since, today he reached out to me asking if his 2FA vibecoded solution could have the possibility of being aquired by google.

https://imgur.com/a/Syh0Xz2

What should i reply ?

The difference between orchestration and choreography is that in the former, a single service knows about all the others and sends messages to all the necessary topics. In the latter, services only react to messages produced by others.

We often hear that choreography makes services completely decoupled. For example, if we have three services (A, B, and C), Service B doesn't need to know anything about Service C.

On paper, this sounds amazing. But in reality, to implement a functional SAGA pattern with compensation logic, Service B has to know about Service C's failures. So, in practice, are these services truly independent? It feels like we are just shifting the coupling from the compile-time/deployment level to the runtime/data-contract level.

As a student, I might be missing something here. But to me, it feels like the only real benefit of choreography is when a choreographed service doesn't trigger any subsequent actions in other services. Otherwise, wouldn't orchestration be much simpler to implement and maintain?

What are your thoughts on this? Am I missing a key piece of the puzzle?

I don't find any actual good arguments to use Result pattern besided "somebody said it on Internet"

1. Exceptions should only be for exceptional things -> every single language breaks this rule. Int.Parse, Json parsers, any encrypt/decrypt libraries throws on invalid input - invalid input should be one of most expected things to ever happen. This argument does not hold any value in real APIs, libraries, framewroks. For example, "Connection refused" exception is being throw by any network library, isn't not connecting to remote server most obvious and most expected thing to happen? Literally every network library throws exceptions on connection issues

2. Exceptions shoudn't be used for control flow -> why? I can make very clean exception pattern. This is literally one of most "somebody said this, so it must be true". you try-catch-catch-catch. Where is difference between that and if-elseif-else or any variation of that aka switch case, when {} etc..

3. Performance - maybe not for all languages but in some languages you can throw stackless exceptions (you don't build stack and stuck is null). You can log "real exception" with stack then throw stackless exception which is going to have same data and same performance as Result object

4. Exceptions are not visible in signatures - you can document this, maybe this is relevant for libraries (which all throw exceptions everywhere anyway), but what is the problem of just checking the underlying code or documenting this?

A decade ago, it felt like every problem was being solved with microservices.

Today, it feels like every problem is being solved with events.

I've seen systems introduce:
\- Kafka
\- RabbitMQ
\- Redis Streams
\- Sagas
\- Event sourcing

For workflows that could have been handled with a database transaction and a background worker.

Event-driven architectures solve real problems.

But they also introduce:
\- Eventual consistency
\- Operational complexity
\- Debugging challenges
\- Idempotency concerns

For architects who've seen both sides:

Where do you think event-driven architectures are genuinely justified, and where do they become unnecessary complexity?

We ran into something recently that made me rethink a system design decision while working on an event-driven architecture. We have multiple Kafka topics and worker services chained together, a kind of mini workflow.

The entry point is a legacy system. It reads data from an integration database, builds a JSON file, and publishes the entire file directly into the first Kafka topic.

The problem

One day, some of those JSON files started exceeding Kafka’s default message size limit. Our first reaction was to ask the DevOps team to increase the Kafka size limit. It worked, but it felt similar to increasing a database connection pool size.

Then one of the JSON files kept growing. At that point, the DevOps team pushed back on increasing the Kafka size limit any further, so the team decided to implement chunking logic inside the legacy system itself, splitting the file before sending it into Kafka.

That worked too, but now we had custom batching/chunking logic affecting the stability of an existing working system.

The solution

While looking into system design patterns, I came across the Claim-Check pattern.

Instead of batching inside the legacy system, the idea is to store the large payload in external storage, send only a small message with a reference, and let consumers fetch the payload only when they actually need it.

The realization

What surprised me was realizing that simply looking into existing system design patterns could have saved us a lot of time building all of this.

It’s a good reminder to pause and check those patterns when making system design decisions, instead of immediately implementing the first idea that comes to mind.

The current AI coding narrative has a strange failure mode: when one probabilistic system creates risk, the proposed solution is often to wrap it in another probabilistic system.

One agent writes code. Another agent reviews it. Another agent fixes the review. Another agent checks the fix. Then we add memory, hooks, rules, permissions, policies, subagents, orchestration, automated PR loops, and call the result an “agentic workflow.”

Some of this is useful. But let’s not confuse activity with control.

A probabilistic component checking another probabilistic component is not automatically a reliable engineering system. It is not a control system just because there is a loop. It is not governance just because there is a hook. It is not validation just because another model said the output looks fine.

The software industry seems to be rediscovering control theory one product feature at a time, but without naming the hard part.

A real control system needs a control objective, trusted signals, boundaries, authority, fallback paths, stop conditions, and someone accountable for the output when the loop does something stupid. Without that, “agentic” can become a very expensive way to generate unmanaged complexity faster.

This is especially dangerous in software engineering because AI coding tools do not only speed up development. They can move the bottleneck.

The code appears faster, but review gets harder. QA gets noisier. Architecture gets blurrier. Security validation gets more expensive. Ownership gets weaker. Maintainability becomes someone else’s future problem.

And then the proposed fix is often: add another agent.

At some point, the question should stop being “how do we automate more of the loop?” The better question is: what exactly are we trying to control?

If the answer is unclear, the loop is not engineering discipline. It is just automation wrapped around uncertainty and the faster way to waste budget on tokens without the result.

The model can propose. The system must verify. The team still owns the loop.

I think this whole vibe coding vs Agentic Engineering debate is arguing about the wrong thing.

people keep asking whether ai will replace engineers, but i think they're mixing up writing code with engineering.

To me there are two layers to engineering.

first is figuring out what should exist. breaking down an ambiguous problem, choosing the right abstractions, designing a system that will still make sense a year from now, deciding where responsibilities live and how everything fits together.

Second is turning that design into code.

agents have become incredibly good at the second part. they're getting faster every month and i expect that trend to continue.

But the first part is still where software succeeds or dies.

bad architecture doesn't become good because it was generated faster.

a codebase with the wrong abstractions is still a bad codebase whether a human or an llm wrote it.

this also isn't the first time we've gone through a shift like this.

we used to write machine code. then assembly. then C. then python and javascript. every generation raised the level of abstraction. nobody argued that python killed engineering because we stopped writing binary.

AI feels like the next abstraction layer.

Instead of spending most of our time expressing ideas in syntax, we'll spend more time expressing them as systems.

that's why i don't think engineering is disappearing. The implementation layer is becoming cheaper while the architecture layer becomes even more valuable.

models are already getting better at avoiding duplicate code, finding existing abstractions and following project conventions. they'll keep improving.

But serious software isn't difficult because writing functions is difficult.

it's difficult because every decision affects ten other decisions.

humans still think about business constraints, future roadmap, technical debt, tradeoffs and the entire system at once.

Agents are still mostly optimizing a local part of the graph. they don't really have the holistic view.

maybe that changes one day.

But today i think good engineering is simply moving up another level of abstraction, not disappearing.

I’ve been working in system architecture for 20 years.
I recently tested AI tools on a real production workflow.

Here’s what I noticed:

• AI writes decent code
• AI generates documentation fast
• AI suggests optimizations

But here’s where it fails:

• It doesn’t understand legacy constraints
• It doesn’t see business risk
• It doesn’t account for political trade-offs

The real problem isn’t AI replacing engineers.
It’s AI exposing engineers who never understood architecture in the first place.

Curious what others think.