It's cliche, but I really do feel reading the Art of Unix Programming gave me a very good sense early on for how to walk this line carefully. Unix programs are high quality - but they're also, ideally, small, and written with an eye to compositionality with the rest of the ecosystem. The best architecture is in the middle, and often the best first draft is a simple series of pipes.

https://www.catb.org/~esr/writings/taoup/html/

(Honest question)

What is the difference between this microservice architecture that gets a lot of hate here?

I'm not the person you're replying to, but here's my take: even though the two look conceptually similar, Unix programs are just a lot simpler. All programs run on the same machine, they read their input, execute, produce output which is piped to the next program, and terminate. Want to change something? Change one of the programs (or your command line), run the command again, that's it.

Microservices are a lot more complicated. You'll need to manage images for each of the services, a fleet of servers on which the services will be deployed, an API for services to communicate together, etc. In many (most?) cases, a monolith architecture will be a lot simpler and work just fine. Once you reach the scale at which you'd actually benefit from a microservice architecture (most companies won't ever reach this scale), you can start hiring devops and other specialists to deal with the additional complexity.

What actually gets hate, I think, is not microservices themselves, but the fact that microservices are often used in contexts where they are completely unnecessary, purely because of big tech cargo culting.

We only think of Unix programs as simple because we have many more abstractions nowadays. But you should compare a Unix program with DOS programs (probably CP/M also but I never wrote those myself) at the time. Poking directly at the hardware, using segmented memory, dealing with interrupts. The idea that a program should be well behaved, should accept things an inputs, should push outputs, and should have a virtual address space are actually huge abstractions over what could just be a block of code run on a spare OS. I'm not saying that microservices are better than monoliths, just that Unix programs aren't as simple as we think they are in a world where we're managing servers like cattle and not like pets.

    > should have a virtual address space

I'm pretty sure that most of the GNU POSIX command line tools were written before virtual address space (VAS) was common. And, as I understand, VAS is hidden from the programmer behind magical malloc().

That's a great question. Some might say it's because of the network - that makes microservices messy and so on. But I dont think so, from what I remember plan9 (the os, successor of unix), Rob Pike wanted to make it so that there is no difference between an object being on the network or outside the network. In unix philosophy, things have the same interface, it's easy to communicate. For microservices it would be REST api which is unique to network things. I honestly see a direct link between these ideas. Unix here is projecting a much nicer, simpler image but nonetheless they seem to overlap a lot. The result in both cases seems to be a hard to debug network of small utilities working together. The saving grace for unix is that you are mostly using stable tools (like ls, cat), everything is on your system so - you don't get to experience the pain of debugging 5 different half-working tools.

Everyone wants to make network objects the same as local objects. Nobody's ever succeeded.

This is the first time that I heard about it. Can you tell me about other examples?

CORBA

https://en.wikipedia.org/wiki/Common_Object_Request_Broker_A...

This predated REST. And the rest is history.

Microservices provide encapsulation and an API interface, but are not composable the way Unix programs are when e.g called by a Unix shell on a Unix OS.

Either microservice A calls into microservice B or there's a glue service calling into both A and B. Either way there's some deep API knowledge and serious development needed.

Compare with a (admittedly trivial, but just doing that is orders of magnitude less complex than web APIs) `ls -1 | grep foo | sed s/foo/bar/g`, exit codes, self-served (--help&al.) or pluggable (man) doc, and other "things are file-ish" aspects, signals (however annoying or broken they can be) and whatnot. There's a whole largely consistent operating system, not just in the purely software sense but in the "how to operate the thing" sense, that enables composition. The closest thing in http land would be REST, maybe, and even that is not quite there.

>> Microservices provide encapsulation and an API interface, but are not composable the way Unix programs are when e.g called by a Unix shell on a Unix OS.

Because the Unix programs all use pipes as their interface. When you simplify and standardize the "API" composition becomes easy. Microservices are more like functions or modules each running as separate processes - if you use the same language for the services and glue you can just compile them all together and call it a program right?

My take:

Unix utilities are stand alone and used as needed for specific tasks. They hardly change much, have no data persistence, usually no config persistence other than runtime params, and don't know about or call each other directly.

Microservices are moving parts in a complex evolving machine with a higher purpose that can and do affect each other.

The problem is that they are microservices in name only. Where a unix utility does a few hundred or a thousand lines of C code for its entirety, a microservice will depend on a complex software stack with a web server, an interpreter, an interface with a database, and so on.

It's easy to forget this complexity, but it comes at a cost in terms of performance and, above all, technical debt. The microservice will probably have to be rewritten in 5 years' time because the software stack will be obsolete. Whereas some unix utilities are over 40 years old.

In theory, only the network boundary. Which allows you to independently scale different parts of the system

In practice, a way of splitting work up between teams. Also it makes it easier to figure out who to blame if something breaks. Also a convenient unit for managerial politics

So because manager X "owns" microservice Y, it's going to stay around so that they have something to manage. Over time the architecture mirrors the organization

If somebody created a complex system of one hundred small unix utilities that all talk to each other over pipes I am sure it would get abd deserve a lot of hate. Unix utilities are nice to do very small, simple things, but there is a limit.

Composability.

This does not mean "it's a small component in a dedicated pipeline". It means "this is a component that's useful in many pipelines".

Microservices require Kafka (or a Kafka equivalent)

You can do something like 0mq, but still need something to coordinate configurations on where service-x is, like etcd.

That's a really good book. Thanks for mentioning it

> Always write the simplest thing you can, but no simpler.

I don't think so. Time and time again the client will insist on stuff like "the customer only needs a single email address/phone number" but you're going to pay for that one later if you do the simple thing and add an "email" column.

Same for addresses.

And a whole bunch of other stuff...you need to normalize the heck out of your DB early on, even if you don't need it now. The code you can mostly make a mess of, just choose a good tech stack.

> You're going to pay for that one later if you do the simple thing and add an "email" column

You'll only pay if the project survives long enough for that new requirement to actually surface, which often it won't.

Go down the simple path to start, and refactor to a more complex solution when it makes sense to do so. If experience tells you the client is definitely going to ask for it later, add a “break condition” that tells you when you need to upgrade. You can put entry points into the code - comments, interfaces - to make it easier to do the upgrade.

In many many cases this doesn't work and it crashes and burns the whole project/startup company when its necessary.

Sometimes messing up your fundamental architecture means that you hit a refactoring your company won't survive (while your competition grabs all the customers who wanted that feature your architecture doesn't allow).

This is where experienced lead engineers earn their worth - they understand which parts cannot be fudged and fixed later and need to be there from the get go.

Normalise the DB from the get go (doesn't really require much effort), then charge for the fact that "actually we have a customer who has 2 email addresses".

That's not a "false" dichotomy, that's an actual dichotomy: it's a real thing. Reading your comment, even tho you don't say it, I get the feeling you'd be with the rallying cry, "Bikeshedders Assemble!" hahaha! :)

It's a false dichotomy because it falsely implies there are only two options. Better than either are the other options which lie in between.

Oh, I see. That's a good point. But I don't think hu3's comment was suggesting there's only two options, just illustrating some possible margins to describe the landscape.

Maybe stavros' was hallucinating that strawman reduction in there, is what I think. Like you don't have to say it's a false dichotomy unless that's the only way you read it. The existence of something between the margins, should be obvious. Anyway, haha! :)

Well, it almost always becomes an actual dichotomy soon, for sufficiently large values of "soon."

Yes, the compounding effects of previous architectural decisions, but not if you take a balanced path, guided by awareness of the two extremes. So it needn't. Hahaha! :)

> Always write the simplest thing you can, but no simpler. Finding that line is where all the art is.

As much as I love simplicity, optimising for peak simplicity isn't always a good use of your time.

Simple enough is often good enough.

IME, it's easier to fix the latter than the former, if only because there is a limit to how large the latter can be possibly become.

Small projects are easier to fix than large projects.

unmaintainable is a weird word used far too much. There are genuine ways it can become unmaintainable of course but the main use I have seen people use it for is "I don't want to understand it and would rather rewrite it"