Nerd-Out: Simplicity vs. Efficiency

So, this is going to be a nerdy post. But hey, I created this blog, so I can write about things that are on my mind, right? I'll try to break down concepts as best as I can, but you're warned – this can get technical.

But let's start from the beginning…

At Magic Pages, I am running a good number of Ghost CMS websites now. The exact number fluctuates, because trials start and end on most days now (yay!), but it's a decent two-digit number.

But simply running a Ghost CMS website isn't the issue. Running them on scale is.

With "scale" I specifically mean that everything is automated. As a business owner, I do not want to lift a finger when somebody signs up. And as a customer, you certainly do not want to wait for me to get out of bed and manually create your website.

Scale means instant.

To achieve this instant provisioning of a new Ghost CMS website, I use Docker under the hood. Docker is a tool (well, a collection of tools really) that lets me put my software into a so-called container.

This container then gets a bunch of configuration – like a domain name, a storage location on the server, etc. But the software underneath is exactly the same, no matter how many of these containers I spin up.

Think about it like a socialist building project (for the lack of a better metaphor…). The apartments in one of these huge apartment blocks all look the same. Heck, in East Germany, they even standardised them throughout the country. Yet, the individual furniture and decoration you and your family bring make these apartments individual homes.

Now, back in early 2023, when I set up the first Magic Pages website I wanted things to be as simple as possible. So, everything that was related to a Ghost CMS website was put into one of these containers.

For our apartment metaphor, it means that you had a kitchen to cook your food. But you also have a little green house to grow vegetables. And a chicken pen on the balcony. And just in case the tap stops working, you have a huge water filtration system in one of the cupboards.

That makes life easy. No need to leave the apartment. Ever.

But it does get a little crowded from time to time. It's simple. But extremely inefficient.

Because, as it turns out, your neighbour is also growing vegetables. And they also have a water filtration system. And chickens. Even though they don't eat eggs. But you know…just in case.

And a month ago, I realised that that's what happened to the Ghost CMS websites on Magic Pages.

The Docker containers were stuffed. A single Ghost CMS website consumed about 700-900 MB of memory. If you scale this, any server quickly becomes quite stuffed. One of the things I absolutely wanted to avoid with Magic Pages.

Thankfully, we were still far away from any real performance issues, but it led me to explore some options.

Inside one of my Ghost CMS Docker stacks, I technically ran two containers. One for the Ghost website itself. And one for a MySQL database, which Ghost uses to store posts, settings, and so on.

Now, the big problem I saw was the fact that the container with the Ghost website only consumed about 100-200 MB of memory out of the 700-900 MB. The rest – about half a Gigabyte – was all due to the MySQL databases.

Woahhh.

Any developer will probably have an idea what my solution was. Exactly – I externalised the entire database server.

Instead of having the green house with all the vegetables in the apartment, Magic Pages now has a separate little backyard, where all the neighbours grow their vegetables next to each other. Everyone gets their little patch of grass assigned and nobody is allowed to draw outside these lines. But there is a central hydration system. The fertilizer is shared. The entire food production is simply more efficient.

So, instead of running countless little MySQL servers, I now run a single big server. And what happened?

Nothing. Performance of the Ghost websites is still superfast.

But the main server's memory consumption was just cut by around 66%.

Now, you might be wondering if I didn't just shift the consumption to another server. Well, not really. See, a MySQL Docker container does not just hold a MySQL database. It also runs its own little operating system and a bunch of stuff along with it.

On the big database server, I just need all of that once. So, a single MySQL database serving Ghost content does not consume around half a Gigabyte of memory any more. It only consumes 100 Megabytes.

Success!