In a forum thread a couple of weeks ago, I mentioned that I was aiming to replace one or two of the servers of the site at some point this year.

None of the servers are currently showing signs of breaking down, nor do they lack performance or features so I'm not in a huge hurry for this. However, the firewall machine is getting to be quite old and it would be great to be able to replace the machine without the stress of having to do it immediately. My expectation for the life of a server is 4-5 years, meaning the firewall machine (deeproot) is far beyond this point; thus this post :)

In addition to deeproot, the web server (fungus) is another machine that's getting a bit on the old side and will be needing a replacement at some point (ideally this year or next).

This means I end up with a few different options for how to proceed with the replacement/upgrade:

1. Upgrade only deeproot, and deal with fungus in a year.
2. Upgrade both deeproot and fungus at the same time.
3. Upgrade both, but with a single (more powerful) machine that runs virtual machines.

There are a couple of factors that come into play in deciding which option I go for:

A. Cost. This is obviously always a concern. FUMBBL runs on hardware hand-picked to give a good mix between cost, reliability and performance. A deeproot replacement would cost roughly $1000 on its own (fairly low-end machine as it doesn't really need super performance on its own). Fungus, on the other hand, will need some more power in terms of CPU cores, memory and storage. I'm estimating roughly $1500 for a replacement. Building a virtual host machine to replace both would likely end up somewhere just north of $2000, depending on configuration. Whichever option I pick here, I'd end up doing a donation drive type thing to cover at least most of the cost.

B. Flexibility. A virtualized system adds flexibility which can be nice. For example, it would allow me to clone a system to do tests without risking breaking things on the live system (one example is upgrading the OS of the web server).

C. Simplicity. Having a bare-metal installation is very simple. Less components mean less things can go wrong.

It's unlikely that I will go with the option to upgrade two servers at the same time (option 2 above), so the choice comes down to a single server upgrade now, or build a VM host (and likely start with migrating deeproot this this new platform, with fungus being done once things have settled down and confirmed to work properly). Virtualization is something that has come up in the past (many years ago), and I still have some reservations about it. In theory, it's a complete no-brainer to virtualize due to the added benefits (backups, ability to move machines to new hardware without trouble, snapshots, etc) but in practice it does add a bit of complexity and there's a loss of performance due to the virtualization (maybe 10-15% or so these days; significantly better than in the past).

There's also the complexity of picking what parts to go with for a virtualization host. Scaling up to a reasonable system that has the capacity to run multiple machines (more CPU cores, more memory, more disk) is also more complex than hand-picking parts for a specific purpose.

Roughly speaking, this is what I'd be looking for in terms of the different machines:

1. Firewall: Cheap CPU (perhaps the i3-7300 at 2 cores/4 threads), 8GB RAM, 2 small SSDs (raid mirror; I like Samsung Pro drives), and some random extra stuff (motherboard, network cards, PSU, cooling). This is likely somewhat overkill and could be reduced in specs
2. Web server: Higher end CPU, 4-8 cores/threads, 16G RAM, larger SSDs (maybe 256GB for FUMBBL) and the extra random assortment of components.
3. Virtualization host: 8-16 threads/cores, 32-64GB RAM, even more storage (2x512GB minimum, perhaps more), extra components (slightly higher end even here; higher capacity PSU, perhaps a different chipset to allow more/faster memory, etc).

If you guys have suggestions or ideas or just simply comments about all this stuff, feel free to post them below :)

PS. I know some of you will suggest cloud hosting but that isn't an option for FUMBBL. It's simply not cost effective. I've looked at this option many many times and nothing will match the cost (both actual cost and cost structure) of self-hosting.