February focus: our big upgrade!
Since omg.lol started a few years ago, the core services (API, database, and application) have always run on just a single server. For the first couple of years, it was a tiny $5 DigitalOcean droplet! It’s been upgraded and expanded a few times since then, and for the most part that strategy has kept up with demand — but now we’re at a point where things are starting to fail under load spikes, and that’s just not cool.
Our steady growth over the past year was met with a surprise leap in membership at the start of 2023. And now, just a month later, we’re starting to see daily (brief) outages. You may not even notice them, but I certainly do, and it’s simply not acceptable.
Rather than continue to grow the current infrastructure vertically, it’s time for omg.lol to take on a more horizontal approach — one that will exceed our current demands as well as allow for rapid scaling when needed. This means global regions, load balancers, multiple edge nodes, and all of that fancypants Enterprise™ stuff, which I normally tend to shrug off, but in this case will warmly embrace (because it’s necessary).
This will be my focus during the month of February, and omg.lol will be on something like a “code freeze” while I complete the work of migrating to the new environment. This is going to be tough for me, because I always want to jump in and fix broken things or make existing things work better (not to mention move forward on the dozen or so new things I have planned). But if I don’t hunker down and strongly prioritize this upgrade, it’ll wind up taking forever, and I worry that our small outages will turn into larger ones.
One thing you can do to help during this time is post any bugs or enhancement/feature requests to GitHub (or if you don’t use GitHub, send them to help@omg.lol). Of course I’ll be keeping an eye on things and anything urgent will receive proper attention, but anything that can wait will get some love as soon as we’re up and running with the new setup.
If you’re curious and want to follow along, I’ll share updates on the new environment and the migration over on social.lol.
Thanks for your patience and support over the next few weeks!
— Adam