An update on Infrastructure Month
Wednesday, March 1 update: everything changed again!
TL;DR: omg.lol has been upgraded to a much nicer technical setup! If you have an external domain pointed at omg.lol, please update your DNS to use this new IP address: 5.78.160.207 5.78.24.5
Eleven days ago, I announced that I’d be spending much of February working on a much-needed update to omg.lol’s infrastructure. Between a spike in growth, routine Mastodon stampedes, and someone who keeps trying to DDoSing us, our poor little server was really starting to feel the strain.
The occasional brief outages were starting to turn into longer and more problematic outages, so I put the pedal to the metal and accelerated our migration. It was a bit hectic, and there were a few rough patches in what was intended to be an otherwise stealth operation, but it’s finished! And we’re now operating with a far more resilient, load-balanced setup. As I type this I’m watching the access logs, and I see an attack underway at this very moment — but the new setup is happily handling all of the traffic regardless.
There’s still some cleanup work to do post-migration, and I’ll be tending to those things very soon (favicons are a mess currently, sorry). It shouldn’t take long to shake out the final remaining crumbs and then it’ll be back to the fun stuff: feature development/expansion and general good times.
External domain DNS
If you use an external domain with any omg.lol services, you probably have it pointed at the old server’s IP address (159.89.247.131). Please change this to our new IP address, which is 5.78.160.207 5.78.24.5. This will speed things up dramatically for you! Instead of trying to coordinate a single point in time for that IP address change, I simply made the old omg.lol server act as another component in the new setup — but because it’s hosted with another provider, it can’t sit behind the same load balancer that the new setup uses. And this means that it will always be a bit slower, so you’ll want to point your domain to the new IP address to take advantage of the new setup. (Soon, I’ll email everyone who hasn’t yet made this change with some info/instructions.)
Report any bugs
If you notice anything weird as a result of the upgrade, please let me know! You can email bugs@omg.lol or log an issue in GitHub (but soon we’ll have another, better way to report bugs and such).
Thanks for your patience over the past few weeks as I’ve worked through this upgrade. ❤️
— Adam