Kitsu's Journey, from a small grey box to a giant datacenter

A watercolor, green-haired foxfolk looking towards the viewer. Trees and mist surround them, a hill in the distance.

Here's a fun fact! Did you know that Kitsu, up until a few weeks ago, was self-hosted? Many Administrators do not know the address of their hosting provider's server farm, let alone where it may be sitting state-wise. It was a pride-point for me knowing where my user's data is – sitting in a small, nice looking box behind the keyboard I use when I'm working. Interesting, huh?

A small mini-PC with 2 front-facing USB-A ports, a USB C port, a headphone jack and a power button, on a small grey box with fabric on the top.

But, that came with a problem. My town is the lucky one that got access to the copper network before the rest of the towns around us did; with a few neighborhoods in particular being without internet, save for satellite and cellular. Heck, even the network connection for cellphones leaves something to be desired there. As the large telecom giants - Rogers & Bell, begin to trench our county for Fibre-optic installation, it was understood that we would be the last to get the internet installed.

The Problem

It's an open joke that Canada has the worst winters, and I take pride of that mockery in some respect, riding our polar bears  (whilst playing "who's lane is it, anyway?') to work and living in our igloos is nice, after all. This, unfortunately, extends to our copper lines, which get strained during the winter months, likely due to the young'uns getting away from school for the snow-covered months. This caused my modem to fail to connect to the servers each and every morning around 3 in the morning; meaning that if there was no intervention by the router or by myself, then my computers would not be able to connect to the internet until someone noticed and rebooted. Luckily, this was an issue that I was able to catch, and had began setting up practices to fix, 3AM alarms going off and phone calls each morning that I did _not_ want to wake up to, but had to regardless, stumbling out of bed to fix it; and sometimes crashing back into bed either way, leaving the server down for upwards of six hours. This was fixed to an excruciatingly long and unforgiving 3 minutes when I implemented a mandatory reboot for my router (after all, the modem was gone otherwise); but even then, it was not a reliable solution. I had to look elsewhere, to figure something out.

Calling the ISP

I love my ISP. They're pretty reliable, and they know what they're talking about – 80% of the time. I use an offshoot of the bigger ISP in our area, who was more than happy to help us out. It was the third call that finally got me a call from a technician from the backbone ISP, who was supposedly going to come to our house, fix our modem and install a new one if needed. I woke up on the couch to a technician calling from a private phone number, and heard that they were aware of this problem. it's been going on for a while now; something that is frustrating, but something we have to deal with. He asked me how I knew it was 3AM - not wanting to explain that I run a plethora of servers, I replied that my computers complain if they lose a connection to the internet (which they do complain publicly) After that brief chat, I knew that something had to change, and soon.

The Move

As I said before, I'm prideful that I know where Kitsu is. It shamed me that I had to move to a service that was away from my own home. With Bell's internet being an astounding 50mbps for CA$80 (an eye-watering price for so astoundingly little), I couldn't afford to move away from our current ISP - going from a 1Gb link to a 50mb link; and thus made the decision that I was going to move Kitsu away from the mainland. KitsuClub and KitsuGay were the services that I moved, and I made sure to go with a provider that everyone in my circles used, a german hosting company named Hetzner. Doing that, I think it made Kitsu a lot better. Kitsu was no longer going down daily, and now that a giant sustained load was lifted from the copper lines, I've noticed that my internet was able to keep itself on for longer than a day - the current best sitting at over 4 days of runtime without a need for a restart. Even then, a lot of Kitsu services are being ran at home; including the blog you're reading this post on right now, our (seldom used) Lemmy instance, and our homepage; KitsuLife. I personally think that in the end, the trade-off is worth it. Even in the future, I might be able to ask Hetzner to say hello to Kitsu, now running happily with neighbors, in a home that is being kept on by dedicated technical experts. It's something I wish I could become, but unfortunately I do not have the budget to be.

All this to say, I hope you're enjoying Kitsu, and I'm sorry for the downtime. Kitsu should be better now; since we're no longer running on a small box at home.