Last Friday, I saw that the links in my blog were outdated. Since it was supposed to be a trivial matter, I logged in to the server and made the changes and then restarted. I got an HTTP 503 error message that basically says the service is unavailable. I tried reverting my changes but the issue was there to say.

Since I was away for the weekend, I only had the opportunity to try and fix things last Tuesday. Well, things only got worse. I received an email that my server was being used in activities suspiciously similar to a Distributed Denial of Service attack. I did a lockdown. It was apparently very effective because even my got locked out of the server.

I thought it was easy to be back up and running since I can easily spin up a new server and move over the data from the old one. It was supposed to be a trivial matter (sounds familiar). Anyway, it was not that easy since complications arose when I tried attaching the old storage device to my new server. The server boots up using the old volume!

By this time, I was at the point of giving up. The crazy thing was that I did not have even one backup of my blog. I know. For someone who is a software engineer and heavily involved with cloud infrastructure, this is basic.

After many tries (including trying all flavors of Linux just to see which one will prevent the server from booting up using the old volume), I finally found the solution. I created a snapshot of the old volume, created an image from that snapshot, and launched a new instance using that image. Before launching the instance, I added a new volume and explicitly defined the mount points so that the old volume will not be accessed during boot time. It worked. It actually worked. I hurriedly archived the whole directory where my blog was stored and downloaded the files.

Since I practically had raw files, I had to resurrect the old blog to export the data. It worked after making minor changes in the configuration. Creating a new server and restoring my data was a breeze.

The lesson? Never skip the backups. My plan now is to always export the data from my blog and store it somewhere safe (onsite and offsite). Aside from that, I will be generating weekly images of the server where my blog is stored.