So, I got the crazy idea to migrate tsumea to a different plan with our webhost since we were running really outdated versions of server software (php, control panel, everything). There were some interesting features that I wanted to test and implement on the site and that required a higher version of php installed and since a lot of things on the old server were many generations out of date, an upgrade was necessary. And it made more sense to just change our hosting plan to a fresh new server already set up, pay a small setup fee, and migrate the gigabytes of files and databases myself rather than updating a whole lot of stuff via ssh.
So, that's what I did. Backing up and moving the data, and starting up on the new server was easy enough, but during the first week up, we were hitting some major issues. You might've noticed that the site was showing a gateway error or was running really, really slowly. I was getting a whole bunch of alerts where everything from the server cpu, memory, apache memory/cpu was getting exhausted. Every time anything reached its limit, apache would shut down and I would have to log in and kickstart it again. Sometimes this happened every 10 minutes. It was particularly strange since we were on a server that had the same specs as the old one. I thought I'd go through the server's logs to see what was showing up and ended up going through each and every error and fixing it, from php script incompatibilities to configuration issues.
After reducing all the logged errors down to just one, there was no improvement on server downtime. I went through our content management system and started to shut down as many modules as possible to try and reduce overhead, and still nothing. It was when I put the site in maintenance mode (where no users are able to visit and resources consumed would be at its most minimum), I noticed that the server hovered at 40-50% capacity. So the server - the allocated memory, cpu, resources etc were at half capacity even when the site wasn't even running. This obviously wasn't the case with the old server, and I would guess that all the new software meant a whole lot of bloat and extra memory / cpu consumption. It meant we needed to get onto a new server with a new plan with more resources.
So, migration #2 happened again a week ago... uploading all the gigabytes of data and files took over 5 hours, setting the file permissions for directories took many additional hours, but it was the 5 hour response time from support on certain technical issues (I couldn't get connected to the database, finding out that they had to set up permissions to allow me to connect, and they did it with the wrong databases to start with). So it took about 5 hours to receive a response each time I had a query, following up on those queries, make any corrections and changes. This took an incredible chunk of time. It probably didn't help that I caught the flu and was in bed with aching bones, dizzy spells, and coughing spasms during all of this.
So last weekend was spent adding and installing all the libraries that were missing on the new server, stuff that managed image processing, right down to the little things such as the uploading progress bar when users upload files. The new server has 2gbs of ram which is near 3 times what we had on the old server and things seem to be running relatively smoothly now.
The unfortunate thing was that the feature I wanted to implement, which needed at least PHP v5.3, didn't end up working exactly as I wanted to.