Last week we moved our Live server from our Old data center to a new one. The data center is in a remote location and we had to manage the show remotely ( of course, some one was out there physically moving the machines).
It was just one Machine. It was already such a challenge.
The planning began almost a month back. The entire planning pivoted around the question “How to keep the downtime to a minimal … And what this minimal would be?”
For any reasonably sized company such things are trivial, taken for granted. But more than a technical exercise, I felt it was a good exercise in Planning and Execution.
We reasoned out thus …
“If downtime should be minimal, then you cant pull the plug and move the machine. which means a clone of the current live machine should be moved first”.
And this automatically leads to
1. Clone the existing machine.
2. check for any IP address hardcoding in the code.
3. keep the clone in continuous sync with the live.
4. move the clone.
5. re-sync the clone with the live.
6. Up the clone and down the main.
Thats about it (I have omitted the specifics).
True to the Murphy’s law “Anything that can go wrong, will”, certain things did go wrong.
But the planning helped in overcoming most surprises. This experience also pointed out some areas where we
could have done better. Overall, I would say it was a smooth transition.
From a technical point of view, I learnt, from A, about the Internet Registrars, Domian servers, the “dig” command, DNS propagation, pointing the Registrar to a new Domain server and stuff like these.
We plan to move the other servers in another two days. We are lot more relaxed and more confident this time.