Task #3362
Updated by REST Automat Admin over 6 years ago
* Install new base server *(DONE)*
** New server mentat-alt.cesnet.cz is ready and installed using Ansible
* Install Mentat system and Warden client on the server *(DONE)*
** Development version of Mentat system is installed on the server using Debian package system
** Warden client is installed on the server and connected to production instance of Warden server
* Perform data and service migration to new server
** Prepare database migration scripts *(DONE)*
** Prepare filesystem migration scripts *(DONE)*
** Migrate
The actual process of migration will be done according to the following checklist:
# Day before migration lower the TTL of relevant DNS records for mentat-hub.cesnet.cz and mentat-alt.cesnet.cz servers
# Presynchronize filesystem data (rsync), so that the actual migration will be much quicker later.
## report attachments
## RRD databases and chart images
## cache files
## persistent state files
## runlog files? (maybe not necessary)
## log files? (maybe not necessary)
# Shut down Warden client on mentat-hub.cesnet.cz and mentat-alt.cesnet.cz servers and let Mentat empty all queues.
# Shut down Mentat systems on mentat-hub.cesnet.cz and mentat-alt.cesnet.cz servers
# Perform database migration
## users
## groups
## filters
## networks
## reports
## event statistics
# Perform filesystem migration - same data as above
# Perform configuration migration
## synchronize content of @/etc/mentat@ configuration directory
# Switch warden client certificates between mentat-hub.cesnet.cz and mentat-alt.cesnet.cz servers
# Switch hostnames and IP addresses between mentat-hub.cesnet.cz and mentat-alt.cesnet.cz servers
# Reboot both servers and pray to your favorite god, or as an atheist sit quietly with your hands in your lap ;-)
# Login to new mentat-hub.cesnet.cz and launch everything
## Launch Mentat backend services (daemons and scripts)
## Launch Warden client and verify messages are being stored into database
## Verify that the web interface is accessible
# Synchronize crontab for root
Event migration might not be necessary. If the Mentat will be running on new server for some time and without any downtimes, we could skip slow migration of events from MongoDB to PostgreSQL.