Actions
Task #3752
closedMigration from MongoDB to PostgreSQL
Start date:
10/10/2017
Due date:
% Done:
100%
Estimated time:
To be discussed:
Description
After many months of rigorous testing we have finally decided to migrate from MongoDB to PostgreSQL. The MongoDB performance and long response times are decreasing the usability of the web interface and even of the reporting subsystem. We will now abandon the MongoDB in favor of PostgreSQL.
The migration will occur in following phases:- Define data models for utility objects (users, groups, etc.)
- Implement new storage service module for utility object management.
- Rewrite all current code working with utility objects.
- Implement new storage service for IDEA event management (maybe it will be possible to use the same service as above, maybe there will have to be different one, highly optimized and with lower overhead).
- Rewrite all current code working with IDEA messages through persistent database storage.
Related issues
Updated by Pavel Kácha over 6 years ago
Notes for future production migration:
- both db engines on the same machine
- user/group management + relevant data – can be migrated independently of event db
- then management GUI
- we will need migration script, which will allow reporter/statistician use Perl DB until also rewritten
- statistician
- simple incremental conversion script from Mongo to PQ + GUI, which already looks into PQ
- then statistician can be written in python, switched and conversion script ditched
- reporter
- simple incremental conversion script from Mongo to PQ + GUI, which already looks into PQ
- then reporter can be written in python, switched and conversion script ditched
- incoming events forked into both old (Mongo) storage and new (PQ) storage daemons (events go simultaneously into both DBs)
- then GUI can be written independently
Updated by Jan Mach almost 6 years ago
- Related to Task #3362: Migrate Mentat system to new hardware added
Updated by Jan Mach almost 6 years ago
- Status changed from In Progress to Closed
- % Done changed from 0 to 100
On tuesday 24.7. 2018 we have successfully migrated our production instance of Mentat system to version of Mentat system that is completely written in Python and uses PostgreSQL as database backend. Therefore this task can be considered complete.
Updated by Jan Mach almost 6 years ago
- Related to Bug #2808: Python and batch_size constraint added
Actions