Config #6251
open
Rethink the indices and planner cost optimizations in light of the current usage patterns
Added by Radko Krkoš over 4 years ago.
Updated over 2 years ago.
Description
When the Mentat was migrated to PotgreSQL, the DB was version 9.6. The current version is v12, with v13 nearing. Tremendous amount of work was done upstream in the database, many of the premises valid then are no longer true. Also, the capabilities of Mentat grew in ways not foreseen or at least not originally intended (thanks to CTI).
We need to go back to drawing board, analyze the analytics processing needs of current Mentat, the processing capabilities of current PostgreSQL, and come up with a new database configuration and indexing schema, addressing the new requirements.
- Related to Bug #6211: Missing indices on aggregated columns added
- To be discussed changed from No to Yes
It seems that most of the current performance problems have proposed solutions that are being implemented. Are there any remaining data access methods with insufficient performance? Perhaps the CTI API was not analyzed thoroughly in this regard?
- To be discussed changed from Yes to No
- Status changed from New to Deferred
- Assignee changed from Radko Krkoš to Pavel Kácha
Based on discussion, currently the performance is adequate. NERD is happy with the improvement for IP search. CTI's opinion is to be queried on some future meeting.
- To be discussed changed from No to Yes
Radko Krkoš wrote in #note-5:
CTI's opinion is to be queried on some future meeting.
Any progress on this?
From CTI Apache log:
1961x /mentat/api/events/dashboard
92x /mentat/api/events/search
43x /mentat/api/events/metadata
6x /mentat/api/events/ID/show
- Status changed from Deferred to In Progress
- Assignee changed from Pavel Kácha to Radko Krkoš
Based on discussion, there are some problems with dashboard performance for CTI's use. Dashboard is based primarily on statistician's output, so the flow of operation should be analyzed. Also, the possibility of replacing statistician by timeline mechanism should be considered.
- To be discussed changed from Yes to No
The feedback from CTI is that the primarily used features of Mentat are:
- dashboard (intensive)
- search (rare)
- event display
Out of which, there seem to be some performance issues with the dashboard. As this component is based on statistician and we are evaluating the possibility of its replacement with the timelines, the use cases should be incorporated in the plan. In the meanwhile a mild analysis & optimization pass can be made on the slow queries of the dashboard.
- Status changed from In Progress to Deferred
This should be postponed until after the statistician situation is dealt with.
Also available in: Atom
PDF