Mentat

Will need (quite long) transition period, as we have Idea events, which are quite old (over year for CESNET networks).
Correct way would be to allow for "envelope" for Idea events, passing through the system, with metadata fields, however that would need quite a redesign. So we'll stay with current model, "injecting" our own fields into suitable place within Idea. Let's start with simply renaming to _Mentat, we might move some fields into more "standardized" positions later.

However it would be nice to also rename database fields, let's drop the "cesnet_" prefix in column names altogether to prevent another possible renames in the future.

As for transition/compatibility, the aim is to start adding fields into new (_Mentat prefixed) names in Enricher, Inspector and Storage, however consider both namespaces (_CESNET and _Mentat) when reading. That could be implemented in various ways (everywhere on use, translation in database layer, replacement by generated fields from metadata table, some combination thereof).

Also, in lib/mentat/stats/idea.py there is some work done with _CESNET (lines 99-103). In my previous commit, I just renamed the key but I am not sure whether it should be also compatible with the _CESNET or not.

Rajmund Hruska wrote in #note-7:

Also, in lib/mentat/stats/idea.py there is some work done with _CESNET (lines 99-103). In my previous commit, I just renamed the key but I am not sure whether it should be also compatible with the _CESNET or not.

From live video: I'm afraid it will have to be more clever - just adding new key would add two types of stats. Solutions (from the top of my head) to be considered:

• If LIST_AGGREGATIONS are considered linearly everywhere, solution might be just to add second line, like:

    [ST_SKEY_ABUSES,     '_CESNET.ResolvedAbuses',        KEY_UNKNOWN],
    [ST_SKEY_ABUSES,     '_Mentat.ResolvedAbuses',        KEY_UNKNOWN],

• Or make middle term multiple and modify statistical logic:

    [ST_SKEY_ABUSES,     ('_CESNET.ResolvedAbuses', '_Mentat.ResolvedAbuses'),        KEY_UNKNOWN],

• Or - of course - hardwire somehow.

Rajmund Hruska wrote in #note-5:

I noticed that in pynspect, there are also _CESNET keys in the test files. Should those keys be renamed to _Mentat?

From live video: They are concerning only tests, only example Idea, and not probably really used anywhere. Would make sense to rename them or get rid of the altogether, but I'd do that after we're ready on our side.

• database schema change
• Alembic migration

Pavel Kácha wrote in #note-10:

Rajmund Hruska wrote in #note-7:

Also, in lib/mentat/stats/idea.py there is some work done with _CESNET (lines 99-103). In my previous commit, I just renamed the key but I am not sure whether it should be also compatible with the _CESNET or not.

From live video: I'm afraid it will have to be more clever - just adding new key would add two types of stats. Solutions (from the top of my head) to be considered:

• If LIST_AGGREGATIONS are considered linearly everywhere, solution might be just to add second line, like:

[...]

• Or make middle term multiple and modify statistical logic:

[...]

• Or - of course - hardwire somehow.

I decided to make tuples - 38233d03

The problem was that the database wasn't migrated. When I ran migration manually by alembic upgrade head, the problem was resolved. Is database migration part of the build at alchemist? I can't find that quickly.

During the meeting, we noticed that there are some indexes with the old name (they contain _cesnet). I updated the migration.

I noticed that there are still old index names in 'Upgrading PostgreSQL from 12.x to 13.x' section in the documentation so I updated the names.