https://homeproj.cesnet.cz/https://homeproj.cesnet.cz/httpauth-login/favicon.ico?16194486082022-01-05T18:53:10ZHomeproj: Redmine for CESNETMentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=359522022-01-05T18:53:10ZRajmund Hruška
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Feedback</i></li><li><strong>Priority</strong> changed from <i>Normal</i> to <i>Low</i></li><li><strong>To be discussed</strong> changed from <i>No</i> to <i>Yes</i></li></ul>There are 2 places where this needs to be solved:
<ul>
<li><strong>thresholded events</strong> - Currently, the thresholding table in the database uses group name. Instead of that, a group ID could be used. The existing records in the database could be migrated by looking up an ID for a given group name.</li>
<li><strong>not reported events</strong> - The events to be reported are searched in the database by group name. The ID could be used instead of that, but that would make <code>_Mentat/ResolvedAbuses</code> much more difficult to comprehend. The idea I had was to store the list of the previous names of a group in the database and then search by this list as well. The previous names of a group are already kind of stored by <code>changelogs_items</code> table in the database, but this seems pretty annoying to parse.</li>
</ul> Mentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=359812022-01-20T08:31:37ZRajmund Hruška
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>In Progress</i></li><li><strong>To be discussed</strong> deleted (<del><i>Yes</i></del>)</li></ul><p>Actually, I will look at the possibility of parsing the previous names from the <code>changelogs_items</code> table.</p> Mentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=360022022-02-01T13:33:24ZRajmund Hruška
<ul></ul><p>Working at this issue made me realize that the following scenario is possible:</p>
<p>There are 2 groups: <strong>GROUP_A</strong> and <strong>GROUP_B</strong>. <strong>GROUP_A</strong> changes its name to <strong>GROUP_A_EDIT</strong> and <strong>GROUP_B</strong> changes its name to <strong>GROUP_A</strong>. Now when reporting for <strong>GROUP_A</strong> (former group <strong>GROUP_B</strong>) the events for <strong>GROUP_A_EDIT</strong> (former <strong>GROUP_A</strong>) will be fetched. I think this is OK though, as those events won't be reported because the sources from those events don't belong to networks owned by <strong>GROUP_A</strong> (former <strong>GROUP_B</strong>).</p> Mentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=360032022-02-01T14:09:43ZRajmund Hruška
<ul><li><strong>% Done</strong> changed from <i>0</i> to <i>10</i></li></ul><p>Rajmund Hruska wrote in <a href="#note-2">#note-2</a>:</p>
<blockquote>
<p>Actually, I will look at the possibility of parsing the previous names from the <code>changelogs_items</code> table.</p>
</blockquote>
<p>I found out that it's actually pretty easy to parse the name from the <code>changelogs_items</code> table.</p> Mentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=360042022-02-01T16:08:07ZPavel Káchaph@cesnet.cz
<ul></ul><p>Rajmund Hruska wrote in <a href="#note-3">#note-3</a>:</p>
<blockquote>
<p>Working at this issue made me realize that the following scenario is possible:</p>
<p>There are 2 groups: <strong>GROUP_A</strong> and <strong>GROUP_B</strong>. <strong>GROUP_A</strong> changes its name to <strong>GROUP_A_EDIT</strong> and <strong>GROUP_B</strong> changes its name to <strong>GROUP_A</strong>. Now when reporting for <strong>GROUP_A</strong> (former group <strong>GROUP_B</strong>) the events for <strong>GROUP_A_EDIT</strong> (former <strong>GROUP_A</strong>) will be fetched. I think this is OK though, as those events won't be reported because the sources from those events don't belong to networks owned by <strong>GROUP_A</strong> (former <strong>GROUP_B</strong>).</p>
</blockquote>
<p>All of this is pretty much best effort, unless we implement more structured logs or put IPs into Idea events.</p>
<p>Just a thought - to keep both exactness AND readability we could put into ResolvedAbuses both, something akin to "ResolvedAbuses": ["<a class="email" href="mailto:abuse@cuni.cz">abuse@cuni.cz</a>[37]"]. It's not nice, but get's the work done, however I'm not sure in how many places we actually do some parsing/extracting. Or, we couls add ResolvedAbusesIDs.</p>
<p>However, are the names <strong>inside JSON</strong> actually a problem? Couldn't it be solved by switching to IDs in events.resolved_abuses (the price is a need for translation for searching I guess).</p> Mentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=360062022-02-02T15:28:43ZRajmund Hruška
<ul><li><strong>Target version</strong> changed from <i>Backlog</i> to <i>2.10</i></li><li><strong>% Done</strong> changed from <i>10</i> to <i>50</i></li></ul><p>Rajmund Hruska wrote in <a href="#note-2">#note-2</a>:</p>
<blockquote>
<p>Actually, I will look at the possibility of parsing the previous names from the <code>changelogs_items</code> table.</p>
</blockquote>
<p>I managed to write a function for searching the not reported events with old names of the groups.</p> Mentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=360072022-02-02T15:31:44ZRajmund Hruška
<ul><li><strong>To be discussed</strong> set to <i>Yes</i></li></ul> Mentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=360282022-02-11T07:35:27ZRajmund Hruška
<ul><li><strong>To be discussed</strong> deleted (<del><i>Yes</i></del>)</li></ul> Mentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=360482022-02-20T14:02:19ZRajmund Hruška
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Feedback</i></li><li><strong>To be discussed</strong> set to <i>Yes</i></li></ul><p>So, what needs to be done next is to migrate <code>events_thresholded</code> table of <code>mentat_events</code> so it will use <code>groupid</code> instead of <code>groupname</code>. However, <code>groups</code> are stored in the <code>mentat_main</code> database, so cross-database query should be executed. I don't have any experience in cross-database querying but I found on the internet that <code>postgres_fdw</code> is the way to go. <a class="user active" href="https://homeproj.cesnet.cz/users/391">Radko Krkoš</a> would you also recommend this approach or another one?</p> Mentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=360542022-03-01T07:58:04ZRajmund Hruška
<ul><li><strong>To be discussed</strong> deleted (<del><i>Yes</i></del>)</li></ul> Mentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=360552022-03-01T07:58:24ZRajmund Hruška
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>In Progress</i></li></ul> Mentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=363152022-07-19T09:25:12ZPavel Káchaph@cesnet.cz
<ul><li><strong>Target version</strong> changed from <i>2.10</i> to <i>2.11</i></li></ul> Mentat - Bug #7200: Reporting events after group name has changedhttps://homeproj.cesnet.cz/issues/7200?journal_id=368102023-06-29T14:26:46ZRajmund Hruška
<ul><li><strong>Target version</strong> changed from <i>2.11</i> to <i>Backlog</i></li></ul>