Project

General

Profile

Actions

Feature #7621

closed

Reporting of post-hoc events detected by FTAS (and other)

Added by Radko Krkoš almost 2 years ago. Updated 7 days ago.

Status:
Closed
Priority:
High
Assignee:
Category:
Development - Core
Target version:
Start date:
01/04/2023
Due date:
% Done:

0%

Estimated time:
To be discussed:
No

Description

The Flow Traffic Analysis System (FTAS) routinely reports incidents that were prevented by automated measures. The prevention mechanism is generally a blocking of specific flows. The affected users should therefore be notified of the blocking and the characteristics of the prevented traffic. IDEA events sent by FTAS do contain the required information and can be transformed into such reports. Automatic way is preferable to the current state - personal warnings by FTAS administrators.
These reports must be distinguished from the existing ones, as they differ in two aspects:
- The recipient is a target of an attack,
- The reports are mostly informative, no action is expected of the recipient.


Related issues

Related to Mentat - Bug #7796: Mentat-reporter doesn't list all created labels in logClosedJakub Judiny10/15/2024

Actions
Actions #1

Updated by Pavel Kácha 7 months ago

  • Priority changed from Normal to High
Actions #2

Updated by Jakub Judiny 5 months ago

  • Assignee set to Jakub Judiny
Actions #3

Updated by Jakub Judiny 5 months ago

  • Status changed from New to In Progress
Actions #4

Updated by Pavel Kácha 5 months ago

Poznámky ze schůzky 2024-07-23:

  • Hlavičky v Idea _Mentat: vedle EventClass, EventSeverity, ResolvedAbuses použijeme TargetClass, TargetSeverity, TargetAbuses
  • V rozhraní Source groups, Target groups (kde to dává smysl zbavujeme se 'abuse')
  • Subclassy, Tresholding a relapse, filtry - bylo by vhodné zachovat, zvlášť pokud to povede k znovupoužití kódu
Actions #5

Updated by Jakub Judiny 4 months ago

  • Target version changed from Backlog to 2.14
Actions #6

Updated by Jakub Judiny 3 months ago

  • Category changed from Design to Development - Core
  • Status changed from In Progress to Resolved
Actions #7

Updated by Rajmund Hruška 2 months ago

Traceback (most recent call last):
  File "/var/mentat/venv/bin/mentat-reporter.py", line 51, in <module>
    MentatReporterScript().run()
  File "/var/mentat/venv/lib/python3.11/site-packages/pyzenkit/baseapp.py", line 1560, in run
    self._stage_evaluate()
  File "/var/mentat/venv/lib/python3.11/site-packages/pyzenkit/baseapp.py", line 1495, in _stage_evaluate
    analysis = self.runlog_analyze(self._prepare_runlog())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/mentat/venv/lib/python3.11/site-packages/pyzenkit/baseapp.py", line 1748, in runlog_analyze
    return self._sub_runlog_analyze(runlog, analysis)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/mentat/venv/lib/python3.11/site-packages/mentat/module/reporter.py", line 549, in _sub_runlog_analyze
    source_stats = severity_stats['Source']
                   ~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 'Source'
Actions #8

Updated by Rajmund Hruška 2 months ago

  • Status changed from Resolved to In Review
Actions #9

Updated by Rajmund Hruška about 2 months ago

I tried hot patching mentat-dev based on 19101ef5 and another issue occurred:

Traceback (most recent call last):
  File "/var/mentat/venv/bin/mentat-reporter.py", line 48, in <module>
    MentatReporterScript().run()
  File "/var/mentat/venv/lib/python3.11/site-packages/pyzenkit/baseapp.py", line 1559, in run
    self._stage_process()
  File "/var/mentat/venv/lib/python3.11/site-packages/pyzenkit/baseapp.py", line 1472, in _stage_process
    self._sub_stage_process()
  File "/var/mentat/venv/lib/python3.11/site-packages/pyzenkit/zenscript.py", line 353, in _sub_stage_process
    self.execute_script_command(cmdname)
  File "/var/mentat/venv/lib/python3.11/site-packages/pyzenkit/zenscript.py", line 410, in execute_script_command
    self.runlog[command_name] = cbk()  # pylint: disable=locally-disabled,not-callable
                                ^^^^^
  File "/var/mentat/venv/lib/python3.11/site-packages/mentat/module/reporter.py", line 389, in cbk_command_report
    result['reports'][group.name] = self._report_for_group(
                                    ^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/mentat/venv/lib/python3.11/site-packages/mentat/module/reporter.py", line 444, in _report_for_group
    result[severity] = reporter.report(
                       ^^^^^^^^^^^^^^^^
  File "/var/mentat/venv/lib/python3.11/site-packages/mentat/reports/event.py", line 187, in report
    events_passed_filters, aggregated_events, fltlog, passed_cnt = self.filter_events(group.name, events_fetched, is_target)
                                                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/mentat/venv/lib/python3.11/site-packages/mentat/reports/event.py", line 696, in filter_events
    filtered_groups, fallback_groups, fltlog = self.filter_one_event(src, event_copy, main_group, fltlog, is_target)
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/mentat/venv/lib/python3.11/site-packages/mentat/reports/event.py", line 621, in filter_one_event
    filtered_groups, fltlog = self._filter_groups(groups, event, fltlog, is_target)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/mentat/venv/lib/python3.11/site-packages/mentat/reports/event.py", line 578, in _filter_groups
    filter_list = self.settings_dict[group].setup_filters(self.filter_parser, self.filter_compiler, is_target)
                  ~~~~~~~~~~~~~~~~~~^^^^^^^
KeyError: 'abuse@xxxxxxx'

The actual name of the group is redacted. It specifically fails on this group though, the error occurred also on the second run.

Actions #10

Updated by Radko Krkoš about 2 months ago

Some reports contain wrong average packet sizes. For example https://mentat-dev.cesnet.cz/mentat/reports/M20241014TL-E7CXD/unauth - reported with 1, which does not compute from original data.

Actions #11

Updated by Radko Krkoš about 2 months ago

Rajmund Hruška wrote in #note-9:

I tried hot patching mentat-dev based on 19101ef5 and another issue occurred:

[...]

The actual name of the group is redacted. It specifically fails on this group though, the error occurred also on the second run.

This may have something to do with the fact, that this specific group is disabled on mentat-dev. There are 9 disabled groups total, but we probably do not hit them all (or at least the offending one is hit first):
https://mentat-dev.cesnet.cz/mentat/groups/list?state=disabled&submit=Search

Actions #12

Updated by Jakub Judiny about 2 months ago

Radko Krkoš wrote in #note-10:

Some reports contain wrong average packet sizes. For example https://mentat-dev.cesnet.cz/mentat/reports/M20241014TL-E7CXD/unauth - reported with 1, which does not compute from original data.

Missing +. Should be fixed now.

Actions #13

Updated by Jakub Judiny about 2 months ago

Radko Krkoš wrote in #note-11:

Rajmund Hruška wrote in #note-9:

I tried hot patching mentat-dev based on 19101ef5 and another issue occurred:

[...]

The actual name of the group is redacted. It specifically fails on this group though, the error occurred also on the second run.

This may have something to do with the fact, that this specific group is disabled on mentat-dev. There are 9 disabled groups total, but we probably do not hit them all (or at least the offending one is hit first):
https://mentat-dev.cesnet.cz/mentat/groups/list?state=disabled&submit=Search

WHOIS mapped IP to a network even though the network was from a disabled group. This should also be fixed now.

Actions #14

Updated by Rajmund Hruška about 2 months ago

  • Related to Bug #7796: Mentat-reporter doesn't list all created labels in log added
Actions #15

Updated by Rajmund Hruška 7 days ago

  • Status changed from In Review to Closed
Actions

Also available in: Atom PDF