Project

General

Profile

Actions

Feature #7577

open

Implement the notion of detector credibility and use it for reporting

Added by Pavel Kácha 8 months ago. Updated 6 days ago.

Status:
In Progress
Priority:
Normal
Category:
Development - Core
Target version:
Start date:
03/23/2022
Due date:
% Done:

50%

Estimated time:
To be discussed:

Description

Some detectors are a bit unreliable and sometimes report false positives. We might consider reporting events from them only if more reliable detector hits, or more of less reliable detectors hit the same IP.
Note that detectors, which produce more types of events, might exhibit different levels of reliability for different types.

Possible solutions:

  • mark the reliability in the inspector
    • pro: allows for high granularity (not only detector)
    • con: not in line with Warden notion of detectors
  • implement db of detectors within Mentat
    • con: needs some (semi)manual means of keeping in line with Warden
  • implement full Warden client management
    • pro: one source of truth
    • con: needs to implement the API on the Warden side
Questions:
  • where and how to store the credibility information (or rules) - Warden or Mentat side? One value or more granular rules?
  • how to combine the values for reporting - two level, linear/nonlinear combination..?
Actions #1

Updated by Rajmund Hruska 6 months ago

  • Assignee set to Rajmund Hruska
Actions #2

Updated by Pavel Kácha 4 months ago

From discussion 26. 7. 2022:

  • own client db (plus relatively independent import script, similar to network info from Negistry)
  • maybe in the future - plugin for Warden clietnt management
  • From Warden we need
    • widening of client info – let's start simply - big float
    • we need API for getting client info
    • later maybe for management
    • clients need new permission bit for client info management
Actions #3

Updated by Pavel Kácha 4 months ago

From discussion 2. 8. 2022:

  • Let's start lightly
    • New db for client data - id, name, desc, credibility
    • (Generated database is not a good point for reusing, as it server different purpose – set of client names from history for search forms is quite different from present set of clients.)
    • New view for clients, similar to Users, Groups, Networks, etc.
Actions #4

Updated by Rajmund Hruska 3 months ago

  • Status changed from New to Feedback
  • Target version changed from Backlog to 2.11
  • % Done changed from 0 to 10
  • To be discussed changed from No to Yes

Pavel Kácha wrote in #note-3:

From discussion 2. 8. 2022:

  • Let's start lightly
    • New db for client data - id, name, desc, credibility
    • (Generated database is not a good point for reusing, as it server different purpose – set of client names from history for search forms is quite different from present set of clients.)
    • New view for clients, similar to Users, Groups, Networks, etc.

I finished this part of the assignment. I also put source attribute to the detectors table and implemented a view for updating detectors. This got me thinking that updating detectors in Warden can be done using this view. When the information about the detectors will be fetched, the new detectors will be created with source Warden. Then, either after updating a detector or periodically, when import script is executed, the updated information will be sent to Warden.

Oh, and just to clarify - are client and detector the same thing? Or is it like cz.cesnet.nemea.filer and cz.cesnet.nemea.bruteforce are two detectors of the client cz.cesnet.nemea?

Actions #6

Updated by Rajmund Hruska 2 months ago

So, I checked the events detected in the last 24 hours.

What happens frequently is an event being detected by cz.cesnet.nemea and cz.casablanca.nemea

Quite a common is an event being detected by both cz.cesnet.tmc.log_analyzer and cz.cesnet.nemea.hoststats
https://mentat-alt.cesnet.cz/mentat/events/6efdb99e-cb4f-485f-adb4-eb9d01b7862e/show
https://mentat-alt.cesnet.cz/mentat/events/3455da72-d9f3-4bb0-bb91-4136cb104469/show

I saw one event being detected by cz.cesnet.nemea.hoststats and cz.cesnet.gc15.
https://mentat-alt.cesnet.cz/mentat/events/d5c8c4d5-93d5-4487-83b5-466de3f2951b/show
https://mentat-alt.cesnet.cz/mentat/events/1663564515_gc15.cesnet.cz_1212_src_ip_13.67.114.156_32/show

Actions #7

Updated by Rajmund Hruska 2 months ago

Rajmund Hruska wrote in #note-6:

So, I checked the events detected in the last 24 hours.

What happens frequently is an event being detected by cz.cesnet.nemea and cz.casablanca.nemea

Quite a common is an event being detected by both cz.cesnet.tmc.log_analyzer and cz.cesnet.nemea.hoststats
https://mentat-alt.cesnet.cz/mentat/events/6efdb99e-cb4f-485f-adb4-eb9d01b7862e/show
https://mentat-alt.cesnet.cz/mentat/events/3455da72-d9f3-4bb0-bb91-4136cb104469/show

I saw one event being detected by cz.cesnet.nemea.hoststats and cz.cesnet.gc15.
https://mentat-alt.cesnet.cz/mentat/events/d5c8c4d5-93d5-4487-83b5-466de3f2951b/show
https://mentat-alt.cesnet.cz/mentat/events/1663564515_gc15.cesnet.cz_1212_src_ip_13.67.114.156_32/show

And it's pretty much the same for 72 hours window.

Actions #8

Updated by Pavel Kácha 2 months ago

Thanks, Rajmund. So I guess we may have a tad of overengineering.

Mainly it means we don't have to hurry with the second phase of the plan – probabilities and calculated decision making in reporter as naive algorithm will be enough for some time.

Actions #9

Updated by Rajmund Hruska 22 days ago

  • % Done changed from 10 to 20

I am currently working on importing detectors data from Warden. Right now, I am sending everything from the database, as it can be seen here: https://hruska.vm.cesnet.cz/warden3/getClients?secret=test01. I am not sure if Mentat should be able to get everything or only a subset of client information.

Actions #10

Updated by Rajmund Hruska 19 days ago

  • % Done changed from 20 to 50

I merged the current work to devel and deployed on mentat-alt. The detectors can be accessed at https://mentat-alt.cesnet.cz/mentat/detectors/list.

Actions #11

Updated by Rajmund Hruska 6 days ago

  • Status changed from Feedback to In Progress
  • To be discussed deleted (Yes)
So based on today's meeting:
  • add hits (similar to filters) and the date when was the client created to detectors management
  • when reporting events, discard events from detectors with credibility lower than 0.5
  • inform about the discarded events in the reports
Actions

Also available in: Atom PDF