Feature #3365

Task #3374: Migrate all core modules from legacy Mentat

Implement mentat-enricher.py module

Added by Jan Mach about 2 years ago. Updated 9 months ago.

Status:ClosedStart date:03/21/2017
Priority:NormalDue date:
Assignee:Jan Mach% Done:

100%

Category:Development - Core
Target version:2.0

Description

Implement daemon module for IDEA message enrichment.

Key features:
  • Modular design
  • Enable easy writing and plugging-in of new enrichment sub-module, possibly from configuration file
Possible enrichment sub-modules:
  • Resolve target abuse contact/group
  • Geolocation
  • Hostname resolving

Associated revisions

Revision 6fa14513
Added by Jan Mach almost 2 years ago

Implemented prototype of mentat-enricher.py module.

Enricher is real-time message processing module with modular architecture and with support for easy implementation and usage of arbitrary enrichment plugins. Plugins can be developed based on predefined base class and dynamically loaded and instantinated upon module startup. (Redmine issue: #3365)

Revision f2ea9dea
Added by Jan Mach over 1 year ago

Feature: Implemented support for generating whois exception files from exception lists.

Exception files can be used to override internal whois resolving for designated networks to different abuse groups without modifying the original database records. This new feature enables users to generate whois exception files from simple text files containing lists of networks, which can be easily generated by anyone. (Redmine issue: #3385, #3365)

Revision ae16929b
Added by Jan Mach over 1 year ago

Implemented support for reloading enrichment plugins.

Message enrichment plugins need the reloading ability to enable fetching fresh information from original data sources. Otherwise the daemon would soon be using obsolete information for example for resolving target abuse groups. (Redmine issue: #3365)

Revision 5f6202ae
Added by Jan Mach over 1 year ago

Fix: Fixed the failing geoip message enrichment plugin.

Message enrichment completely failed in case these was no record in geoip city database for given IP (there was a check for None result type missing).

(Redmine issue: #3365)

History

#1 Updated by Jan Mach about 2 years ago

  • Parent task set to #3374

#2 Updated by Jan Mach over 1 year ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 80

#3 Updated by Jan Mach 9 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 80 to 100

Current state of this module is sufficient for production environment. We are finally releasing 2.0 version of Mentat system, so the period of frantic coding and implementation chaos is over. Any further improvements of this module will be done as they should in separate Redmine issues.

Also available in: Atom PDF