Project

General

Profile

Actions

Bug #6497

closed

Mentat does not store events with missing Node[*].Name

Added by Pavel Kácha over 3 years ago. Updated about 3 years ago.

Status:
Closed
Priority:
Low
Assignee:
Category:
Development - Core
Target version:
Start date:
11/04/2020
Due date:
% Done:

100%

Estimated time:
To be discussed:
No

Description

Node[*].Name is not mandatory, however internal Idea converter seems to expect it and crumbles.

2020-09-03 13:50:28,291 mentat-storage.py [38488] ERROR: Component 'storage': Unable to store IDEA message 'mentat-hub.5009.1598934233.799019.42.159097562.idea' into database: 'Traceback (most recent call last):
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/daemon/component/storage.py", line 260, in cbk_event_message_process
    self.event_gateway(daemon, args)
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/daemon/component/storage.py", line 104, in _event_insert_bulk
    self.event_service.insert_event_bulkci(args['idea'])
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 951, in wrapped_f
    return func(other_self, *args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 1275, in insert_event_bulkci
    self.cursor.insert_event(idea_event)
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 593, in insert_event
    idea_pgsql = mentat.idea.sqldb.Idea(idea_event)
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/idea/sqldb.py", line 178, in __init__
    self.node_name = idea_event.get_detectors()
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/idea/internal.py", line 264, in get_detectors
    return [node['Name'] for node in self.get('Node', [])]
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/idea/internal.py", line 264, in <listcomp>
    return [node['Name'] for node in self.get('Node', [])]
  File "/var/mentat/venv/lib/python3.7/site-packages/typedcols.py", line 192, in __getitem__
    return self.data[key]
KeyError: 'Name'
'

Files

case.idea (755 Bytes) case.idea Pavel Kácha, 09/03/2020 02:50 PM

Related issues

Related to Mentat - Bug #6698: Use of deprecated function in test_sqldb.pyClosedPavel Kácha11/04/2020

Actions
Actions #1

Updated by Pavel Kácha over 3 years ago

Actions #2

Updated by Rajmund Hruška over 3 years ago

  • Status changed from New to In Progress
  • Assignee changed from Rajmund Hruška to Pavel Kácha

Node.Name is expected even at the database level - the column is set to be not null. What solution should I choose: making the column nullable or rejecting events with missing Node.Name?

Actions #3

Updated by Pavel Kácha over 3 years ago

  • Assignee changed from Pavel Kácha to Rajmund Hruška

Hmm, I see... thinking...

It happened in message with two Node fields, and only (last) one of them had Name missing. As for CESNET, it does not make sense to accept events without at least one Node.Name - and it even should not happen, because Warden server checks whether first Node.Name corresponds with client name in certificate.

So it seems to me we can settle on a compromise: Reject events with no Node.Name whatsoever (require at least one), but don't break on events with missing Name in some of the Node's. (Bonus: no change in db needed.)

Actions #4

Updated by Rajmund Hruška over 3 years ago

  • Status changed from In Progress to Feedback
  • Assignee changed from Rajmund Hruška to Pavel Kácha

I made a change so that Nodes with no names are accepted and stored in database as NULL. If there are no node names, an exception is raised.

https://homeproj.cesnet.cz/projects/mentat/repository/mentat-ng/revisions/7e1190fcd9330a4b7a6861568dcb2ed1500af066

Actions #5

Updated by Pavel Kácha over 3 years ago

Wouldn't it make sense to change get_detectors to not return empty Names (None) at all - omit them completely? Code at internal.py:430 would not be needed and sqldb.py:179 would be a tad simpler. Is get_detector used anywhere else, where "change" in semantics may interfere?

Actions #6

Updated by Rajmund Hruška over 3 years ago

Indeed, it gets simpler by not storing empty Names. That was my original idea, but I didn't manage to make it work. Here is the updated code:

https://homeproj.cesnet.cz/projects/mentat/repository/mentat-ng/revisions/2032b53df64f0fe801a7abe37a9b0480749ecf75

Actions #7

Updated by Pavel Kácha over 3 years ago

Ok. What exactly happens when this exception is triggered?

Actions #8

Updated by Rajmund Hruška over 3 years ago

The exception is stored in the log and the message is moved to errors.

Actions #9

Updated by Pavel Kácha over 3 years ago

  • Assignee changed from Pavel Kácha to Rajmund Hruška

Could we wedge in some small unit test?

Actions #10

Updated by Rajmund Hruška over 3 years ago

  • Assignee changed from Rajmund Hruška to Pavel Kácha
Actions #11

Updated by Rajmund Hruška over 3 years ago

  • Status changed from Feedback to Resolved
Actions #12

Updated by Rajmund Hruška over 3 years ago

  • Related to Bug #6698: Use of deprecated function in test_sqldb.py added
Actions #13

Updated by Jan Mach about 3 years ago

  • Status changed from Resolved to Closed

Closing, so that the version can also be closed.

Actions

Also available in: Atom PDF