Project

General

Profile

Actions
Copy link

Bug #7590

closed

/api/events/search sometimes crashes with TypeError: Expected 22 arguments, got 23

Added by Rajmund Hruška over 2 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Rajmund Hruška
Category:
Development - Database
Target version:
2.10
Start date:
06/06/2022
Due date:
% Done:

100%

Estimated time:
To be discussed:
No

Description

There is some weird error which is not replicable with the same URI.


INTERNAL SERVER ERROR

Request: /api/events/search?submit=Search&source_addrs=88.102.216.31&limit=100&dt_from=2022-05-23%2008:28:01
Traceback:
Traceback (most recent call last):
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask_login/utils.py", line 272, in decorated_view
    return func(*args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/vial/view/__init__.py", line 909, in dispatch_request
    items = self.search(form_data)
  File "/var/mentat/venv/lib/python3.7/site-packages/hawat/base.py", line 372, in search
    self._check_search_query_quota()
  File "/var/mentat/venv/lib/python3.7/site-packages/hawat/base.py", line 350, in _check_search_query_quota
    int(flask_login.current_user.get_id())
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 1711, in queries_status
    records = self.cursor.fetchall()
  File "/var/mentat/venv/lib/python3.7/site-packages/psycopg2/extras.py", line 349, in fetchall
    return list(map(nt._make, ts))
  File "/usr/lib/python3.7/collections/__init__.py", line 407, in _make
    raise TypeError(f'Expected {num_fields} arguments, got {len(result)}')
TypeError: Expected 22 arguments, got 23

Right after we have received this error, I tried running the same query but I got different error message.

INTERNAL SERVER ERROR

Request: /api/events/search?submit=Search&source_addrs=88.102.216.31&limit=100&dt_from=2022-05-23%2008:28:01
Traceback:
Traceback (most recent call last):
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask_login/utils.py", line 272, in decorated_view
    return func(*args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/vial/view/__init__.py", line 909, in dispatch_request
    items = self.search(form_data)
  File "/var/mentat/venv/lib/python3.7/site-packages/hawat/base.py", line 372, in search
    self._check_search_query_quota()
  File "/var/mentat/venv/lib/python3.7/site-packages/hawat/base.py", line 350, in _check_search_query_quota
    int(flask_login.current_user.get_id())
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 1711, in queries_status
    records = self.cursor.fetchall()
  File "/var/mentat/venv/lib/python3.7/site-packages/psycopg2/extras.py", line 345, in fetchall
    ts = super(NamedTupleCursor, self).fetchall()
psycopg2.ProgrammingError: no results to fetch

Related issues

Is duplicate of Mentat - Bug #7565: Timeline falls flat on certain queryClosed03/02/2022

Actions
Blocked by Mentat - Bug #6191: Implement thread safety in eventstorage service moduleClosedPavel Kácha01/14/2020

Actions
Actions
Copy link
 #1

Updated by Rajmund Hruška over 2 years ago

A similar error happened in March but with /timeline/search:

Message type: ERROR
Location:     /var/mentat/venv/lib/python3.7/site-packages/vial/app.py:168
Module:       app
Function:     log_exception
Time:         2022-03-29 10:36:49,147

Message:

Exception on /timeline/search?dt_from=2021-04-01+07%3A00%3A00&dt_to=2022-03-29+08%3A00%3A00&source_addrs=147.231.0.0%2F16&source_ports=&submit=Hledat [GET]
Traceback (most recent call last):
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask_login/utils.py", line 272, in decorated_view
    return func(*args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/vial/view/__init__.py", line 1040, in dispatch_request
    self.custom_search(form_data)
  File "/var/mentat/venv/lib/python3.7/site-packages/hawat/blueprints/timeline/__init__.py", line 232, in custom_search
    self._search_events_aggr(fargs, qtype, aggr_name)
  File "/var/mentat/venv/lib/python3.7/site-packages/hawat/blueprints/timeline/__init__.py", line 186, in _search_events_aggr
    dbtoplist = enable_toplist
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 947, in wrapped_f
    return func(other_self, *args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 983, in exc_handle_wrapper
    return func(self, *args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 1381, in search_events_aggr
    result = self.cursor.search_events_aggr(parameters, qtype = qtype, qname = qname, dbtoplist = dbtoplist)
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 715, in search_events_aggr
    return self.cursor.fetchall()
  File "/var/mentat/venv/lib/python3.7/site-packages/psycopg2/extras.py", line 349, in fetchall
    return list(map(nt._make, ts))
  File "/usr/lib/python3.7/collections/__init__.py", line 407, in _make
    raise TypeError(f'Expected {num_fields} arguments, got {len(result)}')
TypeError: Expected 23 arguments, got 3

Actions
Copy link
 #2

Updated by Rajmund Hruška over 2 years ago

And another one:

Message type: CRITICAL
Location:     /var/mentat/venv/lib/python3.7/site-packages/vial/app.py:414
Module:       app
Function:     eh_internal_server_error
Time:         2022-06-06 11:26:43,973

Message:

INTERNAL SERVER ERROR

Request: /timeline/search?dt_from=2022-01-01+17%3A00%3A00&dt_to=2022-03-22+17%3A00%3A00&source_addrs=147.231.0.0%2F16&source_ports=&submit=Hledat
Traceback:
Traceback (most recent call last):
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask_login/utils.py", line 272, in decorated_view
    return func(*args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/vial/view/__init__.py", line 1040, in dispatch_request
    self.custom_search(form_data)
  File "/var/mentat/venv/lib/python3.7/site-packages/hawat/blueprints/timeline/__init__.py", line 232, in custom_search
    self._search_events_aggr(fargs, qtype, aggr_name)
  File "/var/mentat/venv/lib/python3.7/site-packages/hawat/blueprints/timeline/__init__.py", line 186, in _search_events_aggr
    dbtoplist = enable_toplist
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 947, in wrapped_f
    return func(other_self, *args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 983, in exc_handle_wrapper
    return func(self, *args, **kwargs)
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 1381, in search_events_aggr
    result = self.cursor.search_events_aggr(parameters, qtype = qtype, qname = qname, dbtoplist = dbtoplist)
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/services/eventstorage.py", line 715, in search_events_aggr
    return self.cursor.fetchall()
  File "/var/mentat/venv/lib/python3.7/site-packages/psycopg2/extras.py", line 349, in fetchall
    return list(map(nt._make, ts))
  File "/usr/lib/python3.7/collections/__init__.py", line 407, in _make
    raise TypeError(f'Expected {num_fields} arguments, got {len(result)}')
TypeError: Expected 22 arguments, got 3

Actions
Copy link
 #3

Updated by Rajmund Hruška over 2 years ago

  • Related to Bug #7565: Timeline falls flat on certain query added
Actions
Copy link
 #4

Updated by Rajmund Hruška over 2 years ago

  • Category set to Development - Database
  • Status changed from New to Feedback
  • To be discussed changed from No to Yes

Now I am pretty confident that the root of this issue is #6191. What do you think?

Actions
Copy link
 #5

Updated by Rajmund Hruška over 2 years ago

  • Status changed from Feedback to In Review
  • % Done changed from 0 to 100
  • To be discussed changed from Yes to No

#6191 was (hopefully) fixed, so this one should be fixed as well.

Actions
Copy link
 #6

Updated by Rajmund Hruška over 2 years ago

  • Blocked by Bug #6191: Implement thread safety in eventstorage service module added
Actions
Copy link
 #7

Updated by Rajmund Hruška over 2 years ago

  • Related to deleted (Bug #7565: Timeline falls flat on certain query)
Actions
Copy link
 #8

Updated by Rajmund Hruška over 2 years ago

  • Is duplicate of Bug #7565: Timeline falls flat on certain query added
Actions
Copy link
 #9

Updated by Pavel Kácha over 2 years ago

  • Status changed from In Review to Closed
Actions
Copy link

Also available in: Atom PDF