Project

General

Profile

Actions

Bug #6857

closed

Search hosts view generates too broad database queries

Added by Radko Krkoš 7 months ago. Updated 5 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Design
Target version:
Start date:
01/05/2021
Due date:
% Done:

0%

Estimated time:
To be discussed:
No

Description

The Hosts tab in Mentat corresponding to the Search hosts view leads to a database query that is too broad and wastes system resources (and takes too long to execute).
The view allows to display results for a single IP address, yet it generates a DB query such as:

SELECT * FROM events AS "_mentatq(83_eaedjv)_" INNER JOIN events_json USING(id) WHERE "detecttime" >= '2020-12-29T12:00:00+00:00'::timestamptz AND "detecttime" <= '2021-01-05T12:00:00+00:00'::timestamptz

The scope of the query could be decreased, from at least two angles:
1. The returned data set could be filtered to only contain events related to the IP queried.
2. Returning all metadata columns JOINed by the JSON document is redundant, most of the data is read, formatted and transferred, but not processed afterwards.


Related issues

Related to Mentat - Feature #6926: Replace Hosts by TimelineClosedPavel Kácha01/13/2021

Actions
Actions #1

Updated by Radko Krkoš 7 months ago

A test run on mentat-hub for a single IP address not present took 5 minutes and 23 seconds, a rerun did not finish after 12 minutes (the query is heavily IO bound).

Actions #2

Updated by Radko Krkoš 7 months ago

  • Assignee changed from Jan Mach to Radko Krkoš
Actions #3

Updated by Pavel Kácha 7 months ago

Well, question is - isn't this usecase already covered (also in a more optimized way) by Timeline?

Actions #4

Updated by Radko Krkoš 7 months ago

Exactly, but I cannot answer that, hence the To be discussed flag.

Actions #5

Updated by Radko Krkoš 7 months ago

  • Status changed from New to Deferred
  • Assignee deleted (Radko Krkoš)
  • To be discussed changed from Yes to No

Based on videocall, this is a (somewhat) deprecated feature, it will be configured to only be accessible by admins and its role taken over by the Timeline feature. One day, this might be revisited and the implementation improved, if distinct (from Timeline) use cases are found.

Actions #6

Updated by Pavel Kácha 7 months ago

Actions #7

Updated by Pavel Kácha 5 months ago

  • Status changed from Deferred to Closed

Originally deferred, but closing - if we decide to revive it, it will have to be reviewed thoroughly and reworked anyway.

Actions

Also available in: Atom PDF