Project

General

Profile

Actions

Feature #4398

closed

Use established lists of possible values for suitable columns instead of dynamic enumerations

Added by Radko Krkoš almost 3 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Research and analysis
Target version:
Start date:
10/24/2018
Due date:
% Done:

0%

Estimated time:
To be discussed:

Description

Certain columns, such as:
  • Category,
  • Source type,
  • Target type,
  • Node type,
  • CESNET event class,
  • CESNET event severity,

can only contain a well-known set of values. This list must be compiled to allow for #4383. As that is the case, maybe we could replace the dynamic enumeration of present values with the pre-defined set.
RFC - Please discuss. Maybe a different list of columns is applicable. Be advised: it is possible to extend the ENUM data types in PostgreSQL but not to reorder it (without dumping and recreating the whole table - serious downtime).


Related issues

Is duplicate of Mentat - Feature #4383: Consider switching string classification fields to enumsRejectedJan Mach10/19/2018

Actions
Actions #1

Updated by Radko Krkoš almost 3 years ago

  • Related to Feature #4383: Consider switching string classification fields to enums added
Actions #2

Updated by Jan Mach almost 3 years ago

I think that this is not possible due to the nature of IDEA format. There is a possibility for different that defined values to occur for example within the Category field, data format must therefore allow storing arbitrary values, not only those in the specification. Additionally, some of the values like EventClass might seem to be picked from constant set of values, this is sadly only illusion. These sets are deployment specific and can change with addition of new data sources.

Actions #3

Updated by Pavel Kácha almost 3 years ago

I consider a dynamic evaluation a feature for users - there are only searchable values in the search pulldowns, not all values (event not sensible ones). So unless I misunderstand, I don't think it's a good idea.

However, Mek, if we implement #4383, we will cease to support arbitrary values no matter what. That does not depend on having computed or static lists.

Actions #4

Updated by Radko Krkoš almost 3 years ago

Jan Mach wrote:

I think that this is not possible due to the nature of IDEA format. There is a possibility for different that defined values to occur for example within the Category field, data format must therefore allow storing arbitrary values, not only those in the specification. Additionally, some of the values like EventClass might seem to be picked from constant set of values, this is sadly only illusion. These sets are deployment specific and can change with addition of new data sources.

This would mean that #4383 is essentially flawed and not a way to go.

Have you considered the remark in the last sentence? ENUM data types in PostgreSQL are extensible - if a new value is observed, it can be added. This could even work as a sanity check. For example, what other values for severity are there except for those well-known? Same for categories. In theory, any value is possible, in reality, new ones are not added often, we pick from a well-known set.

Inspector (or importer?) could then mark events using a non-standard value in any of the ENUM columns.

Only problem would be with those ENUMs where order is important (as is with categories - we would like to filter for all of the same first part in compound ones - i.e. Abusive.* - just like described in #4348) as reordering ENUM values is nontrivial, efficiently requiring table dump, retype and table restore. An example would be adding a "very low" severity into ("low", "medium", "high", "critical").

Do you see ENUMs viable at least for non-ordered sets of values? What about the checks? Is that a non-feature?

Actions #5

Updated by Radko Krkoš almost 3 years ago

Pavel Kácha wrote:

I consider a dynamic evaluation a feature for users - there are only searchable values in the search pulldowns, not all values (event not sensible ones).

I see. Then there is the issue that a "no value" option is present even for NOT NULL columns - this would return no data. Similarly for the "any value" option - meaningless filtering condition for a NOT NULL column - all entries match.

So unless I misunderstand, I don't think it's a good idea.
However, Mek, if we implement #4383, we will cease to support arbitrary values no matter what. That does not depend on having computed or static lists.

I think we need to discuss this, you two obviously have different understanding of this issue.

Actions #6

Updated by Radko Krkoš over 2 years ago

  • Status changed from New to Feedback
Actions #7

Updated by Radko Krkoš over 2 years ago

  • Related to deleted (Feature #4383: Consider switching string classification fields to enums)
Actions #8

Updated by Radko Krkoš over 2 years ago

  • Is duplicate of Feature #4383: Consider switching string classification fields to enums added
Actions #9

Updated by Radko Krkoš over 2 years ago

  • Status changed from Feedback to Closed

Closing as duplicate. All issues have been discussed.

Actions #10

Updated by Jan Mach over 2 years ago

  • Target version changed from Backlog to Rejected
Actions

Also available in: Atom PDF