Project

General

Profile

Actions

Bug #1479

closed

Core, Storage/Mongo - _set_last_filter

Added by Radomír Orkáč over 10 years ago. Updated over 10 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Development - Core
Target version:
Start date:
03/30/2014
Due date:
% Done:

0%

Estimated time:
To be discussed:

Description

Ahoj Meku,

resim bug report #1450 - Poměrně malé dotazy trvají neúměrně dlouho:

"Není problém s {"Alert.Analyzer.@analyzerid":""}? Proč tam vlastně je?"

A po dlouhem hledani se mi zda, ze v mongodb dotazu chyba neni. K chybnemu vypisu dojde az zde:

lib/Mentat/Storage/Mongo.pm

sub _set_last_filter(;$$$$$$)
{
    my $self = shift;
    confess "Instance method not invoked on object instance" unless blessed($self);
    my ($type, $filter, $projection, $sort, $limit, $skip) = @_;

    $self->{LAST_FILTER} = '';

    my ($fstr, $pstr, $sstr, $fcorrected);
    eval {
        $fstr   = ($filter)?encode_json($filter):'';
        $pstr   = ($projection)?encode_json($projection):'';
        $sstr   = ($sort)?encode_json($sort):'';
    };
    if ($@ and $@ =~ /allow_blessed|convert_blessed/) {
        $fcorrected = clone($filter);
        $fcorrected = $self->_unbless_filter($fcorrected);
        $fstr       = encode_json($fcorrected);
        $sstr       = ($sort)?encode_json($sort):'';
    }

    $self->{LAST_FILTER}  = "[$type] Filter '$fstr'";
    $self->{LAST_FILTER} .= " with projection '$pstr'" if $pstr;
    $self->{LAST_FILTER} .= " with sorting '$sstr'" if $sstr;
    $self->{LAST_FILTER} .= " with limit '$limit'" if $limit;
    $self->{LAST_FILTER} .= " with skip '$skip'" if $skip;
    return $self;
}

Vstup:

Sun Mar 30 17:46:11 2014 mentat apache2[18920]: INFO: Filter: ( Alert/Analyzer/@analyzerid EQ "au1-pickup" AND Alert/Analyzer/@name EQ "LaBrea" )
Sun Mar 30 17:46:12 2014 mentat apache2[18920]: INFO: Filter: $VAR1 = {
          '$and' => [
                      {
                        'Alert.Analyzer.@analyzerid' => 'au1-pickup'
                      },
                      {
                        'Alert.Analyzer.@name' => 'LaBrea'
                      }
                    ]
        };

Sun Mar 30 17:46:12 2014 mentat apache2[18920]: INFO: Last filter before:

Bezproblemovy vystup:

Sun Mar 30 17:46:12 2014 mentat apache2[18920]: INFO: Last filter after: [FIND] Filter '{"$and":[{"Alert.Analyzer.@analyzerid":"au1-pickup"},
{"Alert.Analyzer.@name":"LaBrea"}]}' with sorting '{"Alert.DetectTime.@ntpstamp":-1}' with limit '30'

Vstup:

Sun Mar 30 17:47:06 2014 mentat apache2[18921]: INFO: Filter: (Alert/Source/Node/Address/ip EQ "1.93.34.214" OR Alert/Target/Node/Address/ip EQ
"195.113.254.3") AND ( Alert/Analyzer/@analyzerid EQ "au1-pickup" AND Alert/Analyzer/@name EQ "LaBrea" )
Sun Mar 30 17:47:07 2014 mentat apache2[18921]: INFO: Filter: $VAR1 = {
          '$and' => [
                      {
                        '$or' => [
                                   {
                                     'Alert.Source.Node.Address.ipv4.ip' => bless( {
                                                                                     'subtype' => 0,
                                                                                     'data' => ']" 
                                                                                   }, 'MongoDB::BSON::Binary' )
                                   },
                                   {
                                     'Alert.Target.Node.Address.ipv4.ip' => bless( {
                                                                                     'subtype' => 0,
                                                                                     'data' => '�'
                                                                                   }, 'MongoDB::BSON::Binary' )
                                   }
                                 ]
                      },
                      {
                        '$and' => [
                                    {
                                      'Alert.Analyzer.@analyzerid' => 'au1-pickup'
                                    },
                                    {
                                      'Alert.Analyzer.@name' => 'LaBrea'
                                    }
                                  ]
                      }
                    ]
        };

Sun Mar 30 17:47:07 2014 mentat apache2[18921]: INFO: Last filter before:

Problemovy vystup:

Sun Mar 30 17:47:07 2014 mentat apache2[18921]: INFO: Last filter after: [FIND] Filter '{"$and":[{"$or":[{"Alert.Source.Node.Address.ipv4.ip":"015D22D6"},
{"Alert.Target.Node.Address.ipv4.ip":"C371FE03"}]},{"$and":[{"Alert.Analyzer.@analyzerid":""},{"Alert.Analyzer.@name":""}]}]}' with sorting
'{"Alert.DetectTime.@ntpstamp":-1}' with limit '30'

Mej se, R.


Related issues

Blocks Mentat - Bug #1450: Poměrně malé dotazy trvají neúměrně dlouhoClosedRadomír Orkáč03/11/2014

Actions
Actions #1

Updated by Radomír Orkáč over 10 years ago

  • Assignee set to Jan Mach
Actions #2

Updated by Jan Mach over 10 years ago

  • Status changed from New to Feedback
  • Assignee changed from Jan Mach to Radomír Orkáč

Podíval jsem se na to a opravdu jsem našel chybu ve svém kódu, ale bohužel se netýkala problému s pomalými dotazy. Chyba spočívala v tom, že se chybně převáděl dotaz z hashe do stringu, ale tento převod sloužil pouze k zobrazení uživateli. Přidáním výpisu jsem ověřil, že databáze skutečně provádí správný dotaz, takže problém stále přetrvává. Opravuje commit commit:067130d7.

S původní chybou:

Mon Mar 31 09:20:16 2014 omnius hawat_server.pl[14256]: INFO: [MMSM] Executing MongoDB command: [FIND] Filter '{"$and":[{"$or":[{"Ale
rt.Source.Node.Address.ipv4.ip":"015D22D6"},{"Alert.Target.Node.Address.ipv4.ip":"C371FE03"}]},{"$and":[{"Alert.Analyzer.@analyzerid" 
:""},{"Alert.Analyzer.@name":""}]}]}' with sorting '{"Alert.DetectTime.@ntpstamp":-1}' with limit '30'
Mon Mar 31 09:20:16 2014 omnius hawat_server.pl[14256]: INFO: [MMSM] Executing MongoDB filter (RAW): $VAR1 = {
          '$and' => [
                      {
                        '$or' => [
                                   {
                                     'Alert.Source.Node.Address.ipv4.ip' => bless( {
                                                                                     'subtype' => 0,
                                                                                     'data' => '^A]"<D6>'
                                                                                   }, 'MongoDB::BSON::Binary' )
                                   },
                                   {
                                     'Alert.Target.Node.Address.ipv4.ip' => bless( {
                                                                                     'subtype' => 0,
                                                                                     'data' => '<C3>q<FE>^C'
                                                                                   }, 'MongoDB::BSON::Binary' )
                                   }
                                 ]
                      },
                      {
                        '$and' => [
                                    {
                                      'Alert.Analyzer.@analyzerid' => 'au1-pickup'
                                    },
                                    {
                                      'Alert.Analyzer.@name' => 'LaBrea'
                                    }
                                  ]
                      }
                    ]
        };

Jak je vidět z druhého výpisu (Dump přesně toho, co šlo do databáze), tak se vždy pokládal správný dotaz.

Svou chybu jsem tedy opravil a provedl merge a push opravy do devel větve.

Actions #3

Updated by Pavel Kácha over 10 years ago

  • Category set to Development - Core
  • Status changed from Feedback to Closed
  • Assignee changed from Radomír Orkáč to Jan Mach
  • Target version set to 0.6
Actions

Also available in: Atom PDF