Mentat

Feature: Prototype implementation of next generation web user interface for Mentat system.

This commit represents huge amount of work and introduces new Python-based version of Hawat web user interface for Mentat system. It is built on top of the great Flask framework. Currently it is still work in progress and many features are implemented only as dummy stubs, however the core functionality is almost finished. Following is a list of current key features:

• Focus on modular architecture using Flask`s blueprint feature. * Multi-layered configurability (defaults, multiple file overrides). * Plugin system. * Internationalization support. * Support for multiple authentication systems.

This commit provides working albeit hollow starting point for future web interface development. IT IS NOT READY FOR PRODUCTION USAGE!!!

(Redmine issue: #3734,#3361)

Implemented demonstration whois and geoip lookup functionality in Hawat web user interface.

The lookup results are not styled yet, only proof of concept. (Redmine issue: #3734)

Added some initial CSS styles to geoip and whois search view results.

(Redmine issue: #3734)

Implemented generic view endpoint protection mechanism for Jinja templates.

Objects in Jinja templates can now be protected with permission protection mechanism. This can be used for hiding links, that are inaccessible to current user etc. As a proof of concept raw results output of geoip and whois search are now restricted only to users with developer role. (Redmine issue: #3734)

Added new helper methods into Jinja global namespace.

Added following helpers into Jinja global namespace to be used in templates:

• babel_format_datetime ⇒ flask_babel.format_datetime
• babel_format_timedelta ⇒ flask_babel.format_timedelta
• get_current_utc ⇒ datetime.datetime.utcnow

These helpers can be used for localized formating of timestamps in template views.

(Redmine issue: #3734)

The hawat application factory now returns production level interface by default instead of development level.

The default configuration object for Hawat application factory is now hawat.config.ProductionConfig. (Redmine issue: #3734)

Implemented framework for easy development of item CRUD views.

Application now provides easy to extend View classes for quick development of create,update,view,delete,list views. (Redmine issue: #3734)

Implemented prototype CRUDL views for user account management.

(Redmine issue: #3734)

Translation work into czech language. Full coverage of all currently existing strings.

(Redmine issue: #3734)

Improved documentation of hawat.config module.

(Redmine issue: #3734,#3361)

Improved and unified output of form errors using predefined site macro.

(Redmine issue: #3734)

PostgreSQL migration: Reimplemented DBAL, authentication layer and user management modules in Hawat.

Base on the previous work the DBAL in Hawat was reimplemented to use the new mentat.services.sqlstorage module and sqlalchemy backend. The authentication layer was rewritten to use the new dabase interface. The user management plugin was rewritten to use new database interface.

(Redmine issue: #3752,#3734)

Revision of the item CRUDL framework in Hawat.

The item manipulation framework (create,read,update,delete,list) was revised, simplified and better documented. All changes were implemented and tested on existing user management pluggable module. (Redmine issue: #3734,#3361)

Implemented prototype pluggable module for group management.

The module contains all CRUDL views, forms and templates to enable group management. (Redmine issue: #3734)

Moved string output conversion from hawat.models.user.GuiUserModel to the base class.

(Redmine issue: #3734)

Implemented prototype pluggable module for reporting filter management.

The module contains all CRUDL views, forms and templates to enable reporting filter management. (Redmine issue: #3734)

Implemented prototype pluggable module for network record management.

The module contains all CRUDL views, forms and templates to enable network record management. (Redmine issue: #3734)

Fix: Fixed minor bug in attribute name.

(Redmine issue: #3734)

Added post-action hooks to support additional tasks after successfull item create/update/delete action.

This will allow performing additional tasks like sending email about newly created user accounts, etc. (Redmine issue: #3734)

Implemented prefered timezone selection feature.

Users may now choose prefered timezone, which will be used when displaying time data. (Redmine issue: #3734)

Implemented prefered locale selection feature.

Users may now choose prefered locale, which will be used for application language localization and datetime formating. (Redmine issue: #3734)

Documentation improvements.

(Redmine issue: #3361,#3734)

Refactoring: Renamed get_current_utc and get_current_local Jinja template helpers to get_datetime_utc and get_datetime_local respectively.

(Redmine issue: #3734)

Added current datetime timestamp to global Jinja template variables.

This timestamp more or less represents the generation time of the page and may serve as common baseline for multiple time operations. (Redmine issue: #3734)

Greatly improved translation coverage of existing web interface content.

(Redmine issue: #3734)

Added limits to the size of selectpicker widget.

(Redmine issue: #3734)

Added user account role management feature for web interface.

(Redmine issue: #3734)

Created macros for rendering form items and modified all existing forms to use them.

This will enable very easy modification of rendering of all form items. (Redmine issue: #3734)

Removed locale and timezone from database GroupModel.

This feature would introduce unnecessary complexity and will not be implemented. User might be member of multiple groups and it would not be clear which settings to use. (Redmine issue: #3734)

Minor style improvements in user account detail view.

(Redmine issue: #3734)

Added link to user account update to the detail view.

(Redmine issue: #3734)

Added link to group update to the detail view.

(Redmine issue: #3734)

Added link to network record update to the detail view.

(Redmine issue: #3734)

Added link to reporting filter update to the detail view.

(Redmine issue: #3734)

Added link to “My account” to main menu for quick access to the profile of currently logged in user.

(Redmine issue: #3734)

Implemented group membership and group management feature into user accounts.

User accounts can now be added into groups and can be designated as group managers. (Redmine issue: #3734)

Implemented user membership and user managers feature into groups.

This feature is a reverse direction to the previous commit. User accounts can now be added into groups and can be designated as group managers. (Redmine issue: #3734)

Limited number of values to be displayed in multi variant of bootstrap-select.

(Redmine issue: #3734)

Implemented last login time remembering for user accounts.

(Redmine issue: #3734)

Translated the “before” string in item detail view.

(Redmine issue: #3734)

Implemented parent - children relationship for groups.

(Redmine issue: #3734)

Improved implementation of network record management.

• Implemented network specific abuse contacts.
• Implemented linking between groups and asociated networks.
• Hardened network record form.

(Redmine issue: #3734)

Implemented very simple skeleton of a script for quick SQL database initialization.

This script is a very simple helper capable of creating appropriate user accounts and databases. (Redmine issue: #3734)

Added the dobefore() hook to item create/update/delete base views.

(Redmine issue: #3734)

Implemented prototype of reporting settings management.

(Redmine issue: #3734)

Improved breadcrumbs navigation for object management views.

(Redmine issue: #3734)

Design improvevent in list, show and creatupdate views.

Added delete action to item detail view, moved actions to toolbar in item list view, changed layout and page width in item list view. (Redmine issue: #3734)

Design improvevents in list, show and creatupdate views.

Added delete action to item detail view, moved actions to toolbar in item list view, changed layout and page width in item list view. (Redmine issue: #3734)

Implemented skeleton of environment based user account registration page.

For the purposes of prototyping. (Redmine issue: #3734)

Added separate side-menus for authenticated and anonymous users.

The original menu was dependent on auth_env blueprint and did not allow customizations. New menus allow any authentication modules to append any number of menu entries. Additionally, the menu rendering macro was moved to design blueprint and all macros were commented. ((Redmine issue: #3734)

Implemented prototype of environment based registration process.

Work in progress, needs polishing. (Redmine issue: #3734)

Activated account existence checking feature.

(Redmine issue: #3734)

Switched auth_env login process to request environment variable.

So far the auth_env module worked in development mode and performed login for hardcoded user. This is no longer necessary and this feature was removed. (Redmine issue: #3734)

Fix: Added missig return statement.

(Redmine issue: #3734)

Updated default configurations for Hawat.

Added better default value for default email sender, commented local Hawat configuration file. (Redmine issue: #3734)

Implemented basic sending of account registration information emails.

(Redmine issue: #3734)

Moved flask_login bootstrapping and logout feature to application core.

This will simplify the creation of additional authentication modules, because they have to take care only of custom login. As a side-effect the flask_login is now mandatory authentication provider, but this step brings an order into otherwise potentially chaotic environment. (Redmine issue: #3734)

Rerouted auth_env module from auth to auth_env namespace, made default login view configurable via application configuration.

(Redmine issue: #3734)

Fix: Fixed broken action cancelation button.

Up to this point it was not possible to cancel action without valid form. Check for button press was after form validation, which made the whole feature useless. (Redmine issue: #3734)

Changed class of flash messages produced by flask_login to 'info’.

(Redmine issue: #3734)

Implemented the feature for redirection back to original page.

After performing certain actions (item create/update/delete, login, locale change) an attempt will be made to SECURELY redirect user back to original page beforefalling back to default redirection target. (Redmine issue: #3734)

Refactoring: Renamed HawatCRUDLView to HawatDbmodelView.

(Redmine issue: #3734)

Removed now unnecessary and obsolete methods that were supplying redirection targets in item manipulation views.

(Redmine issue: #3734)

Removed forgotten test printouts in redirection feature.

(Redmine issue: #3734)

Improved base item create/update/delete views.

Moved common features into common base class. Tweaked action status messages. (Redmine issue: #3734)

Rewritten the inheritance tree for user account forms to move common fields into parent classes.

(Redmine issue: #3734)

Added menu link for easy switching users with auth_dev authentication module.

(Redmine issue: #3734)

Added specific view for displaying user account of currently logged-in user.

(Redmine issue: #3734)

Fix: Fixed typo in dictionary key name.

(Redmine issue: #3734)

Added support for Bootstrap tooltips.

(Redmine issue: #3734)

Improved user experience with geoip and whois database lookup pluggable modules.

Improved feedback messages and result templates for better user experience, unified module code. (Redmine issue: #3734)

Greatly improved translation coverage of existing web interface content.

(Redmine issue: #3734)

Fix: Prevent redirection to item detail page after item deletion.

(Redmine issue: #3734)

Item form hardening by adding more validators.

(Redmine issue: #3734)

Removed the 'abuses’ attribute from network record model.

Target emails will be stored in settings_reporting object. (Redmine issue: #3734)

Fix: Fixed the syntax error.

(Redmine issue: #3734)

Added developer role protected printout of raw item data to creatupdate views.

The printout is implemented using Jinja macro for reusability. (Redmine issue: #3734)

Added information about page generation time into page footer.

(Redmine issue: #3734)

Moved global jinja variable to application core instead of bae view class.

(Redmine issue: #3734)

Only admin may now change the group attribute for group networks.

(Redmine issue: #3734)

Included template context into dobefore() and doafter() method arguments.

(Redmine issue: #3734)

Redesigned callback hooks in CRUDL base views.

• Renamed dobefore() hook method to do_before_action().
• Renamed doafter() hook method to do_after_action().
• Added the do_before_render() hook method to enable custom modifications before template rendering.

(Redmine issue: #3734)

Enhanced implementation of filter management module.

The filter management plugable module is now fully implemented. (Redmine issue: #3734)

Fix: Fixed name errors after renaming custom Jinja functions.

(Redmine issue: #3734)

Small tweaks in a design of devtools module view.

(Redmine issue: #3734)

Simplifications and improvements in mentat.stats.rrd statistical library.

• Simplified definition of list of generated charts.
• Unified code for generating chart and sparkchart specifications.
• Added lookup() method for looking up list of charts that should be available.
• Added 'last 2 years’ chart (main reason, we need a list of charts for user interface).

(Redmine issue: #3734)

Changed default size of sparkchart images (made it smaller).

(Redmine issue: #3734)

Added more data into chart metadata structure.

The reason was to enable better rendering of performance statistics view in web interface. (Redmine issue: #3734)

Implemented Jinja template helper for detecting existence of arbitrary file.

(Redmine issue: #3734)

Implemented base view for serving arbitrary files from arbitrary directories.

(Redmine issue: #3734)

Implemented prototype of performance statistics module for Hawat.

This module is capable of displaying system performace statistic charts based on RRD library. Charts are already pregenerated by the mentat-statistician.py module, this module only renders the result page. Currently the location of chart directory is hardcoded and not configurable. This should be fixed in a similar way to for example geoip service configuration, so that there is only single configuration for chart location. (Redmine issue: #3734)

Fix: Fixed incorrect endpoint name for sparkchart image links.

(Redmine issue: #3734)

Implemented data model and migration script for event statistics.

The model represents PostgreSQL representation of MongoDB based event statistics. The current migration script was enhanced to migrate event statistics objects. (Redmine issue: #3734)

Implemented prototype module for viewing overall event statistics.

The Dashboards module in Hawat user interface now contains prototype implementation for calculating and viewing overall event processing statistics. The module design is based on original Perl-based interface with many enhancements both on the surface (interface) and in the backend code. The work is not yet finished, some of the tasks that remain to be done are:

• Links from data tables to other modules of the interface.
• Missing sorting arrows in data table headers.
• Interface tweaks.
• Chart descriptions and/or help texts.
• Search form tweaks or changes (current implementation is maybe too powerfull for general user).

(Redmine issue: #3734)

Implemented internal and SQL model for representing event reports.

This commit includes updates in MongoDB2SQL migration script. (Redmine issue: #3734)

Implemented working prototype of report access module for Hawat.

This implementation still needs lots of polishing and there are following known major issues:

• invalid encoding of report message text coming from migrated MongoDB records
• some parts of the interface are not working yet (report remailing, report deletion)
• unauthorized access to reports is not working
• report data browser is not yet implemented
• report search result statistics are not yet implemented

These issues will be resolved in separate commits. (Redmine issue: #3734)

Improved interface translations.

(Redmine issue: #3734)

Implemented conversion library for converting IDEA messages into dict structures containing only primitive data types.

These structures will contain only most primitive data types (int, str, bool). These will be then stored into PostgreSQL`s native data type jsonb. These conversions are basically reverse operation to what mentat.idea.internal library does.

(Redmine issue: #3734)

Changed datatype of _CESNET.StorageTime attribute of IDEA message from integer to timestamp.

The numeric representation was kept in the schema for legacy purposes, but now it will be unified with other datetime attributes. As a result, the mentat.idea.internal or other compatible library is a required base class for handling IDEA messages in Mentat framework (because the idea.lite and idea.base do not properly define the custom _CESNET.StorageTime attribute).

(Redmine issue: #3734)

Implemented SQL model for representing IDEA events.

This commit contains following work:

• SQL data model for representing IDEA events in PostgreSQL.
• SQL schema for IDEA event table and indices.
• Event storage service - database abstraction layer for working with IDEA events.
• Migration script for converting events from MongoDB collection into PostgreSQL table.

This code is based on initial prototype and work done by Radko Krkoš <radko.krkos@cesnet.cz> and he deserves credit for a lot of research and pilot work.

(Redmine issue: #3734)

Fix: Fixed unit tests after changing mentat.idea.mongodb data model.

The _CESNET.StorageTime attribute is now a timestamp instead of integer. (Redmine issue: #3734)

Implemented working prototype of event access module for Hawat.

This implementation still needs lots of polishing and there are following known major issues:

• event search form is not yet fully implemented
• event search result page is only prototype
• event detail page is only prototype

These issues will be resolved in separate commits. (Redmine issue: #3734)

Big improvements in IDEA event module for Hawat.

• Event search form is almost complete. The only thing missing is implementation of logical operation 'or’ and perhaps ability to show and hide unnecessary form elements.
• Event search result list was improved, but interlinking back to search form is in prototype state. There are many ways how this feature can be implemented and this will be polished after discussion and more testing.
• Event detail page design was improved, but interlinking back to search form is missing and waiting for resolution from previous paragraph to be consistent.
• Ability to search according to protocols was added to event search form.
• Created translations for the whole module.

• Event delete action
• Event download action

This commit introduces a lot of new work and not everything is yet in final state. However the live demonstration and hands-on experience is needed to move things forward.

(Redmine issue: #3734)

Implemented event download feature for saving IDEA events into JSON files.

(Redmine issue: #3734)

Implemented working prototype of Mentat system status module for Hawat.

This module is based on the same library as mentat-controller.py and can provide access to following information (non exhaustive list):

• status of real-time modules (runnning, not running, ...)
• status of cronjob modules (enabled, disabled)
• PID file analysis
• log file analysis
• ... and many more

The idea is to provide administrator access to information about the state of the whole system. Current implementation is only a skeleton and fraction of possible features, they will be implemented in future versions.

(Redmine issue: #3734)

Next iteration in event search form design.

Following experimental changes have been made in event search form:

• highlighting of matched searches in result table
• removed row index column and event delete action
• information about current page and range of displayed entries was moved to the pager
• created different styles of dropdown search action buttons within result table to evaluate best approach
• displaying last SQL query for debugging purposes
• redesigned event search form (hiding of unnecessary fields, explicit search according to storage time, removed logop between source_addrs and target_addrs)

This is an experimental work to allow testing the interface on development server, some things may yet change. (Redmine issue: #3734)

Improved web interface translations.

(Redmine issue: #3734)

Fix: Corrected the name of itemset 'detector’ cache file.

(Redmine issue: #3734)

Implemented working prototype of Mentat database status module for Hawat.

The idea is to provide administrator access to information about the state of the whole database. Current implementation is only a proof of concept and fraction of possible features, they will be implemented in future versions.

(Redmine issue: #3734)

Minor improvements in web interface translations into Czech.

(Redmine issue: #3734)

Minor improvements in Hawat menus.

• Made certain texts in the menu hidden on smaller displays, so that the menu does not take two rows.
• Menu reorganization - moved status overviews from MORE section to ADMIN section of menu.
• Minor menu translation tweaks.

(Redmine issue: #3734)

Fix: Fixed the call for Jinja2 default() filter.

(Redmine issue: #3734)

Next iteration in IDEA event search form design.

• Added the ability to search according to the 'source|target|detector types’ and 'description’.
• Added the ability to search according to the host address|ports|types. Host is an alias for Source OR Target.
• Minor design improvements in main menu and other page widgets.
• Updated mentat-precache.py according to changes mentioned above.
• Added new 'allow_empty’ feature to mentat-precache.py.

(Redmine issue: #3752,#3734)

Next iteration in IDEA event search form, result page and detail page design.

This commit contains following design changes and fixes:

• Fixed the issue with duplicated HTML identifiers for datetime selects.
• Written macros for rendering search dropdown buttons for source addresses, categories and abuse groups.
• Modified accessors in internal IDEA representation.
• Implemented implicit sorting by storage time in case it was chosen in the search form.
• Additional minor design fixes and tweaks.

(Redmine issue: #3734)

Improvements in 'dbstatus’ Hawat pluggable module.

Improved implementation, design, output and coding style of 'dbstatus’ pluggable module. The view page now display basic characteristics of event table (number of rows, oldest and newest record, table size, ...) as well as the values of most important PostgreSQL configurations. (Redmine issue: #3734)

Improved web interface translations.

(Redmine issue: #3734)

Fix: Fixed minor typo in translation.

(Redmine issue: #3734)

Fix: Fixed bugs in database status view template.

(Redmine issue: #3734)

Improvements in 'status’ Hawat pluggable module.

Improved coding style of the blueprint. Improved output of the status view page. The page now provides more detailed information about module and cronjob status as well as detailed information about discovered log files, pid files and processes. This should still be considered as a draft, there are many more improvements yet to be done, especialy the regarding the design of the page. (Redmine issue: #3734)

Improved czech web interface translations.

(Redmine issue: #3734)

Implemented actual resource authorization in Hawat web interface.

So far the access to various resources in Hawat web interface was protected only by authentication. This was to simplify the development process. As the first stable release approaches, it was time to harden the security. (Redmine issue: #3734)

Implemented unauthenticated access to reports.

All reports generated by mentat-reporter.py are now accessible via unauthenticated URL protected by security token. This feature enables easy sharing of security information even with people without full access to web interface. (Redmine issue: #3734)

Improved czech web interface translations.

(Redmine issue: #3734)

Fix: Exception should not be raised in user_loader callback in Flask-Login.

(Redmine issue: #3734)

Upgraded FontAwesome from major version 4 to major version 5.

(Redmine issue: #3734)

Implemented the pager feature into BaseSearchView, implemented the pager feature into report searching Hawat module.

(Redmine issue: #3734)

Fix: Fixed the bug in searching for port numbers using event search form.

The PostgreSQL database complained, that it is unable to serach for integers using list of strings. Added explicit conversion from list of strings (received by user form) to list of integers. (Redmine issue: #3752,#3734)

Upgraded third-party web interface libraries.

(Redmine issue: #3734,#3387)

Improved web interface translations.

(Redmine issue: #3734)

Implemented quick measurement of request processing time in Hawat.

Request processing time is measured from the correct request beginning point in Flask (using beforerequest decorator) up to the moment of rendering Jinja2 template for particular page. There is a bit of error, because some tasks performed after page rendering might not get measured, but this feature is intended to measure mainly length of database queries. For most use cases this solution should be enough, there is always debug toolbar for developers. (Redmine issue: #3734)

Added list of reporting filters to group detail view.

(Redmine issue: #3734)

Added constants for role names to Hawat`s constant module.

(Redmine issue: #3734)

Improvements related to account registration process using auth_env pluggable module for Hawat.

• Changed validator for user login from email to string containing restricted set of characters.
• Renamed field 'Login (eppn)' to just 'Login’. It made no sense to be used that was for example with Basic or certificate based authentication service.
• Implemented sendind of confirmation emails after successfull registration to system admins and to the user that performed the registration. The emails are generated using Jinja2 templates.
• Implemented sending of confirmation emails after successfull account activation to the user and as BCC to system admins.
• Added missing rollback() calls to base View implementations to resolve SQLAlchemy errors after unsuccessfull database operation.

(Redmine issue: #3734)

Improved web interface translations to Czech language.

(Redmine issue: #3734)

Translated informational emails, that are sent to the user and admins after account registration and activation.

• Registration email sent to the user is localized into the locale selected by the user during registration.
• Registration email sent to the admins is localized into the default application locale.
• Activation email sent to the user is localized into the user`s locale.

(Redmine issue: #3734)

Implemented new base view CreateForView.

This new base view anables implementing view for maintainer with restricted permissions. Using these views particular user can be given permission to create child items for particular parent item. For example so far only the administrator was able to create new networks of filters. (Redmine issue: #3734)

Implemented new views for creating new networks and filters for particular group.

Based on previous commit new feature was implemented, that will enable group maintainers to add new filters and networks to that particular group (so far administration privileges were necessary). (Redmine issue: #3734)

Improved coding style and content of flash messages of Hawat object management modules.

(Redmine issue: #3734,#3443)

Template design improvements for better presentation of user/group/settings detail page.

(Redmine issue: #3734)

Updated Czech translations of Hawat web interface.

(Redmine issue: #3734)

Huge overhaul of Hawat`s Flask-based framework and all it`s modules.

This commit introduces huge amount of work that went into revision of existing Flask-based framework, on top of which the web interface and all of it`s modules are implemented. The process of registering modules and views into the application was greatly simplified (this will be used for further improvements of application menu). Code quality was improved using Pylint and Pyflakes tools. Documentation was improved. Many code features were refactored for performance, optimization and code style reasons. (Redmine issue: #3734,#3443)

Implemented automated hiding of menu entries based on permissions of current user.

(Redmine issue: #3734)

Small tweaks in main menu entry organization.

(Redmine issue: #3734)

Implemented support for item action menu into application menu code.

The application menu in hawat.menu module was improved to also support item action menus. These menus are currently on item detail page and contain various contextual item actions like editing or deletion. (Redmine issue: #3734)

Implemented views for enabling/disabling reporting filters.

(Redmine issue: #3734)

Fix: Fixed switched templates for item enabling and disabling actions.

(Redmine issue: #3734)

Localized HTTP error pages.

(Redmine issue: #3734)

Fix: The actual SQL query displayed after event searching is now being displayed only to developers.

(Redmine issue: #3734)

Improved Czech web interface translations.

(Redmine issue: #3734)

Implemented prototype of reporting statistics dashboard.

Missing feature that needs to be implemented is filtering based on abuse groups. Currently any user can view dashboards for any abuse group reports. Aside from administrator user should be able to view dashboards only for groups he/she is member of. (Redmine issue: #3734)

Fix: Fixed two minor bugs - invalid icon name and incorrect Jinja macro that failed in certain conditions.

(Redmine issue: #3734)

Improved variable dumping macros for Jinja2 templates.

(Redmine issue: #3734)

Fix: Fixed invalid icon name.

(Redmine issue: #3734)

Fix: Used common Jinja macro for rendering alert messages wherever possible.

(Redmine issue: #3734)

Disabled broken and unfinished Javascript feature.

(Redmine issue: #3734)

Enhanced event search form to accept comma separated list of addresses for source, target and host form fields.

Now it is possible to enter multiple values to each of the mentioned form fields and all values will be automatically ORed. (Redmine issue: #3734)

Added mechanism for propagating information from build system to web interface.

For now we want to be able to display information about current build version in the footer of the web page. (Redmine issue: #3734)

Minor improvements in web interface footer layout.

(Redmine issue: #3734)

Minor tweaks in Hawat`s top menu bar.

Removed unnecesary title. Added tooltip to locale switcher. (Redmine issue: #3734)

Fix: Fixed invalid HTTP error code in error page.

(Redmine issue: #3734)

Fix: The IP4R PostgreSQL extension does not recognize certain network addressess.

User was presented with HTTP 500 error page without knowing what is actually wrong. Because the error is encountered at database level and the value passess form validation, it was necessary to analyze the exception thrown by database and present user with meaningfull message. For example 195.113.134.0/14 is not a valid network address from the point of view of IP4R extension, because there are more 1s than necessary and the extension does no adjusting on its own. This pathc solves only the error presentation to the user, actual solution stil has to be implemented. (Redmine issue: #3752,#3734)

Implemented access restrictions to reports and reporting dashboards.

Only administrator can access any reports and view reporting dashboards for all groups. Otherwise user may view reporting data only for groups he/she is member of. (Redmine issue: #3734)

Moved do_before_action hook to more appropriate place.

This hook should execute after each successfull form submission (except cancel action), otherwise it is not much usefull. (Redmine issue: #3734)

Improved filter management module.

• more information is displayed in filter list view
• fixed invalid permissions for filter create view
• fixed bug in filter creation and update: when basic filter was created, filtering rule was not generated from form data

(Redmine issue: #3734)

Improved network management module.

• more information is displayed in network list view
• fixed invalid permissions for network create view

(Redmine issue: #3734)

Unified display of development data on various views.

Variable dumps for development pyurposes were unified among multiple views. (Redmine issue: #3734)

Implemented percentage displays in statistical tables.

(Redmine issue: #3734)

Huge improvements in reporter module and report views in Hawat.

• Improved report model to enable parent reports, so we can link extra reports with parent summary reports.
• Fixed/improved statistical calculations for reports.
• Greatly improved report view visuals.
• Greatly improved report search form.

This commit contains backwards compatibility breaking features including necesity do alter database schema. (Redmine issue: #3734)

Improved report show and search view visuals.

Added new data displays, widgets and tooltips. (Redmine issue: #3734)

Minor fixes and improvements in report show and search view visuals.

(Redmine issue: #3734)

More web interface UI and UX improvements.

This patch focuses more on improving look and feel when working with event search form and event detail view. The interlinking with other parts of the system was improved when working from event detail page. Event action search dropdown functionality was also improved. (Redmine issue: #3734)

More improvements in event search view.

Enriched event search form with option to search according to inspection errors. Added special search options for administrators. Additional minor tweaks and improvements. (Redmine issue: #3734)

Tweaked event severity and classification display in event search and show views.

(Redmine issue: #3734)

Activated Traget column for large monitor sizes in event search result table.

(Redmine issue: #3734)

Improved czech web interface translations.

(Redmine issue: #3734)

Unified form submission buttons into button-toolbar widget.

(Redmine issue: #3734)

Improvements in the dbstatus pluggable module for Hawat.

The module is now capable of displaying information about following tables within the events database: events, thresholds, events_thresholded. (Redmine issue: #3734)

Improved result visualisation in the dbstatus pluggable module for Hawat.

Added charts and other graphical items to improve result visualisation. (Redmine issue: #3734)

Improved czech web interface translations.

(Redmine issue: #3734)

Moved everall event dashboard view from separate blueprint to events blueprint.

This way it will be similar to implementation of reports dashboard, which is also in reports blueprint. (Redmine issue: #3734)

Fix: Fixed link to incorrect dashboard view.

Overall event dashboards can not be calculated for particular group, link was meant to pount to reporting dashboard. (Redmine issue: #3734)

Removed unnecessary default value and unused reporting dashboard form.

The default value for searching reports is not necessary, because there is a result limit and also result is ordered in descending order, so the latest reports will always come on top. (Redmine issue: #3734)

Improved/fixed czech web interface and report translations.

(Redmine issue: #3734)

Replaced 'unassigned’ and 'undisclosed’ string it event search result table in severity and group columns with icon (to make the table more compact).

(Redmine issue: #3734)

Huge improvements in Hawat web interface.

This patch introduces huge amount of work related to Hawat web interface. I was just clicking through the interface and trying to improve everything that popped out at the moment. There are following improvements:

• Enabled DataTable localication plugins.
• Implemented common layout for object management views (list views).
• Redesigned application menu implementation.
• Enhanced application menu to support contextual item action menus in listing tables. All administration list views now use it to generate contextual action toolbars.
• Menu object is now used even to generate breadcrumb menus.
• Simplified working with view icons and their propagation into action menus.
• Implemented AJAX spinner and used AJAX to fetch raw report data to web interface. There is stil a lot of room for improvements, the idea is for the user to be able to inteligently browse through the loaded data in some way.
• Redesigned object administration views (users, groups, filters, networks).
• Cleaned up list of built-in icons.
• Improved web interface translations after the changes in this commit.
• Fixed the bug in serving report data files.

(Redmine issue: #3734,#3443)

Permission system redesign.

The maintainer permission is now used as slightly less powerfull administrator. Added permission for any role. It is now enforced wherever possible. Added prototype dump of all registered view permissions, however permission enforced by authorize_item_action callback are missing in the list. (Redmine issue: #3734)

Added new Jinja2 function for getting reference to view class.

This method can be usefull for accessing view metadata stored as class properties like icons, titles, etc. (Redmine issue: #3734)

Added ability for base view to check that current user can access given endpoint.

This can be used to dynamically hide inaccessible parts of web interface. This method also return false in case the endpoint does not exist within the application, so you may use it to hide links to another pluggable modules. (Redmine issue: #3734)

Fix: Improved _layout_list.html base template to be more resistant to nonexistence of data for breadcrumbs and action menu widgets.

(Redmine issue: #3734)

Implemented new Hawat pluggable module: changelogs.

Hawat can now track item changes and display them through new changelogs pluggable module. This feature can be used for monitoring changes done on various items like user and group account settings. This patch contains complete work including enhancements of database model. (Redmine issue: #3734)

Fix: Minor fixes and improvements.

(Redmine issue: #3734)

Improved czech web interface translations.

(Redmine issue: #3734)

Added logging of directly served files.

(Redmine issue: #3734)

Enhanced events search form to support search parameter negations.

Certain search parameters now support negations, so it is possible to for example negate category selection. (Redmine issue: #3734)

Added new action button for searching related events from report detail view.

(Redmine issue: #3734)

Improved czech web interface translations.

(Redmine issue: #3734)

Fix: Fixed broken event dashboard view.

(Redmine issue: #3734)