Task #1017

Testing

Added by Jan Mach almost 6 years ago. Updated 8 months ago.

Status:In ProgressStart date:05/31/2013
Priority:LowDue date:
Assignee:Jan Mach% Done:

0%

Category:Development - Core
Target version:Long running

Description

Long running task to record work progress

Continually develop and improve test code to ensure quality and stability of the project. Ideally every class and module should be covered with unit tests or other type of test code.

Associated revisions

Revision 7c3d1a9b
Added by Jan Mach almost 2 years ago

Improved unit tests for mentat.filtering.jpath module.

(Redmine issues: #3392 and #1017)

Revision efaa9467
Added by Jan Mach almost 2 years ago

Stability improvements in library mentat.filtering.jpath.

Improved error handling in JPath library, better input validation. Additions include unit tests. (Redmine issues: #3392 and #1017)

Revision 6803d78a
Added by Jan Mach almost 2 years ago

Ensured allowed character set for node names in JPath.

Explicit list of allowed characters instead of "\w”, including unit tests and documentation. (Redmine issues: #3392, #3361 and #1017)

Revision d214e620
Added by Jan Mach almost 2 years ago

Bugfix: JPath library was not working with IDEA messages.

The JPath library did not work with IDEA messages, there are no dicts and lists. Isinstance checks had to be updated accordingly. Added assertions for working with IDEA messages into unit test case to prevent this from happening in the future. (Redmine issues: #3392 and #1017)

Revision bf0598d3
Added by Jan Mach almost 2 years ago

Implemented support for conditional value overwriting and uniqueness.

The jpath_set() method now optionally enables user to specify, whether already existing value should be overwritten or not, and whether value should be unique or not. The uniqueness option works only for lists at the end of JPath. (Redmine issues: #3392, #3361 and #1017, enables #3372 and #3396)

Revision 8496ca27
Added by Jan Mach almost 2 years ago

Implemented support for default argument values in inspection actions.

When not explicitly set and appropriate, default value will be used as and inspection action agument. Updated log line styles to match. Updated demo and default configuration files to provide clues of new options. Updated unit tests to verify new functionality (visually, currently without assertions. (Redmine issues: #3373 and #1017)

Revision ce01f6d7
Added by Jan Mach almost 2 years ago

Huge revission of mentat.filtering library.

This commit represents huge revision of mentat.filtering library. Codebase has been checked, unit tests have been improved and documentation has been written to make order in current state of things. Based on this filtering language improvements will be designed. (Redmine issues: #3393, #1017 and #3361)

Revision 5b01f6fa
Added by Jan Mach almost 2 years ago

Implemented internal IDEA message representation class.

The default idea.lite library for representing IDEA messages does not take into account custom subkeys, that can appear in messages handled by Mentat. This patch introduces prototype of new library, which is based on idea.lite and takes custom keys into account. (Redmine issues: #3364 and #1017)

Revision 4164e95a
Added by Jan Mach almost 2 years ago

Improvements in mentat.idea.internal library.

Greatly improved code reusability by employing typedef generator approach like in the underlying typedcol and idea.lite library. Additionally the documentation and unit tests were both improved, documentation can now be generated using Sphinx-doc tool. (Redmine issues: #3364, #3361 and #1017)

Revision daafd215
Added by Jan Mach almost 2 years ago

Implemented module for converting IDEA messages to and from MongoDB.

There are classess in mentat.idea.mongodb module for easy conversion of IDEA messages to and from MongoDB representation. Appropriate documentation and unit tests were also created. (Redmine issue: #3364, #3361 and #1017)

NOTE: There is probably bug in current Perl-based library for message conversion, because all unit test conversions related to timestamps from database messages stored by legacy code fail. There is always 1 or 2 hour time difference. This issue is not yet fixed.

Revision ae6231f0
Added by Jan Mach almost 2 years ago

Performance optimization of mentat.filtering.jpath.jpath_parse() function.

Implemented jpath_parse_c() as a caching variant of jpath_parse() function. Benchmarking confirmed huge performance improvement, see mentat.filtering.benchmark.bench_jpath.py for details. Another small improvement was archived by moving compilation of chunk regular expression to global module variable (14% performance increase). All module functions now use this caching variant internally. Also implemented cache_size() and cache_clear() functions for cache management. Beware, that jpath_parse_c() function does not make deep copy of returned value from cache (for performance reasons). Treat those values as read only, or suffer the consequences. (Redmine issues: #3392 and #1017)

Revision 6830a9e4
Added by Jan Mach almost 2 years ago

Fixed the problem with bson.BSON.encode being unable to encode typedcol.TypedList objects.

In commit 9865c900af39a98c2c5254b06ed4f132dded6929 the mentat.idea.mongodb was unable to store IDEA messages into MongoDB. The issue was with bson.BSON encoder, which was hardcoded in a way that handled any unknown object as dict. We were not able to convince the encoder to treat TypedList objects as lists, so we had to use different approach and supply appropriate data structure. The mentat.idea.mongodb.IdeaIn convertor now produces data structure composed of simple dicts and lists instead of TypedDicts and TypedLists.

Current implementation should however be considered as prototype and proof of concept, because it probably will be possible to write it in more elegant way. The current problem is, that the idea.base.idea_typedef contains hardcoded calls for typedcol.typed_list(), which are not customizable from outside of the module via flavour mechanism. The addon feature was used to monkeypatch these definitions. This is of course not optimal solution, because any changes in underlying library must be propagated manually into mongodb library.

Additionally, IDEA messages stored in database contain some additional attributes, that are database specific and internal and should be stripped upor retrieving from database. Currently this must be done manually using truncate() function call, more optimal solution would be to incorporate this into typedcol library and strip these attributes during object instantination/conversion process.

(Redmine issues: #3364 and #1017)

Revision 25b51380
Added by Jan Mach almost 2 years ago

Finished prototype of mentat-storage.py module.

This commit introduces finished working prototype of mentat-storage.py real-time message processing module including appropriate unit tests and basic documentation work. Key features are possible customization of target database and collection, usage of core database configuration file, which can be overriden with local config file, or command line options. Messages are currently stored in database one by one, however batch processing will possibly be implemented in the future.

(Redmine issues: #3364, #1017 and #3361)

Revision 2a8951ea
Added by Jan Mach almost 2 years ago

Improved unit tests for mentat-inspector module.

Implemented test suite for better testing daemon components. (Redmine issue: #1017)

Revision 01a731e4
Added by Jan Mach almost 2 years ago

Changed default configuration for mentat-storage.py module.

The mentat-storage.py module will most likelly be the last module in processing chain, so it should delete the messages by default, othewise the whole procesing chain will hang. Additionally, there was a small fix that needed to be done in module unit test file. (Redmine issue: #3364 and #1017)

Revision 76f98298
Added by Jan Mach almost 2 years ago

Fixed the mentat.daemon.component.commiter unit test.

(Redmine issue: #1017)

Revision 97980d3f
Added by Jan Mach almost 2 years ago

Bugfix: Filer daemon component logs unsuccessfull file operations.

If the filesystem operation was not succesfull, the exception was catched to enable recovery, but it was not recorded into logfile or anywhere else. (Redmine issues: #3374 and #1017)

Revision 57db1217
Added by Jan Mach almost 2 years ago

Fixed all unit tests to working state.

All currently implemented unit tests are working, even if only in most basic manner. This enables us to implement automated testing into our build system. (Redmine issues: #1017 and 3387)

Revision 71a3c3f8
Added by Jan Mach almost 2 years ago

Fix: Fixed the unit test for geoip service.

Test script attempts to obtain geoip databases before execution of test code, so that it is runnable anywhere and database files do not have to be part of the distribution. (Redmine issue: #1017)

Revision fdff65d6
Added by Jan Mach over 1 year ago

Fix: Changed call to subprocess.call() method to support older versions of Python3.

(Redmine issue: #1017)

Revision 5f761799
Added by Jan Mach over 1 year ago

Overhaul of IDEA statistic calculation library and RRD manipulation library.

Improved code, documentation and unit tests in IDEA and RRD libraries related to mentat-statistician.py. (Redmine issue: #3379, #1017, #3361)

Revision 8c6542f4
Added by Jan Mach over 1 year ago

Migration, refactoring and code style unification to support latest PyZenKit and Pynspect libraries.

This huge update represents a lot of work on many parts of the system. After major upgrade in PyZenKit and Pynspect libraries it was necessary to bring this project up to date. There were many changes in the code, lot of refactoring and while all this was done, code quality was improved, documentation written and unit tests modified. Many of these changes simply make sure, that the code works with latest versions of both libraries. This commit is supposed to be a single major feature upgrade, after this only smaller commits will follow to polish this huge amount of work.

(Redmine issues: #3374, #1017, #3361, #3387, #3443)

Revision 42279600
Added by Jan Mach over 1 year ago

Small tweaks in whois module unit test file.

(Redmine issue: #1017)

Revision b9675265
Added by Jan Mach over 1 year ago

Fix: Fixed unit tests.

All unit tests now pass, even though they still need a lot of work and in some case do not do much of a testing. But at least the testing feature can be now enabled on the automated build system. As the unit tests will get improved in time, the deployment process will get more and more reliable. (Redmine issue: #1017,#3387)

Revision efd27fdc
Added by Jan Mach over 1 year ago

Fix: Fixed broken unit test.

Unit test for mentat.script.fetcher was broken on machines, where mentat is not installed and /etc/metat/core directory does not exist. (Redmine issue: #1017)

Revision 1ebf2392
Added by Jan Mach over 1 year ago

Fixed failing unit test for application StoragePlugin.

(Redmine issue: #1017)

Revision 6931b441
Added by Jan Mach over 1 year ago

Improved geoip, whois and storage services to fully use Mentat core configurations.

The mentat.geoip, mentat.whois and mentat.storage services were enhanced to fully understand and use Mentat core configurations. The main idea is that the configuration of these services should be centralized for the whole Mentat ecosystem and any module or component, that wishes to use these services should be able to easily obtain correctly configured instance and without the knowledge of the configuration directory schema and syntax. This patch introduces following new concepts:

  • Each service has corresponding service manager that is capable of understanding Mentat core configuration tree schema and syntax and quickly provide fully configured service instance.
  • Each service has ability to store reference to fully configured service instance at module level (like a singleton pattern) and any code can get a hold of this reference.

This patch also contains improvements in documentation of all above mentioned modules and also many improvements in unit testing framework, which was used to verify correct functionality. (Redmine issue: #1017,#3361)

Revision 43a61dcf
Added by Jan Mach over 1 year ago

Fix: Unit tests for whois enricher plugin must use and populate different database for all tests.

(Redmine issue: #1017,#3387)

Revision eea78bfe
Added by Jan Mach over 1 year ago

Hotfix: Disabled broken tests to let automated build pass.

(Redmine issue: #1017,#3387)

Revision 9bac77f8
Added by Jan Mach about 1 year ago

Fixed broken unit test.

(Redmine issue: #1017)

Revision 9e7ffaa5
Added by Jan Mach about 1 year ago

Fix: Fixed broken unit tests.

(Redmine issue: #1017)

Revision 4739e206
Added by Jan Mach about 1 year ago

Fix: Fixed missing PostgreSQL database initialization in fetcher unit test.

The unit test for fetcher script was missing the SQL schema creation and PostgreSQL extension registration. (Redmine issue: #1017)

Revision c25d926e
Added by Jan Mach about 1 year ago

Fix: Disabled build breaking tests.

Test in mentat.system module need more thinking, they were used for development and debugging, however they fail on deployment server (because those locations do not exist). Testing this module will be little bit tricky and will require to create fake filesystem structures. (Redmine issue: #1017,#3387)

Revision d2679872
Added by Jan Mach about 1 year ago

Fix: Minor fixes in unit tests.

(Redmine issue: #1017)

Revision df8b6802
Added by Jan Mach about 1 year ago

Improved event reporting library.

  • Updated documentation of the library classes and its methods.
  • Fixed minor issues reported by pylint utility.
  • Improved unit test coverage of the whole library.

(Redmine issue: #3361,#1017,#3443)

Revision 80a99798
Added by Jan Mach about 1 year ago

Disabled badly written unit test for fetcher script.

This unit test caused complications on automated deployment system and needs to be rewritten. (Redmine issue: #3387,#1017)

Revision 04f7785d
Added by Jan Mach 11 months ago

Fix: Fixed failing unit test for Hawat web interface.

(Redmine issue: #1017)

Revision 3ae644ab
Added by Jan Mach 11 months ago

Added valid maxDiff variable name from unit test framework to list of valid names in Pylint to globally ignore this warning.

(Redmine issue: #3443,#1017)

Revision 367c7d38
Added by Jan Mach 11 months ago

Fix: Fixed unit tests broken after previous commit.

(Redmine issue: #1017)

Revision 5d1ad7d8
Added by Jan Mach 8 months ago

Fix: Fixed broken unit test after previous commits.

(Redmine issue: #1017)

Revision 4debb542
Added by Jan Mach 8 months ago

Fix: Fixed unit tests broken after implementing timezone selection feature into reports.

(Redmine issue: #1017)

Revision 35524f5d
Added by Jan Mach 8 months ago

Fix: Added missing constant definition.

(Redmine issue: #1017,#3443)

Revision 7204ad02
Added by Jan Mach 8 months ago

Fix: Fixed broken committer daemon module.

Added missing value to event map definition. (Redmine issue: #1017,#3443)

Revision 03126e10
Added by Jan Mach 8 months ago

Feature: Improved unit tests for mentat-informant.py.

Added unit test for testing method for saving attachments into JSON file. (Redmine issue: #1017,#4237,#4229)

Revision 5aa6ad5c
Added by Jan Mach 7 months ago

Fix: Fixed broken unit tests for PostgreSQL service.

On some versions of PostgreSQL the mogrify method returns differently formatted strings (extra space in after comma in lists). This needed to be fixed. (Redmine issue: #1017)

Revision 80af4ce7
Added by Jan Mach 6 months ago

Improved coding style in mentat.services.test_eventstorage test module.

(Redmine issue: #1017,#3443)

Revision 908e9b8f
Added by Jan Mach 6 months ago

Deleted unit tests for MongoDB related code.

This step will greatly simplify the requirements for our automated build and deployment system. The MongoDB related code is still there for the purposes of data migration. (Redmine issue: #4225,#3387,#1017)

Revision 74526c1b
Added by Jan Mach 6 months ago

Fix: Fixed broken unit test after changing default value for max attachment size.

(Redmine issue: #1017)

Revision 2c9fb952
Added by Jan Mach 6 months ago

Improved coding style, unit tests and doumentation of IDEA message enrichment plugins.

(Redmine issue: #4262,#1017,#3443)

Revision f7d5114b
Added by Jan Mach 6 months ago

Improvements in coding style, documentation and unit tests of mentat-enricher.py module.

(Redmine issue: #4213,#1017,#3361,#3443)

Revision 6ebe8bb2
Added by Jan Mach 3 months ago

Fix: Fixed broken unit tests after previous commit.

(Redmine issue: #1017)

Revision 15cfc12c
Added by Jan Mach 22 days ago

Fix: Unit tests must run with APP_ROOT_PATH set.

(Redmine issue: #1017,#3443)

Revision c682ea63
Added by Jan Mach 22 days ago

Fix: Fixed DemoFetcherScript demonstration and unit test.

(Redmine issue: #1017,#3443)

Revision 96894e03
Added by Jan Mach 14 days ago

Fixed broken daemon component unit tests.

Changes were necessary after previous commits. (Redmine issue: #1017,#3443)

History

#1 Updated by Jan Mach almost 2 years ago

  • Tracker changed from Feature to Task

#2 Updated by Jan Mach almost 2 years ago

  • Subject changed from Jednotkové testy to Testing
  • Description updated (diff)
  • Status changed from New to In Progress

#3 Updated by Jan Mach 8 months ago

  • Priority changed from Normal to Low

Also available in: Atom PDF