Mentat

precache is a module which is run by cron - it's a post-processing module. mentat-controller --command stop only stops real-time modules. So mentat-controller doesn't wait for precache.py to finish.

The precache module is usually running about 7.5 minutes. But sometimes it's running more than 10 minutes. The cron job is running every 10 minutes and I think this module (precache) is not protected against running multiple times so there might be some race conditions.

This might be an issue for multiple post-processing modules. For example, mentat-reporter can also run more than 10 minutes, but there is a file mutex, so the other run results in an error:

Traceback (most recent call last):
  File "/var/mentat/venv/bin/mentat-reporter.py", line 51, in <module>
    MentatReporterScript().run()
  File "/var/mentat/venv/lib/python3.7/site-packages/pyzenkit/baseapp.py", line 1559, in run
    self._stage_process()
  File "/var/mentat/venv/lib/python3.7/site-packages/pyzenkit/baseapp.py", line 1472, in _stage_process
    self._sub_stage_process()
  File "/var/mentat/venv/lib/python3.7/site-packages/pyzenkit/zenscript.py", line 353, in _sub_stage_process
    self.execute_script_command(cmdname)
  File "/var/mentat/venv/lib/python3.7/site-packages/pyzenkit/zenscript.py", line 410, in execute_script_command
    self.runlog[command_name] = cbk()  # pylint: disable=locally-disabled,not-callable
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/module/reporter.py", line 343, in cbk_command_report
    with SimpleFlock("/var/tmp/mentat-reporter.py", 5):
  File "/var/mentat/venv/lib/python3.7/site-packages/mentat/module/reporter.py", line 119, in __enter__
    fcntl.flock(self._fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
BlockingIOError: [Errno 11] Resource temporarily unavailable

Rajmund Hruška wrote in #note-5:

I will change the precache cron to run every hour instead of every 10 minutes. At first, I will just change it manually on mentat-alt. But I don't think this will help with the total execution time. I might be wrong though.

I was indeed wrong. On mentat-alt, the module is now running every hour and the runtime is about 10 minutes. So, I think it would be a good idea to change the configuration file in the repository.