Project

General

Profile

Actions

Bug #7644

closed

Some post-processing modules are not protected against race conditions

Added by Rajmund Hruška over 1 year ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Category:
Development - Core
Target version:
Start date:
04/17/2023
Due date:
% Done:

100%

Estimated time:
To be discussed:

Description

mentat-reporter has a file mutex so it should be fine. But for example mentat-precache doesn't have any and it's probably the case for multiple modules.


Related issues

Related to Mentat - Bug #7624: precache module throws error when database is shutting downClosedRajmund Hruška01/31/2023

Actions
Actions #1

Updated by Rajmund Hruška over 1 year ago

  • Related to Bug #7624: precache module throws error when database is shutting down added
Actions #2

Updated by Rajmund Hruška over 1 year ago

  • To be discussed changed from No to Yes

In mentat-reporter, https://github.com/derpston/python-simpleflock is used. It's not imported as a module, the source code from the repository is copied to our source code. What I don't like about the code is that it throws an exception if the lock cannot by acquired, and the error is emailed to system administrators. Is it necessary to send the emails?

Also, there are two approaches:
1. the script should detect if another instance of itself is running
2. file lock - can be implemented in the python script, like simpleflock, or maybe we can use flock from the system, so it would be something like */5 * * * * /usr/bin/flock /tmp/my.lock /usr/local/bin/myjob.

I like the second approach with flock, as it's the easiest and fastest to implement

Actions #3

Updated by Pavel Kácha over 1 year ago

Rajmund Hruška wrote in #note-2:

2. file lock - can be implemented in the python script, like simpleflock, or maybe we can use flock from the system, so it would be something like */5 * * * * /usr/bin/flock /tmp/my.lock /usr/local/bin/myjob.

I like the second approach with flock, as it's the easiest and fastest to implement

If the lock is meant to guard only the cronjob as a whole, I agree, flock is completely pragmatic and sane thing to use.

Actions #4

Updated by Rajmund Hruška over 1 year ago

  • Status changed from New to In Progress
  • Assignee set to Rajmund Hruška
  • Target version changed from Backlog to 2.11

The thing is, this will only affect executing jobs from crontab. It would still be possible to have multiple instances running at the same time if the script was executed manually. But I think it's OK. The scripts are not meant to be executed manually.

Actions #5

Updated by Rajmund Hruška over 1 year ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

In e3ccafa4 I added flock to the scripts running multiple times a day. The ones running just once a day remained unchanged.

Actions #6

Updated by Rajmund Hruška over 1 year ago

  • Status changed from Resolved to Closed
  • To be discussed deleted (Yes)
Actions #7

Updated by Rajmund Hruška over 1 year ago

  • Status changed from Closed to In Review
Actions #8

Updated by Rajmund Hruška over 1 year ago

  • Status changed from In Review to Closed
Actions

Also available in: Atom PDF