Bug #7591
closedTests fail because remaining connection slots are reserved for non-replication superuser connections
0%
Description
This happens both in vagrantbox and GitLab CI/CD.
======================================================================
ERROR: Perform the tests of module service.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2285, in _wrap_pool_connect
return fn()
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 363, in connect
return _ConnectionFairy._checkout(self)
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 773, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 492, in checkout
rec = pool._do_get()
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/pool/impl.py", line 139, in _do_get
self._dec_overflow()
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit__
exc_value, with_traceback=exc_tb,
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
raise exception
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/pool/impl.py", line 136, in _do_get
return self._create_connection()
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 657, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit__
exc_value, with_traceback=exc_tb,
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
raise exception
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 652, in __connect
connection = pool._invoke_creator(self)
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/var/mentat/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 488, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/var/mentat/venv/lib/python3.7/site-packages/psycopg2/__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: connection to server at "localhost" (::1), port 5432 failed: FATAL: remaining connection slots are reserved for non-replication superuser connections
Files
Updated by Rajmund Hruška over 2 years ago
Alchemist/Buildbot has in the /etc/postgresql/12/main/postgresql.conf max_connections = 200 and default is 100.
Updated by Rajmund Hruška over 2 years ago
- Status changed from New to Feedback
- Assignee set to Rajmund Hruška
- To be discussed changed from No to Yes
A simple solution is just changing the number in configuration file like this: sed -i '/max_connections/c\max_connections = 200' /etc/postgresql/13/main/postgresql.conf. I used that in the GitLab CI/CD config file and it can be added in the vagrant script. Is that good enough?
Updated by Rajmund Hruška over 2 years ago
In the Vagrant, I changed max_connections to 25, I ran tests and then I executed SELECT * FROM pg_stat_activity; in psql. It seems that the connections are in the state idle as shown on the attached picture.
Updated by Radko Krkoš over 2 years ago
Rajmund Hruska wrote in #note-3:
In the Vagrant, I changed
max_connectionsto 25, I ran tests and then I executedSELECT * FROM pg_stat_activity;inpsql. It seems that the connections are in the stateidleas shown on the attached picture.
Based on the successive timestamps "state_change" it seems that connection is made for each test separately and then not closed.
Updated by Rajmund Hruška over 2 years ago
It seems that the problem lies in the lib/hawat/test/__init__.py. I put the session.close() at in the teardown_db method but that didn't help.
def teardown_db(self):
with self.app.app_context():
hawat.db.db_get().drop_all()
hawat.db.db_session().close()
Updated by Rajmund Hruška over 2 years ago
Honestly, I don't know how to solve this one. Disposing of connection pool with hawat.db.db_get().get_engine(self.app).dispose() also didn't help.
Updated by Radko Krkoš over 2 years ago
Rajmund Hruska wrote in #note-7:
Honestly, I don't know how to solve this one. Disposing of connection pool with
hawat.db.db_get().get_engine(self.app).dispose()also didn't help.
OK. Do you want someone else to look at it?
Updated by Rajmund Hruška over 2 years ago
Updated by Rajmund Hruška over 2 years ago
- Assignee changed from Rajmund Hruška to Radko Krkoš
Updated by Pavel Kácha over 2 years ago
- Target version changed from 2.10 to 2.11
Updated by Rajmund Hruška over 2 years ago
I ran the tests with nose2 and unittest and default Postgres configuration file, where max_connections = 100. I didn't come across the problem with the connections, so it really is an issue with nose. I really thought I had the same issue when I ran tests with nose2 a couple of weeks ago, but it seems fine now.
Updated by Radko Krkoš over 2 years ago
Rajmund Hruska wrote in #note-13:
I ran the tests with
nose2andunittestand default Postgres configuration file, wheremax_connections = 100. I didn't come across the problem with the connections, so it really is an issue withnose. I really thought I had the same issue when I ran tests withnose2a couple of weeks ago, but it seems fine now.
Good news then.
Updated by Pavel Kácha over 2 years ago
- Target version changed from 2.11 to 2.10