Project

General

Profile

Actions
Copy link

Bug #7613

closed

Various fields insist on interpreting whitespace

Added by Pavel Kácha over 1 year ago. Updated 10 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Jakub Maloštik
Category:
Development - GUI
Target version:
2.11
Start date:
10/21/2022
Due date:
% Done:

0%

Estimated time:
To be discussed:

Description

At least user and group management search forms keep whitespace when searching. As both username and group name must not contain whitespace, this is unnecessary and annoying on typos.

May make sense to review other forms.

Related issues

Related to Mentat - Bug #7686: Whitespace(s) around IPs cause parsing problemClosedJakub Judiny10/25/2023

Actions
Actions
Copy link
 #1

Updated by Jakub Maloštik about 1 year ago

  • Assignee set to Jakub Maloštik
Actions
Copy link
 #2

Updated by Jakub Maloštik about 1 year ago

While resolving this, I noticed that there are pretty weak constraints for users' full names. For example, names can (and do) contain two consecutive spaces, which makes searching harder. Should a constraint for this be added in this issue, should a new issue be created or should this be completely disregarded? (in case this should be dealt with, is a migration for changing existing names for users a good solution?)

Actions
Copy link
 #3

Updated by Jakub Maloštik about 1 year ago

  • To be discussed changed from No to Yes
Actions
Copy link
 #4

Updated by Jakub Maloštik about 1 year ago

A similar thing goes for the group name, it is nowhere enforced that the group name cannot contain a space.

Actions
Copy link
 #5

Updated by Rajmund Hruška about 1 year ago

If group name contains multiple spaces, only one space is displayed in GUI. I don't know if that's a feature or a bug, I am just letting you know.

https://mentat-alt.cesnet.cz/mentat/groups/340/show

Actions
Copy link
 #6

Updated by Rajmund Hruška 12 months ago

When imported abuse group has multiple email addresses, the resulting name of the group is the concatenation of all addresses using ',' as a separator. Example of such group can be found here - https://mentat-alt.cesnet.cz/mentat/groups/188/show. But the migration would render them invalid.

Actions
Copy link
 #7

Updated by Pavel Kácha 12 months ago

Rajmund Hruška wrote in #note-6:

When imported abuse group has multiple email addresses, the resulting name of the group is the concatenation of all addresses using ',' as a separator. Example of such group can be found here - https://mentat-alt.cesnet.cz/mentat/groups/188/show. But the migration would render them invalid.

Twas probably me on the meeting to suggest normalising group names similar to usernames. Ok, so how about underscore as a separator?

Actions
Copy link
 #8

Updated by Rajmund Hruška 12 months ago

Pavel Kácha wrote in #note-7:

Rajmund Hruška wrote in #note-6:

When imported abuse group has multiple email addresses, the resulting name of the group is the concatenation of all addresses using ',' as a separator. Example of such group can be found here - https://mentat-alt.cesnet.cz/mentat/groups/188/show. But the migration would render them invalid.

Twas probably me on the meeting to suggest normalising group names similar to usernames. Ok, so how about underscore as a separator?

I think underscore is OK. So, Jakub Maloštik can you change the separator in netmngr.py script?

Actions
Copy link
 #9

Updated by Jakub Maloštik 12 months ago

Rajmund Hruška wrote in #note-8:

Pavel Kácha wrote in #note-7:

Rajmund Hruška wrote in #note-6:

When imported abuse group has multiple email addresses, the resulting name of the group is the concatenation of all addresses using ',' as a separator. Example of such group can be found here - https://mentat-alt.cesnet.cz/mentat/groups/188/show. But the migration would render them invalid.

Twas probably me on the meeting to suggest normalising group names similar to usernames. Ok, so how about underscore as a separator?

I think underscore is OK. So, Jakub Maloštik can you change the separator in netmngr.py script?

Since Nentat seems to allow for the + symbol in email addresses it is also replaced with '_'. Done.

Actions
Copy link
 #10

Updated by Rajmund Hruška 12 months ago

  • Status changed from New to Resolved
Actions
Copy link
 #11

Updated by Rajmund Hruška 12 months ago

  • Status changed from Resolved to Feedback

Jakub Maloštik, did you push all commits to homeproj?

Actions
Copy link
 #12

Updated by Jakub Maloštik 12 months ago

Rajmund Hruška wrote in #note-11:

Jakub Maloštik, did you push all commits to homeproj?

I believe so. Did I forget something?

Actions
Copy link
 #13

Updated by Rajmund Hruška 11 months ago

Jakub Maloštik wrote in #note-12:

Rajmund Hruška wrote in #note-11:

Jakub Maloštik, did you push all commits to homeproj?

I believe so. Did I forget something?

Sorry, I didn't notice c7d0bd2c, as it wasn't linked to this issue.

Actions
Copy link
 #14

Updated by Rajmund Hruška 11 months ago

  • Status changed from Feedback to In Progress

On the devel branch, I built the machine from scratch and then imported the data from negistry whois file (located at /var/mentat/whois-negistry.json on mentat-hub/alt) using mentat-netmngr --command update --whois-file whois-negistry.json --source negistry. After that, I merged your branch and ran hawat-cli db upgrade. I checked the web interface at mentat/groups/list and I got 500 error.

[Mon Jun 05 12:47:28.923381 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036] [2023-06-05 12:47:28,923] DEBUG in base: Loading ACL identity for user 'admin'., referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.931199 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036] mod_wsgi (pid=32514): Exception occurred processing WSGI script '/vagrant/bin/mentat-hawat-vagrant.wsgi'., referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.932898 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036] Traceback (most recent call last):, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.933287 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 2552, in __call__, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.933402 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     return self.wsgi_app(environ, start_response), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.933421 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 2532, in wsgi_app, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.933427 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     response = self.handle_exception(e), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.933437 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 2529, in wsgi_app, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.933443 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     response = self.full_dispatch_request(), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.933452 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1825, in full_dispatch_request, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.933458 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     rv = self.handle_user_exception(e), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.933467 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/flask/app.py", line 1823, in full_dispatch_request, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.933474 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     rv = self.dispatch_request(), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.933483 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/flask_debugtoolbar/__init__.py", line 142, in dispatch_request, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.933850 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     return view_func(**req.view_args), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.934037 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/flask_login/utils.py", line 290, in decorated_view, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.934045 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     return current_app.ensure_sync(func)(*args, **kwargs), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.934054 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/flask_principal.py", line 199, in _decorated, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.934061 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     rv = f(*args, **kw), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.934069 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/flask/views.py", line 107, in view, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.934075 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     return current_app.ensure_sync(self.dispatch_request)(**kwargs), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.934083 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/vagrant/lib/hawat/view/__init__.py", line 1197, in dispatch_request, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.934304 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     form = self.get_search_form(flask.request.args), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.934464 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/vagrant/lib/hawat/blueprints/groups/__init__.py", line 128, in get_search_form, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.934570 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     meta={'csrf': False}, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935038 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/wtforms/form.py", line 208, in __call__, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935077 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     return type.__call__(cls, *args, **kwargs), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935088 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/vagrant/lib/hawat/blueprints/groups/forms.py", line 295, in __init__, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935094 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     super().__init__(*args, **kwargs), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935103 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/flask_wtf/form.py", line 73, in __init__, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935108 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     super().__init__(formdata=formdata, **kwargs), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935117 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/wtforms/form.py", line 286, in __init__, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935122 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     self.process(formdata, obj, data=data, **kwargs), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935131 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/wtforms/form.py", line 127, in process, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935148 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     field.process(formdata, data, extra_filters=field_extra_filters), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935157 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]   File "/var/mentat/venv/lib/python3.7/site-packages/wtforms/fields/core.py", line 334, in process, referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935163 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036]     self.data = filter(self.data), referer: https://localhost:4443/mentat/
[Mon Jun 05 12:47:28.935194 2023] [wsgi:error] [pid 32514:tid 140328837236480] [client 10.0.2.2:46036] TypeError: descriptor 'strip' requires a 'str' object but received a 'NoneType', referer: https://localhost:4443/mentat/
Actions
Copy link
 #15

Updated by Jakub Maloštik 11 months ago

I wasn't able to reproduce the problem but tried to fix it with the revision 12f2055f.

Actions
Copy link
 #16

Updated by Jakub Maloštik 11 months ago

  • Status changed from In Progress to Resolved
Actions
Copy link
 #17

Updated by Rajmund Hruška 11 months ago

  • Status changed from Resolved to Feedback

I accidentally found some case when the migration doesn't work.

mentat_main=# update groups set name = '?abuse@zcu.cz' where name ~ 'zcu';
UPDATE 1
mentat_main=# update groups set name = '?abuse@zcu.cz' where name ~ 'ssa';
ERROR:  duplicate key value violates unique constraint "ix_groups_name" 
DETAIL:  Key (name)=(?abuse@zcu.cz) already exists.
mentat_main=# update groups set name = '!abuse@zcu.cz' where name ~ 'ssa';
UPDATE 1
mentat_main=# select name from groups where name ~ 'zcu';
     name      
---------------
 ?abuse@zcu.cz
 !abuse@zcu.cz
(2 rows)

And after running the migration:

mentat_main=# select name from groups where name ~ 'zcu';
            name            
----------------------------
 ?abuse@zcu.cz
 INVALID_NAME__abuse@zcu_cz
(2 rows)

It seems like the issue is in '?' character. When I ran the migration with these groups:

mentat_main=# select name from groups where name ~ '\?.*';
              name              
--------------------------------
 ?abuse@zcu.cz
 ?abuse@rokycany.nemocnicepk.cz
 ?abuse@muni.cz
 ?vtp-abuse@fi.muni.cz

nothing happened. After prepending '!':

mentat_main=# update groups set name = '!' || name where name ~ '\?.*';
UPDATE 4
mentat_main=# select name from groups where name ~ '\!.*';
              name               
---------------------------------
 !?abuse@zcu.cz
 !?abuse@muni.cz
 !?abuse@rokycany.nemocnicepk.cz
 !?vtp-abuse@fi.muni.cz
(4 rows)

and running migration again, it worked:
mentat_main=# select name from groups where name ~ 'INVALID';
                     name                     
----------------------------------------------
 INVALID_NAME___abuse@muni_cz
 INVALID_NAME__abuse@zcu_cz
 INVALID_NAME___abuse@zcu_cz
 INVALID_NAME___abuse@rokycany_nemocnicepk_cz
 INVALID_NAME___vtp-abuse@fi_muni_cz
(5 rows)

Actions
Copy link
 #18

Updated by Rajmund Hruška 10 months ago

  • Status changed from Feedback to In Review
  • Target version changed from Backlog to 2.11
  • To be discussed deleted (Yes)
Actions
Copy link
 #19

Updated by Rajmund Hruška 10 months ago

  • Status changed from In Review to Closed
Actions
Copy link
 #20

Updated by Jakub Judiny 6 months ago

  • Related to Bug #7686: Whitespace(s) around IPs cause parsing problem added
Actions
Copy link

Also available in: Atom PDF