Bug #7553
closedWrong display of national characters in preloaded data in user registration form
100%
Description
The user registration form at: https://mentat-hub.cesnet.cz/mentat/auth_env/register displays the national characters in full name wrongly. The input can be corrected and then the right representation is visible in user management, so the trouble seems to lie in load from identity provider. The case when the name is not fixed by user was not tested. The bug is also present on mentat-alt.
Updated by Pavel Kácha about 2 years ago
- Priority changed from High to Normal
- Target version changed from Backlog to 2.10
Updated by Radko Krkoš almost 2 years ago
Also observed in e-mail (see Full name):
Return-Path: <mentat@mentat-alt.cesnet.cz>
X-Original-To: krkos@office2.cesnet.cz
Delivered-To: krkos@office2.cesnet.cz
Received: from postino.cesnet.cz (postino.cesnet.cz [IPv6:2001:718:1:101::144:242])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by office2.cesnet.cz (Postfix) with ESMTPS id 915A3400076
for <krkos@office2.cesnet.cz>; Thu, 12 May 2022 14:09:10 +0200 (CEST)
Received: by postino.cesnet.cz (Postfix, from userid 110)
id 8CFEC580090; Thu, 12 May 2022 14:09:10 +0200 (CEST)
X-Spam-Status: No
X-Spam-cesnet-Status: No, score=-1.7 required=5.0 tests=BAYES_00,SPF_HELO_NONE,
SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no
version=3.4.6
X-Spam-Level:
X-Spam-Virus: No
X-Spam-Report:
* -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1%
* [score: 0.0000]
* 0.0 SPF_NONE SPF: sender does not publish an SPF Record
* 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record
* -0.0 T_SCC_BODY_TEXT_LINE No description available.
* 0.2 TXREP TXREP: Score normalizing based on sender's reputation
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on postino.cesnet.cz
X-postfwd: rbl:wl.cesnet.cz:<mentat-alt.cesnet.cz is listed on CESNET welcomelist since 1652341206.0.>
X-postfwd: rbl:wl.cesnet.cz:<mentat-alt.cesnet.cz is listed on CESNET welcomelist since 1652341206.0.>
X-postfwd: rbl:wl.cesnet.cz:<mentat-alt.cesnet.cz is listed on CESNET welcomelist since 1652341206.0.>
X-postfwd: rbl:wl.cesnet.cz:<mentat-alt.cesnet.cz is listed on CESNET welcomelist since 1652341206.0.>
Received: from mentat-alt.cesnet.cz (mentat-alt.cesnet.cz [195.113.161.44])
by postino.cesnet.cz (Postfix) with ESMTP id 9D4D758008E;
Thu, 12 May 2022 14:09:09 +0200 (CEST)
Received: by mentat-alt.cesnet.cz (Postfix)
id 9393034C007B; Thu, 12 May 2022 14:09:09 +0200 (CEST)
Delivered-To: root@mentat-alt.cesnet.cz
Received: from mentat-alt.cesnet.cz (localhost [IPv6:::1])
by mentat-alt.cesnet.cz (Postfix) with ESMTP id 8F9E234C007A
for <root@mentat-alt.cesnet.cz>; Thu, 12 May 2022 14:09:09 +0200 (CEST)
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Subject: [Mentat] Feedback for report - M20220512EL-IA39J
From: mentat@mentat-alt.cesnet.cz
To: root@mentat-alt.cesnet.cz
Date: Thu, 12 May 2022 14:09:09 +0200
Message-ID: <165235734950.19524.7826368553439102015@mentat-alt.cesnet.cz>
Reply-To: Rajmund.Hruska@cesnet.cz
Dear administrator,
Feedback for report "M20220512EL-IA39J (regular_recon-scanning, ip:
2001:67c:1220:8b4:565:f268:b312:22a4)" has been send by user:
Login: hruska@cesnet.cz
Full name: Rajmund Hubert Hruška
Email: Rajmund.Hruska@cesnet.cz
Organization: CESNET, z. s. p. o.
Message by user:
TEST by Rajmund
Link to report:
https://mentat.domain.org/mentat/reports/M20220512EL-IA39J/unauth#regular_recon-scanning
Have a nice day
-- Mentat System
Updated by Rajmund Hruška almost 2 years ago
- Status changed from New to In Progress
So far I have concluded that the fullname given by the environment variable cn is encoded in UTF-8. I printed the bytes of the string given by the environment with this code ":".join("{:02x}".format(ord(c)) for c in s) and I used a online convertor to convert my name to bytes (https://onlineutf8tools.com/convert-utf8-to-bytes). I got the same sequence of bytes.
What I don't know yet is why it's displayed in this way. And why it's stored in the database as a wrong string.
Updated by Rajmund Hruška almost 2 years ago
- % Done changed from 0 to 40
I don't really know why, but this seems to be working:
item.fullname = flask.request.environ['cn'].encode("iso-8859-1").decode()
Updated by Radko Krkoš almost 2 years ago
Rajmund Hruska wrote in #note-7:
I don't really know why, but this seems to be working:
Yeah, incorrectly declared encoding somewhere. The mentioned code looks like a valid workaround to me. If you are not into more detective work, I am OK with using that.
Updated by Rajmund Hruška almost 2 years ago
- Status changed from In Progress to Resolved
- % Done changed from 40 to 100
Radko Krkoš wrote in #note-8:
Rajmund Hruska wrote in #note-7:
I don't really know why, but this seems to be working:
Yeah, incorrectly declared encoding somewhere. The mentioned code looks like a valid workaround to me. If you are not into more detective work, I am OK with using that.
Great! So it should be fixed in 4c89be6b.
Nothing much happens when the name isn't fixed by the user. For example on mentat-hub there is only one user for whom the full name wasn't fixed - https://mentat-hub.cesnet.cz/mentat/users/169/show.
Updated by Radko Krkoš almost 2 years ago
Rajmund Hruska wrote in #note-9:
Radko Krkoš wrote in #note-8:
Rajmund Hruska wrote in #note-7:
I don't really know why, but this seems to be working:
Yeah, incorrectly declared encoding somewhere. The mentioned code looks like a valid workaround to me. If you are not into more detective work, I am OK with using that.
Great! So it should be fixed in 4c89be6b.
Nothing much happens when the name isn't fixed by the user. For example on mentat-hub there is only one user for whom the full name wasn't fixed - https://mentat-hub.cesnet.cz/mentat/users/169/show.
And the same goes for organization name, based on your example. Please, evaluate the fix for all parameters taken from flask environment.
Updated by Rajmund Hruška almost 2 years ago
Radko Krkoš wrote in #note-10:
Rajmund Hruska wrote in #note-9:
Radko Krkoš wrote in #note-8:
Rajmund Hruska wrote in #note-7:
I don't really know why, but this seems to be working:
Yeah, incorrectly declared encoding somewhere. The mentioned code looks like a valid workaround to me. If you are not into more detective work, I am OK with using that.
Great! So it should be fixed in 4c89be6b.
Nothing much happens when the name isn't fixed by the user. For example on mentat-hub there is only one user for whom the full name wasn't fixed - https://mentat-hub.cesnet.cz/mentat/users/169/show.
And the same goes for organization name, based on your example. Please, evaluate the fix for all parameters taken from flask environment.
Yes, I noticed just a few seconds after writing the previous post I fixed that as well. The other used variables are login and email address so there shouldn't be any problem with those.
Updated by Pavel Kácha almost 2 years ago
- Status changed from Resolved to In Progress
- Assignee changed from Rajmund Hruška to Radko Krkoš
Radko, could you try implement migration for these columns?
Updated by Radko Krkoš over 1 year ago
Pavel Kácha wrote in #note-12:
Radko, could you try implement migration for these columns?
As discussed, a migration without plpython3u is available in branch krkos-bug-#7553-auth-demangle_wo_plpython3u.
Updated by Rajmund Hruška over 1 year ago
- Status changed from In Progress to In Review
Merged into devel and deployed on mentat-alt. The migration was executed without any errors.