infra/tickets

Fork 1

FAS login occasionally fails with HTTP 400. #13280

New issue

Closed

opened 2026-04-20 17:14:23 +00:00 by rokejulianlockhart · 20 comments

rokejulianlockhart commented

2026-04-20 17:14:23 +00:00

Description of request

At id.fedoraproject.org/openidc/Continue?ipsilon_transaction_id=88cb46e1-8b98-4f74-82b9-82135336eccb, as I frequently do, I observed the undermentioned, upon attempted, unsuccessful login:

400 – Bad Request

Invalid Transaction ID

If the error persists and if you think this is an error, contact Fedora Infra to resolve the problem.

I've yet to acquire HAR, because it rarely occurs consequtively, despite frequently recurring. Regardless, this has been previously reported elsewhere, many times:

However, no tickets, since the transferral from Pagure, remain open, to track this.

My Environment

#!/usr/bin/env sh
rpm -qf $(command -v firefox-nightly) \
	--queryformat "$(
		cat <<'EOF'
Name: %{NAME}
Version: %{VERSION}
Release: %{RELEASE}
Architecture: %{ARCH}
Install Date: %{INSTALLTIME:date}
Size: %{SIZE}
Signature: %{SIGPGP:pgpsig}
Source RPM: %{SOURCERPM}
Build Date: %{BUILDTIME:date}
Build Host: %{BUILDHOST}
Packager: %{PACKAGER}
Vendor: %{VENDOR}
EOF
	)"$'\n' | yq -P

Name: firefox-nightly
Version: 151.0a1
Release: 20260420091406
Architecture: x86_64
Install Date: Mon 20 Apr 2026 17:57:44 BST
Size: 384505308
Signature: (none)
Source RPM: firefox-nightly-151.0a1-20260420091406.src.rpm
Build Date: Mon 20 Apr 2026 12:24:33 BST
Build Host: b3a886767332
Packager: (none)
Vendor: Mozilla

### Description of request At [`id.fedoraproject.org/openidc/Continue?ipsilon_transaction_id=88cb46e1-8b98-4f74-82b9-82135336eccb`](https://id.fedoraproject.org/openidc/Continue?ipsilon_transaction_id=88cb46e1-8b98-4f74-82b9-82135336eccb), as I frequently do, I observed the undermentioned, upon attempted, unsuccessful login: > # 400 – Bad Request > > Invalid Transaction ID > > If the error persists and if you think this is an error, contact <a href="https://forge.fedoraproject.org/infra/tickets/issues">Fedora Infra</a> to resolve the problem. I've yet to acquire HAR, because it rarely occurs consequtively, despite frequently recurring. Regardless, this has been previously reported elsewhere, many times: - [`issues/12294`](https://forge.fedoraproject.org/infra/tickets/issues/12294#issuecomment-392665) - [`issues/12715`](https://forge.fedoraproject.org/infra/tickets/issues/12715#issuecomment-392663) - [`issues/12534`](https://forge.fedoraproject.org/infra/tickets/issues/12534#issuecomment-268161) However, no tickets, since the transferral from Pagure, remain open, to track this. #### My Environment 1. <p></p><details><p></p> ~~~sh #!/usr/bin/env sh rpm -qf $(command -v firefox-nightly) \ --queryformat "$( cat <<'EOF' Name: %{NAME} Version: %{VERSION} Release: %{RELEASE} Architecture: %{ARCH} Install Date: %{INSTALLTIME:date} Size: %{SIZE} Signature: %{SIGPGP:pgpsig} Source RPM: %{SOURCERPM} Build Date: %{BUILDTIME:date} Build Host: %{BUILDHOST} Packager: %{PACKAGER} Vendor: %{VENDOR} EOF )"$'\n' | yq -P ~~~ <p></p></details><p></p> 1. <blockquote> ~~~YAML Name: firefox-nightly Version: 151.0a1 Release: 20260420091406 Architecture: x86_64 Install Date: Mon 20 Apr 2026 17:57:44 BST Size: 384505308 Signature: (none) Source RPM: firefox-nightly-151.0a1-20260420091406.src.rpm Build Date: Mon 20 Apr 2026 12:24:33 BST Build Host: b3a886767332 Packager: (none) Vendor: Mozilla ~~~ </blockquote>

rokejulianlockhart added spent time

2026-04-20 17:14:45 +00:00

20 minutes

james added the

points

Gain

Medium

labels

2026-04-20 19:15:04 +00:00

james commented

2026-04-20 19:49:03 +00:00

Member

We see this occasionally, where it fixes itself if you go back and try again ... and it maybe happens more often if you leave the window open before doing the FAS authentication.

Nobody really knows why, and we're hoping that it'll get fixed as part of the migration to keycloak (hopefully soon, post freeze, now). I can leave this issue open for a bit.

We see this occasionally, where it fixes itself if you go back and try again ... and it maybe happens more often if you leave the window open before doing the FAS authentication. Nobody really knows why, and we're hoping that it'll get fixed as part of the migration to keycloak (hopefully soon, post freeze, now). I can leave this issue open for a bit.

👍 1

rokejulianlockhart commented

2026-04-20 20:09:51 +00:00

Author

@james, thanks. I shall link this to issues/13188, then.

@james, thanks. I shall link this to [`issues/13188`](https://forge.fedoraproject.org/infra/tickets/issues/13188#issue-99945), then.

smoliicek referenced this issue

2026-04-29 19:11:04 +00:00

Error when logging into Fedora accounts: 400 - Bad Request, invalid transaction id #13298

smoliicek referenced this issue

2026-05-13 19:42:02 +00:00

400 error when trying to log into Fedora Discussions #13342

rokejulianlockhart commented

2026-05-15 00:16:52 +00:00

Author

In retrospect, the undermentioned is irrelevant:

Strangely, I've had the error page also cause:

…if at all relevant!

In retrospect, the undermentioned is irrelevant: <p></p><details><p></p> Strangely, I've had the error page also cause: 1. [`crash-stats.mozilla.org/report/index/76e72c01-153f-4ba2-a629-8cb910260515`](https://crash-stats.mozilla.org/report/index/76e72c01-153f-4ba2-a629-8cb910260515) 1. [`crash-stats.mozilla.org/report/index/4ed413c3-6df3-4978-9979-96ceb0260515`](https://crash-stats.mozilla.org/report/index/4ed413c3-6df3-4978-9979-96ceb0260515) 1. [`crash-stats.mozilla.org/report/index/73f72c59-1d4c-4c23-9ea3-d631b0260515`](https://crash-stats.mozilla.org/report/index/73f72c59-1d4c-4c23-9ea3-d631b0260515) …if at all relevant! <video src="/attachments/dafed6de-85e9-4e06-8b00-c073d6c92653" title="@{'Date'='20260515T011038+0100'; 'Type'='Video'; 'Origin'='" controls></video> <p></p></details><p></p>

@{'Date'='20260515T011038+0100'; 'Type'='Video'; 'Origin'='.webm

2.3 MiB

smoliicek referenced this issue

2026-06-17 10:22:42 +00:00

Can't log in to my FAS account #13414

smoliicek referenced this issue

2026-06-29 19:20:47 +00:00

Cannot login to bugziilla #13438

rokejulianlockhart commented

2026-06-30 13:44:30 +00:00

Author

This recently occurred, at id.fedoraproject.org/saml2/SSO/Continue?ipsilon_transaction_id=b766d92d-002c-4da0-9526-fddf8881fc38:

<!doctype html>
<html>

    <head>
        <meta charset="UTF-8">
        <title>Bad Request</title>
        <link href="https://apps.fedoraproject.org/global/fedora-bootstrap-1.5.0/fedora-bootstrap.min.css" rel="stylesheet">
        <link href='/ui/res/login.css' rel='stylesheet' type='text/css'>
        <meta name="generator" content="Ipsilon">
    </head>

    <body>
        <div class="navbar navbar-light masthead">
            <div class="container justify-content-between"><img alt="logo" src="/ui/res/FedoraAccounts.png" height=40px />
                <div></div>
            </div>
        </div>
        <div class="bodycontent py-5 min-height-50">
            <div class="container">
                <h1 class="text-center"><strong>400 - Bad Request</strong></h1>
                <p class="text-center">Invalid transaction id</p>
                <p class="text-center">If the error persists and if you think this is an error, contact <a href="https://forge.fedoraproject.org/infra/tickets/issues">Fedora Infra</a> to resolve the problem.</p>
            </div>
        </div>

…when attempting to authenticate at pagure.io/login/?next=https://pagure.io/ipsilon/issue/398.

bugzilla.mozilla.org/show_bug.cgi?id=2051153#c3 and bugzilla.mozilla.org/show_bug.cgi?id=2051153#c3 made this difficult to diagnose:

However, because I couldn't get the entire flow to reproduce the failure (although ipsilon_transaction_id=870287d6-9603-4741-805b-35f51abda1d8 occurred soon afterward), even the -jsconsole didn't assist — I solely observe a request for the FAS logo:

My Environment

The Output Of about:support, at ./attachments/ ¹ ²

This recently occurred, at [`id.fedoraproject.org/saml2/SSO/Continue?ipsilon_transaction_id=b766d92d-002c-4da0-9526-fddf8881fc38`](https://id.fedoraproject.org/saml2/SSO/Continue?ipsilon_transaction_id=b766d92d-002c-4da0-9526-fddf8881fc38#:~:text=400%20%2D%20Bad%20Request,Infra%20to%20resolve%20the%20problem%2E): <blockquote> ~~~HTML <!doctype html> <html> <head> <meta charset="UTF-8"> <title>Bad Request</title> <link href="https://apps.fedoraproject.org/global/fedora-bootstrap-1.5.0/fedora-bootstrap.min.css" rel="stylesheet"> <link href='/ui/res/login.css' rel='stylesheet' type='text/css'> <meta name="generator" content="Ipsilon"> </head> <body> <div class="navbar navbar-light masthead"> <div class="container justify-content-between"><img alt="logo" src="/ui/res/FedoraAccounts.png" height=40px /> <div></div> </div> </div> <div class="bodycontent py-5 min-height-50"> <div class="container"> <h1 class="text-center"><strong>400 - Bad Request</strong></h1> <p class="text-center">Invalid transaction id</p> <p class="text-center">If the error persists and if you think this is an error, contact <a href="https://forge.fedoraproject.org/infra/tickets/issues">Fedora Infra</a> to resolve the problem.</p> </div> </div> ~~~ </blockquote> …when attempting to authenticate at [`pagure.io/login/?next=https://pagure.io/ipsilon/issue/398`](https://pagure.io/login/?next=https://pagure.io/ipsilon/issue/398). [`bugzilla.mozilla.org/show_bug.cgi?id=2051153#c3`](https://bugzilla.mozilla.org/show_bug.cgi?id=2051153#c3) and [`bugzilla.mozilla.org/show_bug.cgi?id=2051153#c3`](https://bugzilla.mozilla.org/show_bug.cgi?id=2043908#c0) made this difficult to diagnose: ![`Screenshot_20260630_142620`](/attachments/7b48f944-5244-4d0b-b24c-253bfdbd5a13) However, because I couldn't get the entire flow to reproduce the failure (although [`ipsilon_transaction_id=870287d6-9603-4741-805b-35f51abda1d8`](https://id.fedoraproject.org/openidc/Continue?ipsilon_transaction_id=870287d6-9603-4741-805b-35f51abda1d8) occurred soon afterward), even the `-jsconsole` didn't assist — I solely observe a request for the FAS logo: ![`Screenshot_20260630_142800`](/attachments/ad453175-1521-4ca1-b2e0-187ace10d8da) #### My Environment [The Output Of `about:support`, at `./attachments/`](/attachments/6cda71fc-0dcd-498b-b157-b3edff266bb6) [^1] [^2] [^1]: [`codeberg.org/forgejo/forgejo/issues/13259`](https://codeberg.org/forgejo/forgejo/issues/13259#issue-6033575) [^2]: [`codeberg.org/forgejo/forgejo/issues/13260`](https://codeberg.org/forgejo/forgejo/issues/13260#issue-6033620)

Screenshot_20260630_142800.png

59 KiB

e5b6af4611e5a9acab7ff65a0d034146dc6625e32fc8b5fac6f8efd36856cea51783e075f2367e80c618c9da7592e61dba5712cb5a01e9a5e53a444094ebb219.txt.JSON

169 KiB

Screenshot_20260630_142620.png

80 KiB

catanzaro commented

2026-06-30 14:50:38 +00:00

I hit the same error just now. It's a very common problem, at least for me.

👍 2

kevin commented

2026-07-31 16:04:12 +00:00

Owner

I think we figured out what this problem was and have worked around it.

If any of you still see it, please let us know.

I think we figured out what this problem was and have worked around it. If any of you still see it, please let us know.

rokejulianlockhart commented

2026-07-31 16:13:33 +00:00

Author

@kevin, unfortunately, this continues to consistently reproduce on every first login:

@kevin, unfortunately, this continues to consistently reproduce on every first login: ![`Screenshot_20260731_171237`](/attachments/cd2be84b-7095-46bf-87b4-36234071a71e)

Screenshot_20260731_171237.png

76 KiB

frostyx commented

2026-07-31 16:28:42 +00:00

It happened to me ~1 hour ago too

kevin commented

2026-07-31 16:35:50 +00:00

Owner

sigh. I had to reboot one of the backends and you may have testing right when it rebooted.

Can you try again now?

sigh. I had to reboot one of the backends and you may have testing _right_ when it rebooted. Can you try again now?

kevin commented

2026-07-31 16:43:45 +00:00

Owner

ok, next question: Do you have 'advanced tracking protection' enabled? If you disable it does the problem go away?

rokejulianlockhart commented

2026-07-31 20:17:10 +00:00

Author

@kevin, about:config returns:

privacy.trackingprotection.enabled: false
privacy.trackingprotection.socialtracking.enabled: false
privacy.trackingprotection.pbmode.enabled: false
privacy.trackingprotection.fingerprinting.enabled: false
privacy.trackingprotection.allow_list.hasUserInteractedWithETPSettings: true

…whereas about:preferences#etpCustomize:~:text=Customize%20tracking%20protection returns:

Additionally, I attempted to acquire HAR again, but it ceased to reproduce, again! ¹

/attachments/0203651b-5fd4-4afe-9179-4531ebc69105 ↩︎

@kevin, `about:config` returns: ~~~YAML privacy.trackingprotection.enabled: false privacy.trackingprotection.socialtracking.enabled: false privacy.trackingprotection.pbmode.enabled: false privacy.trackingprotection.fingerprinting.enabled: false privacy.trackingprotection.allow_list.hasUserInteractedWithETPSettings: true ~~~ …whereas `about:preferences#etpCustomize:~:text=Customize%20tracking%20protection` returns: ![image](/attachments/7eb2e2b0-1a70-46a7-b03a-45dba907d602) Additionally, I attempted to acquire HAR again, but it ceased to reproduce, again! [^1] [^1]: [`/attachments/0203651b-5fd4-4afe-9179-4531ebc69105`](/attachments/0203651b-5fd4-4afe-9179-4531ebc69105)

Screencast_20260731_211334.mp4

260 KiB

image.png

96 KiB

frostyx commented

2026-07-31 22:00:02 +00:00

I use the defaults that come with my https://zen-browser.app/ but I have many, many trackingprotection related settings in my about:config. I'll try to disable them and see.

I use the defaults that come with my https://zen-browser.app/ but I have many, many `trackingprotection` related settings in my `about:config`. I'll try to disable them and see.

kevin referenced this issue

2026-08-01 15:19:04 +00:00

Authentication issues #13482

marcindulak commented

2026-08-04 15:11:38 +00:00

Was getting that "400 – Bad Request" when trying to FAS login in on https://bodhi.fedoraproject.org/updates/FEDORA-2026-2551d0d66b and https://src.fedoraproject.org/rpms/python-ase

Tried from both Google Chrome (Version 150.0.7871.186 (Official Build) (64-bit)) and Firefox (153.0.1), also in private windows.

Made 5 failed attempts, then cleaned all https://src.fedoraproject.org cookies, and succeeded to login onto it after two more failed attempts in Google chrome.

Was getting that "400 – Bad Request" when trying to FAS login in on https://bodhi.fedoraproject.org/updates/FEDORA-2026-2551d0d66b and https://src.fedoraproject.org/rpms/python-ase Tried from both Google Chrome (Version 150.0.7871.186 (Official Build) (64-bit)) and Firefox (153.0.1), also in private windows. Made 5 failed attempts, then cleaned all https://src.fedoraproject.org cookies, and succeeded to login onto it after two more failed attempts in Google chrome. ![image](/attachments/a8e80f04-96c4-4a31-aaf9-69a8ef521f84)

image.png

125 KiB

abompard commented

2026-08-06 17:23:08 +00:00

Member

Hey folks. I have spent a few hours on this, I think I may know where it's coming from. I think it's a bug in Ipsilon.

Ipsilon defines pages as objects, which are exposed by CherryPy. However, in ipsilon.login.common.LoginFormBase, it stores the current transaction as an instance variable in self.trans. I think this object gets overwritten if another login happens in parallel, because this does not look thread-safe. As a result the transaction id does not match the cookie anymore and is considered invalid.

I'll keep looking at this tomorrow and attempt to fix it, but I wanted to write this down here so others know we're working on it (and I don't forget everything on Monday ;-) )

Hey folks. I have spent a few hours on this, I think I may know where it's coming from. I think it's a bug in Ipsilon. Ipsilon defines pages as objects, which are exposed by CherryPy. However, in `ipsilon.login.common.LoginFormBase`, it stores the current transaction as an instance variable in `self.trans`. I think this object gets overwritten if another login happens in parallel, because this does not look thread-safe. As a result the transaction id does not match the cookie anymore and is considered invalid. I'll keep looking at this tomorrow and attempt to fix it, but I wanted to write this down here so others know we're working on it (and I don't forget everything on Monday ;-) )

👍 1

abompard commented

2026-08-06 18:04:26 +00:00

Member

I've set mod_wgi to 1 thread only (and added 2 processes to compensate). Let's see if it helps.

I've set `mod_wgi` to 1 thread only (and added 2 processes to compensate). Let's see if it helps.

kevin commented

2026-08-06 22:25:47 +00:00

Owner

Oops. I ran playbooks and I think it overrode your changes. ;( Sorry about that...

abompard commented

2026-08-07 06:43:28 +00:00

Member

No worries, I've put it back.
I'll commit it to ansible too because we should probably observe this for a few days / week(s).

No worries, I've put it back. I'll commit it to ansible too because we should probably observe this for a few days / week(s).

abompard commented

2026-08-10 11:02:56 +00:00

Member

OK I think that fixed the issue. Of course the proper fix would be to have Ipsilon use thread-local variables instead of class attributes, using for example cherrypy.thread_data, but that would be a much larger change that would impact multiple places in Ipsilon and should be properly tested. For now, I think using processes instead of threads is sufficient, as we plan to replace Ipsilon soonish.

OK I *think* that fixed the issue. Of course the proper fix would be to have Ipsilon use thread-local variables instead of class attributes, using for example `cherrypy.thread_data`, but that would be a much larger change that would impact multiple places in Ipsilon and should be properly tested. For now, I think using processes instead of threads is sufficient, as we plan to replace Ipsilon soonish.

❤️ 1