Switch from Pagure to Forge for blocker review discussions #296

Closed
opened 2026-01-20 13:26:09 +00:00 by kparal · 21 comments
Owner

Currently we support only Pagure for doing async blocker review discussions. For production, we use the blocker-review repo.

Since we're migrating from Pagure to Forge, we would need a change in blockerbugs that would drop Pagure support and introduce Forge support for blocker discussions.

This means:
a) implementing reading, creating and editing a Forge ticket through its API
b) figuring out how to get notified when a Forge ticket is updated (in Pagure, we do this through a webhook)
c) figuring out authentication with Forge, and testing for privileged members
d) fixing ticket formatting if needed

Don't forget that most of the changes can be tested through staging instances - https://stg.pagure.io , https://bugzilla.stage.redhat.com and https://forge.stg.fedoraproject.org .

Implementation checklist:

  • Merge #299 and deploy to staging
  • Test it properly
  • Migrate https://pagure.io/fedora-qa/blocker-review to https://forge.fedoraproject.org/quality/blocker-review . Update the README with latest version from BBA.
  • Merge to master, tag a new release, and deploy to production
  • Recreate discussions that currently point to Pagure. The currently open migrated tickets will likely need to be manually marked as duplicates and closed.
  • Update references to blocker-review (from wiki pages etc)
  • Announce the change publicly
  • Drop obsolete variables from Infra
Currently we support only Pagure for doing async blocker review discussions. For production, we use the [blocker-review](https://pagure.io/fedora-qa/blocker-review) repo. Since we're migrating from Pagure to Forge, we would need a change in blockerbugs that would drop Pagure support and introduce Forge support for blocker discussions. This means: a) implementing reading, creating and editing a Forge ticket through its API b) figuring out how to get notified when a Forge ticket is updated (in Pagure, we do this through a webhook) c) figuring out authentication with Forge, and testing for privileged members d) fixing ticket formatting if needed Don't forget that most of the changes can be tested through staging instances - https://stg.pagure.io , https://bugzilla.stage.redhat.com and https://forge.stg.fedoraproject.org . **Implementation checklist:** - [x] Merge #299 and deploy to staging - [x] Test it properly - [x] Migrate https://pagure.io/fedora-qa/blocker-review to https://forge.fedoraproject.org/quality/blocker-review . Update the README with latest version from BBA. - [x] Merge to master, tag a new release, and deploy to production - [x] Recreate discussions that currently point to Pagure. The currently open migrated tickets will likely need to be manually marked as duplicates and closed. - [x] Update references to blocker-review (from wiki pages etc) - [x] Announce the change publicly - [x] Drop obsolete variables from Infra
kparal added this to the Next BB tasks project 2026-01-20 13:26:10 +00:00
jgroman added reference fix/296-blocker-review-voting-on-forge 2026-01-27 09:06:34 +00:00
kparal modified the project from Sprint 2 to Sprint 3 2026-02-10 15:00:56 +00:00
Owner
  • The code currently lives in branch fix/296-blocker-review-voting-on-forge
  • Forge bot is configured using new constants with FORGEJO_ prefix in config.py, there is also new DISCUSSION_SYSTEM option, which needs to be changed from "pagure" to "forgejo"
  • Users currently get authorized to vote by being members of the team set in FORGEJO_ADMIN_TEAM (unfortunate name, needs refactoring :) )
- The code currently lives in branch `fix/296-blocker-review-voting-on-forge` - Forge bot is configured using new constants with `FORGEJO_` prefix in `config.py`, there is also new `DISCUSSION_SYSTEM` option, which needs to be changed from "pagure" to "forgejo" - Users currently get authorized to vote by being members of the team set in `FORGEJO_ADMIN_TEAM` (unfortunate name, needs refactoring :) )
adamwill modified the project from Sprint 3 to Sprint 4 2026-02-24 16:31:55 +00:00
jgroman modified the project from Sprint 4 to Sprint 5 2026-03-10 12:27:26 +00:00
Author
Owner

Some thoughts during my quick inspection today:

  • Pagure support is still kept in (and it involves some overhead code to differentiate between the two systems), do you have a preference? I'd either drop Pagure immediately, or (in case we want to play defensive), after a few weeks once Forgejo integration is verified to work in production. The latter is of course safer, but given that we'll likely push this to production after F44 cycle, we won't have many tickets to validate this with for the next 3-4 months, so I'm not sure if that helps us at all 🙂 I currently lean a bit towards just nuking Pagure immediately, which will make the new code simpler (including CLI commands). What about you?

Also, let's create a PR so that I can put some direct comments on particular lines?

Some thoughts during my quick inspection today: * Pagure support is still kept in (and it involves some overhead code to differentiate between the two systems), do you have a preference? I'd either drop Pagure immediately, or (in case we want to play defensive), after a few weeks once Forgejo integration is verified to work in production. The latter is of course safer, but given that we'll likely push this to production after F44 cycle, we won't have many tickets to validate this with for the next 3-4 months, so I'm not sure if that helps us at all 🙂 I currently lean a bit towards just nuking Pagure immediately, which will make the new code simpler (including CLI commands). What about you? Also, let's create a PR so that I can put some direct comments on particular lines?
Owner

I was just playing it safe and keeping backwards compatibility with Pagure just in case. If that is not really required we can remove Pagure specific code for sure.
Let's create PR after possible Actions upgrade is evaluated (#298) since that would change required code considerably.

I was just playing it safe and keeping backwards compatibility with Pagure just in case. If that is not really required we can remove Pagure specific code for sure. Let's create PR after possible Actions upgrade is evaluated (https://forge.fedoraproject.org/quality/blockerbugs/issues/298) since that would change required code considerably.
Author
Owner

Let's create PR after possible Actions upgrade is evaluated (#298) since that would change required code considerably.

I wouldn't delay this with that ticket. It might take a long time to evaluate it and it might be a complete rewrite of the process. We can be much quicker if we finalize this and push to production, because you have it almost all written already, as it seems. We can consider #298 as a possible future improvement.

> Let's create PR after possible Actions upgrade is evaluated (#298) since that would change required code considerably. I wouldn't delay this with that ticket. It might take a long time to evaluate it and it might be a complete rewrite of the process. We can be much quicker if we finalize this and push to production, because you have it almost all written already, as it seems. We can consider #298 as a possible future improvement.
Author
Owner

In our today's team call, we agreed that it makes no sense to carry forward Pagure integration as well. Jaroslav, could you please adjust the PR to drop Pagure support? It should make some code paths simpler. Thanks!

In our today's team call, we agreed that it makes no sense to carry forward Pagure integration as well. Jaroslav, could you please adjust the PR to drop Pagure support? It should make some code paths simpler. Thanks!
Owner

Ack, will do.

Ack, will do.
kparal modified the project from Sprint 5 to Sprint 6 2026-03-24 15:13:54 +00:00
adamwill modified the project from Sprint 6 to Sprint 7 2026-04-07 15:10:53 +00:00
Author
Owner

TODO: In Forgejo 15.0 access tokens can now be limited to just specific repositories. It might be a good idea to limit the blockerbot access just to the blocker-review repo. (Once Forgejo 15.0 is deployed in Fedora).

TODO: In Forgejo 15.0 [access tokens can now be limited to just specific repositories](https://forgejo.org/2026-04-release-v15-0/#repository-specific-access-tokens). It might be a good idea to limit the blockerbot access just to the blocker-review repo. (Once Forgejo 15.0 is deployed in Fedora).
adamwill modified the project from Sprint 7 to Sprint 8 2026-04-21 15:13:18 +00:00
Owner

the staging deployment should be working and ready for testing now.

the staging deployment should be working and ready for testing now.
Author
Owner

We've hit a snag with checking admin vote commands privileges (AGREED, REVOTE). Previously, we checked membership in a particular Pagure group, and those members could issue admin vote commands. We tried to replicate the same approach in Forge, where we have two Quality teams - Owners (admin access to the organization and all our repos), and Members (admin access to all our repos). The idea is that Quality Members can issue admin vote commands (in the blocker-review repo).

At the same time, I'm trying to have the bot (blockerbot) just minimal access itself (both interactive and API), so that an attacker can't misuse it much if it gains access to it.

But it's not easy to have both at the same time. Current constraints:

  • Blockerbot can only verify org team membership (e.g. Quality Members), if it's a member of that org itself.
  • Blockerbot can verify general org membership (but not down to a particular team), even if it isn't a member of that org itself.
  • If we make blockerbot a member of Quality Members, it will have admin access to all our repos (interactively). For an API token, we can restrict it just to rw access to all repos. If the account or API token is hijacked, it can be used to do unpleasant stuff.
    • For API token approach, there's another option - have two different tokens. The first token would grant read access to Quality org (we need that to check team membership), the second token would grant rw access to the blocker-review repo selectively (not all our repos). The downside is that BBA would need to handle two tokens instead of one, and that this only restricts API token misuse - a full account hijacking still allows everything interactively (including creating new token with full privileges, of course).

Our current options (that we're aware of):

A) Give blockerbot Quality Members membership, use one token with rw:issues and read:org. This is the easiest, but least safe. No more code changes needed, but both the account and the API token can be misused widely.

B) Give blockerbot Quality Members membership, use two tokens, one with read:org, and the second with rw:issues just for the blocker-review repo. This is a bit safer, the API tokens can't be misused (outside of the blocker-review repo). The full account hijack still allows everything.

C) Create a specific new Quality team just for blockerbot, with very restricted access. We just need blockerbot to be a part of Quality, it doesn't need to have write permissions anywhere (just blocker-review repo is enough). We can have a special-tailored team just for it. This would be ideal, but it's not self-service. We would need to ask Fedora Infra to set up two new FAS groups (one is the primary source of truth, the second just clones the members and is used for Forge - I have no idea why they do it this way, but they do), create the Forge group, and set up the connections. Because we might have some needs for future adjustments, I feel a bit bad for bothering the Infra team every time.

D) Don't give blockerbot Quality membership, check global Quality membership for admin vote command privileges. If we don't require to check a particular Quality team, but we rely just on arbitrary Quality membership (i.e. any team, as long as it's inside Quality), we don't need blockerbot to be a part of Quality, all the troubles from above are gone. The downside is, obviously, that anyone inside Quality can issue admin vote commands. Currently that's not a problem at all, because Members is used exactly for that, and there's no other team (except Owners). But if we wanted, in the future, create some other team, with limited access, for some specific group of people, all of those would be able to issue admin vote commands as well. Please note that I'm currently not aware of any specific use case we might want to achieve by having another team (if we want to give elevated access to a specific repo to a specific person, that can be done inside that very repo through Collaborators feature, it doesn't need handling through org teams). So perhaps this is just a theoretical concern. But if we decide to go this way, it will require code changes if we ever encounter this situation and want to create another team.

E) Check FAS directly. Instead of fighting Forge and its tokens and memberships, we can query a FAS group directly to see its members. The Forge teams are directly mapped from FAS anyway. But this would require a new code (with some form of authentication, because FAS account/group info can't be accessed publicly), and we don't know how difficult this can be.

F) Keep the admin list defined in Blockerbugs directly. Similarly to Testdays, in the admin interface we could define a list of FAS users eligible for admin vote commands. It's quite a bit of work, and increases maintenance, though.

There might be more approaches that didn't occur to us. Do you see any other?

@adamwill @jgroman Do you have any preference for these options?

For me, the technically best is C), but it complicates the setup and creates more work for Infra. I'd love if this (and any future changes) were in our hands instead. So I might suggest D), because it feels good enough for now, and should be a very easy change. E) might be also easy, we just don't know.

We've hit a snag with checking admin vote commands privileges (AGREED, REVOTE). Previously, we checked membership in a particular Pagure group, and those members could issue admin vote commands. We tried to replicate the same approach in Forge, where we have two Quality teams - Owners (admin access to the organization and all our repos), and Members (admin access to all our repos). The idea is that Quality Members can issue admin vote commands (in the blocker-review repo). At the same time, I'm trying to have the bot (blockerbot) just minimal access itself (both interactive and API), so that an attacker can't misuse it much if it gains access to it. But it's not easy to have both at the same time. Current constraints: * Blockerbot can only verify **org team membership** (e.g. Quality Members), if it's a member of that org itself. * Blockerbot can verify **general org membership** (but not down to a particular team), even if it isn't a member of that org itself. * If we make blockerbot a member of Quality Members, it will have **admin access to all our repos** (interactively). For an API token, we can restrict it just to rw access to all repos. If the account or API token is hijacked, it can be used to do unpleasant stuff. * For API token approach, there's another option - have two different tokens. The first token would grant read access to Quality org (we need that to check team membership), the second token would grant rw access to the blocker-review repo selectively (not all our repos). The downside is that BBA would need to handle two tokens instead of one, and that this only restricts API token misuse - a full account hijacking still allows everything interactively (including creating new token with full privileges, of course). Our current options (that we're aware of): **A) Give blockerbot Quality Members membership, use one token with rw:issues and read:org.** This is the easiest, but least safe. No more code changes needed, but both the account and the API token can be misused widely. **B) Give blockerbot Quality Members membership, use two tokens, one with read:org, and the second with rw:issues just for the blocker-review repo.** This is a bit safer, the API tokens can't be misused (outside of the blocker-review repo). The full account hijack still allows everything. **C) Create a specific new Quality team just for blockerbot, with very restricted access.** We just need blockerbot to be a part of Quality, it doesn't need to have write permissions anywhere (just blocker-review repo is enough). We can have a special-tailored team just for it. This would be ideal, but it's not self-service. We would need to ask Fedora Infra to set up two new FAS groups (one is the primary source of truth, the second just clones the members and is used for Forge - I have no idea why they do it this way, but they do), create the Forge group, and set up the connections. Because we might have some needs for future adjustments, I feel a bit bad for bothering the Infra team every time. **D) Don't give blockerbot Quality membership, check global Quality membership for admin vote command privileges.** If we don't require to check a particular Quality _team_, but we rely just on arbitrary Quality membership (i.e. any team, as long as it's inside Quality), we don't need blockerbot to be a part of Quality, all the troubles from above are gone. The downside is, obviously, that anyone inside Quality can issue admin vote commands. Currently that's not a problem at all, because Members is used exactly for that, and there's no other team (except Owners). But if we wanted, in the future, create some other team, with limited access, for some specific group of people, all of those would be able to issue admin vote commands as well. Please note that I'm currently not aware of any specific use case we might want to achieve by having another team (if we want to give elevated access to a specific repo to a specific person, that can be done inside that very repo through Collaborators feature, it doesn't need handling through org teams). So perhaps this is just a theoretical concern. But if we decide to go this way, it will require code changes if we ever encounter this situation and want to create another team. **E) Check FAS directly.** Instead of fighting Forge and its tokens and memberships, we can query a FAS group directly to see its members. The Forge teams are directly mapped from FAS anyway. But this would require a new code (with some form of authentication, because FAS account/group info can't be accessed publicly), and we don't know how difficult this can be. **F) Keep the admin list defined in Blockerbugs directly.** Similarly to Testdays, in the admin interface we could define a list of FAS users eligible for admin vote commands. It's quite a bit of work, and increases maintenance, though. There might be more approaches that didn't occur to us. Do you see any other? @adamwill @jgroman Do you have any preference for these options? For me, the technically best is C), but it complicates the setup and creates more work for Infra. I'd love if this (and any future changes) were in our hands instead. So I might suggest D), because it feels good enough for now, and should be a very easy change. E) might be also easy, we just don't know.
Owner

In ideal conditions I would also prefer C (dedicated team) but as not to burden infra team with extra work I'd say let's do D (just query Quality org public members).
BTW: I suspect E (querying FAS) would require Kerberos auth.

In ideal conditions I would also prefer C (dedicated team) but as not to burden infra team with extra work I'd say let's do D (just query Quality org public members). BTW: I suspect E (querying FAS) would require Kerberos auth.
Owner

I'd also prefer D or E. We've had extremely loose permissions for relatively unimportant things before - like the blockerbugs admin interface having no auth on it at all, and anonymous editing in the wiki "Test Results" space - and it has not been a problem. Any abuse of membership would be pretty visible, too (anyone subscribed to the ticket would see the inappropriate action).

I'd also prefer D or E. We've had extremely loose permissions for relatively unimportant things before - like the blockerbugs admin interface having no auth on it at all, and anonymous editing in the wiki "Test Results" space - and it has not been a problem. Any abuse of membership would be pretty visible, too (anyone subscribed to the ticket would see the inappropriate action).
Owner

I guess we could also note that the admin commands in blockerbugs don't currently really mean much...and the 'real' status change action - which is to put a string in the Whiteboard field in Bugzilla - is also loosely-constrained. There are lots and lots of people who can set the Whiteboard field on any bug in BZ, and of course the user who reported any blocker can also set it. So far I can't think of a single case of it being done maliciously.

I guess we could also note that the admin commands in blockerbugs don't currently really *mean* much...and the 'real' status change action - which is to put a string in the Whiteboard field in Bugzilla - is also loosely-constrained. There are lots and lots of people who can set the Whiteboard field on any bug in BZ, and of course the user who reported any blocker can also set it. So far I can't think of a single case of it being done maliciously.
Author
Owner

We went with D) and it seems to be working.

We went with **D)** and it seems to be working.
Author
Owner

@adamwill @psklenar @lruzicka
Folks, the (hopefully) final version is now deployed in staging. Can you please test it a bit before we move it to production?

Blockerbugs: https://qa.stg.fedoraproject.org/blockerbugs/
Bugs are retrieved from: https://bugzilla.stage.redhat.com/
Review tickets: https://forge.stg.fedoraproject.org/quality/blocker-review/issues

The latest version in stg bugzilla is 40, so use F40BetaBlocker and similar keywords when proposing any bugs. Don't worry, all emails are disabled in staging, you can do completely what you want there. If you use the Propose button in BBA, it gets added to the UI immediately, you don't need to wait for the 30 minute refresh.

Please try to propose some bugs and vote on them in the tickets, use AGREED and REVOTE etc.

If no one can see any issue, we can promote this to production. Thanks!

@adamwill Any opinion whether we should migrate all the old 2000 voting tickets from https://pagure.io/fedora-qa/blocker-review , when setting up our production Forge blocker-review repo?

@adamwill @psklenar @lruzicka Folks, the (hopefully) final version is now deployed in staging. Can you please test it a bit before we move it to production? Blockerbugs: https://qa.stg.fedoraproject.org/blockerbugs/ Bugs are retrieved from: https://bugzilla.stage.redhat.com/ Review tickets: https://forge.stg.fedoraproject.org/quality/blocker-review/issues The latest version in stg bugzilla is 40, so use **F40**BetaBlocker and similar keywords when proposing any bugs. Don't worry, all emails are disabled in staging, you can do completely what you want there. If you use the Propose button in BBA, it gets added to the UI immediately, you don't need to wait for the 30 minute refresh. Please try to propose some bugs and vote on them in the tickets, use AGREED and REVOTE etc. If no one can see any issue, we can promote this to production. Thanks! @adamwill Any opinion whether we should migrate all the old 2000 voting tickets from https://pagure.io/fedora-qa/blocker-review , when setting up our production Forge blocker-review repo?
Owner

Thanks!

@adamwill Any opinion whether we should migrate all the old 2000 voting tickets from https://pagure.io/fedora-qa/blocker-review , when setting up our production Forge blocker-review repo?

mmm...I think we probably should give it a shot. There's valuable historic info in there. It's good to have a record of why decisions were made. If it turns out to be a real pain we could maybe live without it, though.

Thanks! > @adamwill Any opinion whether we should migrate all the old 2000 voting tickets from https://pagure.io/fedora-qa/blocker-review , when setting up our production Forge blocker-review repo? mmm...I think we probably should give it a shot. There's valuable historic info in there. It's good to have a record of why decisions were made. If it turns out to be a real pain we could maybe live without it, though.
Author
Owner

Alright, I'll try. (I added an implementation checklist to the first post, so that everyone can keep track of the progress).

Alright, I'll try. (I added an implementation checklist to the first post, so that everyone can keep track of the progress).
Author
Owner
**BBA version 1.15 tagged and deployed to production.** Publicly announced here: * https://lists.fedoraproject.org/archives/list/test-announce@lists.fedoraproject.org/thread/I6EY57ECOCT5SQ2S2MYFCVU4MC3ZW4N7/ * https://discussion.fedoraproject.org/t/blocker-review-tickets-migrated-to-forge/190381
Author
Owner

Infra ansible cleanup ticket:
infra/ansible#3324

Infra ansible cleanup ticket: https://forge.fedoraproject.org/infra/ansible/pulls/3324
Author
Owner

I believe we can finally consider this done, yay 🎉

This has been through many many milestone, might deserve even more story points. Changing and closing.

I believe we can finally consider this done, yay 🎉️ This has been through many many milestone, might deserve even more story points. Changing and closing.
Owner

I guess we can open a new ticket for testing in staging and then deploying to prod?

I guess we can open a new ticket for testing in staging and then deploying to prod?
Author
Owner

It's already deployed to production 😆️ See the checklist in the first comment.

It's already deployed to production 😆️ See the checklist in the first comment.
Sign in to join this conversation.
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
quality/blockerbugs#296
No description provided.