Support running tests for MRs in dist-git

duffy commented

2023-07-20 18:44:34 +00:00

It would be nice to have Fedora OpenQA support for running tests on merge requests to dist-git.

adamwill commented

2023-07-20 23:57:36 +00:00

Owner

This is a bit more feasible than the packit thing, I think. dist-git MRs are more obviously good things to test for Fedora, and dist-git is 'closer' to the Fedora ecosystem, which should mean it's easier to handle the communication problems (I'm pretty sure we get fedmsgs from dist-git, and Fedora CI has already figured out how to get results shown in dist-git so we should be able to piggyback off that knowledge).

It's still a "nice-to-have" though. It's been on my 'that'd be cool' list for a while and it's good to have a ticket for it, but I can't promise to get to it any time soon. If anyone else wants to take a crack at it I'm certainly happy to help.

This is a bit more feasible than [the packit thing](https://pagure.io/fedora-qa/fedora_openqa/issue/103), I think. dist-git MRs are more obviously good things to test for Fedora, and dist-git is 'closer' to the Fedora ecosystem, which should mean it's easier to handle the communication problems (I'm pretty sure we get fedmsgs from dist-git, and Fedora CI has already figured out how to get results shown in dist-git so we should be able to piggyback off that knowledge). It's still a "nice-to-have" though. It's been on my 'that'd be cool' list for a while and it's good to have a ticket for it, but I can't promise to get to it any time *soon*. If anyone else wants to take a crack at it I'm certainly happy to help.

adamwill commented

2023-07-20 23:57:49 +00:00

Owner

Metadata Update from @adamwill:

Issue priority set to: Low
Issue tagged with: enhancement

**Metadata Update from @adamwill**: - Issue priority set to: Low - Issue tagged with: enhancement

adamwill added the

Priority

Low

label

2026-02-06 01:03:15 +00:00

adamwill commented

2026-06-18 09:24:03 +00:00

Owner

So someone asked for this during Flock, and with Flock energy, I kinda did it - you actually can do this now, kinda. If you comment /openqa test on a dist-git PR, tests on the scratch build that Packit-CI (thanks!) built for it should get scheduled on openQA staging. There are many caveats, though!

This has been tested like twice
There's a mechanism to try and handle when the scratch build isn't done yet, but that hasn't even been tested once
Results aren't reported back to the PR yet. That turned out to be much harder than actually running the tests. I just about finished writing it, but I wrote it entirely blind and have not had time to test it yet so it's likely broken. I wanted to get to deploying, testing and fixing it in the last few days, but got sidetracked (from this sidetrack...) by the openssl 4 update
There are probably awkward corner cases I didn't discover / think about yet

Still, progress!

So someone asked for this during Flock, and with Flock energy, I [kinda did it](https://forge.fedoraproject.org/quality/fedora_openqa/src/branch/schedule-distgit-prs) - you actually can do this now, kinda. If you comment `/openqa test` on a dist-git PR, tests on the scratch build that Packit-CI (thanks!) built for it should get scheduled on openQA staging. There are many caveats, though! 1. This has been tested like twice 2. There's a mechanism to try and handle when the scratch build isn't done yet, but that hasn't even been tested once 3. Results aren't reported back to the PR yet. That turned out to be much harder than actually running the tests. I just about finished writing it, but I wrote it entirely blind and have not had time to test it yet so it's likely broken. I wanted to get to deploying, testing and fixing it in the last few days, but got sidetracked (from this sidetrack...) by the openssl 4 update 4. There are probably awkward corner cases I didn't discover / think about yet Still, progress!

adamwill self-assigned this

2026-06-18 09:24:16 +00:00

adamwill added the

labels

2026-06-18 09:24:55 +00:00

adamwill commented

2026-06-18 09:26:25 +00:00

Owner

Oh, forgot to mention: this is implemented as an "on request" thing rather than a "test every PR" thing for a couple of reasons:

Resources (maybe we have enough to test every PR, but I didn't check yet)
Known failure cases: we know the tests will fail on cases where multiple PRs need to be tested against each other (e.g. soname bumps), and I hate setting up something that we know will definitely cause false failures. I'd only consider making this automated once we have some kinda mechanism for associating multiple related PRs in place, so we can test them together

Oh, forgot to mention: this is implemented as an "on request" thing rather than a "test every PR" thing for a couple of reasons: 1. Resources (maybe we have enough to test every PR, but I didn't check yet) 2. Known failure cases: we *know* the tests will fail on cases where multiple PRs need to be tested against each other (e.g. soname bumps), and I hate setting up something that we *know* will *definitely* cause false failures. I'd only consider making this automated once we have some kinda mechanism for associating multiple related PRs in place, so we can test them together

adamwill added this to the Sprint 11 project

2026-06-23 07:56:10 +00:00

adamwill commented

2026-06-24 11:12:28 +00:00

Owner

I've done more work on this but it's now a bit blocked by the issues with staging fedora-messaging. Once that's all cleared up I can get back to finishing this up.

I've done more work on this but it's now a bit blocked by the [issues with staging fedora-messaging](https://forge.fedoraproject.org/infra/tickets/issues/13422). Once that's all cleared up I can get back to finishing this up.

adamwill commented

2026-06-30 18:29:52 +00:00

Owner

This is now basically working, with status reporting included - see how this test PR has a 'success' status for 'Fedora openQA tests - scratch build 146822887'. But I still need to do some polishing and check that various scenarios work, and write some tests. Moving to sprint 12.

This is now basically working, with status reporting included - see how [this test PR](https://src.fedoraproject.org/rpms/fedfind/pull-request/2) has a 'success' status for 'Fedora openQA tests - scratch build 146822887'. But I still need to do some polishing and check that various scenarios work, and write some tests. Moving to sprint 12.

adamwill modified the project from Sprint 11 to Sprint 12

2026-06-30 18:30:01 +00:00

adamwill referenced this issue from a commit

2026-07-03 21:43:27 +00:00

consumer, scheduler: support testing dist-git PRs (#104)

adamwill referenced this issue

2026-07-03 21:43:34 +00:00

consumer, scheduler: support testing dist-git PRs (#104) #143

adamwill referenced this issue from a commit

2026-07-03 21:57:38 +00:00

consumer, scheduler: support testing dist-git PRs (#104)

adamwill referenced this issue from a commit