Skip mass rebuilds when dist-git is ahead of the tagged build #13051

jnsamyak · 2026-08-10T13:57:39Z

jnsamyak commented

2026-08-10 13:57:39 +00:00

Avoid rebuilding packages whose rawhide HEAD has non-releng commits
beyond the latest build in desttag, so failed-gating content is not
bumped into the rebuild tag. Log skips to skipped_distgit_mismatch.txt.

Fixes: releng/tickets#13428

Signed-off-by: Samyak Jain samyak.jn11@gmail.com

Avoid rebuilding packages whose rawhide HEAD has non-releng commits beyond the latest build in desttag, so failed-gating content is not bumped into the rebuild tag. Log skips to skipped_distgit_mismatch.txt. Fixes: https://forge.fedoraproject.org/releng/tickets/issues/13428 Signed-off-by: Samyak Jain <samyak.jn11@gmail.com>

jnsamyak added 1 commit

2026-08-10 13:57:39 +00:00

Skip mass rebuilds when dist-git is ahead of the tagged build c93d4ef0a1

Avoid rebuilding packages whose rawhide HEAD has non-releng commits
beyond the latest build in desttag, so failed-gating content is not
bumped into the rebuild tag. Log skips to skipped_distgit_mismatch.txt.

Fixes: releng/tickets#13428

Signed-off-by: Samyak Jain <samyak.jn11@gmail.com>

jnsamyak requested reviews from Owners, members

2026-08-10 13:57:39 +00:00

jnsamyak referenced this pull request from releng/tickets

2026-08-10 13:58:02 +00:00

Mass rebuild scripts should check Bodhi and not bump/build anything that failed gating #13428

adamwill reviewed

2026-08-20 17:16:05 +00:00

adamwill left a comment

Aside from a couple of notes this looks good to me, though it's a bit hard to reason about. Might be a good idea to throw an AI review bot at it?

release-process/mass-rebuilds/mass_rebuild.py Outdated

					
				@ -40,0 +67,4 @@

				    if not tagged_commit:

				        return True

				    try:

				        head = subprocess.check_output(

this file already has runme and runmeoutput wrappers for subprocess; perhaps those should be used for at least some of these calls?

this file already has `runme` and `runmeoutput` wrappers for subprocess; perhaps those should be used for at least some of these calls?

With the new changes I switched the rev-parse/ log prrobes to runmeoutput with retrie to 1. Left merge-base --is-ancestor as a direct subprocess.call exit status 1 means “not an ancestor” (a normal skip), not a failed command.

jnsamyak marked this conversation as resolved

release-process/mass-rebuilds/mass_rebuild.py Outdated

					
				@ -40,0 +74,4 @@

				            ['git', 'rev-parse', '--verify', tagged_commit + '^{commit}'],

				            cwd=pkgdir, text=True, stderr=subprocess.DEVNULL).strip()

				    except (subprocess.CalledProcessError, FileNotFoundError, OSError):

				        return False

I think it would be good for all the False returns to log / output, so we know why we're refusing to rebuild this particular thing. That would also help catch if one of the checks turns out to be not really correct.

I think it would be good for all the `False` returns to log / output, so we know *why* we're refusing to rebuild this particular thing. That would also help catch if one of the checks turns out to be not really correct.

Ok made this change each False path now prints a specific skip reason before returning.

jnsamyak marked this conversation as resolved

jnsamyak commented

2026-08-24 06:52:36 +00:00

@adamwill need to add ai-review bot here for that, I think it is not configured for this tooling repository for now

jnsamyak added 1 commit

2026-08-24 13:09:32 +00:00

use runmeoutput for git probes and log skip reasons 16b375392b

Signed-off-by: Samyak Jain <samyak.jn11@gmail.com>

adamwill commented

2026-08-24 17:53:03 +00:00

you can always invoke a review manually somehow - ai-code-review has ways to do it from the command line, and you can also just ask your favorite clanker (claude code, opencode, whatever) to do it interactively.

jnsamyak force-pushed ft-mass-tag-gating from 16b375392b to 938062af36

Orphaned CI / Run Tox Checks for Orphaned tools (pull_request) Failing after 1m19s

Details

2026-08-25 12:40:24 +00:00

Compare

jnsamyak force-pushed ft-mass-tag-gating from 938062af36

Orphaned CI / Run Tox Checks for Orphaned tools (pull_request) Failing after 1m19s

Details

to b9f175feef

Orphaned CI / Run Tox Checks for Orphaned tools (pull_request) Failing after 1m23s

Details

2026-09-07 06:19:46 +00:00

Compare

jnsamyak commented

2026-09-07 06:21:25 +00:00

Did a self-review pass before merging. Caught a few things and fixed them in the follow-up commit:

Race in skip report writing was checking if the file exists then opening it separately.
Silent Koji fallback ewhen listTagged doesn't return a source URL, we fall back to getBuild. If that also fails (network hiccup, Koji maintenance), the exception was eaten silently. Now it logs a warning to stderr. During a 20k-package run, you really want to see why packages are getting skipped.
Cosmetic NVR guard added
Two new test casesadded for a releng bumpspec stacked on top of a maintainer commit to prove the whole chain is checked, not just the top commit, and divergent history (force-push scenario) to prove merge-base --is-ancestor catches it. Both were working correctly already, just untested.

Did a self-review pass before merging. Caught a few things and fixed them in the follow-up commit: - Race in skip report writing was checking if the file exists then opening it separately. - Silent Koji fallback ewhen listTagged doesn't return a source URL, we fall back to getBuild. If that also fails (network hiccup, Koji maintenance), the exception was eaten silently. Now it logs a warning to stderr. During a 20k-package run, you really want to see why packages are getting skipped. - Cosmetic NVR guard added - Two new test casesadded for a releng bumpspec stacked on top of a maintainer commit to prove the whole chain is checked, not just the top commit, and divergent history (force-push scenario) to prove merge-base --is-ancestor catches it. Both were working correctly already, just untested.

jnsamyak merged commit d13191f8b3 into main

2026-09-07 06:21:36 +00:00

jnsamyak deleted branch ft-mass-tag-gating

2026-09-07 06:21:37 +00:00

Sign in to join this conversation.

No reviewers

releng/members

releng/Owners