releng/tooling

Fork 13

Add ccp.py - compose-critical package list generation script #13039

Merged

kevin merged 1 commit from adamwill/releng-tooling:add-ccp into main

2026-07-08 00:17:22 +00:00

adamwill commented

2026-06-30 18:57:33 +00:00

Member

This adds ccp.py - a script for generating the "compose-critical
package list" for a given compose. It's a bit of a WIP but it
does already work and give you a fairly useful list (as a flat
text file for now).

Signed-off-by: Adam Williamson awilliam@redhat.com

This adds ccp.py - a script for generating the "compose-critical package list" for a given compose. It's a bit of a WIP but it does already work and give you a fairly useful list (as a flat text file for now). Signed-off-by: Adam Williamson <awilliam@redhat.com>

adamwill added 1 commit

2026-06-30 18:57:34 +00:00

Add ccp.py - compose-critical package list generation script b30d42a9bc

This adds ccp.py - a script for generating the "compose-critical
package list" for a given compose. It's a bit of a WIP but it
does already work and give you a fairly useful list (as a flat
text file for now).

Signed-off-by: Adam Williamson <awilliam@redhat.com>

adamwill requested reviews from members, Owners

2026-06-30 18:57:34 +00:00

adamwill commented

2026-06-30 19:00:37 +00:00

Author

Member

I have a few ideas for things to do with this:

Use it to extend or replace the use of critpath groups for scheduling openQA tests - we should run openQA tests on all 'compose-critical packages' for relevant images (so, server tests for server images, kde tests for kde images etc.)
Use it to identify particularly 'important' rmdepcheck failures for Rawhide and Branched - breaking deps of a compose-critical package is a higher-priority issue
Tie it into the orphan retirement process somehow so we treat "compose-critical package is getting retired" as a priority issue and work hard to find a maintainer or make it no-longer-critical (CC @gotmax23 )

I have a few ideas for things to do with this: * Use it to extend or replace the use of critpath groups for scheduling openQA tests - we should run openQA tests on all 'compose-critical packages' for relevant images (so, server tests for server images, kde tests for kde images etc.) * Use it to identify particularly 'important' [rmdepcheck](https://forge.fedoraproject.org/quality/rmdepcheck) failures for Rawhide and Branched - breaking deps of a compose-critical package is a higher-priority issue * Tie it into the orphan retirement process somehow so we treat "compose-critical package is getting retired" as a priority issue and work hard to find a maintainer or make it no-longer-critical (CC @gotmax23 )

adamwill referenced this pull request from releng/tickets

2026-06-30 19:04:35 +00:00

Track 'compose critical packages' similar to critical-path #12943

adamwill referenced this pull request from quality/paperwork

2026-06-30 19:05:36 +00:00

Write compose-critical package script #68

adamwill referenced this pull request from quality/tickets

2026-06-30 19:07:54 +00:00

Enhance compose-critical package script output #911

kevin commented

2026-07-04 16:57:36 +00:00

Owner

Seems to do what it says on the tin. ;)
Might be nice to handle --help and explain what a 'compose top level' is... or just add an example?
I wonder if it could/should just default to latest rawhide?

Might also dump the file in the cwd instead of var tmp? and note that it wrote the file out so you know where it is.

All minor stuff, happy to merge this and improve or if you want to polish it some thats fine too.

Seems to do what it says on the tin. ;) Might be nice to handle --help and explain what a 'compose top level' is... or just add an example? I wonder if it could/should just default to latest rawhide? Might also dump the file in the cwd instead of var tmp? and note that it wrote the file out so you know where it is. All minor stuff, happy to merge this and improve or if you want to polish it some thats fine too.

adamwill force-pushed add-ccp from b30d42a9bc to 8d1eedd35d

2026-07-06 23:41:46 +00:00

Compare

adamwill commented

2026-07-06 23:51:35 +00:00

Author

Member

tweaked it to address those notes, and it also outputs a json file organized by variant now.

adamwill commented

2026-07-06 23:53:47 +00:00

Author

Member

I keep changing my mind on whether we want to care about the distinction between packages that are in the buildroot and packages that are in the produced image or not. Currently we care about them most of the way through the script, but don't actually put that information in either of the output files. WDYT? Should I just simplify it and not care about the distinction, change the output to include the info, or what?

kevin commented

2026-07-07 17:38:35 +00:00

Owner

I think we should care. If we don't I think down the road we may be sorry we didn't.

Unless it's a gigantic hassle I suppose to log that...

I think we should care. If we don't I think down the road we may be sorry we didn't. Unless it's a gigantic hassle I suppose to log that...

adamwill commented

2026-07-07 18:05:10 +00:00

Author

Member

It's not at all a giant hassle to output it, if anything it just makes consuming the data slightly more annoying if you just want both types of package in one bucket. But it's no big deal, just a couple of extra lines. I'll tweak the script to output it in separate buckets.

It's not at all a giant hassle to output it, if anything it just makes *consuming* the data slightly more annoying if you just want both types of package in one bucket. But it's no big deal, just a couple of extra lines. I'll tweak the script to output it in separate buckets.

adamwill commented

2026-07-07 19:51:57 +00:00

Author

Member

Adjusted to drop the now-pretty-useless data container class and output by variant, arch and image/buildroot, e.g.:

{
  "Everything": {
    "aarch64": {
      "buildroot": [
        "R-srpm-macros",
        "add-determinism",
...

Adjusted to drop the now-pretty-useless data container class and output by variant, arch and image/buildroot, e.g.: ``` { "Everything": { "aarch64": { "buildroot": [ "R-srpm-macros", "add-determinism", ... ```

adamwill commented

2026-07-07 19:57:38 +00:00

Author

Member

oh, hmm, one thing I forgot to mention/discuss: this gives us binary packages. critpath.py can output either binary or source, and we use it in source mode.

For one of the purposes I'm intending to use this for (rmdepcheck gating on rawhide), binary is actually what we want, but for other purposes, we probably want source package names.

I probably need to add a mode that can discover the source package names...

oh, hmm, one thing I forgot to mention/discuss: this gives us *binary* packages. critpath.py can output either binary or source, and we use it in source mode. For one of the purposes I'm intending to use this for (rmdepcheck gating on rawhide), binary is actually what we want, but for other purposes, we probably want source package names. I probably need to add a mode that can discover the source package names...