quality/tickets

Fork 0

Adopt a compose differ and make it useful for comparing release candidates #905

New issue

Open

opened 2026-05-18 16:59:26 +00:00 by kparal · 5 comments

kparal commented

2026-05-18 16:59:26 +00:00

Owner

During Fedora 44 cycle, I used a modified fedora-compose-differ to inform about changes between RCs, like here.

Adam later noted that we have a similar tool producing outputs like this for every nightly compose. It's part of Releng's compose-utils.

It would be handy if we took one of those tools, tweaked it a little bit for our purposes, and used it for easy RC diff generation and announcement. In the first step, I'd like to have it ready for manual generation. Making it part of the RC announcement automatically would be be a follow-up task.

What I'd like to see:

Have the tool part of Fedora (Quality or not) repos
Make it accept two arbitrary URLs as input args for comparison
It would be handy if it could list all RCs (their compose ID and their label) for a given release. For example for F44 it would list Fedora-44-20260422.1 with label RC-1.7 (from metadata/composeinfo.json). (There were more dirs in there, but are already cleaned up). That would make it fast to determine which directories I want to compare.

During Fedora 44 cycle, I used a modified [fedora-compose-differ](https://github.com/jgroman/fedora-compose-differ) to inform about changes between RCs, like [here](https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/45JZRSF4ZCQG4GJ3XH7JIDHO2QMPLSWK/). Adam later noted that we have a similar tool producing outputs like [this](https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/4X6FIRAMPF4KA633RQ4VVMTJ7L3EDDI3/) for every nightly compose. It's part of [Releng's compose-utils](https://pagure.io/compose-utils/blob/master/f/compose_utils/changelog.py). It would be handy if we took one of those tools, tweaked it a little bit for our purposes, and used it for easy RC diff generation and announcement. In the first step, I'd like to have it ready for manual generation. Making it part of the RC announcement automatically would be be a follow-up task. What I'd like to see: 1. Have the tool part of Fedora (Quality or not) repos 2. Make it accept two arbitrary URLs as input args for comparison 3. It would be handy if it could list all RCs (their compose ID and their label) for a given release. For example for [F44](https://kojipkgs.fedoraproject.org/compose/44/) it would list `Fedora-44-20260422.1` with label `RC-1.7` (from `metadata/composeinfo.json`). (There were more dirs in there, but are already cleaned up). That would make it fast to determine which directories I want to compare.

kparal added the

labels

2026-05-18 16:59:26 +00:00

adamwill commented

2026-05-18 21:09:45 +00:00

Owner

The releng one is https://pagure.io/compose-utils/blob/master/f/compose_utils/changelog.py . It's packaged as compose-changelog in compose-utils and called by https://forge.fedoraproject.org/releng/pungi-fedora/src/branch/main/nightly.sh for rawhide composes.

adamwill commented

2026-05-18 21:18:27 +00:00

Owner

You can find RCs from the metadata-archive 'series' file. e.g. https://kojipkgs.fedoraproject.org/compose/metadata-archive/series/Fedora-44 for F44. The format is 'compose_id [label]'; only candidate composes (for Beta and Final) have labels. Labels for beta candidates start with Beta, labels for final candidates start with RC.

You can find RCs from the metadata-archive 'series' file. e.g. https://kojipkgs.fedoraproject.org/compose/metadata-archive/series/Fedora-44 for F44. The format is 'compose_id [label]'; only candidate composes (for Beta and Final) have labels. Labels for beta candidates start with `Beta`, labels for final candidates start with `RC`.

adamwill commented

2026-05-19 01:15:40 +00:00

Owner

we could actually probably enhance the release candidate build script to generate a diff from the previous RC, using the metadata-archive.

kparal commented

2026-05-19 07:25:44 +00:00

Author

Owner

The releng one is https://pagure.io/compose-utils/blob/master/f/compose_utils/changelog.py

Thanks, I updated the original post.

You can find RCs from the metadata-archive 'series' file. e.g. https://kojipkgs.fedoraproject.org/compose/metadata-archive/series/Fedora-44 for F44.

Oooh, that is nice. My only concern is whether that file is updated immediately after a compose is done. Looking at the Rawhide dir and its timestamps, it seems to have 30 minutes delay, which is not that bad, I guess.

we could actually probably enhance the release candidate build script to generate a diff from the previous RC

Yes, that would be nice. I'd like to include a diff to previous RC automatically, but that feels like the second step. The first step is to allow manual generation by feeding it two URLs (or compose IDs) as input. I'm fine with doing it manually in the beginning, as I've done for F44, I believe it was very useful. I also need to be able to compare two arbitrary composes, not just two directly sequential ones, because that's often useful (example).

> The releng one is https://pagure.io/compose-utils/blob/master/f/compose_utils/changelog.py Thanks, I updated the original post. > You can find RCs from the metadata-archive 'series' file. e.g. https://kojipkgs.fedoraproject.org/compose/metadata-archive/series/Fedora-44 for F44. Oooh, that is nice. My only concern is whether that file is updated immediately after a compose is done. Looking at the Rawhide dir and its timestamps, it seems to have 30 minutes delay, which is not that bad, I guess. > we could actually probably enhance the release candidate build script to generate a diff from the previous RC Yes, that would be nice. I'd like to include a diff to previous RC automatically, but that feels like the second step. The first step is to allow manual generation by feeding it two URLs (or compose IDs) as input. I'm fine with doing it manually in the beginning, as I've done for F44, I believe it was very useful. I also need to be able to compare two arbitrary composes, not just two directly sequential ones, because that's often useful ([example](https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/45JZRSF4ZCQG4GJ3XH7JIDHO2QMPLSWK/)).

adamwill commented

2026-05-19 16:34:46 +00:00

Owner

Oooh, that is nice. My only concern is whether that file is updated immediately after a compose is done. Looking at the Rawhide dir and its timestamps, it seems to have 30 minutes delay, which is not that bad, I guess.

It's done by the compose scripts themselves, e.g. here for Rawhide. I'm not sure why you're seeing a thirty minute delay.

So, the file I linked before is the inner implementation - the outer CLI layer is compose-changelog. I think in theory my suggestion here would be to enhance that to add any features you want it to have, and then use it. It should be possible to have the script that builds candidate composes find the previous candidate (if any) from the series metadata and generate a changelog automatically, like the Rawhide script does using the 'latest' symlink.

One feature I guess that might be missing is the ability to run on remote composes; the current script assumes filesystem access. It's backed by productmd.compose which itself assumes local filesystem access, so I guess we'd want to enhance compose-changelog to be able to download the metadata files from a remote compose to a local temporary directory before feeding them to productmd.

> Oooh, that is nice. My only concern is whether that file is updated immediately after a compose is done. Looking at the Rawhide dir and its timestamps, it seems to have 30 minutes delay, which is not that bad, I guess. It's done by the compose scripts themselves, e.g. [here](https://forge.fedoraproject.org/releng/pungi-fedora/src/branch/main/release-candidate.sh#L209) for Rawhide. I'm not sure why you're seeing a thirty minute delay. So, the file I linked before is the inner implementation - the outer CLI layer is [compose-changelog](https://pagure.io/compose-utils/blob/master/f/bin/compose-changelog). I think in theory my suggestion here would be to enhance that to add any features you want it to have, and then use it. It should be possible to have [the script that builds candidate composes](https://forge.fedoraproject.org/releng/pungi-fedora/src/branch/main/release-candidate.sh) find the previous candidate (if any) from the series metadata and generate a changelog automatically, like the [Rawhide script does](https://forge.fedoraproject.org/releng/pungi-fedora/src/branch/main/nightly.sh#L87) using the 'latest' symlink. One feature I guess that might be missing is the ability to run on remote composes; the current script assumes filesystem access. It's backed by [productmd.compose](https://github.com/release-engineering/productmd/blob/master/productmd/compose.py) which itself assumes local filesystem access, so I guess we'd want to enhance `compose-changelog` to be able to download the metadata files from a remote compose to a local temporary directory before feeding them to productmd.

No labels

Rows
Columns