Packaging two Python scripts, one of which is a wrapper for the
other, is pretty awkward. But more importantly, we can make the
EL "wrapper" work better with a refactor, too. It's now a stand
alone script called `rdc-compose`. The original script is renamed
to `rdc-repos`.
The CLI scripts now share a lot of bits via `shared.py`. We put
handy run-it-locally wrappers for each CLI script at the top
level; you *can* install the package properly and you'll get
`rdc-repos` and `rdc-compose` executables but we'll likely rarely
do that.
`rdc-compose` will now run faster because it will reuse the same
metadata between variants (previously it got re-downloaded with
every execution of `rmdepcheck.py`) and it can now nicely collect
up all the repoclosure strings before using the same parse-and-
exit code as rmdepcheck, which solves a lot of awkward issues
around output.
We also add tests covering `rdc-compose`, as part of which it made
sense to factor out the Pungi config URL discovery (which I meant
to do anyway). We fix a bug that was exposed by writing the tests:
we have to check variants that depend on variants we have packages
for, even if we don't have packages for those variants.
This isn't great git hygiene, sorry. Should have made the refactor,
the addition of tests and the bug fix into separate commits. But
it's awkward and time-consuming to unpick now and I don't think
it's a good use of time.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
After much discussion with @yselkowitz, this should be the best
option for ELN testing, I hope.
ELN (like other ELs) has multiple variants with repositories.
There is a mapping in Pungi config of relationships between the
variants. Each variant is expected to be repoclosure-complete,
possibly in itself, possibly with reference to some or all of
the other variants. Each variant contains only a subset of the
packages in the buildroot.
We have been asked to make it so that, given an arbitrary set of
"packages under test" for a given EL, the packages it contains
from each variant should be tested separately against that
variant, so we fail if replacing the packages in the variant with
the matching packages from the set under test would cause new
repoclosure issues in that variant, or the packages under test
for that variant have installability issues when tested against
the variant and any others it "depends" on.
Packages from the set under test that are not in any variant
should be ignored, as repoclosure of and installability within
the buildroot is not currently desired to be tested.
This wrapper script takes as inputs a compose that we should test
against, and a filesystem path containing the packages to be
tested. The compose is usually expected to be the latest compose
of the EL under test. Using the metadata, we discover all variants
in the compose that have repositories (except Buildroot, which is
ignored). We then list out the binary packages by name in each
variant repository.
We then split the set of packages-under-test into several temporary
variant repositories: each package is placed in the repo for all
variants of which the identically-named package in the compose is a
part, if any.
We then get and parse a copy of the variant mapping from the
Pungi config. Then, for each "populated" variant (that is, a
variant for which the packages-under-test set contains at least
one package), we run rmdepcheck, with the compose repos for the
variant plus any variants it "depends" on as the baserepos, the
local temporary variant repository as the newrepo, and the local
temp repos for all "depended-on" variants as addrepos. All
rmdepcheck output is passed through.
We track the return codes of each rmdepcheck run and return the
sum of each unique return code.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
This adds a 'non-checked base repo' concept as part of addressing
the ELN scenario covered in #37. A 'non-checked base repo' is just
like a base repo, but it is not included in the `--check` arg when
generating the modified repoclosure. So it's always available to
the depsolver, and it is modified just like a regular base repo
(unlike a 'non-modified base repo'), but we don't actually check
for repoclosure issues in it.
This is intended to be used in EL scenarios where there are many
interdependent variant repositories. We may be testing a set of
packages that are each destined for a different set of the variant
repositories. We want to test each variant repository separately.
Some repositories 'depend' on others - e.g. packages in Extras can
depend on packages in BaseOS, Appstream and CRB. So when testing
the repository of packages destined for the Extras variant, we
should keep the BaseOS, Appstream and CRB variant repos "in scope"
for both the "base repositories" set and the "new package
repositories" set.
For the "new package repositories" set we can use `--addrepos`
to include the other variant repos but not check them (when doing
the installability phase). But we need the equivalent on the
"base repositories" side: we need to be able to specify repos
that will be modified, but not checked during the repoclosure
phase. This adds that feature.
Signed-off-by: Adam Williamson <awilliam@redhat.com>
