quality/rmdepcheck
5
0
Fork 1
Code Issues 10 Pull requests 1 Projects Releases Packages Activity Actions

Refactor into a module with two CLI scripts, add tests, fix bug
All checks were successful
CI via Tox / tox (pull_request) Successful in 1m47s
Details

Browse source 
Packaging two Python scripts, one of which is a wrapper for the
other, is pretty awkward. But more importantly, we can make the
EL "wrapper" work better with a refactor, too. It's now a stand
alone script called `rdc-compose`. The original script is renamed
to `rdc-repos`.

The CLI scripts now share a lot of bits via `shared.py`. We put
handy run-it-locally wrappers for each CLI script at the top
level; you *can* install the package properly and you'll get
`rdc-repos` and `rdc-compose` executables but we'll likely rarely
do that.

`rdc-compose` will now run faster because it will reuse the same
metadata between variants (previously it got re-downloaded with
every execution of `rmdepcheck.py`) and it can now nicely collect
up all the repoclosure strings before using the same parse-and-
exit code as rmdepcheck, which solves a lot of awkward issues
around output.

We also add tests covering `rdc-compose`, as part of which it made
sense to factor out the Pungi config URL discovery (which I meant
to do anyway). We fix a bug that was exposed by writing the tests:
we have to check variants that depend on variants we have packages
for, even if we don't have packages for those variants.

This isn't great git hygiene, sorry. Should have made the refactor,
the addition of tests and the bug fix into separate commits. But
it's awkward and time-consuming to unpick now and I don't think
it's a good use of time.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
This commit is contained in:
Adam Williamson 2026-05-08 17:32:11 -07:00
commit 4cfe070ed0
118 changed files with 1803 additions and 1197 deletions
Download patch file Download diff file Expand all files Collapse all files

2
.forgejo/workflows/ci.yml
View file

@ -10,7 +10,7 @@ jobs:
image: quay.io/fedora/fedora:latest
steps:
- name: Install required packages
run: dnf -y install nodejs tox git
run: dnf -y install nodejs tox git createrepo_c
- uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8
with:
fetch-depth: 0

75
README.md
View file

@ -1,13 +1,18 @@
# rmdepcheck
rmdepcheck is an RPM dependency check tool based on a repository metadata modification approach.
It works by comparing a checked repository to one or more base repositories. First, checks are run
on the base repositories as-is. Next, we re-run the checks, but with the checked repository
available and using dnf's `excludepkgs` option to hide from the base repositories all packages from
the same source RPM(s) as the package(s) in the checked repositories removed. The results of the
two runs are compared. New failures should indicate problems introduced by the checked repositories.
Also, some relevant checks are run on the checked repositories with reference to the modified base
repositories.
rmdepcheck is an RPM dependency check tool based on a repository metadata modification approach. It
exists as an installable library, but is not usually intended to be installed or imported; you are
meant to use one of the two CLI interfaces, which can be run directly with no need for installation.
## rdc-repos
rdc-repos is the original form of rmdepcheck. It works by comparing a checked repository to one
or more base repositories. First, checks are run on the base repositories as-is. Next, we re-run
the checks, but with the checked repository available and using dnf's `excludepkgs` option to hide
from the base repositories all packages from the same source RPM(s) as the package(s) in the
checked repositories removed. The results of the two runs are compared. New failures should
indicate problems introduced by the checked repositories. Also, some relevant checks are run on the
checked repositories with reference to the modified base repositories.
Optionally, other types of repository can be specified:
@ -24,8 +29,8 @@ HighAvailability variant can depend on packages in the BaseOS and Appstream vari
testing an update containing packages destined for all three variants, which we split into three
repositories. When testing the HA repository, we would want to have the "new package" BaseOS and
Appstream repositories "in scope" but not checked ("additional new repositories"), and the "base"
BaseOS and Appstream repositories "in scope" and modified, but not checked
("non-checked base repositories").
BaseOS and Appstream repositories "in scope" and modified, but not checked ("non-checked base
repositories").
Additional new repositories might also be used for multilib scenarios; it may be desirable to use
such an additional repository for packages for the multilib arch(es), if e.g. installability of
@ -35,28 +40,23 @@ An alternative mode allows simply testing the consequences of *removing* a list
entirely; in this mode, in the second step, we exclude all binary packages built from the specified
source packages. The installability check is skipped in this context.
## rdc-el-wrapper
## rdc-compose
rdc-el-wrapper is an opinionated wrapper for running rmdepcheck on Enterprise Linux. The EL case
is complicated because EL composes contain multiple variants with package repositories. These
contain a subset of all packages in the buildroot. Each variant is expected to be repoclosure-
complete with regard to itself and possibly some or all of the other variants; the mapping for
this is kept in Pungi configuration.
Because of this we cannot just test the packages-under-test against the buildroot, as we do for
Fedora. We must split the packages-under-test set into per-variant repositories, with each package
included in each repository it should be part of (we assume this mapping is not changed from the
compose we're testing against, and discover it from that). We then test each of the split repos
against the same variant repo from the compose, with other variant repos in scope on both sides,
as per the Pungi mapping. This is a lot of work but should provide the most accurate and useful
results.
rdc-compose is alternative interface which takes a distribution compose URL (this must be a Pungi
/ productmd-style compose) and a local filesystem path containing the packages under test as inputs.
It discovers the variants contained in the compose, splits the packages-under-test into per-variant
repositories, and then checks those against the target compose appropriately in much the same way
described above, testing appropriate sets of variants against each other according to the rules on
which variants can depend on packages from which other variants. This is a lot of work but should
provide the most accurate and useful results.
## Requirements
rmdepcheck has no run-time Python dependencies outside the standard library. Its only external
dependency is dnf. It checks for dnf, and will exit early with an error if it is not found.
rdc-repos has no run-time Python dependencies outside the standard library. rdc-compose requires
the requests and urllib3 libraries. Both scripts require dnf; rdc-compose also requires createrepo.
The scripts checks for their dependencies, and will exit early with an error if they are not found.
rmdepcheck is written primarily for Red Hat-family distributions, but should in theory be usable
anywhere these utilities can be installed (and forward slashes act as directory separators).
anywhere these dependencies can be installed (and forward slashes act as directory separators).
If you use a version of dnf older than 5.2.15.0, you may see false failures for 'rich' dependencies,
as older dnf versions did not handle these correctly. Use 5.4.0.0 or newer for the best handling
@ -74,7 +74,7 @@ directly from PyPI with pip and other tools.
Simple usage looks like this:
```
rmdepcheck https://a.base.repo.example/repo,file:///another/baserepo file:///the/testedrepo
rdc-repos https://a.base.repo.example/repo,file:///another/baserepo file:///the/testedrepo
```
The repositories are specified as a comma-separated list. Repositories should be specified as URLs,
@ -82,7 +82,7 @@ but for convenience, file:// can be omitted. Only file:// , http:// and https://
For the alternative 'removal' mode, usage looks like:
```
rmdepcheck --removes https://a.base.repo.example/repo,file:///another/baserepo sourcepkg1,sourcepkg2
rdc-repos --removes https://a.base.repo.example/repo,file:///another/baserepo sourcepkg1,sourcepkg2
```
This tests removing sourcepkg1 and sourcepkg2, and all binary packages built from them, from the
@ -91,20 +91,17 @@ base repositories.
If you want to test repositories containing packages of an arch that does not match the system on
which you are testing, pass `--arch <arch>`, where `<arch>` is the arch you wish to test.
For more complex usage, see `rmdepcheck --help`.
For more complex usage, see `rdc-repos --help`.
Note rmdepcheck is really only intended for use as a script, not as an importable library. If you
want to use it as a library go ahead, but this isn't a supported use case and bugs in it may not
be addressed.
rdc-el-wrapper usage looks like this:
rdc-compose usage looks like this:
```
rdc-el-wrapper https://kojipkgs.fedoraproject.org/compose/eln/latest-Fedora-eln/compose /the/testedrepo
rdc-compose https://kojipkgs.fedoraproject.org/compose/eln/latest-Fedora-eln/compose /the/testedrepo
```
An optional `--arch` arg can be passed as for rmdepcheck. Note that you **MUST NOT** include
file:// in the tested repo arg, and it **MUST** be a local filesystem path. rdc-el-wrapper does
not support remote tested repos.
file:// in the tested repo arg, and it **MUST** be a local filesystem path. rdc-compose does not
support remote tested repos.It does not actually need to be a repository; it just needs to be a
directory with RPMs in it.
### Testing build dependencies
@ -118,7 +115,7 @@ of all source packages will be unresolvable and the tool will run slowly and pro
Example:
```
rmdepcheck https://a.base.repo.example/binaryrepo,https://a.base.repo.example/sourcerepo file:///the/testedrepo
rdc-repos https://a.base.repo.example/binaryrepo,https://a.base.repo.example/sourcerepo file:///the/testedrepo
```
## License

1
install.requires
View file

@ -1 +1,2 @@
requests
urllib3

5
pyproject.toml
View file

@ -35,7 +35,8 @@ Issues = "https://codeberg.org/AdamWill/rmdepcheck/issues"
Changelog = "https://codeberg.org/AdamWill/rmdepcheck/src/branch/main/CHANGELOG.md"
[project.scripts]
rmdepcheck = "rmdepcheck:main"
rdc-repos = "rmdepcheck.repos:main"
rdc-compose = "rmdepcheck.compose:main"
[build-system]
requires = ["setuptools>=40.6.0", "setuptools-git", "wheel"]
@ -47,7 +48,7 @@ branch = true
source_pkgs = ["rmdepcheck"]
[tool.coverage.paths]
source = [".", ".tox/**/site-packages"]
source = ["src", ".tox/**/site-packages"]
[tool.coverage.report]
show_missing = true

14
rdc-compose Executable file
View file

@ -0,0 +1,14 @@
#!/usr/bin/python3
"""Convenience wrapper for executing rdc-compose from a git checkout."""
import os
import sys
# add src subdirectory directory to module import path
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), "src"))
from rmdepcheck.compose import main # pylint: disable=wrong-import-position
if __name__ == "__main__":
main()

256
rdc-el-wrapper.py
View file

@ -1,256 +0,0 @@
#!/usr/bin/python3
# Copyright Red Hat
#
# This file is part of rmdepcheck.
#
# rmdepcheck is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation, either version 3 of
# the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Author(s): Adam Williamson <awilliam@redhat.com>
"""Opinionated rmdepcheck wrapper that handles the awkward Enterprise
Linux case, where we have to test against multiple shipped variants
with complex lookaside relationships, all filtered from a shared build
root.
"""
# Standard libraries
import argparse
import ast
import os
import pathlib
import platform
import re
import shutil
import subprocess
import sys
import tempfile
from functools import partial
from urllib.parse import urlparse
from requests import Session
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry
VrDict = dict[str, str]
DNFTEMP = tempfile.TemporaryDirectory(prefix="rdcelndnf", dir="/var/tmp")
DNFARGS = ["dnf", "--setopt", f"cachedir={DNFTEMP.name}", "-q", "--disablerepo=*"]
HERE = os.path.abspath(os.path.dirname(__file__))
SESSION = Session()
SUBPCAPTURE = partial(subprocess.run, capture_output=True, text=True, check=False)
def split_put(put: pathlib.Path, repotemp: pathlib.Path, vrs: VrDict) -> set[str]:
"""Splits a directory containing the packages to be tested into
multiple repositories per variant. put ('packages under test')
is the filesystem path containing packages under test (it cannot
be an HTTP URL, in this opinionated wrapper). repotemp is a Path
to a temporary directory to store the split repos. vrs is a dict
with variant IDs as keys and full compose repo URLs as values.
For each package under test, we get its name, look for a package
of the same name in each of the variant repositories, and if
there is one, we put the package in a subdirectory of repotemp,
named for the variant. At the end, we run createrepo in each of
the subdirectories we created, and return their names.
"""
if not (put.exists() and put.is_dir()):
raise ValueError(f"split_put: invalid put value {put}")
put = put.absolute()
# generate per-variant package name lists
pkgs = {}
for variant, repo in vrs.items():
args = DNFARGS + [
"--repofrompath", f"{variant},{repo}", "repoquery", "--queryformat", "%{name}\n"
]
res = SUBPCAPTURE(args)
res.check_returncode()
pkgs[variant] = res.stdout.splitlines()
# track which variants we 'populate'
populated = set()
for _file in os.listdir(put):
if _file.endswith(".rpm"):
pkgname = _file.rsplit("-", 2)[0]
for (variant, pkgnames) in pkgs.items():
if pkgname in pkgnames:
populated.add(variant)
os.makedirs(repotemp / variant, exist_ok=True)
try:
# hardlink
os.link(put / _file, repotemp / variant / _file)
except OSError:
# copy
shutil.copy2(put / _file, repotemp / variant / _file)
for variant in populated:
subprocess.run(("createrepo", str(repotemp / variant)), capture_output=True, check=True)
return populated
def get_variant_repos(compose: str, arch: str) -> VrDict:
"""Discover variants with repositories in a compose, return a
dict with variant IDs as keys and repo URLs as values.
"""
resp = SESSION.get(f"{compose}/metadata/composeinfo.json")
resp.raise_for_status()
ci = resp.json()
variants = ci["payload"]["variants"]
ret = {}
for variant in variants:
if variants[variant]["id"].lower() == "buildroot":
continue
repopath = variants[variant]["paths"].get("repository", {}).get(arch, "")
if not repopath:
continue
ret[variants[variant]["id"]] = f"{compose}/{repopath}"
return ret
def get_val():
"""Get the variants_as_lookaside list which tells us which
variants pull in which other variants. FIXME: this is currently
ELN specific.
"""
resp = SESSION.get(
"https://forge.fedoraproject.org/releng/pungi-fedora/raw/branch/eln/fedora/override.conf"
)
resp.raise_for_status()
override = resp.text
# next three lines suggested by AI
match = re.search(r"variant_as_lookaside\s*=\s*(\[.*?\])", override, re.DOTALL)
if not match:
raise ValueError("Could not find variant_as_lookaside in configuration")
# eek
return ast.literal_eval(match.group(1))
def url_check(arg: str) -> str:
"""Check arg is an http or https URL."""
parsed = urlparse(arg)
if parsed.scheme in ("http", "https"):
return arg
if parsed.scheme:
raise ValueError(f"Unsupported URL scheme {parsed.scheme} in {arg}")
raise ValueError(f"No URL scheme in {arg}")
def check_arch(arg: str) -> str:
"""Use host arch if no value passed."""
if arg:
return arg
return platform.machine()
def check_utils() -> None:
"""Check required utilities are installed."""
missing = []
for prog in (("dnf", "--version"), ("createrepo", "--version")):
try:
subprocess.run(prog, stdout=subprocess.DEVNULL, check=True)
except FileNotFoundError:
missing.append(prog[0])
if missing:
sys.exit("Please install missing required utilities: " + " ".join(missing))
def parse_args() -> argparse.Namespace:
"""Parse arguments with argparse."""
parser = argparse.ArgumentParser(
description=("Opinionated rmdepcheck wrapper for EL use cases.")
)
parser.add_argument(
"compose",
type=url_check,
help="The URL(s) of a compose to discover variant repos from and check against",
)
parser.add_argument(
"repo",
type=pathlib.Path,
help="Local filesystem path containing packages to test",
)
parser.add_argument(
"--arch",
type=check_arch,
default="",
help="Arch to operate on. Must match arch of passed repository",
)
return parser.parse_args()
def main() -> None:
"""Main loop."""
try:
# set up requests session with retries
retries = Retry(
total=5,
backoff_factor=0.5,
status_forcelist=[502, 503, 504],
allowed_methods={'GET'},
)
SESSION.mount('https://', HTTPAdapter(max_retries=retries))
# check DNF is installed
check_utils()
# set up args
args = parse_args()
if f"--forcearch={args.arch}" not in DNFARGS:
DNFARGS.append(f"--forcearch={args.arch}")
# find variants, do the repo split
vrs = get_variant_repos(args.compose, args.arch)
val = get_val()
with tempfile.TemporaryDirectory(prefix="rdcelnrepo", dir="/var/tmp") as repotd:
repotemp = pathlib.Path(repotd)
variants = split_put(args.repo, repotemp, vrs)
rets = set()
for variant in variants:
# figure out what other variants to pull in per val
depvars = set()
for pair in val:
if pair[0] == variant:
depvars.add(pair[1])
# put the compose repos for all variants in basearg, even
# ones we didn't 'populate' locally
basearg = vrs[variant]
ncbasearg = ",".join(vrs[var] for var in depvars)
newarg = str(repotemp / variant)
# put depended-on variant-split PUT repos that actually
# exist in addrepos
# variants contains only variants we 'populated'
addarg = ",".join(str(repotemp / var) for var in depvars if var in variants)
# run rmdepcheck, track return code
# FIXME: what to do about JSON mode? capture output, combine?
rargs = [f"{HERE}/rmdepcheck.py", "--arch", args.arch]
if addarg:
rargs.extend(("--addrepos", addarg))
if ncbasearg:
rargs.extend(("--ncbaserepos", ncbasearg))
rargs.extend((basearg, newarg))
rets.add(subprocess.run(rargs, check=False).returncode)
# we exit the sum of all *unique* return codes
sys.exit(sum(rets))
except KeyboardInterrupt:
sys.stderr.write("Interrupted, exiting...\n")
sys.exit(1)
if __name__ == "__main__": # pragma: no cover
main()
# vim: set textwidth=100 ts=8 et sw=4:

14
rdc-repos Executable file
View file

@ -0,0 +1,14 @@
#!/usr/bin/python3
"""Convenience wrapper for executing rdc-repos from a git checkout."""
import os
import sys
# add src subdirectory directory to module import path
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), "src"))
from rmdepcheck.repos import main # pylint: disable=wrong-import-position
if __name__ == "__main__":
main()

445
rmdepcheck.py
View file

@ -1,445 +0,0 @@
#!/usr/bin/python3
# Copyright Red Hat
#
# This file is part of rmdepcheck.
#
# rmdepcheck is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation, either version 3 of
# the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Author(s): Adam Williamson <awilliam@redhat.com>
"""RPM package installability and reverse-dependency checks using a
repository modification strategy (hence 'rm').
"""
# Standard libraries
import argparse
import hashlib
import json
import os
import platform
import subprocess
import sys
import tempfile
from functools import partial
from typing import Iterable
from urllib.parse import urlparse
# type alias for the tuples produced by parse_repoclosure
# can't properly declare this because type statement was only added in
# 3.12, and TypeAlias is deprecated since 3.12 and wasn't in 3.9
DepTuple = tuple[str, str, str]
# use a fresh temporary cache for each run to avoid collisions between
# runs and polluting the 'real' cache
# pylint: disable-next=consider-using-with
DNFTEMP = tempfile.TemporaryDirectory(prefix="rmdepcheck", dir="/var/tmp")
DNFARGS = ["dnf", "--setopt", f"cachedir={DNFTEMP.name}", "-q", "--disablerepo=*"]
SUBPCAPTURE = partial(subprocess.run, capture_output=True, text=True, check=False)
SUBPCAPTCHECK = partial(subprocess.run, capture_output=True, text=True, check=True)
SUBPCHECK = partial(subprocess.run, check=True)
REPOHASHES: dict[str, str] = {}
def hash_repo(repo: str) -> str:
"""Generate a hash for the repo name, stash it in a dict so we can
map back out later, and return it. This is so we can show the repo
URLs in our final output, as opposed to non-useful made-up repo
names. Not security sensitive.
"""
gothash = hashlib.sha256(repo.encode(encoding="utf-8")).hexdigest()[:8]
REPOHASHES[gothash] = repo
return gothash
def parse_repoclosure(rc: str) -> list[DepTuple]:
"""Given some `dnf repoclosure` output, parse it into a list of
3-tuples each containing a package name, a repo URL (or generated
repo name if we can't look up the hash, should only happen in
tests) and an unresolved dependency for that package.
"""
out = []
pkg = None
for line in rc.splitlines():
if not line.strip():
continue
if line.strip().startswith("package:"):
# package, repo
elems = line.split()
pkg = (elems[1], REPOHASHES.get(elems[3], elems[3]))
continue
if line.strip().startswith("unresolved deps (") or line.strip().startswith("Error:"):
continue
# anything else is an unresolved dep
if pkg:
out.append(pkg + (line.strip(),))
return out
def format_rc_errors(errors: list[DepTuple]) -> None:
"""Format and print parse_repoclosure-style tuples for humans to
read. Used for final output after we do some diffing on the lists
of tuples.
"""
pkg = ("", "")
for error in errors:
if error[:2] != pkg:
pkg = error[:2]
print(f"package: {error[0]} from {error[1]}")
print(f" {error[2]}")
def get_base_repoclosure(
baserepos: Iterable[str], ncbaserepos: Iterable[str], nmbaserepos: Iterable[str]
) -> str:
"""Gets the reference repoclosure text. Base repos, non-checked
base repos and non-modified base repos are available to the
solver, but only the base repos are checked.
"""
cmdargs = DNFARGS + ["repoclosure"]
for repo in list(baserepos) + list(nmbaserepos) + list(ncbaserepos):
cmdargs.extend(["--repofrompath", f"{hash_repo(repo)},{repo}"])
cmdargs.append("--check")
# only check the repos that will be modified
cmdargs.append(",".join([hash_repo(baserepo) for baserepo in baserepos]))
return SUBPCAPTURE(cmdargs).stdout
def get_modified_and_new_repoclosure(
baserepos: list[str],
ncbaserepos: list[str],
nmbaserepos: list[str],
newrepos: list[str],
removes: Iterable[str],
) -> tuple[str, str]:
"""Runs repoclosure with new repo included and excludepkgs used
for modified base repos, and returns the modified repoclosure
text. Non-modified base repos, modified base repos and non-checked
base repos after modification, and new repos are available to
the solver; only the base repos are checked in the "modified"
check and only the first new repo is checked in the "new" check.
"""
queryargs = DNFARGS + ["repoquery", "--queryformat", "%{source_name},%{full_nevra}\n"]
rcargs = DNFARGS + ["repoclosure"]
for mrepo in baserepos + ncbaserepos:
# figure out what to exclude
excludes = []
args = queryargs + ["--repofrompath", f"{hash_repo(mrepo)},{mrepo}"]
out = SUBPCAPTURE(args).stdout
for line in out.splitlines():
# sname, nevr
elems = line.split(",")
if len(elems) != 2:
continue
if elems[0] in removes:
excludes.append(elems[1])
# add each modified base repo with excludepkgs set to the
# list we discovered above, in the repoclosure args
rcargs.extend(["--repofrompath", f"{hash_repo(mrepo)},{mrepo}"])
# there is a theoretical risk we might exceed MAX_ARG_STRLEN
# here, which is usually 131072:
# https://stackoverflow.com/a/29802900/4460661
# still, 131072 is enough for 2621 50-character NEVRs...
rcargs.extend(["--setopt", f"{hash_repo(mrepo)}.excludepkgs={','.join(excludes)}"])
# now add the non-modified base repos and new package repos
for repo in nmbaserepos + newrepos:
rcargs.extend(["--repofrompath", f"{hash_repo(repo)},{repo}"])
# finally, add the check arg
rcargs.append("--check")
rcargs.append(",".join([hash_repo(baserepo) for baserepo in baserepos]))
# get the modified repoclosure
mod = SUBPCAPTURE(rcargs).stdout
new = ""
if newrepos:
# the first newrepo is the checked repo, only check that
rcargs[-1] = hash_repo(newrepos[0])
# get the new repoclosure
new = SUBPCAPTURE(rcargs).stdout
return (mod, new)
def get_source_packages(repos: Iterable[str]) -> set[str]:
"""Finds and returns the source package names for all packages in
the repositories specified, as a set.
"""
args = list(DNFARGS)
for repo in repos:
args.extend(["--repofrompath", f"{hash_repo(repo)},{repo}"])
args.extend(["repoquery", "--qf", "%{sourcerpm} "])
srpms = SUBPCAPTCHECK(args).stdout.split()
return {srpm.rsplit("-", 2)[0] for srpm in srpms}
def handle_preexisting(fixederrors: list[DepTuple], newrc: list[DepTuple]) -> list[DepTuple]:
"""Catch cases where an error 'moved' from baserc to newrc - a
package in the set under test previously had a dependency issue,
and the new build does not fix it. We should not report this as a
fixed issue in the old repo and a new issue in the repo under
test. Note this modifies the passed lists in-place. See:
https://forge.fedoraproject.org/quality/rmdepcheck/issues/17
"""
preexisting = []
if fixederrors and newrc:
# iterate over copies so we can modify originals on the fly
fecopy = fixederrors.copy()
nrcopy = newrc.copy()
for fixeddep in fecopy:
# see if we can find a broken dep in newrc - the set of
# errors in the repository under test - that exactly
# matches this one, and is for the same package name
newmatch = [
newdep
for newdep in nrcopy
if newdep[0].rsplit("-", 2)[0] == fixeddep[0].rsplit("-", 2)[0]
and newdep[2] == fixeddep[2]
]
if len(newmatch) == 1:
# if so, remove the dep from both lists and add it
# to the new return list so we report it correctly
fixederrors.remove(fixeddep)
newrc.remove(newmatch[0])
preexisting.append(newmatch[0])
return preexisting
def url_check(arg: str) -> str:
"""Check arg is a file, http or https URL. Automatically converts
local paths to file:// URIs."""
# If it's an existing local path, convert to file:// URI
if os.path.exists(arg):
return f"file://{os.path.abspath(arg)}"
parsed = urlparse(arg)
if parsed.scheme in ("http", "https", "file"):
return arg
if parsed.scheme:
raise ValueError(f"Unsupported URL scheme {parsed.scheme} in {arg}")
raise ValueError(f"No URL scheme in {arg}")
def comma_url(arg: str) -> list[str]:
"""Check arg is a comma-separated list of URLs and return them
all as a list. If arg is the empty string, return empty list.
"""
if arg == "":
return []
split = arg.split(",")
for item in split:
try:
url_check(item)
except ValueError as err:
newerr = str(err) + f"from {arg}"
raise ValueError(newerr) from err
return split
def comma_list(arg: str) -> list[str]:
"""Handle a comma-separated list, return as a list."""
if arg == "":
return []
return arg.split(",")
def check_arch(arg: str) -> str:
"""Use host arch if no value passed."""
if arg:
return arg
return platform.machine()
def parse_args() -> argparse.Namespace:
"""Parse arguments with argparse."""
parser = argparse.ArgumentParser(
description=("Reverse dependency check implemented as a repoclosure diff.")
)
parser.add_argument(
"--addrepos",
type=comma_url,
default="",
help="The URL(s) of additional repositories containing new packages to be tested "
"(comma-separated). This is mainly intended for multilib cases: i.e. for testing "
"x86_64 package sets it should contain the matching i686 packages. It will be "
"available to the repoclosure check, but will be ignored by the installability check",
)
parser.add_argument(
"--ncbaserepos",
type=comma_url,
default="",
help="The URL(s) of non-checked base repositories to compare against (comma-separated). "
"These repositories *will* be modified as part of testing, but they will not be checked "
"for repoclosure. They should be repositories whose packages would be replaced with those "
"from the new package repo(s), and which we want to be available to the dependency solver "
"when checking the repoclosure of other repositories, but which we are for some reason not "
"interested in checking the repoclosure of - e.g. the buildroot for Fedora ELN",
)
parser.add_argument(
"--nmbaserepos",
type=comma_url,
default="",
help="The URL(s) of non-modified base repositories to compare against (comma-separated). "
"These repositories *will not* be modified as part of testing. They should be repositories "
"whose packages would *not* be replaced with those from the new package repo(s) - e.g. "
"the frozen release repository for a stable release, which is never changed",
)
parser.add_argument(
"--json",
action="store_true",
help="Output results in JSON rather than human-readable format",
)
parser.add_argument(
"--onlyerrors",
action="store_true",
help="Only print messages about problems caused, not about things that would be fixed",
)
parser.add_argument(
"--removes",
action="store_true",
help="Alternative mode: test removal (only) of all binary packages in the baserepos built "
"from the source package(s) specified as the final argument (a comma-separated list)",
)
parser.add_argument(
"--arch",
type=check_arch,
default="",
help="Arch to operate on. Must match arch of passed repositories. Defaults to host arch",
)
parser.add_argument(
"baserepos",
type=comma_url,
help="The URL(s) of base repositories to compare against (comma-separated). "
"These repositories *will* be modified as part of testing. They should be "
"repositories whose packages would be replaced with those from the new package "
"repo(s) - e.g. the main repository for a development release, or the updates repository "
"for a stable release",
)
# I wanted to do this with parse_known_args, but it messes up --help. aw
if "--removes" in sys.argv:
parser.add_argument(
"removes",
type=comma_list,
help="A comma-separated list of source packages to test the removal of",
)
else:
parser.add_argument(
"repo",
metavar="repo_or_removes",
type=url_check,
help="The URL of the repo containing the main set of new packages to be tested, "
"or a comma-separated list of source packages to test the removal of (if --removes "
"is passed)",
)
args = parser.parse_args()
if args.removes:
args.repo = ""
else:
args.removes = ""
return args
def check_dnf() -> None:
"""Check DNF is installed."""
try:
subprocess.run(("dnf", "--version"), stdout=subprocess.DEVNULL, check=True)
except FileNotFoundError:
sys.exit("Please install missing required utilities: dnf")
def main() -> None:
"""Main loop."""
try:
check_dnf()
exitcode = 0
args = parse_args()
if f"--forcearch={args.arch}" not in DNFARGS:
DNFARGS.append(f"--forcearch={args.arch}")
if args.removes:
newrepos = []
sources = args.removes
iut = "the specified source package removals"
else:
newrepos = [args.repo] + args.addrepos
# find source package(s) of our tested repo(s)
# whether to include addrepos is arguable, but should usually be moot
sources = get_source_packages(newrepos)
iut = "the tested packages"
baseraw = get_base_repoclosure(args.baserepos, args.ncbaserepos, args.nmbaserepos)
baserc = parse_repoclosure(baseraw)
# get the modified rpmclosure output
modraw, newraw = get_modified_and_new_repoclosure(
args.baserepos, args.ncbaserepos, args.nmbaserepos, newrepos, sources
)
modrc = parse_repoclosure(modraw)
newrc = []
if newraw:
newrc = parse_repoclosure(newraw)
newerrors = [dep for dep in modrc if dep not in baserc]
fixederrors = [dep for dep in baserc if dep not in modrc]
preexisting = handle_preexisting(fixederrors, newrc)
# output
if args.json:
jsonout = {}
if newerrors:
if args.json:
jsonout["newerrors"] = [list(err) for err in newerrors]
else:
print(f"Dependencies of other packages that would be BROKEN by {iut}:")
format_rc_errors(newerrors)
exitcode += 1
if newrc:
if args.json:
jsonout["installability"] = [list(err) for err in newrc]
else:
print("")
print("New dependency problems in the tested packages themselves:")
format_rc_errors(newrc)
exitcode += 2
if preexisting:
if args.json:
jsonout["installability_preexisting"] = [list(err) for err in preexisting]
else:
print("")
print("Pre-existing dependency problems in the tested packages themselves:")
format_rc_errors(preexisting)
exitcode += 4
if fixederrors:
if args.json:
jsonout["fixederrors"] = [list(err) for err in fixederrors]
elif not args.onlyerrors:
print("")
print(f"Dependencies of other packages that would be FIXED by {iut}:")
format_rc_errors(fixederrors)
if args.json:
json.dump(jsonout, sys.stdout, indent=4)
sys.stdout.write("\n")
sys.exit(exitcode)
except KeyboardInterrupt:
sys.stderr.write("Interrupted, exiting...\n")
sys.exit(1)
if __name__ == "__main__": # pragma: no cover
main()
# vim: set textwidth=100 ts=8 et sw=4:

29
src/rmdepcheck/__init__.py Normal file
View file

@ -0,0 +1,29 @@
#!/usr/bin/python3
# Copyright Red Hat
#
# This file is part of rmdepcheck.
#
# rmdepcheck is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation, either version 3 of
# the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Author(s): Adam Williamson <awilliam@redhat.com>
"""RPM package installability and reverse-dependency checks using a
repository modification strategy (hence 'rm').
"""
__version__ = "1.0.0"
# vim: set textwidth=100 ts=8 et sw=4:

279
src/rmdepcheck/compose.py Normal file
View file

@ -0,0 +1,279 @@
# Copyright Red Hat
#
# This file is part of rmdepcheck.
#
# rmdepcheck is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation, either version 3 of
# the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Author(s): Adam Williamson <awilliam@redhat.com>
"""RPM package installability and reverse-dependency checks using a
repository modification strategy (hence 'rm'). This module contains
the rdc-compose CLI which operates against composes and contains the
logic for splitting and testing variants appropriately.
"""
# Standard libraries
import argparse
import ast
import os
import pathlib
import re
import shutil
import subprocess
import sys
import tempfile
from requests import Session
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from rmdepcheck.shared import (
SUBPCAPTURE,
get_dnf_args,
check_arch,
url_check,
check_utils,
get_source_packages,
get_base_repoclosure,
get_modified_and_new_repoclosure,
parse_display_exit,
)
VrDict = dict[str, str]
def split_put(
put: pathlib.Path, repotemp: pathlib.Path, vrs: VrDict, dnfargs: list[str]
) -> set[str]:
"""Splits a directory containing the packages to be tested into
multiple repositories per variant. put ('packages under test')
is the filesystem path containing packages under test (it cannot
be an HTTP URL, on this path). repotemp is a Path to a temporary
directory to store the split repos. vrs is a dict with variant IDs
as keys and full compose repo URLs as values. For each package
under test, we get its name, look for a package of the same name
in each of the variant repositories, and if there is one, we put
the package in a subdirectory of repotemp, named for the variant.
At the end, we run createrepo in each of the subdirectories we
created, and return their names.
"""
if not (put.exists() and put.is_dir()):
raise ValueError(f"split_put: invalid put value {put}")
put = put.absolute()
# generate per-variant package name lists
pkgs = {}
for variant, repo in vrs.items():
args = dnfargs + [
"--repofrompath",
f"{variant},{repo}",
"repoquery",
"--queryformat",
"%{name}\n",
]
res = SUBPCAPTURE(args)
res.check_returncode()
pkgs[variant] = res.stdout.splitlines()
# track which variants we 'populate'
populated = set()
for _file in os.listdir(put):
if _file.endswith(".rpm"):
pkgname = _file.rsplit("-", 2)[0]
for variant, pkgnames in pkgs.items():
if pkgname in pkgnames:
populated.add(variant)
os.makedirs(repotemp / variant, exist_ok=True)
try:
# hardlink
os.link(put / _file, repotemp / variant / _file)
except OSError:
# copy
shutil.copy2(put / _file, repotemp / variant / _file)
for variant in populated:
subprocess.run(("createrepo", str(repotemp / variant)), capture_output=True, check=True)
return populated
def get_variant_repos(compose: str, ci: dict, arch: str) -> VrDict:
"""Discover variants with repositories in a compose, return a
dict with variant IDs as keys and repo URLs as values.
"""
variants = ci.get("payload", {}).get("variants", {})
ret = {}
for variant in variants:
vid = variants[variant].get("id", variant)
# as of 2026-05 yselkowitz says he does not want to know about
# buildroot-only dep issues
if vid.lower() == "buildroot":
continue
repopath = variants[variant].get("paths", {}).get("repository", {}).get(arch, "")
if not repopath:
continue
ret[vid] = f"{compose}/{repopath}"
return ret
def pungi_config_url(ci: dict) -> str:
"""Return the URL with the appropriate Pungi config for the
release. Here be magic knowledge.
"""
version = ci.get("payload", {}).get("release", {}).get("version", "")
short = ci.get("payload", {}).get("release", {}).get("short", "")
if version.lower() == "eln" and short.lower() == "fedora":
# pylint: disable-next=line-too-long
return "https://forge.fedoraproject.org/releng/pungi-fedora/raw/branch/eln/fedora/override.conf"
raise ValueError(f"Don't know Pungi config URL for {short} {version}")
def get_val(ci: dict, session: Session) -> dict:
"""Get the variants_as_lookaside list which tells us which
variants pull in which other variants.
"""
url = pungi_config_url(ci)
resp = session.get(url)
resp.raise_for_status()
override = resp.text
# next three lines suggested by AI
match = re.search(r"variant_as_lookaside\s*=\s*(\[.*?\])", override, re.DOTALL)
if not match:
raise ValueError(f"Could not find variant_as_lookaside in configuration at {url}")
# eek
return ast.literal_eval(match.group(1))
def url_check_http(arg: str) -> str:
"""Check arg is an http or https URL."""
return url_check(arg, fileok=False)
def parse_args() -> argparse.Namespace:
"""Parse arguments with argparse."""
parser = argparse.ArgumentParser(
description="Reverse dependency check implemented as a repoclosure diff. Checks against "
"a compose. Intended for Enterprise Linux use cases."
)
parser.add_argument(
"compose",
type=url_check_http,
help="The URL(s) of a compose to discover variant repos from and check against",
)
parser.add_argument(
"repo",
type=pathlib.Path,
help="Local filesystem path containing packages to test",
)
parser.add_argument(
"--arch",
type=check_arch,
default="",
help="Arch to operate on. Must match arch of passed repository",
)
parser.add_argument(
"--json",
action="store_true",
help="Output results in JSON rather than human-readable format",
)
return parser.parse_args()
def main() -> None: # pylint: disable=too-many-locals
"""Main loop."""
try:
# set up requests session with retries
session = Session()
retries = Retry(
total=5,
backoff_factor=0.5,
status_forcelist=[502, 503, 504],
allowed_methods={"GET"},
)
session.mount("https://", HTTPAdapter(max_retries=retries))
# check DNF and createrepo are installed
check_utils((("dnf", "--version"), ("createrepo", "--version")))
# set up args
args = parse_args()
# repoclosure output strings
basercs = ""
modrcs = ""
newrcs = ""
# get composeinfo
resp = session.get(f"{args.compose}/metadata/composeinfo.json")
resp.raise_for_status()
ci = resp.json()
# find variants, do the repo split
vrs = get_variant_repos(args.compose, ci, args.arch)
val = get_val(ci, session)
with (
tempfile.TemporaryDirectory(prefix="rdccomposerepo", dir="/var/tmp") as repotd,
tempfile.TemporaryDirectory(prefix="rdccomposednf", dir="/var/tmp") as dnftemp,
):
dnfargs = get_dnf_args(dnftemp, args.arch)
repotemp = pathlib.Path(repotd)
# this is "all the variants with packages in the new repo"
variants = split_put(args.repo, repotemp, vrs, dnfargs)
# we have to check all variants we have new packages in
# *and* all variants that depend on them
tocheck = set(variants)
for pair in val:
if pair[0] in variants:
tocheck.add(pair[1])
for variant in tocheck:
baserepo = vrs[variant]
# figure out what other variants to pull in per val
depvars = set()
for pair in val:
if pair[0] == variant:
depvars.add(pair[1])
# put the compose repos for all depended-on variants in
# ncbaserepos, even ones we didn't 'populate' locally
ncbaserepos = [vrs[var] for var in depvars]
newrepo = ""
newrepos = []
if variant in variants:
# first newrepo is the PUT repo for this variant,
# *if* it exists
newrepo = str(repotemp / variant)
newrepos.append(newrepo)
# additional repos are depended-on variant-split PUT
# repos that actually exist
# variants contains only variants we 'populated'
addrepos = [str(repotemp / var) for var in depvars if var in variants]
newrepos.extend(addrepos)
baserc = get_base_repoclosure([baserepo], ncbaserepos, [], dnfargs)
basercs = f"{basercs}\n{baserc}"
# get the modified rpmclosure output
sources = get_source_packages(newrepos, dnfargs)
modrc, newrc = get_modified_and_new_repoclosure(
[baserepo], ncbaserepos, [], newrepo, addrepos, sources, dnfargs
)
modrcs = f"{modrcs}\n{modrc}"
if newrc:
newrcs = f"{newrcs}\n{newrc}"
# output
parse_display_exit(basercs, modrcs, newrcs, "the tested packages", args.json, False, True)
except KeyboardInterrupt:
sys.stderr.write("Interrupted, exiting...\n")
sys.exit(1)
# vim: set textwidth=100 ts=8 et sw=4:

204
src/rmdepcheck/repos.py Normal file
View file

@ -0,0 +1,204 @@
# Copyright Red Hat
#
# This file is part of rmdepcheck.
#
# rmdepcheck is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation, either version 3 of
# the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Author(s): Adam Williamson <awilliam@redhat.com>
"""RPM package installability and reverse-dependency checks using a
repository modification strategy (hence 'rm'). This module contains
the rdc-repos CLI which operates against repositories.
"""
# Standard libraries
import argparse
import sys
import tempfile
from rmdepcheck.shared import (
get_dnf_args,
check_arch,
url_check,
check_utils,
get_source_packages,
get_base_repoclosure,
get_modified_and_new_repoclosure,
parse_display_exit,
)
def url_check_file(arg: str) -> str:
"""Check arg is a file, http or https URL. Automatically converts
local paths to file:// URIs."""
return url_check(arg, fileok=True)
def comma_url(arg: str) -> list[str]:
"""Check arg is a comma-separated list of URLs and return them
all as a list. If arg is the empty string, return empty list.
"""
if arg == "":
return []
split = arg.split(",")
for item in split:
try:
url_check_file(item)
except ValueError as err:
newerr = str(err) + f"from {arg}"
raise ValueError(newerr) from err
return split
def comma_list(arg: str) -> list[str]:
"""Handle a comma-separated list, return as a list."""
if arg == "":
return []
return arg.split(",")
def parse_args() -> argparse.Namespace:
"""Parse arguments with argparse."""
parser = argparse.ArgumentParser(
description=("Reverse dependency check implemented as a repoclosure diff.")
)
parser.add_argument(
"--addrepos",
type=comma_url,
default="",
help="The URL(s) of additional repositories containing new packages to be tested "
"(comma-separated). This is mainly intended for multilib cases: i.e. for testing "
"x86_64 package sets it should contain the matching i686 packages. It will be "
"available to the repoclosure check, but will be ignored by the installability check",
)
parser.add_argument(
"--ncbaserepos",
type=comma_url,
default="",
help="The URL(s) of non-checked base repositories to compare against (comma-separated). "
"These repositories *will* be modified as part of testing, but they will not be checked "
"for repoclosure. They should be repositories whose packages would be replaced with those "
"from the new package repo(s), and which we want to be available to the dependency solver "
"when checking the repoclosure of other repositories, but which we are for some reason not "
"interested in checking the repoclosure of - e.g. the buildroot for Fedora ELN",
)
parser.add_argument(
"--nmbaserepos",
type=comma_url,
default="",
help="The URL(s) of non-modified base repositories to compare against (comma-separated). "
"These repositories *will not* be modified as part of testing. They should be repositories "
"whose packages would *not* be replaced with those from the new package repo(s) - e.g. "
"the frozen release repository for a stable release, which is never changed",
)
parser.add_argument(
"--json",
action="store_true",
help="Output results in JSON rather than human-readable format",
)
parser.add_argument(
"--onlyerrors",
action="store_true",
help="Only print messages about problems caused, not about things that would be fixed",
)
parser.add_argument(
"--removes",
action="store_true",
help="Alternative mode: test removal (only) of all binary packages in the baserepos built "
"from the source package(s) specified as the final argument (a comma-separated list)",
)
parser.add_argument(
"--arch",
type=check_arch,
default="",
help="Arch to operate on. Must match arch of passed repositories. Defaults to host arch",
)
parser.add_argument(
"baserepos",
type=comma_url,
help="The URL(s) of base repositories to compare against (comma-separated). "
"These repositories *will* be modified as part of testing. They should be "
"repositories whose packages would be replaced with those from the new package "
"repo(s) - e.g. the main repository for a development release, or the updates repository "
"for a stable release",
)
# I wanted to do this with parse_known_args, but it messes up --help. aw
if "--removes" in sys.argv:
parser.add_argument(
"removes",
type=comma_list,
help="A comma-separated list of source packages to test the removal of",
)
else:
parser.add_argument(
"repo",
metavar="repo_or_removes",
type=url_check_file,
help="The URL of the repo containing the main set of new packages to be tested, "
"or a comma-separated list of source packages to test the removal of (if --removes "
"is passed)",
)
args = parser.parse_args()
if args.removes:
args.repo = ""
else:
args.removes = ""
return args
def main() -> None:
"""Main loop."""
try:
check_utils((("dnf", "--version"),))
args = parse_args()
with tempfile.TemporaryDirectory(prefix="rdcrepos", dir="/var/tmp") as dnftemp:
dnfargs = get_dnf_args(dnftemp, args.arch)
if args.removes:
newrepo = ""
sources = args.removes
iut = "the specified source package removals"
else:
newrepo = args.repo
newrepos = [newrepo] + args.addrepos
# find source package(s) of our tested repo(s)
# whether to include addrepos is arguable, but should usually be moot
sources = get_source_packages(newrepos, dnfargs)
iut = "the tested packages"
baseraw = get_base_repoclosure(
args.baserepos, args.ncbaserepos, args.nmbaserepos, dnfargs
)
# get the modified rpmclosure output
modraw, newraw = get_modified_and_new_repoclosure(
args.baserepos,
args.ncbaserepos,
args.nmbaserepos,
newrepo,
args.addrepos,
sources,
dnfargs,
)
# output
parse_display_exit(baseraw, modraw, newraw, iut, args.json, args.onlyerrors, False)
except KeyboardInterrupt:
sys.stderr.write("Interrupted, exiting...\n")
sys.exit(1)
# vim: set textwidth=100 ts=8 et sw=4:

341
src/rmdepcheck/shared.py Normal file
View file

@ -0,0 +1,341 @@
# Copyright Red Hat
#
# This file is part of rmdepcheck.
#
# rmdepcheck is free software; you can redistribute it
# and/or modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation, either version 3 of
# the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Author(s): Adam Williamson <awilliam@redhat.com>
"""RPM package installability and reverse-dependency checks using a
repository modification strategy (hence 'rm') This module contains
functions and constants shared between the two main scripts.
"""
import hashlib
import json
import os
import platform
import subprocess
import sys
from functools import partial
from typing import Iterable
from urllib.parse import urlparse
# type alias for the tuples produced by parse_repoclosure
# can't properly declare this because type statement was only added in
# 3.12, and TypeAlias is deprecated since 3.12 and wasn't in 3.9
DepTuple = tuple[str, str, str]
# use a fresh temporary cache for each run to avoid collisions between
# runs and polluting the 'real' cache
# pylint: disable-next=consider-using-with
SUBPCAPTURE = partial(subprocess.run, capture_output=True, text=True, check=False)
SUBPCAPTCHECK = partial(subprocess.run, capture_output=True, text=True, check=True)
SUBPCHECK = partial(subprocess.run, check=True) # pylint: disable=invalid-name
REPOHASHES: dict[str, str] = {}
def get_dnf_args(cachedir: str, arch: str) -> list[str]:
"""Construct our basic dnf args."""
return [
"dnf",
"--setopt",
f"cachedir={cachedir}",
"-q",
"--disablerepo=*",
f"--forcearch={arch}",
]
def check_arch(arg: str) -> str:
"""Use host arch if no value passed."""
if arg:
return arg
return platform.machine()
def url_check(arg: str, fileok: bool) -> str:
"""Check arg is a file, http or https URL. Automatically converts
local paths to file:// URIs."""
# If it's an existing local path, convert to file:// URI
if os.path.exists(arg):
if fileok:
return f"file://{os.path.abspath(arg)}"
raise ValueError("Value must be an HTTP or HTTPS URL")
parsed = urlparse(arg)
schemes = ["http", "https"]
if fileok:
schemes.append("file")
if parsed.scheme in schemes:
return arg
if parsed.scheme:
raise ValueError(f"Unsupported URL scheme {parsed.scheme} in {arg}")
raise ValueError(f"No URL scheme in {arg}")
def check_utils(utils: tuple[tuple[str, ...], ...]) -> None:
"""Check required utilities are installed."""
missing = []
for prog in utils:
try:
subprocess.run(prog, stdout=subprocess.DEVNULL, check=True)
except FileNotFoundError:
missing.append(prog[0])
if missing:
sys.exit("Please install missing required utilities: " + " ".join(missing))
def get_source_packages(repos: Iterable[str], dnfargs: list[str]) -> set[str]:
"""Finds and returns the source package names for all packages in
the repositories specified, as a set.
"""
gspargs = list(dnfargs)
for repo in repos:
gspargs.extend(["--repofrompath", f"{hash_repo(repo)},{repo}"])
gspargs.extend(["repoquery", "--qf", "%{sourcerpm} "])
srpms = SUBPCAPTCHECK(gspargs).stdout.split()
return {srpm.rsplit("-", 2)[0] for srpm in srpms}
def hash_repo(repo: str) -> str:
"""Generate a hash for the repo name, stash it in a dict so we can
map back out later, and return it. This is so we can show the repo
URLs in our final output, as opposed to non-useful made-up repo
names. Not security sensitive.
"""
gothash = hashlib.sha256(repo.encode(encoding="utf-8")).hexdigest()[:8]
REPOHASHES[gothash] = repo
return gothash
def parse_repoclosure(rc: str) -> list[DepTuple]:
"""Given some `dnf repoclosure` output, parse it into a list of
3-tuples each containing a package name, a repo URL (or generated
repo name if we can't look up the hash, should only happen in
tests) and an unresolved dependency for that package.
"""
out = []
pkg = None
for line in rc.splitlines():
if not line.strip():
continue
if line.strip().startswith("package:"):
# package, repo
elems = line.split()
pkg = (elems[1], REPOHASHES.get(elems[3], elems[3]))
continue
if line.strip().startswith("unresolved deps (") or line.strip().startswith("Error:"):
continue
# anything else is an unresolved dep
if pkg:
out.append(pkg + (line.strip(),))
return out
def format_rc_errors(errors: list[DepTuple], form: str = "pass") -> None:
"""Format and print parse_repoclosure-style tuples for humans to
read. Used for final output after we do some diffing on the lists
of tuples.
"""
pkg = ("", "")
for error in errors:
if error[:2] != pkg:
pkg = error[:2]
if form == "split":
print(f"package: {error[0]} from {error[1].split('/')[-1]}")
elif form == "strip":
print(f"package: {error[0]}")
else:
print(f"package: {error[0]} from {error[1]}")
print(f" {error[2]}")
def get_base_repoclosure(
baserepos: Iterable[str],
ncbaserepos: Iterable[str],
nmbaserepos: Iterable[str],
dnfargs: list[str],
) -> str:
"""Gets the reference repoclosure text. Base repos, non-checked
base repos and non-modified base repos are available to the
solver, but only the base repos are checked.
"""
cmdargs = dnfargs + ["repoclosure"]
for repo in list(baserepos) + list(nmbaserepos) + list(ncbaserepos):
cmdargs.extend(["--repofrompath", f"{hash_repo(repo)},{repo}"])
cmdargs.append("--check")
# only check the repos that will be modified
cmdargs.append(",".join([hash_repo(baserepo) for baserepo in baserepos]))
return SUBPCAPTURE(cmdargs).stdout
# pylint: disable-next=too-many-arguments,too-many-locals,too-many-positional-arguments
def get_modified_and_new_repoclosure(
baserepos: list[str],
ncbaserepos: list[str],
nmbaserepos: list[str],
newrepo: str,
addrepos: list[str],
removes: Iterable[str],
dnfargs: list[str],
) -> tuple[str, str]:
"""Runs repoclosure with new repo included and excludepkgs used
for modified base repos, and returns the modified repoclosure
text. Non-modified base repos, modified base repos and non-checked
base repos after modification, and new repos are available to
the solver; only the base repos are checked in the "modified"
check and only the first new repo is checked in the "new" check.
"""
queryargs = dnfargs + ["repoquery", "--queryformat", "%{source_name},%{full_nevra}\n"]
rcargs = dnfargs + ["repoclosure"]
for mrepo in baserepos + ncbaserepos:
# figure out what to exclude
excludes = []
args = queryargs + ["--repofrompath", f"{hash_repo(mrepo)},{mrepo}"]
for line in SUBPCAPTURE(args).stdout.splitlines():
# sname, nevr
elems = line.split(",")
if len(elems) != 2:
continue
if elems[0] in removes:
excludes.append(elems[1])
# add each modified base repo with excludepkgs set to the
# list we discovered above, in the repoclosure args
rcargs.extend(["--repofrompath", f"{hash_repo(mrepo)},{mrepo}"])
# there is a theoretical risk we might exceed MAX_ARG_STRLEN
# here, which is usually 131072:
# https://stackoverflow.com/a/29802900/4460661
# still, 131072 is enough for 2621 50-character NEVRs...
rcargs.extend(["--setopt", f"{hash_repo(mrepo)}.excludepkgs={','.join(excludes)}"])
# now add the non-modified base repos and new package repos
for repo in nmbaserepos + [nr for nr in [newrepo] if nr] + addrepos:
rcargs.extend(["--repofrompath", f"{hash_repo(repo)},{repo}"])
# finally, add the check arg
rcargs.append("--check")
rcargs.append(",".join([hash_repo(baserepo) for baserepo in baserepos]))
# get the modified repoclosure
mod = SUBPCAPTURE(rcargs).stdout
new = ""
if newrepo:
rcargs[-1] = hash_repo(newrepo)
# get the new repoclosure
new = SUBPCAPTURE(rcargs).stdout
return (mod, new)
def handle_preexisting(fixederrors: list[DepTuple], newrc: list[DepTuple]) -> list[DepTuple]:
"""Catch cases where an error 'moved' from baserc to newrc - a
package in the set under test previously had a dependency issue,
and the new build does not fix it. We should not report this as a
fixed issue in the old repo and a new issue in the repo under
test. Note this modifies the passed lists in-place. See:
https://forge.fedoraproject.org/quality/rmdepcheck/issues/17
"""
preexisting = []
if fixederrors and newrc:
# iterate over copies so we can modify originals on the fly
fecopy = fixederrors.copy()
nrcopy = newrc.copy()
for fixeddep in fecopy:
# see if we can find a broken dep in newrc - the set of
# errors in the repository under test - that exactly
# matches this one, and is for the same package name
newmatch = [
newdep
for newdep in nrcopy
if newdep[0].rsplit("-", 2)[0] == fixeddep[0].rsplit("-", 2)[0]
and newdep[2] == fixeddep[2]
]
if len(newmatch) == 1:
# if so, remove the dep from both lists and add it
# to the new return list so we report it correctly
fixederrors.remove(fixeddep)
newrc.remove(newmatch[0])
preexisting.append(newmatch[0])
return preexisting
# pylint: disable-next=too-many-branches,too-many-arguments,too-many-positional-arguments
def parse_display_exit(
baseraw: str,
modraw: str,
newraw: str,
iut: str,
dojson: bool,
onlyerrors: bool,
splitrepo: bool,
) -> None:
"""Take the raw rmdepcheck output strings, parse them, produce
human-readable or JSON output, and exit with an appropriate code.
"""
exitcode = 0
baserc = parse_repoclosure(baseraw)
modrc = parse_repoclosure(modraw)
newrc = []
if newraw:
newrc = parse_repoclosure(newraw)
newerrors = [dep for dep in modrc if dep not in baserc]
fixederrors = [dep for dep in baserc if dep not in modrc]
preexisting = handle_preexisting(fixederrors, newrc)
# output
if dojson:
jsonout = {}
if newerrors:
if dojson:
jsonout["newerrors"] = [list(err) for err in newerrors]
else:
print(f"Dependencies of other packages that would be BROKEN by {iut}:")
format_rc_errors(newerrors)
exitcode += 1
if newrc:
if dojson:
jsonout["installability"] = [list(err) for err in newrc]
else:
print("")
print("New dependency problems in the tested packages themselves:")
if splitrepo:
format_rc_errors(newrc, form="split")
else:
format_rc_errors(newrc, form="strip")
exitcode += 2
if preexisting:
if dojson:
jsonout["installability_preexisting"] = [list(err) for err in preexisting]
else:
print("")
print("Pre-existing dependency problems in the tested packages themselves:")
format_rc_errors(preexisting)
exitcode += 4
if fixederrors:
if dojson:
jsonout["fixederrors"] = [list(err) for err in fixederrors]
elif not onlyerrors:
print("")
print(f"Dependencies of other packages that would be FIXED by {iut}:")
format_rc_errors(fixederrors)
if dojson:
json.dump(jsonout, sys.stdout, indent=4)
sys.stdout.write("\n")
sys.exit(exitcode)
# vim: set textwidth=100 ts=8 et sw=4:

52
tests/conftest.py Normal file
View file

@ -0,0 +1,52 @@
# Copyright Red Hat
#
# This file is part of rmdepcheck.
#
# rmdepcheck is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Author: Adam Williamson <awilliam@redhat.com>
"""Test configuration and fixtures."""
import os
import subprocess
import time
from urllib.error import URLError, HTTPError
from urllib.request import urlopen
import pytest
@pytest.yield_fixture(scope="session")
def http(request): # pylint: disable=unused-argument
"""Run a SimpleHTTPServer that sits in front of our mock ELN
compose. We just do this with subprocess as we need it to run
parallel to the tests and this is really the easiest way. Note
we also stash override.conf in the compose dir, it's just
convenient that way.
"""
root = os.path.join(os.path.dirname(os.path.realpath(__file__)), "testdata", "repos", "eln")
args = ("python3", "-m", "http.server", "5001")
proc = subprocess.Popen(args, cwd=root) # pylint: disable=consider-using-with
# block until the server is actually running
resp = None
while not resp:
try:
resp = urlopen("http://localhost:5001/BaseOS") # pylint: disable=consider-using-with
except (ValueError, URLError, HTTPError):
time.sleep(0.1)
yield
# teardown
proc.kill()

666
tests/test_rmdepcheck.py
View file

@ -23,296 +23,414 @@
"""Tests for rmdepcheck."""
import os
import pathlib
import sys
import tempfile
from unittest import mock
from rmdepcheck.shared import (
get_dnf_args,
check_arch,
url_check,
check_utils,
get_source_packages,
hash_repo,
parse_repoclosure,
format_rc_errors,
get_base_repoclosure,
get_modified_and_new_repoclosure,
handle_preexisting,
)
from rmdepcheck import repos, compose
import pytest
import rmdepcheck
HERE = os.path.abspath(os.path.dirname(__file__))
TESTDATA = f"{HERE}/testdata"
REPOS = f"{TESTDATA}/repos"
def test_parse_repoclosure():
with open(f"{TESTDATA}/test_parse_repoclosure.txt", "r", encoding="utf-8") as fh:
rctext = fh.read()
assert rmdepcheck.parse_repoclosure(rctext) == [
("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python(abi) = 3.13"),
("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(wxpython) >= 4"),
("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(xnat) >= 0.3.3"),
("python3-x3dh-1.0.4-3.fc43.noarch", "baserepo1", "python3.14dist(pydantic) >= 1.7.4"),
]
class TestShared:
"""Unit tests for things from rmdepcheck.shared."""
def test_parse_repoclosure(self):
with open(f"{TESTDATA}/test_parse_repoclosure.txt", "r", encoding="utf-8") as fh:
rctext = fh.read()
assert parse_repoclosure(rctext) == [
("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python(abi) = 3.13"),
("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(wxpython) >= 4"),
("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(xnat) >= 0.3.3"),
("python3-x3dh-1.0.4-3.fc43.noarch", "baserepo1", "python3.14dist(pydantic) >= 1.7.4"),
]
def test_format_rc_errors(self, capsys):
errs = [
("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python(abi) = 3.13"),
("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(wxpython) >= 4"),
("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(xnat) >= 0.3.3"),
("python3-x3dh-1.0.4-3.fc43.noarch", "baserepo1", "python3.14dist(pydantic) >= 1.7.4"),
]
format_rc_errors(errs)
captured = capsys.readouterr()
with open(f"{TESTDATA}/test_format_rc_errors.txt", "r", encoding="utf-8") as fh:
exptext = fh.read()
assert captured.out == exptext
def test_get_base_repoclosure(self):
repo = f"file://{REPOS}/base"
with open(f"{TESTDATA}/test_get_base_repoclosure.txt", "r", encoding="utf-8") as testfh:
expected = testfh.read()
expected = expected.replace("{HASH}", hash_repo(repo))
with tempfile.TemporaryDirectory(prefix="testrdc", dir="/var/tmp") as dnftemp:
dnfargs = get_dnf_args(dnftemp, "x86_64")
ret = get_base_repoclosure([repo], [], [], dnfargs)
assert ret == expected
def test_get_modified_and_new_repoclosure(self):
brepo = f"file://{REPOS}/base"
nrepo = f"file://{REPOS}/new"
with open(f"{TESTDATA}/test_get_modified_repoclosure.txt", "r", encoding="utf-8") as testfh:
expectedmod = testfh.read()
expectedmod = expectedmod.replace("{HASH}", hash_repo(brepo))
with open(f"{TESTDATA}/test_get_new_repoclosure.txt", "r", encoding="utf-8") as testfh:
expectednew = testfh.read()
expectednew = expectednew.replace("{HASH}", hash_repo(nrepo))
with tempfile.TemporaryDirectory(prefix="testrdc", dir="/var/tmp") as dnftemp:
dnfargs = get_dnf_args(dnftemp, "x86_64")
ret = get_modified_and_new_repoclosure(
[brepo], [], [], nrepo, [], ["aaa", "ccc", "eee", "fff", "ggg"], dnfargs
)
assert ret == (expectedmod, expectednew)
def test_get_modified_and_new_repoclosure_commasafe(self):
"""Check we survive repoquery output lines that are, precisely,
a string from 'removes'. This is a pretty unlikely scenario, but
hey. Any other line with no commas is actually safe because it
will fail the `if elems[0] in removes` check.
"""
qamock = mock.Mock()
qamock.stdout = "ccc\nbbb,bbb-0:1.0-1.x86_64\nccc,ccc-0:1.0-1.x86_64"
rcmock = mock.Mock()
rcmock.stdout = ""
with mock.patch("rmdepcheck.shared.SUBPCAPTURE", side_effect=[qamock, rcmock]):
with tempfile.TemporaryDirectory(prefix="testrdc", dir="/var/tmp") as dnftemp:
dnfargs = get_dnf_args(dnftemp, "x86_64")
# this shouldn't raise an exception
get_modified_and_new_repoclosure(
["file:///foo/bar"], [], [], "", [], ["ccc"], dnfargs
)
def test_get_source_packages(self):
with tempfile.TemporaryDirectory(prefix="testrdc", dir="/var/tmp") as dnftemp:
dnfargs = get_dnf_args(dnftemp, "x86_64")
sources = get_source_packages([f"file://{REPOS}/new"], dnfargs)
assert sources == {"111", "222", "aaa", "ccc", "eee", "fff", "ggg"}
def test_handle_preexisting(self):
fixederrors = [
("libreoffice-langpack-lv-1:26.2.2.1-0.1.eln155.x86_64", "ogrepo", "mythes-lv"),
("libreoffice-langpack-nb-1:26.2.2.1-0.1.eln155.x86_64", "ogrepo", "mythes-nb"),
("libreoffice-langpack-nn-1:26.2.2.1-0.1.eln155.x86_64", "ogrepo", "mythes-nn"),
("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "someotherdep"),
]
newrc = [
("libreoffice-langpack-lv-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-lv"),
("libreoffice-langpack-nb-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nb"),
("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nn"),
("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "foo"),
("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "mythes-lv"),
]
# this should null out the 'matching' errors (the first three in
# each list), but leave the 'non-matching' ones (the others)
ret = handle_preexisting(fixederrors, newrc)
assert fixederrors == [
("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "someotherdep"),
]
assert newrc == [
("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "foo"),
("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "mythes-lv"),
]
# we should return the errors dropped from 'newrc' (so these can
# be reported separately)
assert ret == [
("libreoffice-langpack-lv-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-lv"),
("libreoffice-langpack-nb-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nb"),
("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nn"),
]
def test_url_check(self):
url_check("file:///foo/bar", fileok=True)
with pytest.raises(ValueError):
url_check("file:///foo/bar", fileok=False)
url_check("https://www.some.where", fileok=True)
url_check("http://some.where.insecure", fileok=True)
# Test automatic conversion of existing local paths to file:// URIs
result = url_check(REPOS, fileok=True)
assert result == f"file://{REPOS}"
with pytest.raises(ValueError):
url_check(REPOS, fileok=False)
# Test with relative path that exists
result = url_check("tests", fileok=True)
assert result.startswith("file://")
assert result.endswith("/tests")
with pytest.raises(ValueError):
url_check("ftp://1997.called", fileok=True)
# Non-existent paths without scheme should still raise ValueError
with pytest.raises(ValueError):
url_check("whatisthis", fileok=True)
@mock.patch("platform.machine", return_value="foobar")
def test_check_arch(self, _):
assert check_arch("x86_64") == "x86_64"
assert check_arch("") == "foobar"
@mock.patch("subprocess.run", autospec=True)
def test_check_utils(self, mock_run):
check_utils((("dnf", "--version"),))
mock_run.side_effect = FileNotFoundError
with pytest.raises(SystemExit) as excinfo:
check_utils((("dnf", "--version"),))
assert excinfo.value.code == "Please install missing required utilities: dnf"
def test_format_rc_errors(capsys):
errs = [
("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python(abi) = 3.13"),
("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(wxpython) >= 4"),
("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(xnat) >= 0.3.3"),
("python3-x3dh-1.0.4-3.fc43.noarch", "baserepo1", "python3.14dist(pydantic) >= 1.7.4"),
]
rmdepcheck.format_rc_errors(errs)
captured = capsys.readouterr()
with open(f"{TESTDATA}/test_format_rc_errors.txt", "r", encoding="utf-8") as fh:
exptext = fh.read()
assert captured.out == exptext
class TestRepos:
"""Unit tests for things in rmdepcheck.repos."""
def test_comma_url(self):
assert repos.comma_url("https://www.some.where") == ["https://www.some.where"]
assert repos.comma_url("https://www.some.where,file:///foo/bar") == [
"https://www.some.where",
"file:///foo/bar",
]
with pytest.raises(ValueError):
repos.comma_url("https://www.some.where,ftp://1997.called")
with pytest.raises(ValueError):
repos.comma_url("ftp://1997.called")
with pytest.raises(ValueError):
repos.comma_url("https://www.some.where,whatisthis")
def test_comma_list(self):
assert repos.comma_list("foo,bar") == ["foo", "bar"]
assert repos.comma_list("") == []
@mock.patch("rmdepcheck.repos.check_utils", side_effect=KeyboardInterrupt)
def test_ctrl_c(self, _, capsys):
with pytest.raises(SystemExit) as excinfo:
repos.main()
assert excinfo.value.code == 1
captured = capsys.readouterr()
assert captured.err == "Interrupted, exiting...\n"
def test_get_base_repoclosure():
repo = f"file://{REPOS}/base"
with open(f"{TESTDATA}/test_get_base_repoclosure.txt", "r", encoding="utf-8") as testfh:
expected = testfh.read()
expected = expected.replace("{HASH}", rmdepcheck.hash_repo(repo))
ret = rmdepcheck.get_base_repoclosure([repo], [], [])
assert ret == expected
class TestCompose:
"Unit tests for things in rmdepcheck.compose." ""
def test_split_put_error(self):
with pytest.raises(ValueError) as err:
compose.split_put(
pathlib.Path("/non/existent/place"),
pathlib.Path("/whatever"),
{"foo": "bar"},
"x86_64",
)
assert str(err.value) == "split_put: invalid put value /non/existent/place"
def test_get_variant_repos_arch(self):
fakeci = {
"payload": {
"variants": {
"BaseOS": {
"id": "BaseOS",
"paths": {
"repository": {
"someotherarch": "somepath",
}
},
}
}
}
}
# should not crash
assert not compose.get_variant_repos("somecompose", fakeci, "somearch")
def test_pungi_config_url(self):
fakeci = {
"payload": {
"release": {
"version": "eln",
"short": "Fedora",
}
}
}
ret = compose.pungi_config_url(fakeci)
assert ret == (
"https://forge.fedoraproject.org"
"/releng/pungi-fedora/raw/branch/eln/fedora/override.conf"
)
fakeci["payload"]["release"]["short"] = "CentOS"
fakeci["payload"]["release"]["version"] = "10.1"
with pytest.raises(ValueError) as err:
compose.pungi_config_url(fakeci)
assert str(err.value) == "Don't know Pungi config URL for CentOS 10.1"
def test_get_val_not_found(self):
fakeci = {
"payload": {
"release": {
"version": "eln",
"short": "Fedora",
}
}
}
fakesess = mock.MagicMock()
fakesess.get.return_value.text = "nope, not in here"
with pytest.raises(ValueError) as err:
compose.get_val(fakeci, fakesess)
assert str(err.value) == (
"Could not find variant_as_lookaside in configuration at https://"
"forge.fedoraproject.org/releng/pungi-fedora/raw/branch/eln/fedora/override.conf"
)
@mock.patch("rmdepcheck.compose.check_utils", side_effect=KeyboardInterrupt)
def test_ctrl_c(self, _, capsys):
with pytest.raises(SystemExit) as excinfo:
compose.main()
assert excinfo.value.code == 1
captured = capsys.readouterr()
assert captured.err == "Interrupted, exiting...\n"
def test_get_modified_and_new_repoclosure():
brepo = f"file://{REPOS}/base"
nrepo = f"file://{REPOS}/new"
with open(f"{TESTDATA}/test_get_modified_repoclosure.txt", "r", encoding="utf-8") as testfh:
expectedmod = testfh.read()
expectedmod = expectedmod.replace("{HASH}", rmdepcheck.hash_repo(brepo))
with open(f"{TESTDATA}/test_get_new_repoclosure.txt", "r", encoding="utf-8") as testfh:
expectednew = testfh.read()
expectednew = expectednew.replace("{HASH}", rmdepcheck.hash_repo(nrepo))
ret = rmdepcheck.get_modified_and_new_repoclosure(
[brepo], [], [], [nrepo], ["aaa", "ccc", "eee", "fff", "ggg"]
class TestE2E:
"""End-to-end functional tests for rmdepcheck and compose."""
def test_e2e_null(self, capsys):
"""End-to-end test using an empty repository, to test what happens
when no changes are found.
"""
sys.argv = ["repos.py", f"file://{REPOS}/base", f"file://{REPOS}/empty"]
with pytest.raises(SystemExit) as excinfo:
repos.main()
assert excinfo.value.code == 0
captured = capsys.readouterr()
assert captured.out == ""
@pytest.mark.parametrize("output", ("human", "json", "onlyerrors"))
def test_e2e_devel(self, output, capsys):
"""End-to-end test similar to a Rawhide or Branched case, with
a single modified base repo and a single check repo. Also tests
JSON output.
"""
sys.argv = ["repos.py", f"file://{REPOS}/base", f"file://{REPOS}/new"]
if output == "json":
sys.argv.insert(1, "--json")
if output == "onlyerrors":
sys.argv.insert(1, "--onlyerrors")
with pytest.raises(SystemExit) as excinfo:
repos.main()
assert excinfo.value.code == 3
captured = capsys.readouterr()
if output == "human":
fname = "test_e2e_devel.txt"
elif output == "onlyerrors":
fname = "test_e2e_devel_onlyerrors.txt"
else:
fname = "test_e2e_devel_json.txt"
with open(f"{TESTDATA}/{fname}", "r", encoding="utf-8") as fh:
exptext = fh.read()
exptext = exptext.replace("{REPOS}", REPOS)
assert captured.out == exptext
def test_e2e_updates(self, capsys):
"""End-to-end test similar to a stable Fedora release, with one
non-modified 'frozen' base repo and one modified updates repo.
"""
sys.argv = [
"repos.py",
"--nmbaserepos",
f"file://{REPOS}/base",
f"file://{REPOS}/updates",
f"file://{REPOS}/new",
]
with pytest.raises(SystemExit) as excinfo:
repos.main()
assert excinfo.value.code == 3
captured = capsys.readouterr()
with open(f"{TESTDATA}/test_e2e_updates.txt", "r", encoding="utf-8") as fh:
exptext = fh.read()
exptext = exptext.replace("{REPOS}", REPOS)
assert captured.out == exptext
@mock.patch(
"rmdepcheck.compose.pungi_config_url",
autospec=True,
return_value="http://localhost:5001/override.conf",
)
assert ret == (expectedmod, expectednew)
def test_e2e_eln(self, _, capsys, http): # pylint: disable=unused-argument
"""End-to-end test of the EL mode. This also exercises the non-
checked repository feature. The setup mimics "real" EL, more or
less. There are BaseOS, CRB, Extras and Buildroot repos. BaseOS
contains epa and epb. Extras contains epc. CRB contains epd and
epe. Buildroot contains epf. epe and epf both require epa = 1.0.
The "update" contains epa 2.0, an epb that requires epc, and an
epd that requires epa.
This sets us several expectations. The update breaks both epe and
epf, but we should only see a report about epe, since epf is in
the buildroot and so should be ignored. epb requiring epc is
wrong because BaseOS cannot require packages in Extras; this
should be reported. It should only be reported *once*, though,
when we're checking BaseOS, even though the BaseOS repos will be
in scope for each run (testing non-checked base repos). epd
requiring epa is fine; no error should be reported.
"""
sys.argv = [
"compose.py",
"--arch",
"x86_64",
"http://localhost:5001",
f"{REPOS}/elnnew",
]
with pytest.raises(SystemExit) as excinfo:
compose.main()
assert excinfo.value.code == 3
captured = capsys.readouterr()
with open(f"{TESTDATA}/test_e2e_eln.txt", "r", encoding="utf-8") as fh:
exptext = fh.read()
exptext = exptext.replace("{REPOS}", REPOS)
assert captured.out == exptext
def test_e2e_removes(self, capsys):
"""End-to-end test of the alternate --removes mode."""
sys.argv = ["repos.py", "--removes", f"file://{REPOS}/base", "aaa,eee"]
with pytest.raises(SystemExit) as excinfo:
repos.main()
assert excinfo.value.code == 1
captured = capsys.readouterr()
with open(f"{TESTDATA}/test_e2e_removes.txt", "r", encoding="utf-8") as fh:
exptext = fh.read()
exptext = exptext.replace("{REPOS}", REPOS)
assert captured.out == exptext
@pytest.mark.parametrize("output", ("human", "json"))
def test_e2e_unfixed(self, output, capsys):
"""End-to-end test where the new repository contains a package
with an unfixed pre-existing dependency issue.
"""
sys.argv = ["repos.py", f"file://{REPOS}/base", f"file://{REPOS}/unfixed"]
if output == "json":
sys.argv.insert(1, "--json")
with pytest.raises(SystemExit) as excinfo:
repos.main()
assert excinfo.value.code == 4
captured = capsys.readouterr()
if output == "human":
fname = "test_e2e_unfixed.txt"
else:
fname = "test_e2e_unfixed_json.txt"
with open(f"{TESTDATA}/{fname}", "r", encoding="utf-8") as fh:
exptext = fh.read()
exptext = exptext.replace("{REPOS}", REPOS)
assert captured.out == exptext
def test_get_modified_and_new_repoclosure_commasafe():
"""Check we survive repoquery output lines that are, precisely,
a string from 'removes'. This is a pretty unlikely scenario, but
hey. Any other line with no commas is actually safe because it
will fail the `if elems[0] in removes` check.
"""
qamock = mock.Mock()
qamock.stdout = "ccc\nbbb,bbb-0:1.0-1.x86_64\nccc,ccc-0:1.0-1.x86_64"
rcmock = mock.Mock()
rcmock.stdout = ""
with mock.patch("rmdepcheck.SUBPCAPTURE", side_effect=[qamock, rcmock]):
# this shouldn't raise an exception
rmdepcheck.get_modified_and_new_repoclosure(["file:///foo/bar"], [], [], [], ["ccc"])
def test_get_source_packages():
sources = rmdepcheck.get_source_packages([f"file://{REPOS}/new"])
assert sources == {"111", "222", "aaa", "ccc", "eee", "fff", "ggg"}
def test_handle_preexisting():
fixederrors = [
("libreoffice-langpack-lv-1:26.2.2.1-0.1.eln155.x86_64", "ogrepo", "mythes-lv"),
("libreoffice-langpack-nb-1:26.2.2.1-0.1.eln155.x86_64", "ogrepo", "mythes-nb"),
("libreoffice-langpack-nn-1:26.2.2.1-0.1.eln155.x86_64", "ogrepo", "mythes-nn"),
("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "someotherdep"),
]
newrc = [
("libreoffice-langpack-lv-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-lv"),
("libreoffice-langpack-nb-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nb"),
("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nn"),
("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "foo"),
("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "mythes-lv"),
]
# this should null out the 'matching' errors (the first three in
# each list), but leave the 'non-matching' ones (the others)
ret = rmdepcheck.handle_preexisting(fixederrors, newrc)
assert fixederrors == [
("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "someotherdep"),
]
assert newrc == [
("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "foo"),
("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "mythes-lv"),
]
# we should return the errors dropped from 'newrc' (so these can
# be reported separately)
assert ret == [
("libreoffice-langpack-lv-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-lv"),
("libreoffice-langpack-nb-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nb"),
("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nn"),
]
def test_url_check():
rmdepcheck.url_check("file:///foo/bar")
rmdepcheck.url_check("https://www.some.where")
rmdepcheck.url_check("http://some.where.insecure")
# Test automatic conversion of existing local paths to file:// URIs
result = rmdepcheck.url_check(REPOS)
assert result == f"file://{REPOS}"
# Test with relative path that exists
result = rmdepcheck.url_check("tests")
assert result.startswith("file://")
assert result.endswith("/tests")
with pytest.raises(ValueError):
rmdepcheck.url_check("ftp://1997.called")
# Non-existent paths without scheme should still raise ValueError
with pytest.raises(ValueError):
rmdepcheck.url_check("whatisthis")
def test_comma_url():
assert rmdepcheck.comma_url("https://www.some.where") == ["https://www.some.where"]
assert rmdepcheck.comma_url("https://www.some.where,file:///foo/bar") == [
"https://www.some.where",
"file:///foo/bar",
]
with pytest.raises(ValueError):
rmdepcheck.comma_url("https://www.some.where,ftp://1997.called")
with pytest.raises(ValueError):
rmdepcheck.comma_url("ftp://1997.called")
with pytest.raises(ValueError):
rmdepcheck.comma_url("https://www.some.where,whatisthis")
def test_comma_list():
assert rmdepcheck.comma_list("foo,bar") == ["foo", "bar"]
assert rmdepcheck.comma_list("") == []
@mock.patch("platform.machine", return_value="foobar")
def test_check_arch(_):
assert rmdepcheck.check_arch("x86_64") == "x86_64"
assert rmdepcheck.check_arch("") == "foobar"
@mock.patch("subprocess.run", autospec=True)
def test_check_dnf(mock_run):
rmdepcheck.check_dnf()
mock_run.side_effect = FileNotFoundError
with pytest.raises(SystemExit) as excinfo:
rmdepcheck.check_dnf()
assert excinfo.value.code == "Please install missing required utilities: dnf"
@mock.patch("rmdepcheck.check_dnf", side_effect=KeyboardInterrupt)
def test_ctrl_c(_, capsys):
with pytest.raises(SystemExit) as excinfo:
rmdepcheck.main()
assert excinfo.value.code == 1
captured = capsys.readouterr()
assert captured.err == "Interrupted, exiting...\n"
def test_e2e_null(capsys):
"""End-to-end test using an empty repository, to test what happens
when no changes are found.
"""
sys.argv = ["rmdepcheck.py", f"file://{REPOS}/base", f"file://{REPOS}/empty"]
with pytest.raises(SystemExit) as excinfo:
rmdepcheck.main()
assert excinfo.value.code == 0
captured = capsys.readouterr()
assert captured.out == ""
@pytest.mark.parametrize("output", ("human", "json", "onlyerrors"))
def test_e2e_devel(output, capsys):
"""End-to-end test similar to a Rawhide or Branched case, with
a single modified base repo and a single check repo. Also tests
JSON output.
"""
sys.argv = ["rmdepcheck.py", f"file://{REPOS}/base", f"file://{REPOS}/new"]
if output == "json":
sys.argv.insert(1, "--json")
if output == "onlyerrors":
sys.argv.insert(1, "--onlyerrors")
with pytest.raises(SystemExit) as excinfo:
rmdepcheck.main()
assert excinfo.value.code == 3
captured = capsys.readouterr()
if output == "human":
fname = "test_e2e_devel.txt"
elif output == "onlyerrors":
fname = "test_e2e_devel_onlyerrors.txt"
else:
fname = "test_e2e_devel_json.txt"
with open(f"{TESTDATA}/{fname}", "r", encoding="utf-8") as fh:
exptext = fh.read()
exptext = exptext.replace("{REPOS}", REPOS)
assert captured.out == exptext
def test_e2e_updates(capsys):
"""End-to-end test similar to a stable Fedora release, with one
non-modified 'frozen' base repo and one modified updates repo.
"""
sys.argv = [
"rmdepcheck.py",
"--nmbaserepos",
f"file://{REPOS}/base",
f"file://{REPOS}/updates",
f"file://{REPOS}/new",
]
with pytest.raises(SystemExit) as excinfo:
rmdepcheck.main()
assert excinfo.value.code == 3
captured = capsys.readouterr()
with open(f"{TESTDATA}/test_e2e_updates.txt", "r", encoding="utf-8") as fh:
exptext = fh.read()
exptext = exptext.replace("{REPOS}", REPOS)
assert captured.out == exptext
def test_e2e_unchecked(capsys):
"""End-to-end test with a smaller repo as the base repo and a
larger buildroot repo as a non-checked base repo. The new repo
contains a package that breaks the deps of one package in the base
repo and one package in the non-checked base repo, and itself
requires a package from the non-checked base repo. We expect to
see a failure for the broken base repo package, but no failure for
the broken non-checked base repo package or the new package itself.
"""
sys.argv = [
"rmdepcheck.py",
"--ncbaserepos",
f"file://{REPOS}/elnroot",
f"file://{REPOS}/elnbase",
f"file://{REPOS}/elnnew",
]
with pytest.raises(SystemExit) as excinfo:
rmdepcheck.main()
assert excinfo.value.code == 1
captured = capsys.readouterr()
with open(f"{TESTDATA}/test_e2e_eln.txt", "r", encoding="utf-8") as fh:
exptext = fh.read()
exptext = exptext.replace("{REPOS}", REPOS)
assert captured.out == exptext
def test_e2e_removes(capsys):
"""End-to-end test of the alternate --removes mode."""
sys.argv = ["rmdepcheck.py", "--removes", f"file://{REPOS}/base", "aaa,eee"]
with pytest.raises(SystemExit) as excinfo:
rmdepcheck.main()
assert excinfo.value.code == 1
captured = capsys.readouterr()
with open(f"{TESTDATA}/test_e2e_removes.txt", "r", encoding="utf-8") as fh:
exptext = fh.read()
exptext = exptext.replace("{REPOS}", REPOS)
assert captured.out == exptext
@pytest.mark.parametrize("output", ("human", "json"))
def test_e2e_unfixed(output, capsys):
"""End-to-end test where the new repository contains a package
with an unfixed pre-existing dependency issue.
"""
sys.argv = ["rmdepcheck.py", f"file://{REPOS}/base", f"file://{REPOS}/unfixed"]
if output == "json":
sys.argv.insert(1, "--json")
with pytest.raises(SystemExit) as excinfo:
rmdepcheck.main()
assert excinfo.value.code == 4
captured = capsys.readouterr()
if output == "human":
fname = "test_e2e_unfixed.txt"
else:
fname = "test_e2e_unfixed_json.txt"
with open(f"{TESTDATA}/{fname}", "r", encoding="utf-8") as fh:
exptext = fh.read()
exptext = exptext.replace("{REPOS}", REPOS)
assert captured.out == exptext
# vim: set textwidth=100 ts=8 et sw=4:

BIN
tests/testdata/repos/base/aaa-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/bbb-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/ccc-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/ddd-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/eee-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/fff-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/ggg-1.0-1.i686.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/ggg-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/hhh-1.0-1.i686.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/hhh-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/repodata/1acaeb6689285db3bfe681d374748eed2bc7cd9c1af87b32a27819f776ecacb6-filelists.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/repodata/6868a9258bc5b257d087b3f926cbdbc014866cac98e1c7047bb0de5c8f4e8b63-other.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/repodata/6b696476761c56d8bceb3e8997d31ca6977aaf63901130fe94c708e7d0cf429d-primary.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/base/repodata/725d5a8541dc6ac107257c1ccd264b53cfdfb601e10f1dffe1db40462fcab118-primary.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/base/repodata/934bc8a9464d6232eb2e22aea3275da638dd1d733d9af49f402ac5d0072b505c-filelists.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/base/repodata/9a69e354abcfd9f7de70b62321bab6263e340615e628cbf96518ffe4ab1bc376-other.xml.zst vendored Normal file
View file

Binary file not shown.

30
tests/testdata/repos/base/repodata/repomd.xml vendored
View file

@ -1,27 +1,27 @@
<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
<revision>1777498820</revision>
<revision>1778717667</revision>
<data type="primary">
<checksum type="sha256">6b696476761c56d8bceb3e8997d31ca6977aaf63901130fe94c708e7d0cf429d</checksum>
<open-checksum type="sha256">54c3435b6735114519dd995763d0a654a564872f34118ce620e7148d7488738e</open-checksum>
<location href="repodata/6b696476761c56d8bceb3e8997d31ca6977aaf63901130fe94c708e7d0cf429d-primary.xml.zst"/>
<timestamp>1777498820</timestamp>
<size>1281</size>
<checksum type="sha256">725d5a8541dc6ac107257c1ccd264b53cfdfb601e10f1dffe1db40462fcab118</checksum>
<open-checksum type="sha256">577f2cb77ecdf002fbcfc3d3d3d5972e9574d262a0a32068f5b795556d6a1bcf</open-checksum>
<location href="repodata/725d5a8541dc6ac107257c1ccd264b53cfdfb601e10f1dffe1db40462fcab118-primary.xml.zst"/>
<timestamp>1778717667</timestamp>
<size>1278</size>
<open-size>10594</open-size>
</data>
<data type="filelists">
<checksum type="sha256">1acaeb6689285db3bfe681d374748eed2bc7cd9c1af87b32a27819f776ecacb6</checksum>
<open-checksum type="sha256">05695974b3212a3e0a658e431c4c6eb1cf20b4346365b9a062c4469e7a85c6a1</open-checksum>
<location href="repodata/1acaeb6689285db3bfe681d374748eed2bc7cd9c1af87b32a27819f776ecacb6-filelists.xml.zst"/>
<timestamp>1777498820</timestamp>
<size>621</size>
<checksum type="sha256">934bc8a9464d6232eb2e22aea3275da638dd1d733d9af49f402ac5d0072b505c</checksum>
<open-checksum type="sha256">863daf657e56f2234c416ab3878b88305619ab38614dee37253281a6f38cd296</open-checksum>
<location href="repodata/934bc8a9464d6232eb2e22aea3275da638dd1d733d9af49f402ac5d0072b505c-filelists.xml.zst"/>
<timestamp>1778717667</timestamp>
<size>624</size>
<open-size>1722</open-size>
</data>
<data type="other">
<checksum type="sha256">6868a9258bc5b257d087b3f926cbdbc014866cac98e1c7047bb0de5c8f4e8b63</checksum>
<open-checksum type="sha256">29e4f16b3e7312bdeecc87f41baa5f8200df3793aaab472e131dcc6e010130b8</open-checksum>
<location href="repodata/6868a9258bc5b257d087b3f926cbdbc014866cac98e1c7047bb0de5c8f4e8b63-other.xml.zst"/>
<timestamp>1777498820</timestamp>
<checksum type="sha256">9a69e354abcfd9f7de70b62321bab6263e340615e628cbf96518ffe4ab1bc376</checksum>
<open-checksum type="sha256">b62ee7d9a3bfa4aed63880cf15f0022c25930473ec0a5004388db138c165b285</open-checksum>
<location href="repodata/9a69e354abcfd9f7de70b62321bab6263e340615e628cbf96518ffe4ab1bc376-other.xml.zst"/>
<timestamp>1778717667</timestamp>
<size>710</size>
<open-size>2838</open-size>
</data>

BIN
tests/testdata/repos/eln/AppStream/repodata/69a3730a283b85a4b3cff7d04bfde3b2b234f0607ebc17319d7a8d143a8e066e-primary.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/AppStream/repodata/6b37cc67608a24beaa81e1191d218f2ffd6b1191dceb5c100bac2e66249d518d-other.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/AppStream/repodata/9b07d97dc6ececed89aac0650b67bfb292647fe9fbaca48f629465be5f53f82b-filelists.xml.zst vendored Normal file
View file

Binary file not shown.

28
tests/testdata/repos/eln/AppStream/repodata/repomd.xml vendored Normal file
View file

@ -0,0 +1,28 @@
<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
<revision>1778717670</revision>
<data type="primary">
<checksum type="sha256">69a3730a283b85a4b3cff7d04bfde3b2b234f0607ebc17319d7a8d143a8e066e</checksum>
<open-checksum type="sha256">e1e2ffd2fb1ee76f87b70750d00ca5677a252b397ab6c2389137a0c33e7b359f</open-checksum>
<location href="repodata/69a3730a283b85a4b3cff7d04bfde3b2b234f0607ebc17319d7a8d143a8e066e-primary.xml.zst"/>
<timestamp>1778717670</timestamp>
<size>123</size>
<open-size>167</open-size>
</data>
<data type="filelists">
<checksum type="sha256">9b07d97dc6ececed89aac0650b67bfb292647fe9fbaca48f629465be5f53f82b</checksum>
<open-checksum type="sha256">bf9808b81cb2dbc54b4b8e35adc584ddcaa73bd81f7088d73bf7dbbada961310</open-checksum>
<location href="repodata/9b07d97dc6ececed89aac0650b67bfb292647fe9fbaca48f629465be5f53f82b-filelists.xml.zst"/>
<timestamp>1778717670</timestamp>
<size>118</size>
<open-size>125</open-size>
</data>
<data type="other">
<checksum type="sha256">6b37cc67608a24beaa81e1191d218f2ffd6b1191dceb5c100bac2e66249d518d</checksum>
<open-checksum type="sha256">e0ed5e0054194df036cf09c1a911e15bf2a4e7f26f2a788b6f47d53e80717ccc</open-checksum>
<location href="repodata/6b37cc67608a24beaa81e1191d218f2ffd6b1191dceb5c100bac2e66249d518d-other.xml.zst"/>
<timestamp>1778717670</timestamp>
<size>117</size>
<open-size>121</open-size>
</data>
</repomd>

BIN
tests/testdata/repos/elnbase/epc-1.0-1.x86_64.rpm → tests/testdata/repos/eln/BaseOS/epa-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/BaseOS/epb-1.0-1.x86_64.rpm vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/BaseOS/repodata/73c134c24423339ee3c1b12e215c925e853b58ade283876a72ca7b92fbf246c4-other.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/BaseOS/repodata/9daa049560d116a0c3e824db9385975140f68e97a03ae87214197284cbbbfad8-primary.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/BaseOS/repodata/e9b667b6a1059447627647ab73574fdee0a9c934cc7bfcf739f32edd1afae3d2-filelists.xml.zst vendored Normal file
View file

Binary file not shown.

28
tests/testdata/repos/eln/BaseOS/repodata/repomd.xml vendored Normal file
View file

@ -0,0 +1,28 @@
<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
<revision>1778717670</revision>
<data type="primary">
<checksum type="sha256">9daa049560d116a0c3e824db9385975140f68e97a03ae87214197284cbbbfad8</checksum>
<open-checksum type="sha256">3da88c933c481526801b2d09467c8e54e4edb8f54bf98422684c243650fdee96</open-checksum>
<location href="repodata/9daa049560d116a0c3e824db9385975140f68e97a03ae87214197284cbbbfad8-primary.xml.zst"/>
<timestamp>1778717670</timestamp>
<size>676</size>
<open-size>2163</open-size>
</data>
<data type="filelists">
<checksum type="sha256">e9b667b6a1059447627647ab73574fdee0a9c934cc7bfcf739f32edd1afae3d2</checksum>
<open-checksum type="sha256">de153bb5dc84de1eabec3de3762909234945d17228ca42b53cee3aa4b3da63f7</open-checksum>
<location href="repodata/e9b667b6a1059447627647ab73574fdee0a9c934cc7bfcf739f32edd1afae3d2-filelists.xml.zst"/>
<timestamp>1778717670</timestamp>
<size>279</size>
<open-size>445</open-size>
</data>
<data type="other">
<checksum type="sha256">73c134c24423339ee3c1b12e215c925e853b58ade283876a72ca7b92fbf246c4</checksum>
<open-checksum type="sha256">a1a6a080f1db568019eee61e3b62d74f3dafcb40867b0f2faf218585b7059123</open-checksum>
<location href="repodata/73c134c24423339ee3c1b12e215c925e853b58ade283876a72ca7b92fbf246c4-other.xml.zst"/>
<timestamp>1778717670</timestamp>
<size>352</size>
<open-size>665</open-size>
</data>
</repomd>

BIN
tests/testdata/repos/elnbase/epa-1.0-1.x86_64.rpm → tests/testdata/repos/eln/Buildroot/epe-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/Buildroot/repodata/4c41bd6e67d5b8fc2da462e55f106fcdc729daf0e722ace44fbf1f0688f4cf68-filelists.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/Buildroot/repodata/8d4fa97f0d4d6bdeca349744217b0e9d694aea2cef332765e97c2252c4df6431-primary.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/Buildroot/repodata/f4ecf39d19149d8918b7485eea237aa87723615393e5f15408b577e2114818d7-other.xml.zst vendored Normal file
View file

Binary file not shown.

28
tests/testdata/repos/eln/Buildroot/repodata/repomd.xml vendored Normal file
View file

@ -0,0 +1,28 @@
<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
<revision>1778717671</revision>
<data type="primary">
<checksum type="sha256">8d4fa97f0d4d6bdeca349744217b0e9d694aea2cef332765e97c2252c4df6431</checksum>
<open-checksum type="sha256">803c8426b401a9880c4f3e606fd7648a4d619ad1b9f64f029ba2edf0d922fcff</open-checksum>
<location href="repodata/8d4fa97f0d4d6bdeca349744217b0e9d694aea2cef332765e97c2252c4df6431-primary.xml.zst"/>
<timestamp>1778717671</timestamp>
<size>626</size>
<open-size>1265</open-size>
</data>
<data type="filelists">
<checksum type="sha256">4c41bd6e67d5b8fc2da462e55f106fcdc729daf0e722ace44fbf1f0688f4cf68</checksum>
<open-checksum type="sha256">69b6ea42561116fa83b8c836736c307693c9d425c6b5d8c9793c86bfa15069bd</open-checksum>
<location href="repodata/4c41bd6e67d5b8fc2da462e55f106fcdc729daf0e722ace44fbf1f0688f4cf68-filelists.xml.zst"/>
<timestamp>1778717671</timestamp>
<size>223</size>
<open-size>285</open-size>
</data>
<data type="other">
<checksum type="sha256">f4ecf39d19149d8918b7485eea237aa87723615393e5f15408b577e2114818d7</checksum>
<open-checksum type="sha256">6afc2f6d415b482a37ee91bb209367ff45b7f6f3434775ee219e73abf819ae72</open-checksum>
<location href="repodata/f4ecf39d19149d8918b7485eea237aa87723615393e5f15408b577e2114818d7-other.xml.zst"/>
<timestamp>1778717671</timestamp>
<size>304</size>
<open-size>393</open-size>
</data>
</repomd>

BIN
tests/testdata/repos/elnroot/epd-1.0-1.x86_64.rpm → tests/testdata/repos/eln/CRB/epd-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/elnroot/epb-1.0-1.x86_64.rpm → tests/testdata/repos/eln/CRB/epe-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/CRB/repodata/63510d69370f860af7ae99ffd909ecee7358765b80abb684b7a38f70e2bd977a-other.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/CRB/repodata/a939af998dafa1b5c771494f948ca1a12e7c8e0ba4b324f784caeca60580b944-filelists.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/CRB/repodata/adce22870e107115797acfb5b60eb3ae16af020f137c2dd783447099c23683cb-primary.xml.zst vendored Normal file
View file

Binary file not shown.

28
tests/testdata/repos/eln/CRB/repodata/repomd.xml vendored Normal file
View file

@ -0,0 +1,28 @@
<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
<revision>1778717670</revision>
<data type="primary">
<checksum type="sha256">adce22870e107115797acfb5b60eb3ae16af020f137c2dd783447099c23683cb</checksum>
<open-checksum type="sha256">d0ab817c34b063b3ff19b611f8e4ce132fa18fc1b5f41c424e5f8faa0bc52aa5</open-checksum>
<location href="repodata/adce22870e107115797acfb5b60eb3ae16af020f137c2dd783447099c23683cb-primary.xml.zst"/>
<timestamp>1778717670</timestamp>
<size>704</size>
<open-size>2235</open-size>
</data>
<data type="filelists">
<checksum type="sha256">a939af998dafa1b5c771494f948ca1a12e7c8e0ba4b324f784caeca60580b944</checksum>
<open-checksum type="sha256">d4cbedeac0b70877906e01ff83afedde852c88d56f5bf4477e8f4ec80d02e9e5</open-checksum>
<location href="repodata/a939af998dafa1b5c771494f948ca1a12e7c8e0ba4b324f784caeca60580b944-filelists.xml.zst"/>
<timestamp>1778717670</timestamp>
<size>279</size>
<open-size>445</open-size>
</data>
<data type="other">
<checksum type="sha256">63510d69370f860af7ae99ffd909ecee7358765b80abb684b7a38f70e2bd977a</checksum>
<open-checksum type="sha256">df320cc37c560b6aa899b3344d87a701d307398f662dc4e7437a22adddc0c2b0</open-checksum>
<location href="repodata/63510d69370f860af7ae99ffd909ecee7358765b80abb684b7a38f70e2bd977a-other.xml.zst"/>
<timestamp>1778717670</timestamp>
<size>353</size>
<open-size>665</open-size>
</data>
</repomd>

BIN
tests/testdata/repos/elnroot/epc-1.0-1.x86_64.rpm → tests/testdata/repos/eln/Extras/epc-1.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/Extras/repodata/718eae20ba82ada746eb049df2e88a420766a22c0a835e3ac0d666d1e5acfbaa-filelists.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/Extras/repodata/d852b0588e7a6b861bf02df7aff77fd692bc597986c9fa22323a36fdb2d73a06-primary.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/eln/Extras/repodata/ef66a3ad019261b3603fddafbb1a8aafdc9ebb314370553c518be659d83b0ada-other.xml.zst vendored Normal file
View file

Binary file not shown.

28
tests/testdata/repos/eln/Extras/repodata/repomd.xml vendored Normal file
View file

@ -0,0 +1,28 @@
<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
<revision>1778717670</revision>
<data type="primary">
<checksum type="sha256">d852b0588e7a6b861bf02df7aff77fd692bc597986c9fa22323a36fdb2d73a06</checksum>
<open-checksum type="sha256">323a9591c1a344cf10d2c9c5f541f118eb922e3e1e8a2274072459699647eb50</open-checksum>
<location href="repodata/d852b0588e7a6b861bf02df7aff77fd692bc597986c9fa22323a36fdb2d73a06-primary.xml.zst"/>
<timestamp>1778717670</timestamp>
<size>606</size>
<open-size>1165</open-size>
</data>
<data type="filelists">
<checksum type="sha256">718eae20ba82ada746eb049df2e88a420766a22c0a835e3ac0d666d1e5acfbaa</checksum>
<open-checksum type="sha256">dae4f03d98107c89b4af8380c4a6a7036b4809f7e68d60a3d2fa2a3a3429f010</open-checksum>
<location href="repodata/718eae20ba82ada746eb049df2e88a420766a22c0a835e3ac0d666d1e5acfbaa-filelists.xml.zst"/>
<timestamp>1778717670</timestamp>
<size>224</size>
<open-size>285</open-size>
</data>
<data type="other">
<checksum type="sha256">ef66a3ad019261b3603fddafbb1a8aafdc9ebb314370553c518be659d83b0ada</checksum>
<open-checksum type="sha256">d890073321ddf2010162771341ea12e3de65d0a4ef40a4cafa696d4cea29fe23</open-checksum>
<location href="repodata/ef66a3ad019261b3603fddafbb1a8aafdc9ebb314370553c518be659d83b0ada-other.xml.zst"/>
<timestamp>1778717670</timestamp>
<size>306</size>
<open-size>393</open-size>
</data>
</repomd>

53
tests/testdata/repos/eln/metadata/composeinfo.json vendored Normal file
View file

@ -0,0 +1,53 @@
{
"payload": {
"release": {
"internal": false,
"name": "Fedora",
"short": "Fedora",
"type": "ga",
"version": "eln"
},
"variants": {
"AppStream": {
"id": "AppStream",
"paths": {
"repository": {
"x86_64": "AppStream"
}
}
},
"BaseOS": {
"id": "BaseOS",
"paths": {
"repository": {
"x86_64": "BaseOS"
}
}
},
"Buildroot": {
"id": "Buildroot",
"paths": {
"repository": {
"x86_64": "Buildroot"
}
}
},
"CRB": {
"id": "CRB",
"paths": {
"repository": {
"x86_64": "CRB"
}
}
},
"Extras": {
"id": "Extras",
"paths": {
"repository": {
"x86_64": "Extras"
}
}
}
}
}
}

91
tests/testdata/repos/eln/override.conf vendored Normal file
View file

@ -0,0 +1,91 @@
# This files overrides default variables defined in the configs in
# the ../shared directory.
from images import *
# Fedora signing keys.
sigkeys = ['f577861e', '6d9f90a6']
# Architectures supported by Fedora ELN.
tree_arches = ['aarch64', 'ppc64le', 's390x', 'x86_64']
# For Fedora-ELN, we do not inherit builds from parent tags.
pkgset_koji_inherit = False
# No jigdo needed in Fedora.
create_jigdo = False
# We only build repositories, installer and images in Fedora so far.
skip_phases = [
"createiso",
"live_media",
"live_images",
"ostree",
"osbs",
]
# Enables macboot on x86_64 for all variants and disables upgrade image building
# everywhere.
# Use 4GB image size for all arches.
lorax_options = [
('^.*$', {
'x86_64': {
'nomacboot': True
},
'*': {
'rootfs_size': 4
}
})
]
# Drop the variants we do not care about from the variant_as_lookaside.
variant_as_lookaside = [
("AppStream", "BaseOS"),
("Extras", "BaseOS"),
("Extras", "AppStream"),
("Extras", "CRB"),
("HighAvailability", "BaseOS"),
("HighAvailability", "AppStream"),
("RT", "BaseOS"),
("RT", "AppStream"),
("NFV", "BaseOS"),
("NFV", "AppStream"),
("CRB", "BaseOS"),
("CRB", "AppStream"),
("SAP", "BaseOS"),
("SAP", "AppStream"),
("SAP", "HighAvailability"),
("SAPHANA", "BaseOS"),
("SAPHANA", "AppStream"),
("SAPHANA", "HighAvailability"),
("Buildroot", "BaseOS"),
("Buildroot", "AppStream"),
("Buildroot", "CRB"),
("Buildroot", "Extras"),
("Buildroot", "HighAvailability"),
("Buildroot", "NFV"),
("Buildroot", "RT"),
("Buildroot", "SAP"),
("Buildroot", "SAPHANA"),
]
# No product_id for Fedora.
product_id_allow_missing = False
# These will be inherited by live_media, live_images and image_build
global_release = '!RELEASE_FROM_LABEL_DATE_TYPE_RESPIN'
global_version = 'ELN'
# live_images ignores this in favor of live_target
global_target = 'eln'
# kiwi images need another target that uses old mock chroot
kiwibuild_target = 'eln-kiwi'
# kiwi image global configuration
kiwibuild_description_scm = 'git+https://pagure.io/fedora-kiwi-descriptions.git?#HEAD'
kiwibuild_description_path = 'Fedora-ELN.kiwi'
kiwibuild_version = '11'
kiwibuild_repo_releasever = 'eln'
# <name>-<version>-<ID>.<arch>
kiwibuild_bundle_name_format = '%N-%v-%I.%A'

BIN
tests/testdata/repos/elnbase/repodata/6212147882b929cefe09ce147e26e1020b738656c9517eac9a52a058d7793a31-other.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/elnbase/repodata/e66e2d334117944d094138fdc62fda05db2e252cd29aeffa765a71451a8856dc-filelists.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/elnbase/repodata/f4ed4ad1632ef201cb823cb62f2408918097df27d66aaee819fc22ba37ab262c-primary.xml.zst vendored
View file

Binary file not shown.

28
tests/testdata/repos/elnbase/repodata/repomd.xml vendored
View file

@ -1,28 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
<revision>1777498823</revision>
<data type="primary">
<checksum type="sha256">f4ed4ad1632ef201cb823cb62f2408918097df27d66aaee819fc22ba37ab262c</checksum>
<open-checksum type="sha256">12c7b2df4b3e597c8a412e6b187f0379f9024b0c19db0a6a22615dd1b5b09962</open-checksum>
<location href="repodata/f4ed4ad1632ef201cb823cb62f2408918097df27d66aaee819fc22ba37ab262c-primary.xml.zst"/>
<timestamp>1777498823</timestamp>
<size>699</size>
<open-size>2263</open-size>
</data>
<data type="filelists">
<checksum type="sha256">e66e2d334117944d094138fdc62fda05db2e252cd29aeffa765a71451a8856dc</checksum>
<open-checksum type="sha256">48d1e74e7b0cb3bf5286c6bf23c19000cb94afdf982c113b926688e51b100a36</open-checksum>
<location href="repodata/e66e2d334117944d094138fdc62fda05db2e252cd29aeffa765a71451a8856dc-filelists.xml.zst"/>
<timestamp>1777498823</timestamp>
<size>279</size>
<open-size>445</open-size>
</data>
<data type="other">
<checksum type="sha256">6212147882b929cefe09ce147e26e1020b738656c9517eac9a52a058d7793a31</checksum>
<open-checksum type="sha256">c8b2793ecacedcdba8b304a308130a84ac2613447053514f1e0f040cf5220132</open-checksum>
<location href="repodata/6212147882b929cefe09ce147e26e1020b738656c9517eac9a52a058d7793a31-other.xml.zst"/>
<timestamp>1777498823</timestamp>
<size>351</size>
<open-size>665</open-size>
</data>
</repomd>

BIN
tests/testdata/repos/elnnew/epa-2.0-1.x86_64.rpm vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/elnroot/epa-1.0-1.x86_64.rpm → tests/testdata/repos/elnnew/epb-2.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/elnnew/epc-2.0-1.x86_64.rpm → tests/testdata/repos/elnnew/epd-2.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/elnnew/repodata/029e08c67f723be4b217594371bb7f9f223cb866b934d857d9bddb4234e9c942-filelists.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/elnnew/repodata/6f2f32cc8b75d194c83ef1075936ed263a468f298ce34bf64a112a510aed46f8-filelists.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/elnnew/repodata/b5d3790249f642dafa96a7b5ba61938b50e19cc4c57fe63d7e4206a869c10925-other.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/elnnew/repodata/c083429e71ea2c4bea418159abf7debbf79dd70f21731b1b4acc5195658e7890-other.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/elnnew/repodata/e1fb8cbc0efa75785de59ffca8a3de5dd3928ab2075891beabbdedf2a74608b0-primary.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/elnnew/repodata/ec7341345e04def216d16e0db04b625570797b0142fe7150924d08f34acc1356-primary.xml.zst vendored Normal file
View file

Binary file not shown.

38
tests/testdata/repos/elnnew/repodata/repomd.xml vendored
View file

@ -1,28 +1,28 @@
<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
<revision>1777498823</revision>
<revision>1778717671</revision>
<data type="primary">
<checksum type="sha256">e1fb8cbc0efa75785de59ffca8a3de5dd3928ab2075891beabbdedf2a74608b0</checksum>
<open-checksum type="sha256">edfd3c129d9db24508a6dcd38765281f02cdf98609587ce9ccc3885cd226237f</open-checksum>
<location href="repodata/e1fb8cbc0efa75785de59ffca8a3de5dd3928ab2075891beabbdedf2a74608b0-primary.xml.zst"/>
<timestamp>1777498823</timestamp>
<size>629</size>
<open-size>1265</open-size>
<checksum type="sha256">ec7341345e04def216d16e0db04b625570797b0142fe7150924d08f34acc1356</checksum>
<open-checksum type="sha256">c09d2fb0e387f9b1d76bddbda53a17ff410e7376c32765e8c1883e8aa5125e10</open-checksum>
<location href="repodata/ec7341345e04def216d16e0db04b625570797b0142fe7150924d08f34acc1356-primary.xml.zst"/>
<timestamp>1778717671</timestamp>
<size>764</size>
<open-size>3299</open-size>
</data>
<data type="filelists">
<checksum type="sha256">029e08c67f723be4b217594371bb7f9f223cb866b934d857d9bddb4234e9c942</checksum>
<open-checksum type="sha256">8519cd36f5dcc309402aa70eb2463eebd439a134f410ee2b052afe51dddb6265</open-checksum>
<location href="repodata/029e08c67f723be4b217594371bb7f9f223cb866b934d857d9bddb4234e9c942-filelists.xml.zst"/>
<timestamp>1777498823</timestamp>
<size>226</size>
<open-size>285</open-size>
<checksum type="sha256">6f2f32cc8b75d194c83ef1075936ed263a468f298ce34bf64a112a510aed46f8</checksum>
<open-checksum type="sha256">d1edbf9e8bb2c6455b8b5ad6506b19b3d89bea5e0e4cc07eb3f675ab536a090a</open-checksum>
<location href="repodata/6f2f32cc8b75d194c83ef1075936ed263a468f298ce34bf64a112a510aed46f8-filelists.xml.zst"/>
<timestamp>1778717671</timestamp>
<size>323</size>
<open-size>605</open-size>
</data>
<data type="other">
<checksum type="sha256">c083429e71ea2c4bea418159abf7debbf79dd70f21731b1b4acc5195658e7890</checksum>
<open-checksum type="sha256">bd83ff93ecd4e37aa7dc35a1dfc61a5c8d560c14eea47fe443c86ca184853289</open-checksum>
<location href="repodata/c083429e71ea2c4bea418159abf7debbf79dd70f21731b1b4acc5195658e7890-other.xml.zst"/>
<timestamp>1777498823</timestamp>
<size>305</size>
<open-size>393</open-size>
<checksum type="sha256">b5d3790249f642dafa96a7b5ba61938b50e19cc4c57fe63d7e4206a869c10925</checksum>
<open-checksum type="sha256">7246bec78e312d7765aa885c6e196cf347adf944323c9d7593d57ef92340f763</open-checksum>
<location href="repodata/b5d3790249f642dafa96a7b5ba61938b50e19cc4c57fe63d7e4206a869c10925-other.xml.zst"/>
<timestamp>1778717671</timestamp>
<size>398</size>
<open-size>937</open-size>
</data>
</repomd>

BIN
tests/testdata/repos/elnroot/repodata/081516c8122bf9e116d0314902b635ef595c121ba2e7a9d8d448de7e636c948b-filelists.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/elnroot/repodata/117a29ab84c4728bf8da3b0b942c823be18f4d190e710ed797182925cfad70ba-other.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/elnroot/repodata/5548c16c17c9999a288149e4c028529fea3de98d310a6f6b9aaf5f9913c7ac7b-primary.xml.zst vendored
View file

Binary file not shown.

28
tests/testdata/repos/elnroot/repodata/repomd.xml vendored
View file

@ -1,28 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
<revision>1777498823</revision>
<data type="primary">
<checksum type="sha256">5548c16c17c9999a288149e4c028529fea3de98d310a6f6b9aaf5f9913c7ac7b</checksum>
<open-checksum type="sha256">a1a05ef98f83ff3a3d3f9916306cd31ca7a948599849ec779a95d6f7e8a40214</open-checksum>
<location href="repodata/5548c16c17c9999a288149e4c028529fea3de98d310a6f6b9aaf5f9913c7ac7b-primary.xml.zst"/>
<timestamp>1777498823</timestamp>
<size>830</size>
<open-size>4359</open-size>
</data>
<data type="filelists">
<checksum type="sha256">081516c8122bf9e116d0314902b635ef595c121ba2e7a9d8d448de7e636c948b</checksum>
<open-checksum type="sha256">d25e63e1db6cb9a61a2a98c0ad43abda59e7826aa7551738079e02b16d75a912</open-checksum>
<location href="repodata/081516c8122bf9e116d0314902b635ef595c121ba2e7a9d8d448de7e636c948b-filelists.xml.zst"/>
<timestamp>1777498823</timestamp>
<size>366</size>
<open-size>765</open-size>
</data>
<data type="other">
<checksum type="sha256">117a29ab84c4728bf8da3b0b942c823be18f4d190e710ed797182925cfad70ba</checksum>
<open-checksum type="sha256">a68bfd0ad8980ca8b2f26e4498cba5d9eb2c30885ca8fd1cac84f655687fe63f</open-checksum>
<location href="repodata/117a29ab84c4728bf8da3b0b942c823be18f4d190e710ed797182925cfad70ba-other.xml.zst"/>
<timestamp>1777498823</timestamp>
<size>445</size>
<open-size>1209</open-size>
</data>
</repomd>

8
tests/testdata/repos/empty/repodata/repomd.xml vendored
View file

@ -1,11 +1,11 @@
<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
<revision>1777498823</revision>
<revision>1778717671</revision>
<data type="primary">
<checksum type="sha256">69a3730a283b85a4b3cff7d04bfde3b2b234f0607ebc17319d7a8d143a8e066e</checksum>
<open-checksum type="sha256">e1e2ffd2fb1ee76f87b70750d00ca5677a252b397ab6c2389137a0c33e7b359f</open-checksum>
<location href="repodata/69a3730a283b85a4b3cff7d04bfde3b2b234f0607ebc17319d7a8d143a8e066e-primary.xml.zst"/>
<timestamp>1777498823</timestamp>
<timestamp>1778717671</timestamp>
<size>123</size>
<open-size>167</open-size>
</data>
@ -13,7 +13,7 @@
<checksum type="sha256">9b07d97dc6ececed89aac0650b67bfb292647fe9fbaca48f629465be5f53f82b</checksum>
<open-checksum type="sha256">bf9808b81cb2dbc54b4b8e35adc584ddcaa73bd81f7088d73bf7dbbada961310</open-checksum>
<location href="repodata/9b07d97dc6ececed89aac0650b67bfb292647fe9fbaca48f629465be5f53f82b-filelists.xml.zst"/>
<timestamp>1777498823</timestamp>
<timestamp>1778717671</timestamp>
<size>118</size>
<open-size>125</open-size>
</data>
@ -21,7 +21,7 @@
<checksum type="sha256">6b37cc67608a24beaa81e1191d218f2ffd6b1191dceb5c100bac2e66249d518d</checksum>
<open-checksum type="sha256">e0ed5e0054194df036cf09c1a911e15bf2a4e7f26f2a788b6f47d53e80717ccc</open-checksum>
<location href="repodata/6b37cc67608a24beaa81e1191d218f2ffd6b1191dceb5c100bac2e66249d518d-other.xml.zst"/>
<timestamp>1777498823</timestamp>
<timestamp>1778717671</timestamp>
<size>117</size>
<open-size>121</open-size>
</data>

63
tests/testdata/repos/mkrepos.py vendored
View file

@ -86,24 +86,34 @@ newddd = rpmfluff.SimpleRpmBuild("ddd", "3.0", "1", ["x86_64"])
newddd.add_requires("ccc = 3.0")
unfixed = (newddd,)
# ELN scenario packages. The "shipped repo" will contain only
# epa and epc. The "buildroot repo" will contain all four
# ELN scenario packages
epa = rpmfluff.SimpleRpmBuild("epa", "1.0", "1", ["x86_64"])
epb = rpmfluff.SimpleRpmBuild("epb", "1.0", "1", ["x86_64"])
epc = rpmfluff.SimpleRpmBuild("epc", "1.0", "1", ["x86_64"])
epd = rpmfluff.SimpleRpmBuild("epd", "1.0", "1", ["x86_64"])
alleln = (epa, epb, epc, epd)
shipeln = (epa, epc)
newepc = rpmfluff.SimpleRpmBuild("epc", "2.0", "1", ["x86_64"])
# these will both be broken by the new epc, but only epa is in the
# "shipped repo", epb is not
epa.add_requires("epc = 1.0")
epb.add_requires("epc = 1.0")
# this requirement can only be satisfied in the buildroot repo, we
# should not report that as an error
newepc.add_requires("epd = 1.0")
epe = rpmfluff.SimpleRpmBuild("epe", "1.0", "1", ["x86_64"])
epf = rpmfluff.SimpleRpmBuild("epe", "1.0", "1", ["x86_64"])
alleln = (epa, epb, epc, epd, epf)
baseeln = (epa, epb)
exteln = (epc,)
crbeln = (epd, epe)
rooteln = (epf,)
for pkg in allbase + allupd + allnew + unfixed + alleln + (newepc,):
newepa = rpmfluff.SimpleRpmBuild("epa", "2.0", "1", ["x86_64"])
newepb = rpmfluff.SimpleRpmBuild("epb", "2.0", "1", ["x86_64"])
newepd = rpmfluff.SimpleRpmBuild("epd", "2.0", "1", ["x86_64"])
neweln = (newepa, newepb, newepd)
# these will both be broken by the new epa, but epf should be ignored
# as it's buildroot-only
epe.add_requires("epa = 1.0")
epf.add_requires("epa = 1.0")
# this is invalid as baseos cannot depend on extras, should be caught
newepb.add_requires("epc")
# this is fine
newepd.add_requires("epa")
for pkg in allbase + allupd + allnew + unfixed + alleln + neweln:
pkg.addVendor("Fedora Project")
pkg.addPackager("Fedora Project")
@ -115,16 +125,25 @@ unfixed = rpmfluff.yumrepobuild.YumRepoBuild(unfixed)
unfixed.repoDir = "unfixed"
updates = rpmfluff.yumrepobuild.YumRepoBuild(allupd)
updates.repoDir = "updates"
elnbase = rpmfluff.yumrepobuild.YumRepoBuild(shipeln)
elnbase.repoDir = "elnbase"
elnroot = rpmfluff.yumrepobuild.YumRepoBuild(alleln)
elnroot.repoDir = "elnroot"
elnnew = rpmfluff.yumrepobuild.YumRepoBuild((newepc,))
elnbase = rpmfluff.yumrepobuild.YumRepoBuild(baseeln)
elnbase.repoDir = "eln/BaseOS"
elnext = rpmfluff.yumrepobuild.YumRepoBuild(exteln)
elnext.repoDir = "eln/Extras"
elncrb = rpmfluff.yumrepobuild.YumRepoBuild(crbeln)
elncrb.repoDir = "eln/CRB"
elnapp = rpmfluff.yumrepobuild.YumRepoBuild([])
elnapp.repoDir = "eln/AppStream"
elnroot = rpmfluff.yumrepobuild.YumRepoBuild(rooteln)
elnroot.repoDir = "eln/Buildroot"
elnnew = rpmfluff.yumrepobuild.YumRepoBuild(neweln)
elnnew.repoDir = "elnnew"
empty = rpmfluff.yumrepobuild.YumRepoBuild([])
empty.repoDir = "empty"
alldirs = [r.repoDir for r in (base, new, unfixed, updates, elnbase, elnroot, elnnew, empty)]
alldirs = [
r.repoDir
for r in (base, new, unfixed, updates, elnbase, elnext, elncrb, elnapp, elnroot, elnnew, empty)
]
def cleanup(repos=True):
@ -140,14 +159,18 @@ def cleanup(repos=True):
cleanup()
for _dir in alldirs:
os.mkdir(_dir)
os.makedirs(_dir)
base.make("x86_64", "i686")
new.make("x86_64")
unfixed.make("x86_64")
updates.make("x86_64")
elnbase.make("x86_64")
elnext.make("x86_64")
elncrb.make("x86_64")
elnapp.make("x86_64")
elnroot.make("x86_64")
elnnew.make("x86_64")
empty.make()
cleanup(repos=False)

BIN
tests/testdata/repos/new/111-3.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/new/222-3.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/new/aaa-3.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/new/ccc-3.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/new/eee-3.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/new/fff-3.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/new/ggg-3.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/new/repodata/0a7860c188784cab59dcc483eea7c38d8e9da287f45492897b95bd1a425da7ce-primary.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/new/repodata/182e2add8c3d3069057852ed9944419578aa72b31eb9233b18b6ce68fa86ba05-primary.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/new/repodata/4f693bcdc44f61c53c77e39ee300085d9e54b731fb75a9b664a8e0b9f6516c09-other.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/new/repodata/8e6cd282fee67920e9fd5728c68c86ad8d8ca56f9e8db935bd9358e22f4b86a8-filelists.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/new/repodata/cf45583b0c95946ca4ca3d567a3da5db1183158763f9b2dc006bc980a91f7393-filelists.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/new/repodata/d6dae028f2d94a20a5ada403a89d418b7d31c646d2350605999882725a5b0862-other.xml.zst vendored
View file

Binary file not shown.

30
tests/testdata/repos/new/repodata/repomd.xml vendored
View file

@ -1,28 +1,28 @@
<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
<revision>1777498821</revision>
<revision>1778717669</revision>
<data type="primary">
<checksum type="sha256">182e2add8c3d3069057852ed9944419578aa72b31eb9233b18b6ce68fa86ba05</checksum>
<open-checksum type="sha256">4ffb8600b26cbe3f260b10f2f7c35e17c92839fe66e5f1f13e37ec170f4d0d16</open-checksum>
<location href="repodata/182e2add8c3d3069057852ed9944419578aa72b31eb9233b18b6ce68fa86ba05-primary.xml.zst"/>
<timestamp>1777498821</timestamp>
<size>1069</size>
<checksum type="sha256">0a7860c188784cab59dcc483eea7c38d8e9da287f45492897b95bd1a425da7ce</checksum>
<open-checksum type="sha256">2ba4a2dbf3603d8fde7bd5d1595d18058f834b485a9611f49e18acca29b3cc61</open-checksum>
<location href="repodata/0a7860c188784cab59dcc483eea7c38d8e9da287f45492897b95bd1a425da7ce-primary.xml.zst"/>
<timestamp>1778717669</timestamp>
<size>1067</size>
<open-size>7430</open-size>
</data>
<data type="filelists">
<checksum type="sha256">8e6cd282fee67920e9fd5728c68c86ad8d8ca56f9e8db935bd9358e22f4b86a8</checksum>
<open-checksum type="sha256">601f4c051e7fb1541e74b63516ad2f5fceecafae40b02a15bef878f1e941b159</open-checksum>
<location href="repodata/8e6cd282fee67920e9fd5728c68c86ad8d8ca56f9e8db935bd9358e22f4b86a8-filelists.xml.zst"/>
<timestamp>1777498821</timestamp>
<checksum type="sha256">cf45583b0c95946ca4ca3d567a3da5db1183158763f9b2dc006bc980a91f7393</checksum>
<open-checksum type="sha256">69839c5fe82af86f9d68cf88757f8886943d9d0ca5387636315eb27330670e7c</open-checksum>
<location href="repodata/cf45583b0c95946ca4ca3d567a3da5db1183158763f9b2dc006bc980a91f7393-filelists.xml.zst"/>
<timestamp>1778717669</timestamp>
<size>499</size>
<open-size>1245</open-size>
</data>
<data type="other">
<checksum type="sha256">d6dae028f2d94a20a5ada403a89d418b7d31c646d2350605999882725a5b0862</checksum>
<open-checksum type="sha256">bfbc10574abc73bf3586a122246953f8073cde56afb8b4470d856ffe0d7823b0</open-checksum>
<location href="repodata/d6dae028f2d94a20a5ada403a89d418b7d31c646d2350605999882725a5b0862-other.xml.zst"/>
<timestamp>1777498821</timestamp>
<size>581</size>
<checksum type="sha256">4f693bcdc44f61c53c77e39ee300085d9e54b731fb75a9b664a8e0b9f6516c09</checksum>
<open-checksum type="sha256">c5f2112ab0f5b051aa8dc39e900a07ac7ba66294f08f144708bba996cb5c7881</open-checksum>
<location href="repodata/4f693bcdc44f61c53c77e39ee300085d9e54b731fb75a9b664a8e0b9f6516c09-other.xml.zst"/>
<timestamp>1778717669</timestamp>
<size>580</size>
<open-size>2025</open-size>
</data>
</repomd>

BIN
tests/testdata/repos/unfixed/ddd-3.0-1.x86_64.rpm vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/unfixed/repodata/77dfe7c593c3e37af3f924516b5aa25cc512a5b8070f75fbeb4b1e844973835b-filelists.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/unfixed/repodata/9d54f4c3d3ab7c943b7e15193e2544bcb82bf7b2e8408ee3167134b0a453255d-filelists.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/unfixed/repodata/a583ea09de5b85171eb9110dacfd240e527828cff25324ce894d3dd9c80cf961-other.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/unfixed/repodata/daa25dfd3c818ca6b5104612d827dac24e70a2050986ed49457124116013f4ef-primary.xml.zst vendored
View file

Binary file not shown.

BIN
tests/testdata/repos/unfixed/repodata/dd80d9c03852999e37b04db876c82b0483a6e906c5884b4561d83a97d7c341c7-primary.xml.zst vendored Normal file
View file

Binary file not shown.

BIN
tests/testdata/repos/unfixed/repodata/ef5302dc1a621cb60e1087af579f4e2bb66e668033b107b8706a39f2c37f8026-other.xml.zst vendored Normal file
View file

Binary file not shown.

Some files were not shown because too many files have changed in this diff Show more