diff --git a/.forgejo/workflows/ci.yml b/.forgejo/workflows/ci.yml index 7dcbd95..d3bba7c 100644 --- a/.forgejo/workflows/ci.yml +++ b/.forgejo/workflows/ci.yml @@ -10,7 +10,7 @@ jobs: image: quay.io/fedora/fedora:latest steps: - name: Install required packages - run: dnf -y install nodejs tox git + run: dnf -y install nodejs tox git createrepo_c - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 with: fetch-depth: 0 diff --git a/README.md b/README.md index 6ebe6f1..3979edc 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,18 @@ # rmdepcheck -rmdepcheck is an RPM dependency check tool based on a repository metadata modification approach. -It works by comparing a checked repository to one or more base repositories. First, checks are run -on the base repositories as-is. Next, we re-run the checks, but with the checked repository -available and using dnf's `excludepkgs` option to hide from the base repositories all packages from -the same source RPM(s) as the package(s) in the checked repositories removed. The results of the -two runs are compared. New failures should indicate problems introduced by the checked repositories. -Also, some relevant checks are run on the checked repositories with reference to the modified base -repositories. +rmdepcheck is an RPM dependency check tool based on a repository metadata modification approach. It +exists as an installable library, but is not usually intended to be installed or imported; you are +meant to use one of the two CLI interfaces, which can be run directly with no need for installation. + +## rdc-repos + +rdc-repos is the original form of rmdepcheck. It works by comparing a checked repository to one +or more base repositories. First, checks are run on the base repositories as-is. Next, we re-run +the checks, but with the checked repository available and using dnf's `excludepkgs` option to hide +from the base repositories all packages from the same source RPM(s) as the package(s) in the +checked repositories removed. The results of the two runs are compared. New failures should +indicate problems introduced by the checked repositories. Also, some relevant checks are run on the +checked repositories with reference to the modified base repositories. Optionally, other types of repository can be specified: @@ -24,8 +29,8 @@ HighAvailability variant can depend on packages in the BaseOS and Appstream vari testing an update containing packages destined for all three variants, which we split into three repositories. When testing the HA repository, we would want to have the "new package" BaseOS and Appstream repositories "in scope" but not checked ("additional new repositories"), and the "base" -BaseOS and Appstream repositories "in scope" and modified, but not checked -("non-checked base repositories"). +BaseOS and Appstream repositories "in scope" and modified, but not checked ("non-checked base +repositories"). Additional new repositories might also be used for multilib scenarios; it may be desirable to use such an additional repository for packages for the multilib arch(es), if e.g. installability of @@ -35,28 +40,23 @@ An alternative mode allows simply testing the consequences of *removing* a list entirely; in this mode, in the second step, we exclude all binary packages built from the specified source packages. The installability check is skipped in this context. -## rdc-el-wrapper +## rdc-compose -rdc-el-wrapper is an opinionated wrapper for running rmdepcheck on Enterprise Linux. The EL case -is complicated because EL composes contain multiple variants with package repositories. These -contain a subset of all packages in the buildroot. Each variant is expected to be repoclosure- -complete with regard to itself and possibly some or all of the other variants; the mapping for -this is kept in Pungi configuration. - -Because of this we cannot just test the packages-under-test against the buildroot, as we do for -Fedora. We must split the packages-under-test set into per-variant repositories, with each package -included in each repository it should be part of (we assume this mapping is not changed from the -compose we're testing against, and discover it from that). We then test each of the split repos -against the same variant repo from the compose, with other variant repos in scope on both sides, -as per the Pungi mapping. This is a lot of work but should provide the most accurate and useful -results. +rdc-compose is alternative interface which takes a distribution compose URL (this must be a Pungi +/ productmd-style compose) and a local filesystem path containing the packages under test as inputs. +It discovers the variants contained in the compose, splits the packages-under-test into per-variant +repositories, and then checks those against the target compose appropriately in much the same way +described above, testing appropriate sets of variants against each other according to the rules on +which variants can depend on packages from which other variants. This is a lot of work but should +provide the most accurate and useful results. ## Requirements -rmdepcheck has no run-time Python dependencies outside the standard library. Its only external -dependency is dnf. It checks for dnf, and will exit early with an error if it is not found. +rdc-repos has no run-time Python dependencies outside the standard library. rdc-compose requires +the requests and urllib3 libraries. Both scripts require dnf; rdc-compose also requires createrepo. +The scripts checks for their dependencies, and will exit early with an error if they are not found. rmdepcheck is written primarily for Red Hat-family distributions, but should in theory be usable -anywhere these utilities can be installed (and forward slashes act as directory separators). +anywhere these dependencies can be installed (and forward slashes act as directory separators). If you use a version of dnf older than 5.2.15.0, you may see false failures for 'rich' dependencies, as older dnf versions did not handle these correctly. Use 5.4.0.0 or newer for the best handling @@ -74,7 +74,7 @@ directly from PyPI with pip and other tools. Simple usage looks like this: ``` -rmdepcheck https://a.base.repo.example/repo,file:///another/baserepo file:///the/testedrepo +rdc-repos https://a.base.repo.example/repo,file:///another/baserepo file:///the/testedrepo ``` The repositories are specified as a comma-separated list. Repositories should be specified as URLs, @@ -82,7 +82,7 @@ but for convenience, file:// can be omitted. Only file:// , http:// and https:// For the alternative 'removal' mode, usage looks like: ``` -rmdepcheck --removes https://a.base.repo.example/repo,file:///another/baserepo sourcepkg1,sourcepkg2 +rdc-repos --removes https://a.base.repo.example/repo,file:///another/baserepo sourcepkg1,sourcepkg2 ``` This tests removing sourcepkg1 and sourcepkg2, and all binary packages built from them, from the @@ -91,20 +91,17 @@ base repositories. If you want to test repositories containing packages of an arch that does not match the system on which you are testing, pass `--arch `, where `` is the arch you wish to test. -For more complex usage, see `rmdepcheck --help`. +For more complex usage, see `rdc-repos --help`. -Note rmdepcheck is really only intended for use as a script, not as an importable library. If you -want to use it as a library go ahead, but this isn't a supported use case and bugs in it may not -be addressed. - -rdc-el-wrapper usage looks like this: +rdc-compose usage looks like this: ``` -rdc-el-wrapper https://kojipkgs.fedoraproject.org/compose/eln/latest-Fedora-eln/compose /the/testedrepo +rdc-compose https://kojipkgs.fedoraproject.org/compose/eln/latest-Fedora-eln/compose /the/testedrepo ``` An optional `--arch` arg can be passed as for rmdepcheck. Note that you **MUST NOT** include -file:// in the tested repo arg, and it **MUST** be a local filesystem path. rdc-el-wrapper does -not support remote tested repos. +file:// in the tested repo arg, and it **MUST** be a local filesystem path. rdc-compose does not +support remote tested repos.It does not actually need to be a repository; it just needs to be a +directory with RPMs in it. ### Testing build dependencies @@ -118,7 +115,7 @@ of all source packages will be unresolvable and the tool will run slowly and pro Example: ``` -rmdepcheck https://a.base.repo.example/binaryrepo,https://a.base.repo.example/sourcerepo file:///the/testedrepo +rdc-repos https://a.base.repo.example/binaryrepo,https://a.base.repo.example/sourcerepo file:///the/testedrepo ``` ## License diff --git a/install.requires b/install.requires index f229360..804abb1 100644 --- a/install.requires +++ b/install.requires @@ -1 +1,2 @@ requests +urllib3 diff --git a/pyproject.toml b/pyproject.toml index d2d3dbd..6187694 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -35,7 +35,8 @@ Issues = "https://codeberg.org/AdamWill/rmdepcheck/issues" Changelog = "https://codeberg.org/AdamWill/rmdepcheck/src/branch/main/CHANGELOG.md" [project.scripts] -rmdepcheck = "rmdepcheck:main" +rdc-repos = "rmdepcheck.repos:main" +rdc-compose = "rmdepcheck.compose:main" [build-system] requires = ["setuptools>=40.6.0", "setuptools-git", "wheel"] @@ -47,7 +48,7 @@ branch = true source_pkgs = ["rmdepcheck"] [tool.coverage.paths] -source = [".", ".tox/**/site-packages"] +source = ["src", ".tox/**/site-packages"] [tool.coverage.report] show_missing = true diff --git a/rdc-compose b/rdc-compose new file mode 100755 index 0000000..261612a --- /dev/null +++ b/rdc-compose @@ -0,0 +1,14 @@ +#!/usr/bin/python3 + +"""Convenience wrapper for executing rdc-compose from a git checkout.""" + +import os +import sys + +# add src subdirectory directory to module import path +sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), "src")) + +from rmdepcheck.compose import main # pylint: disable=wrong-import-position + +if __name__ == "__main__": + main() diff --git a/rdc-el-wrapper.py b/rdc-el-wrapper.py deleted file mode 100755 index 30b03d8..0000000 --- a/rdc-el-wrapper.py +++ /dev/null @@ -1,256 +0,0 @@ -#!/usr/bin/python3 - -# Copyright Red Hat -# -# This file is part of rmdepcheck. -# -# rmdepcheck is free software; you can redistribute it -# and/or modify it under the terms of the GNU General Public License -# as published by the Free Software Foundation, either version 3 of -# the License, or (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . -# -# Author(s): Adam Williamson - - -"""Opinionated rmdepcheck wrapper that handles the awkward Enterprise -Linux case, where we have to test against multiple shipped variants -with complex lookaside relationships, all filtered from a shared build -root. -""" - -# Standard libraries - -import argparse -import ast -import os -import pathlib -import platform -import re -import shutil -import subprocess -import sys -import tempfile -from functools import partial -from urllib.parse import urlparse - -from requests import Session -from requests.adapters import HTTPAdapter -from requests.packages.urllib3.util.retry import Retry - -VrDict = dict[str, str] - -DNFTEMP = tempfile.TemporaryDirectory(prefix="rdcelndnf", dir="/var/tmp") -DNFARGS = ["dnf", "--setopt", f"cachedir={DNFTEMP.name}", "-q", "--disablerepo=*"] -HERE = os.path.abspath(os.path.dirname(__file__)) -SESSION = Session() -SUBPCAPTURE = partial(subprocess.run, capture_output=True, text=True, check=False) - - -def split_put(put: pathlib.Path, repotemp: pathlib.Path, vrs: VrDict) -> set[str]: - """Splits a directory containing the packages to be tested into - multiple repositories per variant. put ('packages under test') - is the filesystem path containing packages under test (it cannot - be an HTTP URL, in this opinionated wrapper). repotemp is a Path - to a temporary directory to store the split repos. vrs is a dict - with variant IDs as keys and full compose repo URLs as values. - For each package under test, we get its name, look for a package - of the same name in each of the variant repositories, and if - there is one, we put the package in a subdirectory of repotemp, - named for the variant. At the end, we run createrepo in each of - the subdirectories we created, and return their names. - """ - if not (put.exists() and put.is_dir()): - raise ValueError(f"split_put: invalid put value {put}") - put = put.absolute() - # generate per-variant package name lists - pkgs = {} - for variant, repo in vrs.items(): - args = DNFARGS + [ - "--repofrompath", f"{variant},{repo}", "repoquery", "--queryformat", "%{name}\n" - ] - res = SUBPCAPTURE(args) - res.check_returncode() - pkgs[variant] = res.stdout.splitlines() - # track which variants we 'populate' - populated = set() - for _file in os.listdir(put): - if _file.endswith(".rpm"): - pkgname = _file.rsplit("-", 2)[0] - for (variant, pkgnames) in pkgs.items(): - if pkgname in pkgnames: - populated.add(variant) - os.makedirs(repotemp / variant, exist_ok=True) - try: - # hardlink - os.link(put / _file, repotemp / variant / _file) - except OSError: - # copy - shutil.copy2(put / _file, repotemp / variant / _file) - for variant in populated: - subprocess.run(("createrepo", str(repotemp / variant)), capture_output=True, check=True) - return populated - - -def get_variant_repos(compose: str, arch: str) -> VrDict: - """Discover variants with repositories in a compose, return a - dict with variant IDs as keys and repo URLs as values. - """ - resp = SESSION.get(f"{compose}/metadata/composeinfo.json") - resp.raise_for_status() - ci = resp.json() - variants = ci["payload"]["variants"] - ret = {} - for variant in variants: - if variants[variant]["id"].lower() == "buildroot": - continue - repopath = variants[variant]["paths"].get("repository", {}).get(arch, "") - if not repopath: - continue - ret[variants[variant]["id"]] = f"{compose}/{repopath}" - return ret - - -def get_val(): - """Get the variants_as_lookaside list which tells us which - variants pull in which other variants. FIXME: this is currently - ELN specific. - """ - resp = SESSION.get( - "https://forge.fedoraproject.org/releng/pungi-fedora/raw/branch/eln/fedora/override.conf" - ) - resp.raise_for_status() - override = resp.text - # next three lines suggested by AI - match = re.search(r"variant_as_lookaside\s*=\s*(\[.*?\])", override, re.DOTALL) - if not match: - raise ValueError("Could not find variant_as_lookaside in configuration") - # eek - return ast.literal_eval(match.group(1)) - - -def url_check(arg: str) -> str: - """Check arg is an http or https URL.""" - parsed = urlparse(arg) - if parsed.scheme in ("http", "https"): - return arg - if parsed.scheme: - raise ValueError(f"Unsupported URL scheme {parsed.scheme} in {arg}") - raise ValueError(f"No URL scheme in {arg}") - - -def check_arch(arg: str) -> str: - """Use host arch if no value passed.""" - if arg: - return arg - return platform.machine() - - -def check_utils() -> None: - """Check required utilities are installed.""" - missing = [] - for prog in (("dnf", "--version"), ("createrepo", "--version")): - try: - subprocess.run(prog, stdout=subprocess.DEVNULL, check=True) - except FileNotFoundError: - missing.append(prog[0]) - if missing: - sys.exit("Please install missing required utilities: " + " ".join(missing)) - - -def parse_args() -> argparse.Namespace: - """Parse arguments with argparse.""" - parser = argparse.ArgumentParser( - description=("Opinionated rmdepcheck wrapper for EL use cases.") - ) - parser.add_argument( - "compose", - type=url_check, - help="The URL(s) of a compose to discover variant repos from and check against", - ) - parser.add_argument( - "repo", - type=pathlib.Path, - help="Local filesystem path containing packages to test", - ) - parser.add_argument( - "--arch", - type=check_arch, - default="", - help="Arch to operate on. Must match arch of passed repository", - ) - return parser.parse_args() - - -def main() -> None: - """Main loop.""" - try: - # set up requests session with retries - retries = Retry( - total=5, - backoff_factor=0.5, - status_forcelist=[502, 503, 504], - allowed_methods={'GET'}, - ) - SESSION.mount('https://', HTTPAdapter(max_retries=retries)) - - # check DNF is installed - check_utils() - - # set up args - args = parse_args() - if f"--forcearch={args.arch}" not in DNFARGS: - DNFARGS.append(f"--forcearch={args.arch}") - - # find variants, do the repo split - vrs = get_variant_repos(args.compose, args.arch) - val = get_val() - with tempfile.TemporaryDirectory(prefix="rdcelnrepo", dir="/var/tmp") as repotd: - repotemp = pathlib.Path(repotd) - variants = split_put(args.repo, repotemp, vrs) - - rets = set() - for variant in variants: - # figure out what other variants to pull in per val - depvars = set() - for pair in val: - if pair[0] == variant: - depvars.add(pair[1]) - # put the compose repos for all variants in basearg, even - # ones we didn't 'populate' locally - basearg = vrs[variant] - ncbasearg = ",".join(vrs[var] for var in depvars) - newarg = str(repotemp / variant) - # put depended-on variant-split PUT repos that actually - # exist in addrepos - # variants contains only variants we 'populated' - addarg = ",".join(str(repotemp / var) for var in depvars if var in variants) - # run rmdepcheck, track return code - # FIXME: what to do about JSON mode? capture output, combine? - rargs = [f"{HERE}/rmdepcheck.py", "--arch", args.arch] - if addarg: - rargs.extend(("--addrepos", addarg)) - if ncbasearg: - rargs.extend(("--ncbaserepos", ncbasearg)) - rargs.extend((basearg, newarg)) - rets.add(subprocess.run(rargs, check=False).returncode) - - # we exit the sum of all *unique* return codes - sys.exit(sum(rets)) - - except KeyboardInterrupt: - sys.stderr.write("Interrupted, exiting...\n") - sys.exit(1) - - -if __name__ == "__main__": # pragma: no cover - main() - -# vim: set textwidth=100 ts=8 et sw=4: diff --git a/rdc-repos b/rdc-repos new file mode 100755 index 0000000..bebd111 --- /dev/null +++ b/rdc-repos @@ -0,0 +1,14 @@ +#!/usr/bin/python3 + +"""Convenience wrapper for executing rdc-repos from a git checkout.""" + +import os +import sys + +# add src subdirectory directory to module import path +sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), "src")) + +from rmdepcheck.repos import main # pylint: disable=wrong-import-position + +if __name__ == "__main__": + main() diff --git a/rmdepcheck.py b/rmdepcheck.py deleted file mode 100755 index 41e9c06..0000000 --- a/rmdepcheck.py +++ /dev/null @@ -1,445 +0,0 @@ -#!/usr/bin/python3 - -# Copyright Red Hat -# -# This file is part of rmdepcheck. -# -# rmdepcheck is free software; you can redistribute it -# and/or modify it under the terms of the GNU General Public License -# as published by the Free Software Foundation, either version 3 of -# the License, or (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . -# -# Author(s): Adam Williamson - - -"""RPM package installability and reverse-dependency checks using a -repository modification strategy (hence 'rm'). -""" - -# Standard libraries - -import argparse -import hashlib -import json -import os -import platform -import subprocess -import sys -import tempfile -from functools import partial -from typing import Iterable -from urllib.parse import urlparse - -# type alias for the tuples produced by parse_repoclosure -# can't properly declare this because type statement was only added in -# 3.12, and TypeAlias is deprecated since 3.12 and wasn't in 3.9 -DepTuple = tuple[str, str, str] - -# use a fresh temporary cache for each run to avoid collisions between -# runs and polluting the 'real' cache -# pylint: disable-next=consider-using-with -DNFTEMP = tempfile.TemporaryDirectory(prefix="rmdepcheck", dir="/var/tmp") -DNFARGS = ["dnf", "--setopt", f"cachedir={DNFTEMP.name}", "-q", "--disablerepo=*"] -SUBPCAPTURE = partial(subprocess.run, capture_output=True, text=True, check=False) -SUBPCAPTCHECK = partial(subprocess.run, capture_output=True, text=True, check=True) -SUBPCHECK = partial(subprocess.run, check=True) -REPOHASHES: dict[str, str] = {} - - -def hash_repo(repo: str) -> str: - """Generate a hash for the repo name, stash it in a dict so we can - map back out later, and return it. This is so we can show the repo - URLs in our final output, as opposed to non-useful made-up repo - names. Not security sensitive. - """ - gothash = hashlib.sha256(repo.encode(encoding="utf-8")).hexdigest()[:8] - REPOHASHES[gothash] = repo - return gothash - - -def parse_repoclosure(rc: str) -> list[DepTuple]: - """Given some `dnf repoclosure` output, parse it into a list of - 3-tuples each containing a package name, a repo URL (or generated - repo name if we can't look up the hash, should only happen in - tests) and an unresolved dependency for that package. - """ - out = [] - pkg = None - for line in rc.splitlines(): - if not line.strip(): - continue - if line.strip().startswith("package:"): - # package, repo - elems = line.split() - pkg = (elems[1], REPOHASHES.get(elems[3], elems[3])) - continue - if line.strip().startswith("unresolved deps (") or line.strip().startswith("Error:"): - continue - # anything else is an unresolved dep - if pkg: - out.append(pkg + (line.strip(),)) - return out - - -def format_rc_errors(errors: list[DepTuple]) -> None: - """Format and print parse_repoclosure-style tuples for humans to - read. Used for final output after we do some diffing on the lists - of tuples. - """ - pkg = ("", "") - for error in errors: - if error[:2] != pkg: - pkg = error[:2] - print(f"package: {error[0]} from {error[1]}") - print(f" {error[2]}") - - -def get_base_repoclosure( - baserepos: Iterable[str], ncbaserepos: Iterable[str], nmbaserepos: Iterable[str] -) -> str: - """Gets the reference repoclosure text. Base repos, non-checked - base repos and non-modified base repos are available to the - solver, but only the base repos are checked. - """ - cmdargs = DNFARGS + ["repoclosure"] - for repo in list(baserepos) + list(nmbaserepos) + list(ncbaserepos): - cmdargs.extend(["--repofrompath", f"{hash_repo(repo)},{repo}"]) - cmdargs.append("--check") - # only check the repos that will be modified - cmdargs.append(",".join([hash_repo(baserepo) for baserepo in baserepos])) - return SUBPCAPTURE(cmdargs).stdout - - -def get_modified_and_new_repoclosure( - baserepos: list[str], - ncbaserepos: list[str], - nmbaserepos: list[str], - newrepos: list[str], - removes: Iterable[str], -) -> tuple[str, str]: - """Runs repoclosure with new repo included and excludepkgs used - for modified base repos, and returns the modified repoclosure - text. Non-modified base repos, modified base repos and non-checked - base repos after modification, and new repos are available to - the solver; only the base repos are checked in the "modified" - check and only the first new repo is checked in the "new" check. - """ - queryargs = DNFARGS + ["repoquery", "--queryformat", "%{source_name},%{full_nevra}\n"] - rcargs = DNFARGS + ["repoclosure"] - for mrepo in baserepos + ncbaserepos: - # figure out what to exclude - excludes = [] - args = queryargs + ["--repofrompath", f"{hash_repo(mrepo)},{mrepo}"] - out = SUBPCAPTURE(args).stdout - for line in out.splitlines(): - # sname, nevr - elems = line.split(",") - if len(elems) != 2: - continue - if elems[0] in removes: - excludes.append(elems[1]) - # add each modified base repo with excludepkgs set to the - # list we discovered above, in the repoclosure args - rcargs.extend(["--repofrompath", f"{hash_repo(mrepo)},{mrepo}"]) - # there is a theoretical risk we might exceed MAX_ARG_STRLEN - # here, which is usually 131072: - # https://stackoverflow.com/a/29802900/4460661 - # still, 131072 is enough for 2621 50-character NEVRs... - rcargs.extend(["--setopt", f"{hash_repo(mrepo)}.excludepkgs={','.join(excludes)}"]) - - # now add the non-modified base repos and new package repos - for repo in nmbaserepos + newrepos: - rcargs.extend(["--repofrompath", f"{hash_repo(repo)},{repo}"]) - - # finally, add the check arg - rcargs.append("--check") - rcargs.append(",".join([hash_repo(baserepo) for baserepo in baserepos])) - - # get the modified repoclosure - mod = SUBPCAPTURE(rcargs).stdout - new = "" - if newrepos: - # the first newrepo is the checked repo, only check that - rcargs[-1] = hash_repo(newrepos[0]) - # get the new repoclosure - new = SUBPCAPTURE(rcargs).stdout - return (mod, new) - - -def get_source_packages(repos: Iterable[str]) -> set[str]: - """Finds and returns the source package names for all packages in - the repositories specified, as a set. - """ - args = list(DNFARGS) - for repo in repos: - args.extend(["--repofrompath", f"{hash_repo(repo)},{repo}"]) - args.extend(["repoquery", "--qf", "%{sourcerpm} "]) - srpms = SUBPCAPTCHECK(args).stdout.split() - return {srpm.rsplit("-", 2)[0] for srpm in srpms} - - -def handle_preexisting(fixederrors: list[DepTuple], newrc: list[DepTuple]) -> list[DepTuple]: - """Catch cases where an error 'moved' from baserc to newrc - a - package in the set under test previously had a dependency issue, - and the new build does not fix it. We should not report this as a - fixed issue in the old repo and a new issue in the repo under - test. Note this modifies the passed lists in-place. See: - https://forge.fedoraproject.org/quality/rmdepcheck/issues/17 - """ - preexisting = [] - if fixederrors and newrc: - # iterate over copies so we can modify originals on the fly - fecopy = fixederrors.copy() - nrcopy = newrc.copy() - for fixeddep in fecopy: - # see if we can find a broken dep in newrc - the set of - # errors in the repository under test - that exactly - # matches this one, and is for the same package name - newmatch = [ - newdep - for newdep in nrcopy - if newdep[0].rsplit("-", 2)[0] == fixeddep[0].rsplit("-", 2)[0] - and newdep[2] == fixeddep[2] - ] - if len(newmatch) == 1: - # if so, remove the dep from both lists and add it - # to the new return list so we report it correctly - fixederrors.remove(fixeddep) - newrc.remove(newmatch[0]) - preexisting.append(newmatch[0]) - return preexisting - - -def url_check(arg: str) -> str: - """Check arg is a file, http or https URL. Automatically converts - local paths to file:// URIs.""" - # If it's an existing local path, convert to file:// URI - if os.path.exists(arg): - return f"file://{os.path.abspath(arg)}" - - parsed = urlparse(arg) - if parsed.scheme in ("http", "https", "file"): - return arg - if parsed.scheme: - raise ValueError(f"Unsupported URL scheme {parsed.scheme} in {arg}") - raise ValueError(f"No URL scheme in {arg}") - - -def comma_url(arg: str) -> list[str]: - """Check arg is a comma-separated list of URLs and return them - all as a list. If arg is the empty string, return empty list. - """ - if arg == "": - return [] - split = arg.split(",") - for item in split: - try: - url_check(item) - except ValueError as err: - newerr = str(err) + f"from {arg}" - raise ValueError(newerr) from err - return split - - -def comma_list(arg: str) -> list[str]: - """Handle a comma-separated list, return as a list.""" - if arg == "": - return [] - return arg.split(",") - - -def check_arch(arg: str) -> str: - """Use host arch if no value passed.""" - if arg: - return arg - return platform.machine() - - -def parse_args() -> argparse.Namespace: - """Parse arguments with argparse.""" - parser = argparse.ArgumentParser( - description=("Reverse dependency check implemented as a repoclosure diff.") - ) - parser.add_argument( - "--addrepos", - type=comma_url, - default="", - help="The URL(s) of additional repositories containing new packages to be tested " - "(comma-separated). This is mainly intended for multilib cases: i.e. for testing " - "x86_64 package sets it should contain the matching i686 packages. It will be " - "available to the repoclosure check, but will be ignored by the installability check", - ) - parser.add_argument( - "--ncbaserepos", - type=comma_url, - default="", - help="The URL(s) of non-checked base repositories to compare against (comma-separated). " - "These repositories *will* be modified as part of testing, but they will not be checked " - "for repoclosure. They should be repositories whose packages would be replaced with those " - "from the new package repo(s), and which we want to be available to the dependency solver " - "when checking the repoclosure of other repositories, but which we are for some reason not " - "interested in checking the repoclosure of - e.g. the buildroot for Fedora ELN", - ) - parser.add_argument( - "--nmbaserepos", - type=comma_url, - default="", - help="The URL(s) of non-modified base repositories to compare against (comma-separated). " - "These repositories *will not* be modified as part of testing. They should be repositories " - "whose packages would *not* be replaced with those from the new package repo(s) - e.g. " - "the frozen release repository for a stable release, which is never changed", - ) - parser.add_argument( - "--json", - action="store_true", - help="Output results in JSON rather than human-readable format", - ) - parser.add_argument( - "--onlyerrors", - action="store_true", - help="Only print messages about problems caused, not about things that would be fixed", - ) - parser.add_argument( - "--removes", - action="store_true", - help="Alternative mode: test removal (only) of all binary packages in the baserepos built " - "from the source package(s) specified as the final argument (a comma-separated list)", - ) - parser.add_argument( - "--arch", - type=check_arch, - default="", - help="Arch to operate on. Must match arch of passed repositories. Defaults to host arch", - ) - parser.add_argument( - "baserepos", - type=comma_url, - help="The URL(s) of base repositories to compare against (comma-separated). " - "These repositories *will* be modified as part of testing. They should be " - "repositories whose packages would be replaced with those from the new package " - "repo(s) - e.g. the main repository for a development release, or the updates repository " - "for a stable release", - ) - # I wanted to do this with parse_known_args, but it messes up --help. aw - if "--removes" in sys.argv: - parser.add_argument( - "removes", - type=comma_list, - help="A comma-separated list of source packages to test the removal of", - ) - else: - parser.add_argument( - "repo", - metavar="repo_or_removes", - type=url_check, - help="The URL of the repo containing the main set of new packages to be tested, " - "or a comma-separated list of source packages to test the removal of (if --removes " - "is passed)", - ) - args = parser.parse_args() - if args.removes: - args.repo = "" - else: - args.removes = "" - return args - - -def check_dnf() -> None: - """Check DNF is installed.""" - try: - subprocess.run(("dnf", "--version"), stdout=subprocess.DEVNULL, check=True) - except FileNotFoundError: - sys.exit("Please install missing required utilities: dnf") - - -def main() -> None: - """Main loop.""" - try: - check_dnf() - exitcode = 0 - args = parse_args() - if f"--forcearch={args.arch}" not in DNFARGS: - DNFARGS.append(f"--forcearch={args.arch}") - if args.removes: - newrepos = [] - sources = args.removes - iut = "the specified source package removals" - else: - newrepos = [args.repo] + args.addrepos - # find source package(s) of our tested repo(s) - # whether to include addrepos is arguable, but should usually be moot - sources = get_source_packages(newrepos) - iut = "the tested packages" - - baseraw = get_base_repoclosure(args.baserepos, args.ncbaserepos, args.nmbaserepos) - baserc = parse_repoclosure(baseraw) - - # get the modified rpmclosure output - modraw, newraw = get_modified_and_new_repoclosure( - args.baserepos, args.ncbaserepos, args.nmbaserepos, newrepos, sources - ) - modrc = parse_repoclosure(modraw) - newrc = [] - if newraw: - newrc = parse_repoclosure(newraw) - - newerrors = [dep for dep in modrc if dep not in baserc] - fixederrors = [dep for dep in baserc if dep not in modrc] - preexisting = handle_preexisting(fixederrors, newrc) - - # output - if args.json: - jsonout = {} - if newerrors: - if args.json: - jsonout["newerrors"] = [list(err) for err in newerrors] - else: - print(f"Dependencies of other packages that would be BROKEN by {iut}:") - format_rc_errors(newerrors) - exitcode += 1 - if newrc: - if args.json: - jsonout["installability"] = [list(err) for err in newrc] - else: - print("") - print("New dependency problems in the tested packages themselves:") - format_rc_errors(newrc) - exitcode += 2 - if preexisting: - if args.json: - jsonout["installability_preexisting"] = [list(err) for err in preexisting] - else: - print("") - print("Pre-existing dependency problems in the tested packages themselves:") - format_rc_errors(preexisting) - exitcode += 4 - if fixederrors: - if args.json: - jsonout["fixederrors"] = [list(err) for err in fixederrors] - elif not args.onlyerrors: - print("") - print(f"Dependencies of other packages that would be FIXED by {iut}:") - format_rc_errors(fixederrors) - if args.json: - json.dump(jsonout, sys.stdout, indent=4) - sys.stdout.write("\n") - - sys.exit(exitcode) - - except KeyboardInterrupt: - sys.stderr.write("Interrupted, exiting...\n") - sys.exit(1) - - -if __name__ == "__main__": # pragma: no cover - main() - -# vim: set textwidth=100 ts=8 et sw=4: diff --git a/rmdepcheck.py b/rmdepcheck.py new file mode 120000 index 0000000..e033b75 --- /dev/null +++ b/rmdepcheck.py @@ -0,0 +1 @@ +rdc-repos \ No newline at end of file diff --git a/src/rmdepcheck/__init__.py b/src/rmdepcheck/__init__.py new file mode 100644 index 0000000..ae9f5b5 --- /dev/null +++ b/src/rmdepcheck/__init__.py @@ -0,0 +1,29 @@ +#!/usr/bin/python3 + +# Copyright Red Hat +# +# This file is part of rmdepcheck. +# +# rmdepcheck is free software; you can redistribute it +# and/or modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation, either version 3 of +# the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# Author(s): Adam Williamson + + +"""RPM package installability and reverse-dependency checks using a +repository modification strategy (hence 'rm'). +""" + +__version__ = "1.0.0" + +# vim: set textwidth=100 ts=8 et sw=4: diff --git a/src/rmdepcheck/compose.py b/src/rmdepcheck/compose.py new file mode 100644 index 0000000..31c2b32 --- /dev/null +++ b/src/rmdepcheck/compose.py @@ -0,0 +1,279 @@ +# Copyright Red Hat +# +# This file is part of rmdepcheck. +# +# rmdepcheck is free software; you can redistribute it +# and/or modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation, either version 3 of +# the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# Author(s): Adam Williamson + + +"""RPM package installability and reverse-dependency checks using a +repository modification strategy (hence 'rm'). This module contains +the rdc-compose CLI which operates against composes and contains the +logic for splitting and testing variants appropriately. +""" + +# Standard libraries + +import argparse +import ast +import os +import pathlib +import re +import shutil +import subprocess +import sys +import tempfile + +from requests import Session +from requests.adapters import HTTPAdapter +from urllib3.util.retry import Retry + +from rmdepcheck.shared import ( + SUBPCAPTURE, + get_dnf_args, + check_arch, + url_check, + check_utils, + get_source_packages, + get_base_repoclosure, + get_modified_and_new_repoclosure, + parse_display_exit, +) + +VrDict = dict[str, str] + + +def split_put( + put: pathlib.Path, repotemp: pathlib.Path, vrs: VrDict, dnfargs: list[str] +) -> set[str]: + """Splits a directory containing the packages to be tested into + multiple repositories per variant. put ('packages under test') + is the filesystem path containing packages under test (it cannot + be an HTTP URL, on this path). repotemp is a Path to a temporary + directory to store the split repos. vrs is a dict with variant IDs + as keys and full compose repo URLs as values. For each package + under test, we get its name, look for a package of the same name + in each of the variant repositories, and if there is one, we put + the package in a subdirectory of repotemp, named for the variant. + At the end, we run createrepo in each of the subdirectories we + created, and return their names. + """ + if not (put.exists() and put.is_dir()): + raise ValueError(f"split_put: invalid put value {put}") + put = put.absolute() + # generate per-variant package name lists + pkgs = {} + for variant, repo in vrs.items(): + args = dnfargs + [ + "--repofrompath", + f"{variant},{repo}", + "repoquery", + "--queryformat", + "%{name}\n", + ] + res = SUBPCAPTURE(args) + res.check_returncode() + pkgs[variant] = res.stdout.splitlines() + # track which variants we 'populate' + populated = set() + for _file in os.listdir(put): + if _file.endswith(".rpm"): + pkgname = _file.rsplit("-", 2)[0] + for variant, pkgnames in pkgs.items(): + if pkgname in pkgnames: + populated.add(variant) + os.makedirs(repotemp / variant, exist_ok=True) + try: + # hardlink + os.link(put / _file, repotemp / variant / _file) + except OSError: + # copy + shutil.copy2(put / _file, repotemp / variant / _file) + for variant in populated: + subprocess.run(("createrepo", str(repotemp / variant)), capture_output=True, check=True) + return populated + + +def get_variant_repos(compose: str, ci: dict, arch: str) -> VrDict: + """Discover variants with repositories in a compose, return a + dict with variant IDs as keys and repo URLs as values. + """ + variants = ci.get("payload", {}).get("variants", {}) + ret = {} + for variant in variants: + vid = variants[variant].get("id", variant) + # as of 2026-05 yselkowitz says he does not want to know about + # buildroot-only dep issues + if vid.lower() == "buildroot": + continue + repopath = variants[variant].get("paths", {}).get("repository", {}).get(arch, "") + if not repopath: + continue + ret[vid] = f"{compose}/{repopath}" + return ret + + +def pungi_config_url(ci: dict) -> str: + """Return the URL with the appropriate Pungi config for the + release. Here be magic knowledge. + """ + version = ci.get("payload", {}).get("release", {}).get("version", "") + short = ci.get("payload", {}).get("release", {}).get("short", "") + if version.lower() == "eln" and short.lower() == "fedora": + # pylint: disable-next=line-too-long + return "https://forge.fedoraproject.org/releng/pungi-fedora/raw/branch/eln/fedora/override.conf" + raise ValueError(f"Don't know Pungi config URL for {short} {version}") + + +def get_val(ci: dict, session: Session) -> dict: + """Get the variants_as_lookaside list which tells us which + variants pull in which other variants. + """ + url = pungi_config_url(ci) + resp = session.get(url) + resp.raise_for_status() + override = resp.text + # next three lines suggested by AI + match = re.search(r"variant_as_lookaside\s*=\s*(\[.*?\])", override, re.DOTALL) + if not match: + raise ValueError(f"Could not find variant_as_lookaside in configuration at {url}") + # eek + return ast.literal_eval(match.group(1)) + + +def url_check_http(arg: str) -> str: + """Check arg is an http or https URL.""" + return url_check(arg, fileok=False) + + +def parse_args() -> argparse.Namespace: + """Parse arguments with argparse.""" + parser = argparse.ArgumentParser( + description="Reverse dependency check implemented as a repoclosure diff. Checks against " + "a compose. Intended for Enterprise Linux use cases." + ) + parser.add_argument( + "compose", + type=url_check_http, + help="The URL(s) of a compose to discover variant repos from and check against", + ) + parser.add_argument( + "repo", + type=pathlib.Path, + help="Local filesystem path containing packages to test", + ) + parser.add_argument( + "--arch", + type=check_arch, + default="", + help="Arch to operate on. Must match arch of passed repository", + ) + parser.add_argument( + "--json", + action="store_true", + help="Output results in JSON rather than human-readable format", + ) + return parser.parse_args() + + +def main() -> None: # pylint: disable=too-many-locals + """Main loop.""" + try: + # set up requests session with retries + session = Session() + retries = Retry( + total=5, + backoff_factor=0.5, + status_forcelist=[502, 503, 504], + allowed_methods={"GET"}, + ) + session.mount("https://", HTTPAdapter(max_retries=retries)) + + # check DNF and createrepo are installed + check_utils((("dnf", "--version"), ("createrepo", "--version"))) + + # set up args + args = parse_args() + + # repoclosure output strings + basercs = "" + modrcs = "" + newrcs = "" + # get composeinfo + resp = session.get(f"{args.compose}/metadata/composeinfo.json") + resp.raise_for_status() + ci = resp.json() + # find variants, do the repo split + vrs = get_variant_repos(args.compose, ci, args.arch) + val = get_val(ci, session) + with ( + tempfile.TemporaryDirectory(prefix="rdccomposerepo", dir="/var/tmp") as repotd, + tempfile.TemporaryDirectory(prefix="rdccomposednf", dir="/var/tmp") as dnftemp, + ): + dnfargs = get_dnf_args(dnftemp, args.arch) + repotemp = pathlib.Path(repotd) + # this is "all the variants with packages in the new repo" + variants = split_put(args.repo, repotemp, vrs, dnfargs) + # we have to check all variants we have new packages in + # *and* all variants that depend on them + tocheck = set(variants) + for pair in val: + if pair[0] in variants: + tocheck.add(pair[1]) + + for variant in tocheck: + baserepo = vrs[variant] + # figure out what other variants to pull in per val + depvars = set() + for pair in val: + if pair[0] == variant: + depvars.add(pair[1]) + # put the compose repos for all depended-on variants in + # ncbaserepos, even ones we didn't 'populate' locally + ncbaserepos = [vrs[var] for var in depvars] + newrepo = "" + newrepos = [] + if variant in variants: + # first newrepo is the PUT repo for this variant, + # *if* it exists + newrepo = str(repotemp / variant) + newrepos.append(newrepo) + # additional repos are depended-on variant-split PUT + # repos that actually exist + # variants contains only variants we 'populated' + addrepos = [str(repotemp / var) for var in depvars if var in variants] + newrepos.extend(addrepos) + + baserc = get_base_repoclosure([baserepo], ncbaserepos, [], dnfargs) + basercs = f"{basercs}\n{baserc}" + + # get the modified rpmclosure output + sources = get_source_packages(newrepos, dnfargs) + modrc, newrc = get_modified_and_new_repoclosure( + [baserepo], ncbaserepos, [], newrepo, addrepos, sources, dnfargs + ) + modrcs = f"{modrcs}\n{modrc}" + if newrc: + newrcs = f"{newrcs}\n{newrc}" + + # output + parse_display_exit(basercs, modrcs, newrcs, "the tested packages", args.json, False, True) + + except KeyboardInterrupt: + sys.stderr.write("Interrupted, exiting...\n") + sys.exit(1) + + +# vim: set textwidth=100 ts=8 et sw=4: diff --git a/src/rmdepcheck/repos.py b/src/rmdepcheck/repos.py new file mode 100644 index 0000000..661c1a8 --- /dev/null +++ b/src/rmdepcheck/repos.py @@ -0,0 +1,204 @@ +# Copyright Red Hat +# +# This file is part of rmdepcheck. +# +# rmdepcheck is free software; you can redistribute it +# and/or modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation, either version 3 of +# the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# Author(s): Adam Williamson + + +"""RPM package installability and reverse-dependency checks using a +repository modification strategy (hence 'rm'). This module contains +the rdc-repos CLI which operates against repositories. +""" + +# Standard libraries + +import argparse +import sys +import tempfile + +from rmdepcheck.shared import ( + get_dnf_args, + check_arch, + url_check, + check_utils, + get_source_packages, + get_base_repoclosure, + get_modified_and_new_repoclosure, + parse_display_exit, +) + + +def url_check_file(arg: str) -> str: + """Check arg is a file, http or https URL. Automatically converts + local paths to file:// URIs.""" + return url_check(arg, fileok=True) + + +def comma_url(arg: str) -> list[str]: + """Check arg is a comma-separated list of URLs and return them + all as a list. If arg is the empty string, return empty list. + """ + if arg == "": + return [] + split = arg.split(",") + for item in split: + try: + url_check_file(item) + except ValueError as err: + newerr = str(err) + f"from {arg}" + raise ValueError(newerr) from err + return split + + +def comma_list(arg: str) -> list[str]: + """Handle a comma-separated list, return as a list.""" + if arg == "": + return [] + return arg.split(",") + + +def parse_args() -> argparse.Namespace: + """Parse arguments with argparse.""" + parser = argparse.ArgumentParser( + description=("Reverse dependency check implemented as a repoclosure diff.") + ) + parser.add_argument( + "--addrepos", + type=comma_url, + default="", + help="The URL(s) of additional repositories containing new packages to be tested " + "(comma-separated). This is mainly intended for multilib cases: i.e. for testing " + "x86_64 package sets it should contain the matching i686 packages. It will be " + "available to the repoclosure check, but will be ignored by the installability check", + ) + parser.add_argument( + "--ncbaserepos", + type=comma_url, + default="", + help="The URL(s) of non-checked base repositories to compare against (comma-separated). " + "These repositories *will* be modified as part of testing, but they will not be checked " + "for repoclosure. They should be repositories whose packages would be replaced with those " + "from the new package repo(s), and which we want to be available to the dependency solver " + "when checking the repoclosure of other repositories, but which we are for some reason not " + "interested in checking the repoclosure of - e.g. the buildroot for Fedora ELN", + ) + parser.add_argument( + "--nmbaserepos", + type=comma_url, + default="", + help="The URL(s) of non-modified base repositories to compare against (comma-separated). " + "These repositories *will not* be modified as part of testing. They should be repositories " + "whose packages would *not* be replaced with those from the new package repo(s) - e.g. " + "the frozen release repository for a stable release, which is never changed", + ) + parser.add_argument( + "--json", + action="store_true", + help="Output results in JSON rather than human-readable format", + ) + parser.add_argument( + "--onlyerrors", + action="store_true", + help="Only print messages about problems caused, not about things that would be fixed", + ) + parser.add_argument( + "--removes", + action="store_true", + help="Alternative mode: test removal (only) of all binary packages in the baserepos built " + "from the source package(s) specified as the final argument (a comma-separated list)", + ) + parser.add_argument( + "--arch", + type=check_arch, + default="", + help="Arch to operate on. Must match arch of passed repositories. Defaults to host arch", + ) + parser.add_argument( + "baserepos", + type=comma_url, + help="The URL(s) of base repositories to compare against (comma-separated). " + "These repositories *will* be modified as part of testing. They should be " + "repositories whose packages would be replaced with those from the new package " + "repo(s) - e.g. the main repository for a development release, or the updates repository " + "for a stable release", + ) + # I wanted to do this with parse_known_args, but it messes up --help. aw + if "--removes" in sys.argv: + parser.add_argument( + "removes", + type=comma_list, + help="A comma-separated list of source packages to test the removal of", + ) + else: + parser.add_argument( + "repo", + metavar="repo_or_removes", + type=url_check_file, + help="The URL of the repo containing the main set of new packages to be tested, " + "or a comma-separated list of source packages to test the removal of (if --removes " + "is passed)", + ) + args = parser.parse_args() + if args.removes: + args.repo = "" + else: + args.removes = "" + return args + + +def main() -> None: + """Main loop.""" + try: + check_utils((("dnf", "--version"),)) + args = parse_args() + with tempfile.TemporaryDirectory(prefix="rdcrepos", dir="/var/tmp") as dnftemp: + dnfargs = get_dnf_args(dnftemp, args.arch) + if args.removes: + newrepo = "" + sources = args.removes + iut = "the specified source package removals" + else: + newrepo = args.repo + newrepos = [newrepo] + args.addrepos + # find source package(s) of our tested repo(s) + # whether to include addrepos is arguable, but should usually be moot + sources = get_source_packages(newrepos, dnfargs) + iut = "the tested packages" + + baseraw = get_base_repoclosure( + args.baserepos, args.ncbaserepos, args.nmbaserepos, dnfargs + ) + + # get the modified rpmclosure output + modraw, newraw = get_modified_and_new_repoclosure( + args.baserepos, + args.ncbaserepos, + args.nmbaserepos, + newrepo, + args.addrepos, + sources, + dnfargs, + ) + + # output + parse_display_exit(baseraw, modraw, newraw, iut, args.json, args.onlyerrors, False) + + except KeyboardInterrupt: + sys.stderr.write("Interrupted, exiting...\n") + sys.exit(1) + + +# vim: set textwidth=100 ts=8 et sw=4: diff --git a/src/rmdepcheck/shared.py b/src/rmdepcheck/shared.py new file mode 100644 index 0000000..888b9f0 --- /dev/null +++ b/src/rmdepcheck/shared.py @@ -0,0 +1,341 @@ +# Copyright Red Hat +# +# This file is part of rmdepcheck. +# +# rmdepcheck is free software; you can redistribute it +# and/or modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation, either version 3 of +# the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# Author(s): Adam Williamson + + +"""RPM package installability and reverse-dependency checks using a +repository modification strategy (hence 'rm') This module contains +functions and constants shared between the two main scripts. +""" + +import hashlib +import json +import os +import platform +import subprocess +import sys +from functools import partial +from typing import Iterable +from urllib.parse import urlparse + +# type alias for the tuples produced by parse_repoclosure +# can't properly declare this because type statement was only added in +# 3.12, and TypeAlias is deprecated since 3.12 and wasn't in 3.9 +DepTuple = tuple[str, str, str] + +# use a fresh temporary cache for each run to avoid collisions between +# runs and polluting the 'real' cache +# pylint: disable-next=consider-using-with +SUBPCAPTURE = partial(subprocess.run, capture_output=True, text=True, check=False) +SUBPCAPTCHECK = partial(subprocess.run, capture_output=True, text=True, check=True) +SUBPCHECK = partial(subprocess.run, check=True) # pylint: disable=invalid-name +REPOHASHES: dict[str, str] = {} + + +def get_dnf_args(cachedir: str, arch: str) -> list[str]: + """Construct our basic dnf args.""" + return [ + "dnf", + "--setopt", + f"cachedir={cachedir}", + "-q", + "--disablerepo=*", + f"--forcearch={arch}", + ] + + +def check_arch(arg: str) -> str: + """Use host arch if no value passed.""" + if arg: + return arg + return platform.machine() + + +def url_check(arg: str, fileok: bool) -> str: + """Check arg is a file, http or https URL. Automatically converts + local paths to file:// URIs.""" + # If it's an existing local path, convert to file:// URI + if os.path.exists(arg): + if fileok: + return f"file://{os.path.abspath(arg)}" + raise ValueError("Value must be an HTTP or HTTPS URL") + + parsed = urlparse(arg) + schemes = ["http", "https"] + if fileok: + schemes.append("file") + if parsed.scheme in schemes: + return arg + if parsed.scheme: + raise ValueError(f"Unsupported URL scheme {parsed.scheme} in {arg}") + raise ValueError(f"No URL scheme in {arg}") + + +def check_utils(utils: tuple[tuple[str, ...], ...]) -> None: + """Check required utilities are installed.""" + missing = [] + for prog in utils: + try: + subprocess.run(prog, stdout=subprocess.DEVNULL, check=True) + except FileNotFoundError: + missing.append(prog[0]) + if missing: + sys.exit("Please install missing required utilities: " + " ".join(missing)) + + +def get_source_packages(repos: Iterable[str], dnfargs: list[str]) -> set[str]: + """Finds and returns the source package names for all packages in + the repositories specified, as a set. + """ + gspargs = list(dnfargs) + for repo in repos: + gspargs.extend(["--repofrompath", f"{hash_repo(repo)},{repo}"]) + gspargs.extend(["repoquery", "--qf", "%{sourcerpm} "]) + srpms = SUBPCAPTCHECK(gspargs).stdout.split() + return {srpm.rsplit("-", 2)[0] for srpm in srpms} + + +def hash_repo(repo: str) -> str: + """Generate a hash for the repo name, stash it in a dict so we can + map back out later, and return it. This is so we can show the repo + URLs in our final output, as opposed to non-useful made-up repo + names. Not security sensitive. + """ + gothash = hashlib.sha256(repo.encode(encoding="utf-8")).hexdigest()[:8] + REPOHASHES[gothash] = repo + return gothash + + +def parse_repoclosure(rc: str) -> list[DepTuple]: + """Given some `dnf repoclosure` output, parse it into a list of + 3-tuples each containing a package name, a repo URL (or generated + repo name if we can't look up the hash, should only happen in + tests) and an unresolved dependency for that package. + """ + out = [] + pkg = None + for line in rc.splitlines(): + if not line.strip(): + continue + if line.strip().startswith("package:"): + # package, repo + elems = line.split() + pkg = (elems[1], REPOHASHES.get(elems[3], elems[3])) + continue + if line.strip().startswith("unresolved deps (") or line.strip().startswith("Error:"): + continue + # anything else is an unresolved dep + if pkg: + out.append(pkg + (line.strip(),)) + return out + + +def format_rc_errors(errors: list[DepTuple], form: str = "pass") -> None: + """Format and print parse_repoclosure-style tuples for humans to + read. Used for final output after we do some diffing on the lists + of tuples. + """ + pkg = ("", "") + for error in errors: + if error[:2] != pkg: + pkg = error[:2] + if form == "split": + print(f"package: {error[0]} from {error[1].split('/')[-1]}") + elif form == "strip": + print(f"package: {error[0]}") + else: + print(f"package: {error[0]} from {error[1]}") + print(f" {error[2]}") + + +def get_base_repoclosure( + baserepos: Iterable[str], + ncbaserepos: Iterable[str], + nmbaserepos: Iterable[str], + dnfargs: list[str], +) -> str: + """Gets the reference repoclosure text. Base repos, non-checked + base repos and non-modified base repos are available to the + solver, but only the base repos are checked. + """ + cmdargs = dnfargs + ["repoclosure"] + for repo in list(baserepos) + list(nmbaserepos) + list(ncbaserepos): + cmdargs.extend(["--repofrompath", f"{hash_repo(repo)},{repo}"]) + cmdargs.append("--check") + # only check the repos that will be modified + cmdargs.append(",".join([hash_repo(baserepo) for baserepo in baserepos])) + return SUBPCAPTURE(cmdargs).stdout + + +# pylint: disable-next=too-many-arguments,too-many-locals,too-many-positional-arguments +def get_modified_and_new_repoclosure( + baserepos: list[str], + ncbaserepos: list[str], + nmbaserepos: list[str], + newrepo: str, + addrepos: list[str], + removes: Iterable[str], + dnfargs: list[str], +) -> tuple[str, str]: + """Runs repoclosure with new repo included and excludepkgs used + for modified base repos, and returns the modified repoclosure + text. Non-modified base repos, modified base repos and non-checked + base repos after modification, and new repos are available to + the solver; only the base repos are checked in the "modified" + check and only the first new repo is checked in the "new" check. + """ + queryargs = dnfargs + ["repoquery", "--queryformat", "%{source_name},%{full_nevra}\n"] + rcargs = dnfargs + ["repoclosure"] + for mrepo in baserepos + ncbaserepos: + # figure out what to exclude + excludes = [] + args = queryargs + ["--repofrompath", f"{hash_repo(mrepo)},{mrepo}"] + for line in SUBPCAPTURE(args).stdout.splitlines(): + # sname, nevr + elems = line.split(",") + if len(elems) != 2: + continue + if elems[0] in removes: + excludes.append(elems[1]) + # add each modified base repo with excludepkgs set to the + # list we discovered above, in the repoclosure args + rcargs.extend(["--repofrompath", f"{hash_repo(mrepo)},{mrepo}"]) + # there is a theoretical risk we might exceed MAX_ARG_STRLEN + # here, which is usually 131072: + # https://stackoverflow.com/a/29802900/4460661 + # still, 131072 is enough for 2621 50-character NEVRs... + rcargs.extend(["--setopt", f"{hash_repo(mrepo)}.excludepkgs={','.join(excludes)}"]) + + # now add the non-modified base repos and new package repos + for repo in nmbaserepos + [nr for nr in [newrepo] if nr] + addrepos: + rcargs.extend(["--repofrompath", f"{hash_repo(repo)},{repo}"]) + + # finally, add the check arg + rcargs.append("--check") + rcargs.append(",".join([hash_repo(baserepo) for baserepo in baserepos])) + + # get the modified repoclosure + mod = SUBPCAPTURE(rcargs).stdout + new = "" + if newrepo: + rcargs[-1] = hash_repo(newrepo) + # get the new repoclosure + new = SUBPCAPTURE(rcargs).stdout + return (mod, new) + + +def handle_preexisting(fixederrors: list[DepTuple], newrc: list[DepTuple]) -> list[DepTuple]: + """Catch cases where an error 'moved' from baserc to newrc - a + package in the set under test previously had a dependency issue, + and the new build does not fix it. We should not report this as a + fixed issue in the old repo and a new issue in the repo under + test. Note this modifies the passed lists in-place. See: + https://forge.fedoraproject.org/quality/rmdepcheck/issues/17 + """ + preexisting = [] + if fixederrors and newrc: + # iterate over copies so we can modify originals on the fly + fecopy = fixederrors.copy() + nrcopy = newrc.copy() + for fixeddep in fecopy: + # see if we can find a broken dep in newrc - the set of + # errors in the repository under test - that exactly + # matches this one, and is for the same package name + newmatch = [ + newdep + for newdep in nrcopy + if newdep[0].rsplit("-", 2)[0] == fixeddep[0].rsplit("-", 2)[0] + and newdep[2] == fixeddep[2] + ] + if len(newmatch) == 1: + # if so, remove the dep from both lists and add it + # to the new return list so we report it correctly + fixederrors.remove(fixeddep) + newrc.remove(newmatch[0]) + preexisting.append(newmatch[0]) + return preexisting + + +# pylint: disable-next=too-many-branches,too-many-arguments,too-many-positional-arguments +def parse_display_exit( + baseraw: str, + modraw: str, + newraw: str, + iut: str, + dojson: bool, + onlyerrors: bool, + splitrepo: bool, +) -> None: + """Take the raw rmdepcheck output strings, parse them, produce + human-readable or JSON output, and exit with an appropriate code. + """ + exitcode = 0 + baserc = parse_repoclosure(baseraw) + modrc = parse_repoclosure(modraw) + newrc = [] + if newraw: + newrc = parse_repoclosure(newraw) + + newerrors = [dep for dep in modrc if dep not in baserc] + fixederrors = [dep for dep in baserc if dep not in modrc] + preexisting = handle_preexisting(fixederrors, newrc) + + # output + if dojson: + jsonout = {} + if newerrors: + if dojson: + jsonout["newerrors"] = [list(err) for err in newerrors] + else: + print(f"Dependencies of other packages that would be BROKEN by {iut}:") + format_rc_errors(newerrors) + exitcode += 1 + if newrc: + if dojson: + jsonout["installability"] = [list(err) for err in newrc] + else: + print("") + print("New dependency problems in the tested packages themselves:") + if splitrepo: + format_rc_errors(newrc, form="split") + else: + format_rc_errors(newrc, form="strip") + exitcode += 2 + if preexisting: + if dojson: + jsonout["installability_preexisting"] = [list(err) for err in preexisting] + else: + print("") + print("Pre-existing dependency problems in the tested packages themselves:") + format_rc_errors(preexisting) + exitcode += 4 + if fixederrors: + if dojson: + jsonout["fixederrors"] = [list(err) for err in fixederrors] + elif not onlyerrors: + print("") + print(f"Dependencies of other packages that would be FIXED by {iut}:") + format_rc_errors(fixederrors) + if dojson: + json.dump(jsonout, sys.stdout, indent=4) + sys.stdout.write("\n") + + sys.exit(exitcode) + + +# vim: set textwidth=100 ts=8 et sw=4: diff --git a/tests/conftest.py b/tests/conftest.py new file mode 100644 index 0000000..df26b1c --- /dev/null +++ b/tests/conftest.py @@ -0,0 +1,52 @@ +# Copyright Red Hat +# +# This file is part of rmdepcheck. +# +# rmdepcheck is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# Author: Adam Williamson + +"""Test configuration and fixtures.""" + +import os +import subprocess +import time +from urllib.error import URLError, HTTPError +from urllib.request import urlopen + +import pytest + + +@pytest.yield_fixture(scope="session") +def http(request): # pylint: disable=unused-argument + """Run a SimpleHTTPServer that sits in front of our mock ELN + compose. We just do this with subprocess as we need it to run + parallel to the tests and this is really the easiest way. Note + we also stash override.conf in the compose dir, it's just + convenient that way. + """ + root = os.path.join(os.path.dirname(os.path.realpath(__file__)), "testdata", "repos", "eln") + args = ("python3", "-m", "http.server", "5001") + proc = subprocess.Popen(args, cwd=root) # pylint: disable=consider-using-with + # block until the server is actually running + resp = None + while not resp: + try: + resp = urlopen("http://localhost:5001/BaseOS") # pylint: disable=consider-using-with + except (ValueError, URLError, HTTPError): + time.sleep(0.1) + yield + + # teardown + proc.kill() diff --git a/tests/test_rmdepcheck.py b/tests/test_rmdepcheck.py index 0a0906e..b2ab741 100644 --- a/tests/test_rmdepcheck.py +++ b/tests/test_rmdepcheck.py @@ -23,296 +23,414 @@ """Tests for rmdepcheck.""" import os +import pathlib import sys +import tempfile from unittest import mock +from rmdepcheck.shared import ( + get_dnf_args, + check_arch, + url_check, + check_utils, + get_source_packages, + hash_repo, + parse_repoclosure, + format_rc_errors, + get_base_repoclosure, + get_modified_and_new_repoclosure, + handle_preexisting, +) +from rmdepcheck import repos, compose + import pytest -import rmdepcheck HERE = os.path.abspath(os.path.dirname(__file__)) TESTDATA = f"{HERE}/testdata" REPOS = f"{TESTDATA}/repos" -def test_parse_repoclosure(): - with open(f"{TESTDATA}/test_parse_repoclosure.txt", "r", encoding="utf-8") as fh: - rctext = fh.read() - assert rmdepcheck.parse_repoclosure(rctext) == [ - ("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python(abi) = 3.13"), - ("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(wxpython) >= 4"), - ("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(xnat) >= 0.3.3"), - ("python3-x3dh-1.0.4-3.fc43.noarch", "baserepo1", "python3.14dist(pydantic) >= 1.7.4"), - ] +class TestShared: + """Unit tests for things from rmdepcheck.shared.""" + + def test_parse_repoclosure(self): + with open(f"{TESTDATA}/test_parse_repoclosure.txt", "r", encoding="utf-8") as fh: + rctext = fh.read() + assert parse_repoclosure(rctext) == [ + ("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python(abi) = 3.13"), + ("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(wxpython) >= 4"), + ("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(xnat) >= 0.3.3"), + ("python3-x3dh-1.0.4-3.fc43.noarch", "baserepo1", "python3.14dist(pydantic) >= 1.7.4"), + ] + + def test_format_rc_errors(self, capsys): + errs = [ + ("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python(abi) = 3.13"), + ("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(wxpython) >= 4"), + ("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(xnat) >= 0.3.3"), + ("python3-x3dh-1.0.4-3.fc43.noarch", "baserepo1", "python3.14dist(pydantic) >= 1.7.4"), + ] + format_rc_errors(errs) + captured = capsys.readouterr() + with open(f"{TESTDATA}/test_format_rc_errors.txt", "r", encoding="utf-8") as fh: + exptext = fh.read() + assert captured.out == exptext + + def test_get_base_repoclosure(self): + repo = f"file://{REPOS}/base" + with open(f"{TESTDATA}/test_get_base_repoclosure.txt", "r", encoding="utf-8") as testfh: + expected = testfh.read() + expected = expected.replace("{HASH}", hash_repo(repo)) + with tempfile.TemporaryDirectory(prefix="testrdc", dir="/var/tmp") as dnftemp: + dnfargs = get_dnf_args(dnftemp, "x86_64") + ret = get_base_repoclosure([repo], [], [], dnfargs) + assert ret == expected + + def test_get_modified_and_new_repoclosure(self): + brepo = f"file://{REPOS}/base" + nrepo = f"file://{REPOS}/new" + with open(f"{TESTDATA}/test_get_modified_repoclosure.txt", "r", encoding="utf-8") as testfh: + expectedmod = testfh.read() + expectedmod = expectedmod.replace("{HASH}", hash_repo(brepo)) + with open(f"{TESTDATA}/test_get_new_repoclosure.txt", "r", encoding="utf-8") as testfh: + expectednew = testfh.read() + expectednew = expectednew.replace("{HASH}", hash_repo(nrepo)) + with tempfile.TemporaryDirectory(prefix="testrdc", dir="/var/tmp") as dnftemp: + dnfargs = get_dnf_args(dnftemp, "x86_64") + ret = get_modified_and_new_repoclosure( + [brepo], [], [], nrepo, [], ["aaa", "ccc", "eee", "fff", "ggg"], dnfargs + ) + assert ret == (expectedmod, expectednew) + + def test_get_modified_and_new_repoclosure_commasafe(self): + """Check we survive repoquery output lines that are, precisely, + a string from 'removes'. This is a pretty unlikely scenario, but + hey. Any other line with no commas is actually safe because it + will fail the `if elems[0] in removes` check. + """ + qamock = mock.Mock() + qamock.stdout = "ccc\nbbb,bbb-0:1.0-1.x86_64\nccc,ccc-0:1.0-1.x86_64" + rcmock = mock.Mock() + rcmock.stdout = "" + with mock.patch("rmdepcheck.shared.SUBPCAPTURE", side_effect=[qamock, rcmock]): + with tempfile.TemporaryDirectory(prefix="testrdc", dir="/var/tmp") as dnftemp: + dnfargs = get_dnf_args(dnftemp, "x86_64") + # this shouldn't raise an exception + get_modified_and_new_repoclosure( + ["file:///foo/bar"], [], [], "", [], ["ccc"], dnfargs + ) + + def test_get_source_packages(self): + with tempfile.TemporaryDirectory(prefix="testrdc", dir="/var/tmp") as dnftemp: + dnfargs = get_dnf_args(dnftemp, "x86_64") + sources = get_source_packages([f"file://{REPOS}/new"], dnfargs) + assert sources == {"111", "222", "aaa", "ccc", "eee", "fff", "ggg"} + + def test_handle_preexisting(self): + fixederrors = [ + ("libreoffice-langpack-lv-1:26.2.2.1-0.1.eln155.x86_64", "ogrepo", "mythes-lv"), + ("libreoffice-langpack-nb-1:26.2.2.1-0.1.eln155.x86_64", "ogrepo", "mythes-nb"), + ("libreoffice-langpack-nn-1:26.2.2.1-0.1.eln155.x86_64", "ogrepo", "mythes-nn"), + ("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "someotherdep"), + ] + newrc = [ + ("libreoffice-langpack-lv-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-lv"), + ("libreoffice-langpack-nb-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nb"), + ("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nn"), + ("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "foo"), + ("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "mythes-lv"), + ] + # this should null out the 'matching' errors (the first three in + # each list), but leave the 'non-matching' ones (the others) + ret = handle_preexisting(fixederrors, newrc) + assert fixederrors == [ + ("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "someotherdep"), + ] + assert newrc == [ + ("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "foo"), + ("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "mythes-lv"), + ] + # we should return the errors dropped from 'newrc' (so these can + # be reported separately) + assert ret == [ + ("libreoffice-langpack-lv-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-lv"), + ("libreoffice-langpack-nb-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nb"), + ("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nn"), + ] + + def test_url_check(self): + url_check("file:///foo/bar", fileok=True) + with pytest.raises(ValueError): + url_check("file:///foo/bar", fileok=False) + url_check("https://www.some.where", fileok=True) + url_check("http://some.where.insecure", fileok=True) + # Test automatic conversion of existing local paths to file:// URIs + result = url_check(REPOS, fileok=True) + assert result == f"file://{REPOS}" + with pytest.raises(ValueError): + url_check(REPOS, fileok=False) + # Test with relative path that exists + result = url_check("tests", fileok=True) + assert result.startswith("file://") + assert result.endswith("/tests") + with pytest.raises(ValueError): + url_check("ftp://1997.called", fileok=True) + # Non-existent paths without scheme should still raise ValueError + with pytest.raises(ValueError): + url_check("whatisthis", fileok=True) + + @mock.patch("platform.machine", return_value="foobar") + def test_check_arch(self, _): + assert check_arch("x86_64") == "x86_64" + assert check_arch("") == "foobar" + + @mock.patch("subprocess.run", autospec=True) + def test_check_utils(self, mock_run): + check_utils((("dnf", "--version"),)) + mock_run.side_effect = FileNotFoundError + with pytest.raises(SystemExit) as excinfo: + check_utils((("dnf", "--version"),)) + assert excinfo.value.code == "Please install missing required utilities: dnf" -def test_format_rc_errors(capsys): - errs = [ - ("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python(abi) = 3.13"), - ("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(wxpython) >= 4"), - ("python3-wxnatpy-0.4.0-13.fc42.noarch", "baserepo0", "python3.13dist(xnat) >= 0.3.3"), - ("python3-x3dh-1.0.4-3.fc43.noarch", "baserepo1", "python3.14dist(pydantic) >= 1.7.4"), - ] - rmdepcheck.format_rc_errors(errs) - captured = capsys.readouterr() - with open(f"{TESTDATA}/test_format_rc_errors.txt", "r", encoding="utf-8") as fh: - exptext = fh.read() - assert captured.out == exptext +class TestRepos: + """Unit tests for things in rmdepcheck.repos.""" + + def test_comma_url(self): + assert repos.comma_url("https://www.some.where") == ["https://www.some.where"] + assert repos.comma_url("https://www.some.where,file:///foo/bar") == [ + "https://www.some.where", + "file:///foo/bar", + ] + with pytest.raises(ValueError): + repos.comma_url("https://www.some.where,ftp://1997.called") + with pytest.raises(ValueError): + repos.comma_url("ftp://1997.called") + with pytest.raises(ValueError): + repos.comma_url("https://www.some.where,whatisthis") + + def test_comma_list(self): + assert repos.comma_list("foo,bar") == ["foo", "bar"] + assert repos.comma_list("") == [] + + @mock.patch("rmdepcheck.repos.check_utils", side_effect=KeyboardInterrupt) + def test_ctrl_c(self, _, capsys): + with pytest.raises(SystemExit) as excinfo: + repos.main() + assert excinfo.value.code == 1 + captured = capsys.readouterr() + assert captured.err == "Interrupted, exiting...\n" -def test_get_base_repoclosure(): - repo = f"file://{REPOS}/base" - with open(f"{TESTDATA}/test_get_base_repoclosure.txt", "r", encoding="utf-8") as testfh: - expected = testfh.read() - expected = expected.replace("{HASH}", rmdepcheck.hash_repo(repo)) - ret = rmdepcheck.get_base_repoclosure([repo], [], []) - assert ret == expected +class TestCompose: + "Unit tests for things in rmdepcheck.compose." "" + + def test_split_put_error(self): + with pytest.raises(ValueError) as err: + compose.split_put( + pathlib.Path("/non/existent/place"), + pathlib.Path("/whatever"), + {"foo": "bar"}, + "x86_64", + ) + assert str(err.value) == "split_put: invalid put value /non/existent/place" + + def test_get_variant_repos_arch(self): + fakeci = { + "payload": { + "variants": { + "BaseOS": { + "id": "BaseOS", + "paths": { + "repository": { + "someotherarch": "somepath", + } + }, + } + } + } + } + # should not crash + assert not compose.get_variant_repos("somecompose", fakeci, "somearch") + + def test_pungi_config_url(self): + fakeci = { + "payload": { + "release": { + "version": "eln", + "short": "Fedora", + } + } + } + ret = compose.pungi_config_url(fakeci) + assert ret == ( + "https://forge.fedoraproject.org" + "/releng/pungi-fedora/raw/branch/eln/fedora/override.conf" + ) + fakeci["payload"]["release"]["short"] = "CentOS" + fakeci["payload"]["release"]["version"] = "10.1" + with pytest.raises(ValueError) as err: + compose.pungi_config_url(fakeci) + assert str(err.value) == "Don't know Pungi config URL for CentOS 10.1" + + def test_get_val_not_found(self): + fakeci = { + "payload": { + "release": { + "version": "eln", + "short": "Fedora", + } + } + } + fakesess = mock.MagicMock() + fakesess.get.return_value.text = "nope, not in here" + with pytest.raises(ValueError) as err: + compose.get_val(fakeci, fakesess) + assert str(err.value) == ( + "Could not find variant_as_lookaside in configuration at https://" + "forge.fedoraproject.org/releng/pungi-fedora/raw/branch/eln/fedora/override.conf" + ) + + @mock.patch("rmdepcheck.compose.check_utils", side_effect=KeyboardInterrupt) + def test_ctrl_c(self, _, capsys): + with pytest.raises(SystemExit) as excinfo: + compose.main() + assert excinfo.value.code == 1 + captured = capsys.readouterr() + assert captured.err == "Interrupted, exiting...\n" -def test_get_modified_and_new_repoclosure(): - brepo = f"file://{REPOS}/base" - nrepo = f"file://{REPOS}/new" - with open(f"{TESTDATA}/test_get_modified_repoclosure.txt", "r", encoding="utf-8") as testfh: - expectedmod = testfh.read() - expectedmod = expectedmod.replace("{HASH}", rmdepcheck.hash_repo(brepo)) - with open(f"{TESTDATA}/test_get_new_repoclosure.txt", "r", encoding="utf-8") as testfh: - expectednew = testfh.read() - expectednew = expectednew.replace("{HASH}", rmdepcheck.hash_repo(nrepo)) - ret = rmdepcheck.get_modified_and_new_repoclosure( - [brepo], [], [], [nrepo], ["aaa", "ccc", "eee", "fff", "ggg"] +class TestE2E: + """End-to-end functional tests for rmdepcheck and compose.""" + + def test_e2e_null(self, capsys): + """End-to-end test using an empty repository, to test what happens + when no changes are found. + """ + sys.argv = ["repos.py", f"file://{REPOS}/base", f"file://{REPOS}/empty"] + with pytest.raises(SystemExit) as excinfo: + repos.main() + assert excinfo.value.code == 0 + captured = capsys.readouterr() + assert captured.out == "" + + @pytest.mark.parametrize("output", ("human", "json", "onlyerrors")) + def test_e2e_devel(self, output, capsys): + """End-to-end test similar to a Rawhide or Branched case, with + a single modified base repo and a single check repo. Also tests + JSON output. + """ + sys.argv = ["repos.py", f"file://{REPOS}/base", f"file://{REPOS}/new"] + if output == "json": + sys.argv.insert(1, "--json") + if output == "onlyerrors": + sys.argv.insert(1, "--onlyerrors") + with pytest.raises(SystemExit) as excinfo: + repos.main() + assert excinfo.value.code == 3 + captured = capsys.readouterr() + if output == "human": + fname = "test_e2e_devel.txt" + elif output == "onlyerrors": + fname = "test_e2e_devel_onlyerrors.txt" + else: + fname = "test_e2e_devel_json.txt" + with open(f"{TESTDATA}/{fname}", "r", encoding="utf-8") as fh: + exptext = fh.read() + exptext = exptext.replace("{REPOS}", REPOS) + assert captured.out == exptext + + def test_e2e_updates(self, capsys): + """End-to-end test similar to a stable Fedora release, with one + non-modified 'frozen' base repo and one modified updates repo. + """ + sys.argv = [ + "repos.py", + "--nmbaserepos", + f"file://{REPOS}/base", + f"file://{REPOS}/updates", + f"file://{REPOS}/new", + ] + with pytest.raises(SystemExit) as excinfo: + repos.main() + assert excinfo.value.code == 3 + captured = capsys.readouterr() + with open(f"{TESTDATA}/test_e2e_updates.txt", "r", encoding="utf-8") as fh: + exptext = fh.read() + exptext = exptext.replace("{REPOS}", REPOS) + assert captured.out == exptext + + @mock.patch( + "rmdepcheck.compose.pungi_config_url", + autospec=True, + return_value="http://localhost:5001/override.conf", ) - assert ret == (expectedmod, expectednew) + def test_e2e_eln(self, _, capsys, http): # pylint: disable=unused-argument + """End-to-end test of the EL mode. This also exercises the non- + checked repository feature. The setup mimics "real" EL, more or + less. There are BaseOS, CRB, Extras and Buildroot repos. BaseOS + contains epa and epb. Extras contains epc. CRB contains epd and + epe. Buildroot contains epf. epe and epf both require epa = 1.0. + The "update" contains epa 2.0, an epb that requires epc, and an + epd that requires epa. + + This sets us several expectations. The update breaks both epe and + epf, but we should only see a report about epe, since epf is in + the buildroot and so should be ignored. epb requiring epc is + wrong because BaseOS cannot require packages in Extras; this + should be reported. It should only be reported *once*, though, + when we're checking BaseOS, even though the BaseOS repos will be + in scope for each run (testing non-checked base repos). epd + requiring epa is fine; no error should be reported. + """ + sys.argv = [ + "compose.py", + "--arch", + "x86_64", + "http://localhost:5001", + f"{REPOS}/elnnew", + ] + with pytest.raises(SystemExit) as excinfo: + compose.main() + assert excinfo.value.code == 3 + captured = capsys.readouterr() + with open(f"{TESTDATA}/test_e2e_eln.txt", "r", encoding="utf-8") as fh: + exptext = fh.read() + exptext = exptext.replace("{REPOS}", REPOS) + assert captured.out == exptext + + def test_e2e_removes(self, capsys): + """End-to-end test of the alternate --removes mode.""" + sys.argv = ["repos.py", "--removes", f"file://{REPOS}/base", "aaa,eee"] + with pytest.raises(SystemExit) as excinfo: + repos.main() + assert excinfo.value.code == 1 + captured = capsys.readouterr() + with open(f"{TESTDATA}/test_e2e_removes.txt", "r", encoding="utf-8") as fh: + exptext = fh.read() + exptext = exptext.replace("{REPOS}", REPOS) + assert captured.out == exptext + + @pytest.mark.parametrize("output", ("human", "json")) + def test_e2e_unfixed(self, output, capsys): + """End-to-end test where the new repository contains a package + with an unfixed pre-existing dependency issue. + """ + sys.argv = ["repos.py", f"file://{REPOS}/base", f"file://{REPOS}/unfixed"] + if output == "json": + sys.argv.insert(1, "--json") + with pytest.raises(SystemExit) as excinfo: + repos.main() + assert excinfo.value.code == 4 + captured = capsys.readouterr() + if output == "human": + fname = "test_e2e_unfixed.txt" + else: + fname = "test_e2e_unfixed_json.txt" + with open(f"{TESTDATA}/{fname}", "r", encoding="utf-8") as fh: + exptext = fh.read() + exptext = exptext.replace("{REPOS}", REPOS) + assert captured.out == exptext -def test_get_modified_and_new_repoclosure_commasafe(): - """Check we survive repoquery output lines that are, precisely, - a string from 'removes'. This is a pretty unlikely scenario, but - hey. Any other line with no commas is actually safe because it - will fail the `if elems[0] in removes` check. - """ - qamock = mock.Mock() - qamock.stdout = "ccc\nbbb,bbb-0:1.0-1.x86_64\nccc,ccc-0:1.0-1.x86_64" - rcmock = mock.Mock() - rcmock.stdout = "" - with mock.patch("rmdepcheck.SUBPCAPTURE", side_effect=[qamock, rcmock]): - # this shouldn't raise an exception - rmdepcheck.get_modified_and_new_repoclosure(["file:///foo/bar"], [], [], [], ["ccc"]) - - -def test_get_source_packages(): - sources = rmdepcheck.get_source_packages([f"file://{REPOS}/new"]) - assert sources == {"111", "222", "aaa", "ccc", "eee", "fff", "ggg"} - - -def test_handle_preexisting(): - fixederrors = [ - ("libreoffice-langpack-lv-1:26.2.2.1-0.1.eln155.x86_64", "ogrepo", "mythes-lv"), - ("libreoffice-langpack-nb-1:26.2.2.1-0.1.eln155.x86_64", "ogrepo", "mythes-nb"), - ("libreoffice-langpack-nn-1:26.2.2.1-0.1.eln155.x86_64", "ogrepo", "mythes-nn"), - ("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "someotherdep"), - ] - newrc = [ - ("libreoffice-langpack-lv-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-lv"), - ("libreoffice-langpack-nb-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nb"), - ("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nn"), - ("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "foo"), - ("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "mythes-lv"), - ] - # this should null out the 'matching' errors (the first three in - # each list), but leave the 'non-matching' ones (the others) - ret = rmdepcheck.handle_preexisting(fixederrors, newrc) - assert fixederrors == [ - ("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "someotherdep"), - ] - assert newrc == [ - ("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "foo"), - ("someotherpackage-1.0-1.eln155.x86_64", "newrepo", "mythes-lv"), - ] - # we should return the errors dropped from 'newrc' (so these can - # be reported separately) - assert ret == [ - ("libreoffice-langpack-lv-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-lv"), - ("libreoffice-langpack-nb-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nb"), - ("libreoffice-langpack-nn-1:26.2.2.2-0.1.eln155.x86_64", "newrepo", "mythes-nn"), - ] - - -def test_url_check(): - rmdepcheck.url_check("file:///foo/bar") - rmdepcheck.url_check("https://www.some.where") - rmdepcheck.url_check("http://some.where.insecure") - # Test automatic conversion of existing local paths to file:// URIs - result = rmdepcheck.url_check(REPOS) - assert result == f"file://{REPOS}" - # Test with relative path that exists - result = rmdepcheck.url_check("tests") - assert result.startswith("file://") - assert result.endswith("/tests") - with pytest.raises(ValueError): - rmdepcheck.url_check("ftp://1997.called") - # Non-existent paths without scheme should still raise ValueError - with pytest.raises(ValueError): - rmdepcheck.url_check("whatisthis") - - -def test_comma_url(): - assert rmdepcheck.comma_url("https://www.some.where") == ["https://www.some.where"] - assert rmdepcheck.comma_url("https://www.some.where,file:///foo/bar") == [ - "https://www.some.where", - "file:///foo/bar", - ] - with pytest.raises(ValueError): - rmdepcheck.comma_url("https://www.some.where,ftp://1997.called") - with pytest.raises(ValueError): - rmdepcheck.comma_url("ftp://1997.called") - with pytest.raises(ValueError): - rmdepcheck.comma_url("https://www.some.where,whatisthis") - - -def test_comma_list(): - assert rmdepcheck.comma_list("foo,bar") == ["foo", "bar"] - assert rmdepcheck.comma_list("") == [] - - -@mock.patch("platform.machine", return_value="foobar") -def test_check_arch(_): - assert rmdepcheck.check_arch("x86_64") == "x86_64" - assert rmdepcheck.check_arch("") == "foobar" - - -@mock.patch("subprocess.run", autospec=True) -def test_check_dnf(mock_run): - rmdepcheck.check_dnf() - mock_run.side_effect = FileNotFoundError - with pytest.raises(SystemExit) as excinfo: - rmdepcheck.check_dnf() - assert excinfo.value.code == "Please install missing required utilities: dnf" - - -@mock.patch("rmdepcheck.check_dnf", side_effect=KeyboardInterrupt) -def test_ctrl_c(_, capsys): - with pytest.raises(SystemExit) as excinfo: - rmdepcheck.main() - assert excinfo.value.code == 1 - captured = capsys.readouterr() - assert captured.err == "Interrupted, exiting...\n" - - -def test_e2e_null(capsys): - """End-to-end test using an empty repository, to test what happens - when no changes are found. - """ - sys.argv = ["rmdepcheck.py", f"file://{REPOS}/base", f"file://{REPOS}/empty"] - with pytest.raises(SystemExit) as excinfo: - rmdepcheck.main() - assert excinfo.value.code == 0 - captured = capsys.readouterr() - assert captured.out == "" - - -@pytest.mark.parametrize("output", ("human", "json", "onlyerrors")) -def test_e2e_devel(output, capsys): - """End-to-end test similar to a Rawhide or Branched case, with - a single modified base repo and a single check repo. Also tests - JSON output. - """ - sys.argv = ["rmdepcheck.py", f"file://{REPOS}/base", f"file://{REPOS}/new"] - if output == "json": - sys.argv.insert(1, "--json") - if output == "onlyerrors": - sys.argv.insert(1, "--onlyerrors") - with pytest.raises(SystemExit) as excinfo: - rmdepcheck.main() - assert excinfo.value.code == 3 - captured = capsys.readouterr() - if output == "human": - fname = "test_e2e_devel.txt" - elif output == "onlyerrors": - fname = "test_e2e_devel_onlyerrors.txt" - else: - fname = "test_e2e_devel_json.txt" - with open(f"{TESTDATA}/{fname}", "r", encoding="utf-8") as fh: - exptext = fh.read() - exptext = exptext.replace("{REPOS}", REPOS) - assert captured.out == exptext - - -def test_e2e_updates(capsys): - """End-to-end test similar to a stable Fedora release, with one - non-modified 'frozen' base repo and one modified updates repo. - """ - sys.argv = [ - "rmdepcheck.py", - "--nmbaserepos", - f"file://{REPOS}/base", - f"file://{REPOS}/updates", - f"file://{REPOS}/new", - ] - with pytest.raises(SystemExit) as excinfo: - rmdepcheck.main() - assert excinfo.value.code == 3 - captured = capsys.readouterr() - with open(f"{TESTDATA}/test_e2e_updates.txt", "r", encoding="utf-8") as fh: - exptext = fh.read() - exptext = exptext.replace("{REPOS}", REPOS) - assert captured.out == exptext - - -def test_e2e_unchecked(capsys): - """End-to-end test with a smaller repo as the base repo and a - larger buildroot repo as a non-checked base repo. The new repo - contains a package that breaks the deps of one package in the base - repo and one package in the non-checked base repo, and itself - requires a package from the non-checked base repo. We expect to - see a failure for the broken base repo package, but no failure for - the broken non-checked base repo package or the new package itself. - """ - sys.argv = [ - "rmdepcheck.py", - "--ncbaserepos", - f"file://{REPOS}/elnroot", - f"file://{REPOS}/elnbase", - f"file://{REPOS}/elnnew", - ] - with pytest.raises(SystemExit) as excinfo: - rmdepcheck.main() - assert excinfo.value.code == 1 - captured = capsys.readouterr() - with open(f"{TESTDATA}/test_e2e_eln.txt", "r", encoding="utf-8") as fh: - exptext = fh.read() - exptext = exptext.replace("{REPOS}", REPOS) - assert captured.out == exptext - - -def test_e2e_removes(capsys): - """End-to-end test of the alternate --removes mode.""" - sys.argv = ["rmdepcheck.py", "--removes", f"file://{REPOS}/base", "aaa,eee"] - with pytest.raises(SystemExit) as excinfo: - rmdepcheck.main() - assert excinfo.value.code == 1 - captured = capsys.readouterr() - with open(f"{TESTDATA}/test_e2e_removes.txt", "r", encoding="utf-8") as fh: - exptext = fh.read() - exptext = exptext.replace("{REPOS}", REPOS) - assert captured.out == exptext - - -@pytest.mark.parametrize("output", ("human", "json")) -def test_e2e_unfixed(output, capsys): - """End-to-end test where the new repository contains a package - with an unfixed pre-existing dependency issue. - """ - sys.argv = ["rmdepcheck.py", f"file://{REPOS}/base", f"file://{REPOS}/unfixed"] - if output == "json": - sys.argv.insert(1, "--json") - with pytest.raises(SystemExit) as excinfo: - rmdepcheck.main() - assert excinfo.value.code == 4 - captured = capsys.readouterr() - if output == "human": - fname = "test_e2e_unfixed.txt" - else: - fname = "test_e2e_unfixed_json.txt" - with open(f"{TESTDATA}/{fname}", "r", encoding="utf-8") as fh: - exptext = fh.read() - exptext = exptext.replace("{REPOS}", REPOS) - assert captured.out == exptext +# vim: set textwidth=100 ts=8 et sw=4: diff --git a/tests/testdata/repos/base/aaa-1.0-1.x86_64.rpm b/tests/testdata/repos/base/aaa-1.0-1.x86_64.rpm index 4e10148..b812f3d 100644 Binary files a/tests/testdata/repos/base/aaa-1.0-1.x86_64.rpm and b/tests/testdata/repos/base/aaa-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/base/bbb-1.0-1.x86_64.rpm b/tests/testdata/repos/base/bbb-1.0-1.x86_64.rpm index e6908cb..ef5a74c 100644 Binary files a/tests/testdata/repos/base/bbb-1.0-1.x86_64.rpm and b/tests/testdata/repos/base/bbb-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/base/ccc-1.0-1.x86_64.rpm b/tests/testdata/repos/base/ccc-1.0-1.x86_64.rpm index a6ef61f..2cf34ea 100644 Binary files a/tests/testdata/repos/base/ccc-1.0-1.x86_64.rpm and b/tests/testdata/repos/base/ccc-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/base/ddd-1.0-1.x86_64.rpm b/tests/testdata/repos/base/ddd-1.0-1.x86_64.rpm index 7b8001d..6255838 100644 Binary files a/tests/testdata/repos/base/ddd-1.0-1.x86_64.rpm and b/tests/testdata/repos/base/ddd-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/base/eee-1.0-1.x86_64.rpm b/tests/testdata/repos/base/eee-1.0-1.x86_64.rpm index 9dc4d3e..40a3a8a 100644 Binary files a/tests/testdata/repos/base/eee-1.0-1.x86_64.rpm and b/tests/testdata/repos/base/eee-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/base/fff-1.0-1.x86_64.rpm b/tests/testdata/repos/base/fff-1.0-1.x86_64.rpm index 8820a74..3be5619 100644 Binary files a/tests/testdata/repos/base/fff-1.0-1.x86_64.rpm and b/tests/testdata/repos/base/fff-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/base/ggg-1.0-1.i686.rpm b/tests/testdata/repos/base/ggg-1.0-1.i686.rpm index ce786d6..2bfb68b 100644 Binary files a/tests/testdata/repos/base/ggg-1.0-1.i686.rpm and b/tests/testdata/repos/base/ggg-1.0-1.i686.rpm differ diff --git a/tests/testdata/repos/base/ggg-1.0-1.x86_64.rpm b/tests/testdata/repos/base/ggg-1.0-1.x86_64.rpm index ea25756..3c4f65b 100644 Binary files a/tests/testdata/repos/base/ggg-1.0-1.x86_64.rpm and b/tests/testdata/repos/base/ggg-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/base/hhh-1.0-1.i686.rpm b/tests/testdata/repos/base/hhh-1.0-1.i686.rpm index 53a9597..38ca132 100644 Binary files a/tests/testdata/repos/base/hhh-1.0-1.i686.rpm and b/tests/testdata/repos/base/hhh-1.0-1.i686.rpm differ diff --git a/tests/testdata/repos/base/hhh-1.0-1.x86_64.rpm b/tests/testdata/repos/base/hhh-1.0-1.x86_64.rpm index 0d5a014..993ae81 100644 Binary files a/tests/testdata/repos/base/hhh-1.0-1.x86_64.rpm and b/tests/testdata/repos/base/hhh-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/base/repodata/1acaeb6689285db3bfe681d374748eed2bc7cd9c1af87b32a27819f776ecacb6-filelists.xml.zst b/tests/testdata/repos/base/repodata/1acaeb6689285db3bfe681d374748eed2bc7cd9c1af87b32a27819f776ecacb6-filelists.xml.zst deleted file mode 100644 index 003c3b4..0000000 Binary files a/tests/testdata/repos/base/repodata/1acaeb6689285db3bfe681d374748eed2bc7cd9c1af87b32a27819f776ecacb6-filelists.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/base/repodata/6868a9258bc5b257d087b3f926cbdbc014866cac98e1c7047bb0de5c8f4e8b63-other.xml.zst b/tests/testdata/repos/base/repodata/6868a9258bc5b257d087b3f926cbdbc014866cac98e1c7047bb0de5c8f4e8b63-other.xml.zst deleted file mode 100644 index 967e4ec..0000000 Binary files a/tests/testdata/repos/base/repodata/6868a9258bc5b257d087b3f926cbdbc014866cac98e1c7047bb0de5c8f4e8b63-other.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/base/repodata/6b696476761c56d8bceb3e8997d31ca6977aaf63901130fe94c708e7d0cf429d-primary.xml.zst b/tests/testdata/repos/base/repodata/6b696476761c56d8bceb3e8997d31ca6977aaf63901130fe94c708e7d0cf429d-primary.xml.zst deleted file mode 100644 index ed69a52..0000000 Binary files a/tests/testdata/repos/base/repodata/6b696476761c56d8bceb3e8997d31ca6977aaf63901130fe94c708e7d0cf429d-primary.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/base/repodata/725d5a8541dc6ac107257c1ccd264b53cfdfb601e10f1dffe1db40462fcab118-primary.xml.zst b/tests/testdata/repos/base/repodata/725d5a8541dc6ac107257c1ccd264b53cfdfb601e10f1dffe1db40462fcab118-primary.xml.zst new file mode 100644 index 0000000..d28cf04 Binary files /dev/null and b/tests/testdata/repos/base/repodata/725d5a8541dc6ac107257c1ccd264b53cfdfb601e10f1dffe1db40462fcab118-primary.xml.zst differ diff --git a/tests/testdata/repos/base/repodata/934bc8a9464d6232eb2e22aea3275da638dd1d733d9af49f402ac5d0072b505c-filelists.xml.zst b/tests/testdata/repos/base/repodata/934bc8a9464d6232eb2e22aea3275da638dd1d733d9af49f402ac5d0072b505c-filelists.xml.zst new file mode 100644 index 0000000..9c9ed15 Binary files /dev/null and b/tests/testdata/repos/base/repodata/934bc8a9464d6232eb2e22aea3275da638dd1d733d9af49f402ac5d0072b505c-filelists.xml.zst differ diff --git a/tests/testdata/repos/base/repodata/9a69e354abcfd9f7de70b62321bab6263e340615e628cbf96518ffe4ab1bc376-other.xml.zst b/tests/testdata/repos/base/repodata/9a69e354abcfd9f7de70b62321bab6263e340615e628cbf96518ffe4ab1bc376-other.xml.zst new file mode 100644 index 0000000..7657822 Binary files /dev/null and b/tests/testdata/repos/base/repodata/9a69e354abcfd9f7de70b62321bab6263e340615e628cbf96518ffe4ab1bc376-other.xml.zst differ diff --git a/tests/testdata/repos/base/repodata/repomd.xml b/tests/testdata/repos/base/repodata/repomd.xml index 9aa144c..5a317c1 100644 --- a/tests/testdata/repos/base/repodata/repomd.xml +++ b/tests/testdata/repos/base/repodata/repomd.xml @@ -1,27 +1,27 @@ - 1777498820 + 1778717667 - 6b696476761c56d8bceb3e8997d31ca6977aaf63901130fe94c708e7d0cf429d - 54c3435b6735114519dd995763d0a654a564872f34118ce620e7148d7488738e - - 1777498820 - 1281 + 725d5a8541dc6ac107257c1ccd264b53cfdfb601e10f1dffe1db40462fcab118 + 577f2cb77ecdf002fbcfc3d3d3d5972e9574d262a0a32068f5b795556d6a1bcf + + 1778717667 + 1278 10594 - 1acaeb6689285db3bfe681d374748eed2bc7cd9c1af87b32a27819f776ecacb6 - 05695974b3212a3e0a658e431c4c6eb1cf20b4346365b9a062c4469e7a85c6a1 - - 1777498820 - 621 + 934bc8a9464d6232eb2e22aea3275da638dd1d733d9af49f402ac5d0072b505c + 863daf657e56f2234c416ab3878b88305619ab38614dee37253281a6f38cd296 + + 1778717667 + 624 1722 - 6868a9258bc5b257d087b3f926cbdbc014866cac98e1c7047bb0de5c8f4e8b63 - 29e4f16b3e7312bdeecc87f41baa5f8200df3793aaab472e131dcc6e010130b8 - - 1777498820 + 9a69e354abcfd9f7de70b62321bab6263e340615e628cbf96518ffe4ab1bc376 + b62ee7d9a3bfa4aed63880cf15f0022c25930473ec0a5004388db138c165b285 + + 1778717667 710 2838 diff --git a/tests/testdata/repos/eln/AppStream/repodata/69a3730a283b85a4b3cff7d04bfde3b2b234f0607ebc17319d7a8d143a8e066e-primary.xml.zst b/tests/testdata/repos/eln/AppStream/repodata/69a3730a283b85a4b3cff7d04bfde3b2b234f0607ebc17319d7a8d143a8e066e-primary.xml.zst new file mode 100644 index 0000000..ea97e2c Binary files /dev/null and b/tests/testdata/repos/eln/AppStream/repodata/69a3730a283b85a4b3cff7d04bfde3b2b234f0607ebc17319d7a8d143a8e066e-primary.xml.zst differ diff --git a/tests/testdata/repos/eln/AppStream/repodata/6b37cc67608a24beaa81e1191d218f2ffd6b1191dceb5c100bac2e66249d518d-other.xml.zst b/tests/testdata/repos/eln/AppStream/repodata/6b37cc67608a24beaa81e1191d218f2ffd6b1191dceb5c100bac2e66249d518d-other.xml.zst new file mode 100644 index 0000000..8cb24b4 Binary files /dev/null and b/tests/testdata/repos/eln/AppStream/repodata/6b37cc67608a24beaa81e1191d218f2ffd6b1191dceb5c100bac2e66249d518d-other.xml.zst differ diff --git a/tests/testdata/repos/eln/AppStream/repodata/9b07d97dc6ececed89aac0650b67bfb292647fe9fbaca48f629465be5f53f82b-filelists.xml.zst b/tests/testdata/repos/eln/AppStream/repodata/9b07d97dc6ececed89aac0650b67bfb292647fe9fbaca48f629465be5f53f82b-filelists.xml.zst new file mode 100644 index 0000000..45f26d6 Binary files /dev/null and b/tests/testdata/repos/eln/AppStream/repodata/9b07d97dc6ececed89aac0650b67bfb292647fe9fbaca48f629465be5f53f82b-filelists.xml.zst differ diff --git a/tests/testdata/repos/eln/AppStream/repodata/repomd.xml b/tests/testdata/repos/eln/AppStream/repodata/repomd.xml new file mode 100644 index 0000000..4b6e068 --- /dev/null +++ b/tests/testdata/repos/eln/AppStream/repodata/repomd.xml @@ -0,0 +1,28 @@ + + + 1778717670 + + 69a3730a283b85a4b3cff7d04bfde3b2b234f0607ebc17319d7a8d143a8e066e + e1e2ffd2fb1ee76f87b70750d00ca5677a252b397ab6c2389137a0c33e7b359f + + 1778717670 + 123 + 167 + + + 9b07d97dc6ececed89aac0650b67bfb292647fe9fbaca48f629465be5f53f82b + bf9808b81cb2dbc54b4b8e35adc584ddcaa73bd81f7088d73bf7dbbada961310 + + 1778717670 + 118 + 125 + + + 6b37cc67608a24beaa81e1191d218f2ffd6b1191dceb5c100bac2e66249d518d + e0ed5e0054194df036cf09c1a911e15bf2a4e7f26f2a788b6f47d53e80717ccc + + 1778717670 + 117 + 121 + + diff --git a/tests/testdata/repos/elnbase/epc-1.0-1.x86_64.rpm b/tests/testdata/repos/eln/BaseOS/epa-1.0-1.x86_64.rpm similarity index 90% rename from tests/testdata/repos/elnbase/epc-1.0-1.x86_64.rpm rename to tests/testdata/repos/eln/BaseOS/epa-1.0-1.x86_64.rpm index a142029..65c0895 100644 Binary files a/tests/testdata/repos/elnbase/epc-1.0-1.x86_64.rpm and b/tests/testdata/repos/eln/BaseOS/epa-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/eln/BaseOS/epb-1.0-1.x86_64.rpm b/tests/testdata/repos/eln/BaseOS/epb-1.0-1.x86_64.rpm new file mode 100644 index 0000000..922843a Binary files /dev/null and b/tests/testdata/repos/eln/BaseOS/epb-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/eln/BaseOS/repodata/73c134c24423339ee3c1b12e215c925e853b58ade283876a72ca7b92fbf246c4-other.xml.zst b/tests/testdata/repos/eln/BaseOS/repodata/73c134c24423339ee3c1b12e215c925e853b58ade283876a72ca7b92fbf246c4-other.xml.zst new file mode 100644 index 0000000..8cefdc5 Binary files /dev/null and b/tests/testdata/repos/eln/BaseOS/repodata/73c134c24423339ee3c1b12e215c925e853b58ade283876a72ca7b92fbf246c4-other.xml.zst differ diff --git a/tests/testdata/repos/eln/BaseOS/repodata/9daa049560d116a0c3e824db9385975140f68e97a03ae87214197284cbbbfad8-primary.xml.zst b/tests/testdata/repos/eln/BaseOS/repodata/9daa049560d116a0c3e824db9385975140f68e97a03ae87214197284cbbbfad8-primary.xml.zst new file mode 100644 index 0000000..eaa73f9 Binary files /dev/null and b/tests/testdata/repos/eln/BaseOS/repodata/9daa049560d116a0c3e824db9385975140f68e97a03ae87214197284cbbbfad8-primary.xml.zst differ diff --git a/tests/testdata/repos/eln/BaseOS/repodata/e9b667b6a1059447627647ab73574fdee0a9c934cc7bfcf739f32edd1afae3d2-filelists.xml.zst b/tests/testdata/repos/eln/BaseOS/repodata/e9b667b6a1059447627647ab73574fdee0a9c934cc7bfcf739f32edd1afae3d2-filelists.xml.zst new file mode 100644 index 0000000..d1752f7 Binary files /dev/null and b/tests/testdata/repos/eln/BaseOS/repodata/e9b667b6a1059447627647ab73574fdee0a9c934cc7bfcf739f32edd1afae3d2-filelists.xml.zst differ diff --git a/tests/testdata/repos/eln/BaseOS/repodata/repomd.xml b/tests/testdata/repos/eln/BaseOS/repodata/repomd.xml new file mode 100644 index 0000000..f2882e6 --- /dev/null +++ b/tests/testdata/repos/eln/BaseOS/repodata/repomd.xml @@ -0,0 +1,28 @@ + + + 1778717670 + + 9daa049560d116a0c3e824db9385975140f68e97a03ae87214197284cbbbfad8 + 3da88c933c481526801b2d09467c8e54e4edb8f54bf98422684c243650fdee96 + + 1778717670 + 676 + 2163 + + + e9b667b6a1059447627647ab73574fdee0a9c934cc7bfcf739f32edd1afae3d2 + de153bb5dc84de1eabec3de3762909234945d17228ca42b53cee3aa4b3da63f7 + + 1778717670 + 279 + 445 + + + 73c134c24423339ee3c1b12e215c925e853b58ade283876a72ca7b92fbf246c4 + a1a6a080f1db568019eee61e3b62d74f3dafcb40867b0f2faf218585b7059123 + + 1778717670 + 352 + 665 + + diff --git a/tests/testdata/repos/elnbase/epa-1.0-1.x86_64.rpm b/tests/testdata/repos/eln/Buildroot/epe-1.0-1.x86_64.rpm similarity index 89% rename from tests/testdata/repos/elnbase/epa-1.0-1.x86_64.rpm rename to tests/testdata/repos/eln/Buildroot/epe-1.0-1.x86_64.rpm index 999448a..c286d78 100644 Binary files a/tests/testdata/repos/elnbase/epa-1.0-1.x86_64.rpm and b/tests/testdata/repos/eln/Buildroot/epe-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/eln/Buildroot/repodata/4c41bd6e67d5b8fc2da462e55f106fcdc729daf0e722ace44fbf1f0688f4cf68-filelists.xml.zst b/tests/testdata/repos/eln/Buildroot/repodata/4c41bd6e67d5b8fc2da462e55f106fcdc729daf0e722ace44fbf1f0688f4cf68-filelists.xml.zst new file mode 100644 index 0000000..3358956 Binary files /dev/null and b/tests/testdata/repos/eln/Buildroot/repodata/4c41bd6e67d5b8fc2da462e55f106fcdc729daf0e722ace44fbf1f0688f4cf68-filelists.xml.zst differ diff --git a/tests/testdata/repos/eln/Buildroot/repodata/8d4fa97f0d4d6bdeca349744217b0e9d694aea2cef332765e97c2252c4df6431-primary.xml.zst b/tests/testdata/repos/eln/Buildroot/repodata/8d4fa97f0d4d6bdeca349744217b0e9d694aea2cef332765e97c2252c4df6431-primary.xml.zst new file mode 100644 index 0000000..08eb506 Binary files /dev/null and b/tests/testdata/repos/eln/Buildroot/repodata/8d4fa97f0d4d6bdeca349744217b0e9d694aea2cef332765e97c2252c4df6431-primary.xml.zst differ diff --git a/tests/testdata/repos/eln/Buildroot/repodata/f4ecf39d19149d8918b7485eea237aa87723615393e5f15408b577e2114818d7-other.xml.zst b/tests/testdata/repos/eln/Buildroot/repodata/f4ecf39d19149d8918b7485eea237aa87723615393e5f15408b577e2114818d7-other.xml.zst new file mode 100644 index 0000000..14872e1 Binary files /dev/null and b/tests/testdata/repos/eln/Buildroot/repodata/f4ecf39d19149d8918b7485eea237aa87723615393e5f15408b577e2114818d7-other.xml.zst differ diff --git a/tests/testdata/repos/eln/Buildroot/repodata/repomd.xml b/tests/testdata/repos/eln/Buildroot/repodata/repomd.xml new file mode 100644 index 0000000..8400bbc --- /dev/null +++ b/tests/testdata/repos/eln/Buildroot/repodata/repomd.xml @@ -0,0 +1,28 @@ + + + 1778717671 + + 8d4fa97f0d4d6bdeca349744217b0e9d694aea2cef332765e97c2252c4df6431 + 803c8426b401a9880c4f3e606fd7648a4d619ad1b9f64f029ba2edf0d922fcff + + 1778717671 + 626 + 1265 + + + 4c41bd6e67d5b8fc2da462e55f106fcdc729daf0e722ace44fbf1f0688f4cf68 + 69b6ea42561116fa83b8c836736c307693c9d425c6b5d8c9793c86bfa15069bd + + 1778717671 + 223 + 285 + + + f4ecf39d19149d8918b7485eea237aa87723615393e5f15408b577e2114818d7 + 6afc2f6d415b482a37ee91bb209367ff45b7f6f3434775ee219e73abf819ae72 + + 1778717671 + 304 + 393 + + diff --git a/tests/testdata/repos/elnroot/epd-1.0-1.x86_64.rpm b/tests/testdata/repos/eln/CRB/epd-1.0-1.x86_64.rpm similarity index 91% rename from tests/testdata/repos/elnroot/epd-1.0-1.x86_64.rpm rename to tests/testdata/repos/eln/CRB/epd-1.0-1.x86_64.rpm index f3395da..4e4d859 100644 Binary files a/tests/testdata/repos/elnroot/epd-1.0-1.x86_64.rpm and b/tests/testdata/repos/eln/CRB/epd-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/elnroot/epb-1.0-1.x86_64.rpm b/tests/testdata/repos/eln/CRB/epe-1.0-1.x86_64.rpm similarity index 80% rename from tests/testdata/repos/elnroot/epb-1.0-1.x86_64.rpm rename to tests/testdata/repos/eln/CRB/epe-1.0-1.x86_64.rpm index df6e850..aac5993 100644 Binary files a/tests/testdata/repos/elnroot/epb-1.0-1.x86_64.rpm and b/tests/testdata/repos/eln/CRB/epe-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/eln/CRB/repodata/63510d69370f860af7ae99ffd909ecee7358765b80abb684b7a38f70e2bd977a-other.xml.zst b/tests/testdata/repos/eln/CRB/repodata/63510d69370f860af7ae99ffd909ecee7358765b80abb684b7a38f70e2bd977a-other.xml.zst new file mode 100644 index 0000000..a5c0c23 Binary files /dev/null and b/tests/testdata/repos/eln/CRB/repodata/63510d69370f860af7ae99ffd909ecee7358765b80abb684b7a38f70e2bd977a-other.xml.zst differ diff --git a/tests/testdata/repos/eln/CRB/repodata/a939af998dafa1b5c771494f948ca1a12e7c8e0ba4b324f784caeca60580b944-filelists.xml.zst b/tests/testdata/repos/eln/CRB/repodata/a939af998dafa1b5c771494f948ca1a12e7c8e0ba4b324f784caeca60580b944-filelists.xml.zst new file mode 100644 index 0000000..4ef0c30 Binary files /dev/null and b/tests/testdata/repos/eln/CRB/repodata/a939af998dafa1b5c771494f948ca1a12e7c8e0ba4b324f784caeca60580b944-filelists.xml.zst differ diff --git a/tests/testdata/repos/eln/CRB/repodata/adce22870e107115797acfb5b60eb3ae16af020f137c2dd783447099c23683cb-primary.xml.zst b/tests/testdata/repos/eln/CRB/repodata/adce22870e107115797acfb5b60eb3ae16af020f137c2dd783447099c23683cb-primary.xml.zst new file mode 100644 index 0000000..05e352d Binary files /dev/null and b/tests/testdata/repos/eln/CRB/repodata/adce22870e107115797acfb5b60eb3ae16af020f137c2dd783447099c23683cb-primary.xml.zst differ diff --git a/tests/testdata/repos/eln/CRB/repodata/repomd.xml b/tests/testdata/repos/eln/CRB/repodata/repomd.xml new file mode 100644 index 0000000..59f3838 --- /dev/null +++ b/tests/testdata/repos/eln/CRB/repodata/repomd.xml @@ -0,0 +1,28 @@ + + + 1778717670 + + adce22870e107115797acfb5b60eb3ae16af020f137c2dd783447099c23683cb + d0ab817c34b063b3ff19b611f8e4ce132fa18fc1b5f41c424e5f8faa0bc52aa5 + + 1778717670 + 704 + 2235 + + + a939af998dafa1b5c771494f948ca1a12e7c8e0ba4b324f784caeca60580b944 + d4cbedeac0b70877906e01ff83afedde852c88d56f5bf4477e8f4ec80d02e9e5 + + 1778717670 + 279 + 445 + + + 63510d69370f860af7ae99ffd909ecee7358765b80abb684b7a38f70e2bd977a + df320cc37c560b6aa899b3344d87a701d307398f662dc4e7437a22adddc0c2b0 + + 1778717670 + 353 + 665 + + diff --git a/tests/testdata/repos/elnroot/epc-1.0-1.x86_64.rpm b/tests/testdata/repos/eln/Extras/epc-1.0-1.x86_64.rpm similarity index 93% rename from tests/testdata/repos/elnroot/epc-1.0-1.x86_64.rpm rename to tests/testdata/repos/eln/Extras/epc-1.0-1.x86_64.rpm index a142029..8c20c48 100644 Binary files a/tests/testdata/repos/elnroot/epc-1.0-1.x86_64.rpm and b/tests/testdata/repos/eln/Extras/epc-1.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/eln/Extras/repodata/718eae20ba82ada746eb049df2e88a420766a22c0a835e3ac0d666d1e5acfbaa-filelists.xml.zst b/tests/testdata/repos/eln/Extras/repodata/718eae20ba82ada746eb049df2e88a420766a22c0a835e3ac0d666d1e5acfbaa-filelists.xml.zst new file mode 100644 index 0000000..afa7de8 Binary files /dev/null and b/tests/testdata/repos/eln/Extras/repodata/718eae20ba82ada746eb049df2e88a420766a22c0a835e3ac0d666d1e5acfbaa-filelists.xml.zst differ diff --git a/tests/testdata/repos/eln/Extras/repodata/d852b0588e7a6b861bf02df7aff77fd692bc597986c9fa22323a36fdb2d73a06-primary.xml.zst b/tests/testdata/repos/eln/Extras/repodata/d852b0588e7a6b861bf02df7aff77fd692bc597986c9fa22323a36fdb2d73a06-primary.xml.zst new file mode 100644 index 0000000..2e20e40 Binary files /dev/null and b/tests/testdata/repos/eln/Extras/repodata/d852b0588e7a6b861bf02df7aff77fd692bc597986c9fa22323a36fdb2d73a06-primary.xml.zst differ diff --git a/tests/testdata/repos/eln/Extras/repodata/ef66a3ad019261b3603fddafbb1a8aafdc9ebb314370553c518be659d83b0ada-other.xml.zst b/tests/testdata/repos/eln/Extras/repodata/ef66a3ad019261b3603fddafbb1a8aafdc9ebb314370553c518be659d83b0ada-other.xml.zst new file mode 100644 index 0000000..6d2e57a Binary files /dev/null and b/tests/testdata/repos/eln/Extras/repodata/ef66a3ad019261b3603fddafbb1a8aafdc9ebb314370553c518be659d83b0ada-other.xml.zst differ diff --git a/tests/testdata/repos/eln/Extras/repodata/repomd.xml b/tests/testdata/repos/eln/Extras/repodata/repomd.xml new file mode 100644 index 0000000..16f677e --- /dev/null +++ b/tests/testdata/repos/eln/Extras/repodata/repomd.xml @@ -0,0 +1,28 @@ + + + 1778717670 + + d852b0588e7a6b861bf02df7aff77fd692bc597986c9fa22323a36fdb2d73a06 + 323a9591c1a344cf10d2c9c5f541f118eb922e3e1e8a2274072459699647eb50 + + 1778717670 + 606 + 1165 + + + 718eae20ba82ada746eb049df2e88a420766a22c0a835e3ac0d666d1e5acfbaa + dae4f03d98107c89b4af8380c4a6a7036b4809f7e68d60a3d2fa2a3a3429f010 + + 1778717670 + 224 + 285 + + + ef66a3ad019261b3603fddafbb1a8aafdc9ebb314370553c518be659d83b0ada + d890073321ddf2010162771341ea12e3de65d0a4ef40a4cafa696d4cea29fe23 + + 1778717670 + 306 + 393 + + diff --git a/tests/testdata/repos/eln/metadata/composeinfo.json b/tests/testdata/repos/eln/metadata/composeinfo.json new file mode 100644 index 0000000..ea0ecc8 --- /dev/null +++ b/tests/testdata/repos/eln/metadata/composeinfo.json @@ -0,0 +1,53 @@ +{ + "payload": { + "release": { + "internal": false, + "name": "Fedora", + "short": "Fedora", + "type": "ga", + "version": "eln" + }, + "variants": { + "AppStream": { + "id": "AppStream", + "paths": { + "repository": { + "x86_64": "AppStream" + } + } + }, + "BaseOS": { + "id": "BaseOS", + "paths": { + "repository": { + "x86_64": "BaseOS" + } + } + }, + "Buildroot": { + "id": "Buildroot", + "paths": { + "repository": { + "x86_64": "Buildroot" + } + } + }, + "CRB": { + "id": "CRB", + "paths": { + "repository": { + "x86_64": "CRB" + } + } + }, + "Extras": { + "id": "Extras", + "paths": { + "repository": { + "x86_64": "Extras" + } + } + } + } + } +} diff --git a/tests/testdata/repos/eln/override.conf b/tests/testdata/repos/eln/override.conf new file mode 100644 index 0000000..5553af4 --- /dev/null +++ b/tests/testdata/repos/eln/override.conf @@ -0,0 +1,91 @@ +# This files overrides default variables defined in the configs in +# the ../shared directory. + +from images import * + +# Fedora signing keys. +sigkeys = ['f577861e', '6d9f90a6'] + +# Architectures supported by Fedora ELN. +tree_arches = ['aarch64', 'ppc64le', 's390x', 'x86_64'] + +# For Fedora-ELN, we do not inherit builds from parent tags. +pkgset_koji_inherit = False + +# No jigdo needed in Fedora. +create_jigdo = False + +# We only build repositories, installer and images in Fedora so far. +skip_phases = [ + "createiso", + "live_media", + "live_images", + "ostree", + "osbs", +] + +# Enables macboot on x86_64 for all variants and disables upgrade image building +# everywhere. +# Use 4GB image size for all arches. +lorax_options = [ + ('^.*$', { + 'x86_64': { + 'nomacboot': True + }, + '*': { + 'rootfs_size': 4 + } + }) +] + +# Drop the variants we do not care about from the variant_as_lookaside. +variant_as_lookaside = [ + ("AppStream", "BaseOS"), + ("Extras", "BaseOS"), + ("Extras", "AppStream"), + ("Extras", "CRB"), + ("HighAvailability", "BaseOS"), + ("HighAvailability", "AppStream"), + ("RT", "BaseOS"), + ("RT", "AppStream"), + ("NFV", "BaseOS"), + ("NFV", "AppStream"), + ("CRB", "BaseOS"), + ("CRB", "AppStream"), + ("SAP", "BaseOS"), + ("SAP", "AppStream"), + ("SAP", "HighAvailability"), + ("SAPHANA", "BaseOS"), + ("SAPHANA", "AppStream"), + ("SAPHANA", "HighAvailability"), + ("Buildroot", "BaseOS"), + ("Buildroot", "AppStream"), + ("Buildroot", "CRB"), + ("Buildroot", "Extras"), + ("Buildroot", "HighAvailability"), + ("Buildroot", "NFV"), + ("Buildroot", "RT"), + ("Buildroot", "SAP"), + ("Buildroot", "SAPHANA"), +] + +# No product_id for Fedora. +product_id_allow_missing = False + +# These will be inherited by live_media, live_images and image_build +global_release = '!RELEASE_FROM_LABEL_DATE_TYPE_RESPIN' +global_version = 'ELN' +# live_images ignores this in favor of live_target +global_target = 'eln' + +# kiwi images need another target that uses old mock chroot +kiwibuild_target = 'eln-kiwi' + +# kiwi image global configuration +kiwibuild_description_scm = 'git+https://pagure.io/fedora-kiwi-descriptions.git?#HEAD' +kiwibuild_description_path = 'Fedora-ELN.kiwi' +kiwibuild_version = '11' +kiwibuild_repo_releasever = 'eln' + +# --. +kiwibuild_bundle_name_format = '%N-%v-%I.%A' diff --git a/tests/testdata/repos/elnbase/repodata/6212147882b929cefe09ce147e26e1020b738656c9517eac9a52a058d7793a31-other.xml.zst b/tests/testdata/repos/elnbase/repodata/6212147882b929cefe09ce147e26e1020b738656c9517eac9a52a058d7793a31-other.xml.zst deleted file mode 100644 index b812bc1..0000000 Binary files a/tests/testdata/repos/elnbase/repodata/6212147882b929cefe09ce147e26e1020b738656c9517eac9a52a058d7793a31-other.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/elnbase/repodata/e66e2d334117944d094138fdc62fda05db2e252cd29aeffa765a71451a8856dc-filelists.xml.zst b/tests/testdata/repos/elnbase/repodata/e66e2d334117944d094138fdc62fda05db2e252cd29aeffa765a71451a8856dc-filelists.xml.zst deleted file mode 100644 index 2253c3a..0000000 Binary files a/tests/testdata/repos/elnbase/repodata/e66e2d334117944d094138fdc62fda05db2e252cd29aeffa765a71451a8856dc-filelists.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/elnbase/repodata/f4ed4ad1632ef201cb823cb62f2408918097df27d66aaee819fc22ba37ab262c-primary.xml.zst b/tests/testdata/repos/elnbase/repodata/f4ed4ad1632ef201cb823cb62f2408918097df27d66aaee819fc22ba37ab262c-primary.xml.zst deleted file mode 100644 index d3994c4..0000000 Binary files a/tests/testdata/repos/elnbase/repodata/f4ed4ad1632ef201cb823cb62f2408918097df27d66aaee819fc22ba37ab262c-primary.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/elnbase/repodata/repomd.xml b/tests/testdata/repos/elnbase/repodata/repomd.xml deleted file mode 100644 index 99dfb7b..0000000 --- a/tests/testdata/repos/elnbase/repodata/repomd.xml +++ /dev/null @@ -1,28 +0,0 @@ - - - 1777498823 - - f4ed4ad1632ef201cb823cb62f2408918097df27d66aaee819fc22ba37ab262c - 12c7b2df4b3e597c8a412e6b187f0379f9024b0c19db0a6a22615dd1b5b09962 - - 1777498823 - 699 - 2263 - - - e66e2d334117944d094138fdc62fda05db2e252cd29aeffa765a71451a8856dc - 48d1e74e7b0cb3bf5286c6bf23c19000cb94afdf982c113b926688e51b100a36 - - 1777498823 - 279 - 445 - - - 6212147882b929cefe09ce147e26e1020b738656c9517eac9a52a058d7793a31 - c8b2793ecacedcdba8b304a308130a84ac2613447053514f1e0f040cf5220132 - - 1777498823 - 351 - 665 - - diff --git a/tests/testdata/repos/elnnew/epa-2.0-1.x86_64.rpm b/tests/testdata/repos/elnnew/epa-2.0-1.x86_64.rpm new file mode 100644 index 0000000..0cefec8 Binary files /dev/null and b/tests/testdata/repos/elnnew/epa-2.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/elnroot/epa-1.0-1.x86_64.rpm b/tests/testdata/repos/elnnew/epb-2.0-1.x86_64.rpm similarity index 72% rename from tests/testdata/repos/elnroot/epa-1.0-1.x86_64.rpm rename to tests/testdata/repos/elnnew/epb-2.0-1.x86_64.rpm index 999448a..9f85440 100644 Binary files a/tests/testdata/repos/elnroot/epa-1.0-1.x86_64.rpm and b/tests/testdata/repos/elnnew/epb-2.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/elnnew/epc-2.0-1.x86_64.rpm b/tests/testdata/repos/elnnew/epd-2.0-1.x86_64.rpm similarity index 71% rename from tests/testdata/repos/elnnew/epc-2.0-1.x86_64.rpm rename to tests/testdata/repos/elnnew/epd-2.0-1.x86_64.rpm index 3c861ee..5f29fd3 100644 Binary files a/tests/testdata/repos/elnnew/epc-2.0-1.x86_64.rpm and b/tests/testdata/repos/elnnew/epd-2.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/elnnew/repodata/029e08c67f723be4b217594371bb7f9f223cb866b934d857d9bddb4234e9c942-filelists.xml.zst b/tests/testdata/repos/elnnew/repodata/029e08c67f723be4b217594371bb7f9f223cb866b934d857d9bddb4234e9c942-filelists.xml.zst deleted file mode 100644 index add7236..0000000 Binary files a/tests/testdata/repos/elnnew/repodata/029e08c67f723be4b217594371bb7f9f223cb866b934d857d9bddb4234e9c942-filelists.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/elnnew/repodata/6f2f32cc8b75d194c83ef1075936ed263a468f298ce34bf64a112a510aed46f8-filelists.xml.zst b/tests/testdata/repos/elnnew/repodata/6f2f32cc8b75d194c83ef1075936ed263a468f298ce34bf64a112a510aed46f8-filelists.xml.zst new file mode 100644 index 0000000..aef1824 Binary files /dev/null and b/tests/testdata/repos/elnnew/repodata/6f2f32cc8b75d194c83ef1075936ed263a468f298ce34bf64a112a510aed46f8-filelists.xml.zst differ diff --git a/tests/testdata/repos/elnnew/repodata/b5d3790249f642dafa96a7b5ba61938b50e19cc4c57fe63d7e4206a869c10925-other.xml.zst b/tests/testdata/repos/elnnew/repodata/b5d3790249f642dafa96a7b5ba61938b50e19cc4c57fe63d7e4206a869c10925-other.xml.zst new file mode 100644 index 0000000..3ed2ec5 Binary files /dev/null and b/tests/testdata/repos/elnnew/repodata/b5d3790249f642dafa96a7b5ba61938b50e19cc4c57fe63d7e4206a869c10925-other.xml.zst differ diff --git a/tests/testdata/repos/elnnew/repodata/c083429e71ea2c4bea418159abf7debbf79dd70f21731b1b4acc5195658e7890-other.xml.zst b/tests/testdata/repos/elnnew/repodata/c083429e71ea2c4bea418159abf7debbf79dd70f21731b1b4acc5195658e7890-other.xml.zst deleted file mode 100644 index f8f4ba7..0000000 Binary files a/tests/testdata/repos/elnnew/repodata/c083429e71ea2c4bea418159abf7debbf79dd70f21731b1b4acc5195658e7890-other.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/elnnew/repodata/e1fb8cbc0efa75785de59ffca8a3de5dd3928ab2075891beabbdedf2a74608b0-primary.xml.zst b/tests/testdata/repos/elnnew/repodata/e1fb8cbc0efa75785de59ffca8a3de5dd3928ab2075891beabbdedf2a74608b0-primary.xml.zst deleted file mode 100644 index 0167dc0..0000000 Binary files a/tests/testdata/repos/elnnew/repodata/e1fb8cbc0efa75785de59ffca8a3de5dd3928ab2075891beabbdedf2a74608b0-primary.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/elnnew/repodata/ec7341345e04def216d16e0db04b625570797b0142fe7150924d08f34acc1356-primary.xml.zst b/tests/testdata/repos/elnnew/repodata/ec7341345e04def216d16e0db04b625570797b0142fe7150924d08f34acc1356-primary.xml.zst new file mode 100644 index 0000000..8e7edbf Binary files /dev/null and b/tests/testdata/repos/elnnew/repodata/ec7341345e04def216d16e0db04b625570797b0142fe7150924d08f34acc1356-primary.xml.zst differ diff --git a/tests/testdata/repos/elnnew/repodata/repomd.xml b/tests/testdata/repos/elnnew/repodata/repomd.xml index 76f3a69..d1a5bc8 100644 --- a/tests/testdata/repos/elnnew/repodata/repomd.xml +++ b/tests/testdata/repos/elnnew/repodata/repomd.xml @@ -1,28 +1,28 @@ - 1777498823 + 1778717671 - e1fb8cbc0efa75785de59ffca8a3de5dd3928ab2075891beabbdedf2a74608b0 - edfd3c129d9db24508a6dcd38765281f02cdf98609587ce9ccc3885cd226237f - - 1777498823 - 629 - 1265 + ec7341345e04def216d16e0db04b625570797b0142fe7150924d08f34acc1356 + c09d2fb0e387f9b1d76bddbda53a17ff410e7376c32765e8c1883e8aa5125e10 + + 1778717671 + 764 + 3299 - 029e08c67f723be4b217594371bb7f9f223cb866b934d857d9bddb4234e9c942 - 8519cd36f5dcc309402aa70eb2463eebd439a134f410ee2b052afe51dddb6265 - - 1777498823 - 226 - 285 + 6f2f32cc8b75d194c83ef1075936ed263a468f298ce34bf64a112a510aed46f8 + d1edbf9e8bb2c6455b8b5ad6506b19b3d89bea5e0e4cc07eb3f675ab536a090a + + 1778717671 + 323 + 605 - c083429e71ea2c4bea418159abf7debbf79dd70f21731b1b4acc5195658e7890 - bd83ff93ecd4e37aa7dc35a1dfc61a5c8d560c14eea47fe443c86ca184853289 - - 1777498823 - 305 - 393 + b5d3790249f642dafa96a7b5ba61938b50e19cc4c57fe63d7e4206a869c10925 + 7246bec78e312d7765aa885c6e196cf347adf944323c9d7593d57ef92340f763 + + 1778717671 + 398 + 937 diff --git a/tests/testdata/repos/elnroot/repodata/081516c8122bf9e116d0314902b635ef595c121ba2e7a9d8d448de7e636c948b-filelists.xml.zst b/tests/testdata/repos/elnroot/repodata/081516c8122bf9e116d0314902b635ef595c121ba2e7a9d8d448de7e636c948b-filelists.xml.zst deleted file mode 100644 index 89eb180..0000000 Binary files a/tests/testdata/repos/elnroot/repodata/081516c8122bf9e116d0314902b635ef595c121ba2e7a9d8d448de7e636c948b-filelists.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/elnroot/repodata/117a29ab84c4728bf8da3b0b942c823be18f4d190e710ed797182925cfad70ba-other.xml.zst b/tests/testdata/repos/elnroot/repodata/117a29ab84c4728bf8da3b0b942c823be18f4d190e710ed797182925cfad70ba-other.xml.zst deleted file mode 100644 index f3f39d0..0000000 Binary files a/tests/testdata/repos/elnroot/repodata/117a29ab84c4728bf8da3b0b942c823be18f4d190e710ed797182925cfad70ba-other.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/elnroot/repodata/5548c16c17c9999a288149e4c028529fea3de98d310a6f6b9aaf5f9913c7ac7b-primary.xml.zst b/tests/testdata/repos/elnroot/repodata/5548c16c17c9999a288149e4c028529fea3de98d310a6f6b9aaf5f9913c7ac7b-primary.xml.zst deleted file mode 100644 index a413f1d..0000000 Binary files a/tests/testdata/repos/elnroot/repodata/5548c16c17c9999a288149e4c028529fea3de98d310a6f6b9aaf5f9913c7ac7b-primary.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/elnroot/repodata/repomd.xml b/tests/testdata/repos/elnroot/repodata/repomd.xml deleted file mode 100644 index 8e9d54f..0000000 --- a/tests/testdata/repos/elnroot/repodata/repomd.xml +++ /dev/null @@ -1,28 +0,0 @@ - - - 1777498823 - - 5548c16c17c9999a288149e4c028529fea3de98d310a6f6b9aaf5f9913c7ac7b - a1a05ef98f83ff3a3d3f9916306cd31ca7a948599849ec779a95d6f7e8a40214 - - 1777498823 - 830 - 4359 - - - 081516c8122bf9e116d0314902b635ef595c121ba2e7a9d8d448de7e636c948b - d25e63e1db6cb9a61a2a98c0ad43abda59e7826aa7551738079e02b16d75a912 - - 1777498823 - 366 - 765 - - - 117a29ab84c4728bf8da3b0b942c823be18f4d190e710ed797182925cfad70ba - a68bfd0ad8980ca8b2f26e4498cba5d9eb2c30885ca8fd1cac84f655687fe63f - - 1777498823 - 445 - 1209 - - diff --git a/tests/testdata/repos/empty/repodata/repomd.xml b/tests/testdata/repos/empty/repodata/repomd.xml index eb5dc0c..ab7abf7 100644 --- a/tests/testdata/repos/empty/repodata/repomd.xml +++ b/tests/testdata/repos/empty/repodata/repomd.xml @@ -1,11 +1,11 @@ - 1777498823 + 1778717671 69a3730a283b85a4b3cff7d04bfde3b2b234f0607ebc17319d7a8d143a8e066e e1e2ffd2fb1ee76f87b70750d00ca5677a252b397ab6c2389137a0c33e7b359f - 1777498823 + 1778717671 123 167 @@ -13,7 +13,7 @@ 9b07d97dc6ececed89aac0650b67bfb292647fe9fbaca48f629465be5f53f82b bf9808b81cb2dbc54b4b8e35adc584ddcaa73bd81f7088d73bf7dbbada961310 - 1777498823 + 1778717671 118 125 @@ -21,7 +21,7 @@ 6b37cc67608a24beaa81e1191d218f2ffd6b1191dceb5c100bac2e66249d518d e0ed5e0054194df036cf09c1a911e15bf2a4e7f26f2a788b6f47d53e80717ccc - 1777498823 + 1778717671 117 121 diff --git a/tests/testdata/repos/mkrepos.py b/tests/testdata/repos/mkrepos.py index 2df3938..de827f2 100755 --- a/tests/testdata/repos/mkrepos.py +++ b/tests/testdata/repos/mkrepos.py @@ -86,24 +86,34 @@ newddd = rpmfluff.SimpleRpmBuild("ddd", "3.0", "1", ["x86_64"]) newddd.add_requires("ccc = 3.0") unfixed = (newddd,) -# ELN scenario packages. The "shipped repo" will contain only -# epa and epc. The "buildroot repo" will contain all four +# ELN scenario packages epa = rpmfluff.SimpleRpmBuild("epa", "1.0", "1", ["x86_64"]) epb = rpmfluff.SimpleRpmBuild("epb", "1.0", "1", ["x86_64"]) epc = rpmfluff.SimpleRpmBuild("epc", "1.0", "1", ["x86_64"]) epd = rpmfluff.SimpleRpmBuild("epd", "1.0", "1", ["x86_64"]) -alleln = (epa, epb, epc, epd) -shipeln = (epa, epc) -newepc = rpmfluff.SimpleRpmBuild("epc", "2.0", "1", ["x86_64"]) -# these will both be broken by the new epc, but only epa is in the -# "shipped repo", epb is not -epa.add_requires("epc = 1.0") -epb.add_requires("epc = 1.0") -# this requirement can only be satisfied in the buildroot repo, we -# should not report that as an error -newepc.add_requires("epd = 1.0") +epe = rpmfluff.SimpleRpmBuild("epe", "1.0", "1", ["x86_64"]) +epf = rpmfluff.SimpleRpmBuild("epe", "1.0", "1", ["x86_64"]) +alleln = (epa, epb, epc, epd, epf) +baseeln = (epa, epb) +exteln = (epc,) +crbeln = (epd, epe) +rooteln = (epf,) -for pkg in allbase + allupd + allnew + unfixed + alleln + (newepc,): +newepa = rpmfluff.SimpleRpmBuild("epa", "2.0", "1", ["x86_64"]) +newepb = rpmfluff.SimpleRpmBuild("epb", "2.0", "1", ["x86_64"]) +newepd = rpmfluff.SimpleRpmBuild("epd", "2.0", "1", ["x86_64"]) +neweln = (newepa, newepb, newepd) + +# these will both be broken by the new epa, but epf should be ignored +# as it's buildroot-only +epe.add_requires("epa = 1.0") +epf.add_requires("epa = 1.0") +# this is invalid as baseos cannot depend on extras, should be caught +newepb.add_requires("epc") +# this is fine +newepd.add_requires("epa") + +for pkg in allbase + allupd + allnew + unfixed + alleln + neweln: pkg.addVendor("Fedora Project") pkg.addPackager("Fedora Project") @@ -115,16 +125,25 @@ unfixed = rpmfluff.yumrepobuild.YumRepoBuild(unfixed) unfixed.repoDir = "unfixed" updates = rpmfluff.yumrepobuild.YumRepoBuild(allupd) updates.repoDir = "updates" -elnbase = rpmfluff.yumrepobuild.YumRepoBuild(shipeln) -elnbase.repoDir = "elnbase" -elnroot = rpmfluff.yumrepobuild.YumRepoBuild(alleln) -elnroot.repoDir = "elnroot" -elnnew = rpmfluff.yumrepobuild.YumRepoBuild((newepc,)) +elnbase = rpmfluff.yumrepobuild.YumRepoBuild(baseeln) +elnbase.repoDir = "eln/BaseOS" +elnext = rpmfluff.yumrepobuild.YumRepoBuild(exteln) +elnext.repoDir = "eln/Extras" +elncrb = rpmfluff.yumrepobuild.YumRepoBuild(crbeln) +elncrb.repoDir = "eln/CRB" +elnapp = rpmfluff.yumrepobuild.YumRepoBuild([]) +elnapp.repoDir = "eln/AppStream" +elnroot = rpmfluff.yumrepobuild.YumRepoBuild(rooteln) +elnroot.repoDir = "eln/Buildroot" +elnnew = rpmfluff.yumrepobuild.YumRepoBuild(neweln) elnnew.repoDir = "elnnew" empty = rpmfluff.yumrepobuild.YumRepoBuild([]) empty.repoDir = "empty" -alldirs = [r.repoDir for r in (base, new, unfixed, updates, elnbase, elnroot, elnnew, empty)] +alldirs = [ + r.repoDir + for r in (base, new, unfixed, updates, elnbase, elnext, elncrb, elnapp, elnroot, elnnew, empty) +] def cleanup(repos=True): @@ -140,14 +159,18 @@ def cleanup(repos=True): cleanup() for _dir in alldirs: - os.mkdir(_dir) + os.makedirs(_dir) base.make("x86_64", "i686") new.make("x86_64") unfixed.make("x86_64") updates.make("x86_64") elnbase.make("x86_64") +elnext.make("x86_64") +elncrb.make("x86_64") +elnapp.make("x86_64") elnroot.make("x86_64") elnnew.make("x86_64") empty.make() + cleanup(repos=False) diff --git a/tests/testdata/repos/new/111-3.0-1.x86_64.rpm b/tests/testdata/repos/new/111-3.0-1.x86_64.rpm index 12aa417..96cd37d 100644 Binary files a/tests/testdata/repos/new/111-3.0-1.x86_64.rpm and b/tests/testdata/repos/new/111-3.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/new/222-3.0-1.x86_64.rpm b/tests/testdata/repos/new/222-3.0-1.x86_64.rpm index 1f3796c..6663fc8 100644 Binary files a/tests/testdata/repos/new/222-3.0-1.x86_64.rpm and b/tests/testdata/repos/new/222-3.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/new/aaa-3.0-1.x86_64.rpm b/tests/testdata/repos/new/aaa-3.0-1.x86_64.rpm index 67841e2..3959582 100644 Binary files a/tests/testdata/repos/new/aaa-3.0-1.x86_64.rpm and b/tests/testdata/repos/new/aaa-3.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/new/ccc-3.0-1.x86_64.rpm b/tests/testdata/repos/new/ccc-3.0-1.x86_64.rpm index 940afd3..5ce93b6 100644 Binary files a/tests/testdata/repos/new/ccc-3.0-1.x86_64.rpm and b/tests/testdata/repos/new/ccc-3.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/new/eee-3.0-1.x86_64.rpm b/tests/testdata/repos/new/eee-3.0-1.x86_64.rpm index a289fd6..f1be63b 100644 Binary files a/tests/testdata/repos/new/eee-3.0-1.x86_64.rpm and b/tests/testdata/repos/new/eee-3.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/new/fff-3.0-1.x86_64.rpm b/tests/testdata/repos/new/fff-3.0-1.x86_64.rpm index 8709ce1..83376e1 100644 Binary files a/tests/testdata/repos/new/fff-3.0-1.x86_64.rpm and b/tests/testdata/repos/new/fff-3.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/new/ggg-3.0-1.x86_64.rpm b/tests/testdata/repos/new/ggg-3.0-1.x86_64.rpm index e0082d2..417c532 100644 Binary files a/tests/testdata/repos/new/ggg-3.0-1.x86_64.rpm and b/tests/testdata/repos/new/ggg-3.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/new/repodata/0a7860c188784cab59dcc483eea7c38d8e9da287f45492897b95bd1a425da7ce-primary.xml.zst b/tests/testdata/repos/new/repodata/0a7860c188784cab59dcc483eea7c38d8e9da287f45492897b95bd1a425da7ce-primary.xml.zst new file mode 100644 index 0000000..f9165d5 Binary files /dev/null and b/tests/testdata/repos/new/repodata/0a7860c188784cab59dcc483eea7c38d8e9da287f45492897b95bd1a425da7ce-primary.xml.zst differ diff --git a/tests/testdata/repos/new/repodata/182e2add8c3d3069057852ed9944419578aa72b31eb9233b18b6ce68fa86ba05-primary.xml.zst b/tests/testdata/repos/new/repodata/182e2add8c3d3069057852ed9944419578aa72b31eb9233b18b6ce68fa86ba05-primary.xml.zst deleted file mode 100644 index ca55152..0000000 Binary files a/tests/testdata/repos/new/repodata/182e2add8c3d3069057852ed9944419578aa72b31eb9233b18b6ce68fa86ba05-primary.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/new/repodata/4f693bcdc44f61c53c77e39ee300085d9e54b731fb75a9b664a8e0b9f6516c09-other.xml.zst b/tests/testdata/repos/new/repodata/4f693bcdc44f61c53c77e39ee300085d9e54b731fb75a9b664a8e0b9f6516c09-other.xml.zst new file mode 100644 index 0000000..5272993 Binary files /dev/null and b/tests/testdata/repos/new/repodata/4f693bcdc44f61c53c77e39ee300085d9e54b731fb75a9b664a8e0b9f6516c09-other.xml.zst differ diff --git a/tests/testdata/repos/new/repodata/8e6cd282fee67920e9fd5728c68c86ad8d8ca56f9e8db935bd9358e22f4b86a8-filelists.xml.zst b/tests/testdata/repos/new/repodata/8e6cd282fee67920e9fd5728c68c86ad8d8ca56f9e8db935bd9358e22f4b86a8-filelists.xml.zst deleted file mode 100644 index a8d3c63..0000000 Binary files a/tests/testdata/repos/new/repodata/8e6cd282fee67920e9fd5728c68c86ad8d8ca56f9e8db935bd9358e22f4b86a8-filelists.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/new/repodata/cf45583b0c95946ca4ca3d567a3da5db1183158763f9b2dc006bc980a91f7393-filelists.xml.zst b/tests/testdata/repos/new/repodata/cf45583b0c95946ca4ca3d567a3da5db1183158763f9b2dc006bc980a91f7393-filelists.xml.zst new file mode 100644 index 0000000..0581c18 Binary files /dev/null and b/tests/testdata/repos/new/repodata/cf45583b0c95946ca4ca3d567a3da5db1183158763f9b2dc006bc980a91f7393-filelists.xml.zst differ diff --git a/tests/testdata/repos/new/repodata/d6dae028f2d94a20a5ada403a89d418b7d31c646d2350605999882725a5b0862-other.xml.zst b/tests/testdata/repos/new/repodata/d6dae028f2d94a20a5ada403a89d418b7d31c646d2350605999882725a5b0862-other.xml.zst deleted file mode 100644 index a2b6b67..0000000 Binary files a/tests/testdata/repos/new/repodata/d6dae028f2d94a20a5ada403a89d418b7d31c646d2350605999882725a5b0862-other.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/new/repodata/repomd.xml b/tests/testdata/repos/new/repodata/repomd.xml index 2ca5748..c369103 100644 --- a/tests/testdata/repos/new/repodata/repomd.xml +++ b/tests/testdata/repos/new/repodata/repomd.xml @@ -1,28 +1,28 @@ - 1777498821 + 1778717669 - 182e2add8c3d3069057852ed9944419578aa72b31eb9233b18b6ce68fa86ba05 - 4ffb8600b26cbe3f260b10f2f7c35e17c92839fe66e5f1f13e37ec170f4d0d16 - - 1777498821 - 1069 + 0a7860c188784cab59dcc483eea7c38d8e9da287f45492897b95bd1a425da7ce + 2ba4a2dbf3603d8fde7bd5d1595d18058f834b485a9611f49e18acca29b3cc61 + + 1778717669 + 1067 7430 - 8e6cd282fee67920e9fd5728c68c86ad8d8ca56f9e8db935bd9358e22f4b86a8 - 601f4c051e7fb1541e74b63516ad2f5fceecafae40b02a15bef878f1e941b159 - - 1777498821 + cf45583b0c95946ca4ca3d567a3da5db1183158763f9b2dc006bc980a91f7393 + 69839c5fe82af86f9d68cf88757f8886943d9d0ca5387636315eb27330670e7c + + 1778717669 499 1245 - d6dae028f2d94a20a5ada403a89d418b7d31c646d2350605999882725a5b0862 - bfbc10574abc73bf3586a122246953f8073cde56afb8b4470d856ffe0d7823b0 - - 1777498821 - 581 + 4f693bcdc44f61c53c77e39ee300085d9e54b731fb75a9b664a8e0b9f6516c09 + c5f2112ab0f5b051aa8dc39e900a07ac7ba66294f08f144708bba996cb5c7881 + + 1778717669 + 580 2025 diff --git a/tests/testdata/repos/unfixed/ddd-3.0-1.x86_64.rpm b/tests/testdata/repos/unfixed/ddd-3.0-1.x86_64.rpm index 005aac1..eee3a3c 100644 Binary files a/tests/testdata/repos/unfixed/ddd-3.0-1.x86_64.rpm and b/tests/testdata/repos/unfixed/ddd-3.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/unfixed/repodata/77dfe7c593c3e37af3f924516b5aa25cc512a5b8070f75fbeb4b1e844973835b-filelists.xml.zst b/tests/testdata/repos/unfixed/repodata/77dfe7c593c3e37af3f924516b5aa25cc512a5b8070f75fbeb4b1e844973835b-filelists.xml.zst deleted file mode 100644 index 13d4a57..0000000 Binary files a/tests/testdata/repos/unfixed/repodata/77dfe7c593c3e37af3f924516b5aa25cc512a5b8070f75fbeb4b1e844973835b-filelists.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/unfixed/repodata/9d54f4c3d3ab7c943b7e15193e2544bcb82bf7b2e8408ee3167134b0a453255d-filelists.xml.zst b/tests/testdata/repos/unfixed/repodata/9d54f4c3d3ab7c943b7e15193e2544bcb82bf7b2e8408ee3167134b0a453255d-filelists.xml.zst new file mode 100644 index 0000000..ec47fbc Binary files /dev/null and b/tests/testdata/repos/unfixed/repodata/9d54f4c3d3ab7c943b7e15193e2544bcb82bf7b2e8408ee3167134b0a453255d-filelists.xml.zst differ diff --git a/tests/testdata/repos/unfixed/repodata/a583ea09de5b85171eb9110dacfd240e527828cff25324ce894d3dd9c80cf961-other.xml.zst b/tests/testdata/repos/unfixed/repodata/a583ea09de5b85171eb9110dacfd240e527828cff25324ce894d3dd9c80cf961-other.xml.zst deleted file mode 100644 index 8d15cec..0000000 Binary files a/tests/testdata/repos/unfixed/repodata/a583ea09de5b85171eb9110dacfd240e527828cff25324ce894d3dd9c80cf961-other.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/unfixed/repodata/daa25dfd3c818ca6b5104612d827dac24e70a2050986ed49457124116013f4ef-primary.xml.zst b/tests/testdata/repos/unfixed/repodata/daa25dfd3c818ca6b5104612d827dac24e70a2050986ed49457124116013f4ef-primary.xml.zst deleted file mode 100644 index ef3f0d8..0000000 Binary files a/tests/testdata/repos/unfixed/repodata/daa25dfd3c818ca6b5104612d827dac24e70a2050986ed49457124116013f4ef-primary.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/unfixed/repodata/dd80d9c03852999e37b04db876c82b0483a6e906c5884b4561d83a97d7c341c7-primary.xml.zst b/tests/testdata/repos/unfixed/repodata/dd80d9c03852999e37b04db876c82b0483a6e906c5884b4561d83a97d7c341c7-primary.xml.zst new file mode 100644 index 0000000..2e0a2b6 Binary files /dev/null and b/tests/testdata/repos/unfixed/repodata/dd80d9c03852999e37b04db876c82b0483a6e906c5884b4561d83a97d7c341c7-primary.xml.zst differ diff --git a/tests/testdata/repos/unfixed/repodata/ef5302dc1a621cb60e1087af579f4e2bb66e668033b107b8706a39f2c37f8026-other.xml.zst b/tests/testdata/repos/unfixed/repodata/ef5302dc1a621cb60e1087af579f4e2bb66e668033b107b8706a39f2c37f8026-other.xml.zst new file mode 100644 index 0000000..6f8e429 Binary files /dev/null and b/tests/testdata/repos/unfixed/repodata/ef5302dc1a621cb60e1087af579f4e2bb66e668033b107b8706a39f2c37f8026-other.xml.zst differ diff --git a/tests/testdata/repos/unfixed/repodata/repomd.xml b/tests/testdata/repos/unfixed/repodata/repomd.xml index 1d5c82a..7688873 100644 --- a/tests/testdata/repos/unfixed/repodata/repomd.xml +++ b/tests/testdata/repos/unfixed/repodata/repomd.xml @@ -1,27 +1,27 @@ - 1777498822 + 1778717669 - daa25dfd3c818ca6b5104612d827dac24e70a2050986ed49457124116013f4ef - ee07278548ed0fee3569e6b1aa00a4488ef82c0216d5fb835899edca81d00ebe - - 1777498822 - 626 + dd80d9c03852999e37b04db876c82b0483a6e906c5884b4561d83a97d7c341c7 + 291264d5bc3e5102ea3553e1f3a8017e7bed89c574fde750f0247c85c2003448 + + 1778717669 + 628 1265 - 77dfe7c593c3e37af3f924516b5aa25cc512a5b8070f75fbeb4b1e844973835b - 7e2dfb6357481ec47ff798fc40501850c014472d258cccc9ac49ec9c1d176acb - - 1777498822 + 9d54f4c3d3ab7c943b7e15193e2544bcb82bf7b2e8408ee3167134b0a453255d + 207660f59451194f04a8c198b5244310cf182ecdbaaae5e819f18ad3851acf1d + + 1778717669 225 285 - a583ea09de5b85171eb9110dacfd240e527828cff25324ce894d3dd9c80cf961 - 747fadb7b8374294f06c6ed1c48f21a25d7ac797625c574d13adf81377341014 - - 1777498822 + ef5302dc1a621cb60e1087af579f4e2bb66e668033b107b8706a39f2c37f8026 + 0e8c0475f1c246f3d2dbb831a437572872bb464a55e356b01808ad4009eec9ba + + 1778717669 305 393 diff --git a/tests/testdata/repos/updates/aaa-2.0-1.x86_64.rpm b/tests/testdata/repos/updates/aaa-2.0-1.x86_64.rpm index 942fadc..7a9474b 100644 Binary files a/tests/testdata/repos/updates/aaa-2.0-1.x86_64.rpm and b/tests/testdata/repos/updates/aaa-2.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/updates/bbb-2.0-1.x86_64.rpm b/tests/testdata/repos/updates/bbb-2.0-1.x86_64.rpm index 784a613..eac9d1e 100644 Binary files a/tests/testdata/repos/updates/bbb-2.0-1.x86_64.rpm and b/tests/testdata/repos/updates/bbb-2.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/updates/ccc-2.0-1.x86_64.rpm b/tests/testdata/repos/updates/ccc-2.0-1.x86_64.rpm index 5182af8..1934386 100644 Binary files a/tests/testdata/repos/updates/ccc-2.0-1.x86_64.rpm and b/tests/testdata/repos/updates/ccc-2.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/updates/ddd-2.0-1.x86_64.rpm b/tests/testdata/repos/updates/ddd-2.0-1.x86_64.rpm index 34a8717..3cd07d6 100644 Binary files a/tests/testdata/repos/updates/ddd-2.0-1.x86_64.rpm and b/tests/testdata/repos/updates/ddd-2.0-1.x86_64.rpm differ diff --git a/tests/testdata/repos/updates/repodata/3126b56a66fb9df617bc7a7c1182bf8142d919f9852096bb6f61077a40e6ea2b-filelists.xml.zst b/tests/testdata/repos/updates/repodata/3126b56a66fb9df617bc7a7c1182bf8142d919f9852096bb6f61077a40e6ea2b-filelists.xml.zst new file mode 100644 index 0000000..bd0b310 Binary files /dev/null and b/tests/testdata/repos/updates/repodata/3126b56a66fb9df617bc7a7c1182bf8142d919f9852096bb6f61077a40e6ea2b-filelists.xml.zst differ diff --git a/tests/testdata/repos/updates/repodata/5f112f5ba0bd830d4f7d2b42a6d57c1041d78b661a423c3e00b0b6072fe3d690-other.xml.zst b/tests/testdata/repos/updates/repodata/5f112f5ba0bd830d4f7d2b42a6d57c1041d78b661a423c3e00b0b6072fe3d690-other.xml.zst deleted file mode 100644 index 68a0797..0000000 Binary files a/tests/testdata/repos/updates/repodata/5f112f5ba0bd830d4f7d2b42a6d57c1041d78b661a423c3e00b0b6072fe3d690-other.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/updates/repodata/63c56609259c2a5620e4da4ccf34898028efb88337163d5b44e75a95e684d91a-primary.xml.zst b/tests/testdata/repos/updates/repodata/63c56609259c2a5620e4da4ccf34898028efb88337163d5b44e75a95e684d91a-primary.xml.zst new file mode 100644 index 0000000..ba1613e Binary files /dev/null and b/tests/testdata/repos/updates/repodata/63c56609259c2a5620e4da4ccf34898028efb88337163d5b44e75a95e684d91a-primary.xml.zst differ diff --git a/tests/testdata/repos/updates/repodata/736fcb107df83b5ae000865878a1f670603afb579fbb3179e2ce4d72c9586e48-primary.xml.zst b/tests/testdata/repos/updates/repodata/736fcb107df83b5ae000865878a1f670603afb579fbb3179e2ce4d72c9586e48-primary.xml.zst deleted file mode 100644 index a9baa9b..0000000 Binary files a/tests/testdata/repos/updates/repodata/736fcb107df83b5ae000865878a1f670603afb579fbb3179e2ce4d72c9586e48-primary.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/updates/repodata/ca771b7fed377c578d3d9ffd1b856a9c1c0e5192ad31e2d111d1ae2ed978e34c-filelists.xml.zst b/tests/testdata/repos/updates/repodata/ca771b7fed377c578d3d9ffd1b856a9c1c0e5192ad31e2d111d1ae2ed978e34c-filelists.xml.zst deleted file mode 100644 index e9be59a..0000000 Binary files a/tests/testdata/repos/updates/repodata/ca771b7fed377c578d3d9ffd1b856a9c1c0e5192ad31e2d111d1ae2ed978e34c-filelists.xml.zst and /dev/null differ diff --git a/tests/testdata/repos/updates/repodata/e07157006906a954f6b20bd3998d53515332d76c45c79661ca57763750c7b986-other.xml.zst b/tests/testdata/repos/updates/repodata/e07157006906a954f6b20bd3998d53515332d76c45c79661ca57763750c7b986-other.xml.zst new file mode 100644 index 0000000..9a976b3 Binary files /dev/null and b/tests/testdata/repos/updates/repodata/e07157006906a954f6b20bd3998d53515332d76c45c79661ca57763750c7b986-other.xml.zst differ diff --git a/tests/testdata/repos/updates/repodata/repomd.xml b/tests/testdata/repos/updates/repodata/repomd.xml index caf3cd5..b217b79 100644 --- a/tests/testdata/repos/updates/repodata/repomd.xml +++ b/tests/testdata/repos/updates/repodata/repomd.xml @@ -1,28 +1,28 @@ - 1777498822 + 1778717669 - 736fcb107df83b5ae000865878a1f670603afb579fbb3179e2ce4d72c9586e48 - a301ab11789be905d36e3f1bf6aa8afe8f62be4dd888781f35f040cda6630ede - - 1777498822 - 847 + 63c56609259c2a5620e4da4ccf34898028efb88337163d5b44e75a95e684d91a + e4fdec66dc3b24723eb9411bdeaef01027f5ba2a7d2d65f0095ab9ddf1d53847 + + 1778717669 + 852 4359 - ca771b7fed377c578d3d9ffd1b856a9c1c0e5192ad31e2d111d1ae2ed978e34c - 92c0e6ce6d1a917a7d9104471513ad2d1699245864e08d8fcb5d78260bf0127c - - 1777498822 - 369 + 3126b56a66fb9df617bc7a7c1182bf8142d919f9852096bb6f61077a40e6ea2b + dfcdc6f2d5453c4a32947c10ef20c3bb4c2f8026e61d31e0615f15e029e8e32d + + 1778717669 + 372 765 - 5f112f5ba0bd830d4f7d2b42a6d57c1041d78b661a423c3e00b0b6072fe3d690 - 981a53e7a783d642baf951a1efc14237eb9f11473cae778d566cada8e1a65c54 - - 1777498822 - 447 + e07157006906a954f6b20bd3998d53515332d76c45c79661ca57763750c7b986 + 012806c92cd1914cdd5705f7e268ab930836875587eba1bc5377d25a55fa485b + + 1778717669 + 448 1209 diff --git a/tests/testdata/test_e2e_devel.txt b/tests/testdata/test_e2e_devel.txt index a6bdf38..79cd11f 100644 --- a/tests/testdata/test_e2e_devel.txt +++ b/tests/testdata/test_e2e_devel.txt @@ -5,9 +5,9 @@ package: hhh-1.0-1.i686 from file://{REPOS}/base ggg(x86-32) New dependency problems in the tested packages themselves: -package: 111-3.0-1.x86_64 from file://{REPOS}/new +package: 111-3.0-1.x86_64 nonexistent -package: 222-3.0-1.x86_64 from file://{REPOS}/new +package: 222-3.0-1.x86_64 aaa = 1.0 Dependencies of other packages that would be FIXED by the tested packages: diff --git a/tests/testdata/test_e2e_devel_onlyerrors.txt b/tests/testdata/test_e2e_devel_onlyerrors.txt index 5ae3407..eb07a99 100644 --- a/tests/testdata/test_e2e_devel_onlyerrors.txt +++ b/tests/testdata/test_e2e_devel_onlyerrors.txt @@ -5,7 +5,7 @@ package: hhh-1.0-1.i686 from file://{REPOS}/base ggg(x86-32) New dependency problems in the tested packages themselves: -package: 111-3.0-1.x86_64 from file://{REPOS}/new +package: 111-3.0-1.x86_64 nonexistent -package: 222-3.0-1.x86_64 from file://{REPOS}/new +package: 222-3.0-1.x86_64 aaa = 1.0 diff --git a/tests/testdata/test_e2e_eln.txt b/tests/testdata/test_e2e_eln.txt index de149e1..031660e 100644 --- a/tests/testdata/test_e2e_eln.txt +++ b/tests/testdata/test_e2e_eln.txt @@ -1,3 +1,7 @@ Dependencies of other packages that would be BROKEN by the tested packages: -package: epa-1.0-1.x86_64 from file://{REPOS}/elnbase - epc = 1.0 +package: epe-1.0-1.x86_64 from http://localhost:5001/CRB + epa = 1.0 + +New dependency problems in the tested packages themselves: +package: epb-2.0-1.x86_64 from BaseOS + epc diff --git a/tests/testdata/test_e2e_updates.txt b/tests/testdata/test_e2e_updates.txt index 224ef22..062ba52 100644 --- a/tests/testdata/test_e2e_updates.txt +++ b/tests/testdata/test_e2e_updates.txt @@ -3,5 +3,5 @@ package: bbb-2.0-1.x86_64 from file://{REPOS}/updates aaa = 2.0 New dependency problems in the tested packages themselves: -package: 111-3.0-1.x86_64 from file://{REPOS}/new +package: 111-3.0-1.x86_64 nonexistent diff --git a/tox.ini b/tox.ini index d96cc57..bc5a1a4 100644 --- a/tox.ini +++ b/tox.ini @@ -7,8 +7,6 @@ deps = -r{toxinidir}/tests.requires commands = coverage run -m pytest {posargs} -setenv = - PYTHONPATH = {toxinidir} [testenv:ci] sitepackages = false deps = @@ -16,11 +14,12 @@ deps = skip_install = true ignore_errors = true commands = - black --check rmdepcheck.py tests/test_rmdepcheck.py tests/testdata/repos/mkrepos.py + black --check rdc-repos rdc-compose src/rmdepcheck tests/test_rmdepcheck.py tests/testdata/repos/mkrepos.py coverage combine coverage report coverage xml diff-cover coverage.xml --fail-under=100 diff-quality --violations=pylint --fail-under=100 - mypy rmdepcheck.py - + mypy src/rmdepcheck +setenv = + PYTHONPATH = {toxinidir}/src diff --git a/tox.requires b/tox.requires index 70d78d6..c25f561 100644 --- a/tox.requires +++ b/tox.requires @@ -4,3 +4,6 @@ diff-cover mypy pylint pytest-cov +requests +types-requests +urllib3