Missing repositories in grokmirror manifest

pawelwms commented

2026-02-27 18:58:58 +00:00

Description of request

Hello,

I was testing out a set-up of a mirror of Fedora’s dist-git repositories by following the Grokmirror instructions. While most of it worked without issues and the repos seem to get updates as the upstream is updated, I’ve noticed a few of the repositories are missing from Fedora’s manifest (https://src.fedoraproject.org/grokmirror/manifest.js.gz).

For the time being, I’ve identified these few:

f43-backgrounds
gap-srpm-macros
gnulib-l10n
llvm20
mariadb11.8
mjpegtools
mozjs140
ngtcp2
nodejs24
python-sphinxcontrib-doxylink
rust-pyo3-build-config0.25

I was able to clone these repositories manually from https://src.fedoraproject.org/rpms/<pkg>.git and the repos themselves don’t seem stale.

Would anyone know, why some repositories may be missing from the manifest? Or is it still maintained?

Thank you in advance for any help on this topic!

### Description of request Hello, I was testing out a set-up of a mirror of Fedora’s dist-git repositories by following the Grokmirror instructions. While most of it worked without issues and the repos seem to get updates as the upstream is updated, I’ve noticed a few of the repositories are missing from Fedora’s manifest (`https://src.fedoraproject.org/grokmirror/manifest.js.gz`). For the time being, I’ve identified these few: - f43-backgrounds - gap-srpm-macros - gnulib-l10n - llvm20 - mariadb11.8 - mjpegtools - mozjs140 - ngtcp2 - nodejs24 - python-sphinxcontrib-doxylink - rust-pyo3-build-config0.25 I was able to clone these repositories manually from `https://src.fedoraproject.org/rpms/<pkg>.git` and the repos themselves don’t seem stale. Would anyone know, why some repositories may be missing from the manifest? Or is it still maintained? Thank you in advance for any help on this topic!

kevin added the

labels

2026-03-02 20:03:44 +00:00

pawelwms commented

2026-04-01 18:14:18 +00:00

Author

I've found more missing repos. I'll keep updating this comment, if I find anything more to keep the original one short:

cargo-rpm-macros
dns-root-data
f43-backgrounds
gap-srpm-macros
gnulib-l10n
llvm20
mjpegtools
mozjs140
ngtcp2
nodejs24
python-exceptiongroup
python-geographiclib
python-iplotx
python-sphinxcontrib-doxylink
rubygem-rack-session
rubygem-rackup
rust-aligned
rust-ambient-id
rust-annotate-snippets0.11
rust-anstream0.6
rust-anstyle-parse0.2
rust-anstyle-svg0.1
rust-argfile0.2
rust-as-slice
rust-assert2
rust-assert2-macros
rust-astral_async_http_range_reader
rust-astral_async_zip
rust-astral-pubgrub
rust-astral-reqwest-middleware
rust-astral-reqwest-retry
rust-astral-tl
rust-astral-version-ranges
rust-async-global-executor2
rust-attribute-derive
rust-attribute-derive-macro
rust-av-scenechange
rust-axum-core0.4
rust-axum-macros0.4
rust-axum0.7
rust-bindgen0.71
rust-bitstream-io2
rust-blobby0.3
rust-block-buffer0.10
rust-block-padding0.3
rust-borrow-or-share
rust-camino-tempfile
rust-cargo_metadata0.19
rust-cipher0.4
rust-clang
rust-clap_lex0.7
rust-collection_literals
rust-cpufeatures0.2
rust-crypto-common0.1
rust-cyclonedx-bom
rust-cyclonedx-bom-macros
rust-darling_core0.20
rust-darling_macro0.20
rust-darling0.20
rust-datatest-stable
rust-dlwrap
rust-fax
rust-fax_derive
rust-find-msvc-tools
rust-fluent-uri
rust-foldhash0.1
rust-get-size-derive2
rust-get-size2
rust-getrandom0.3
rust-gif0.13
rust-glycin-common
rust-hashbrown0.15
rust-hashlink0.10
rust-hickory-proto0.24
rust-hickory-resolver0.24
rust-hybrid-array
rust-inout0.1
rust-interpolator
rust-jiter0.10
rust-jiter0.11
rust-kvm-bindings0.11
rust-libspa-sys0.8
rust-libspa0.8
rust-libsqlite3-sys0.28
rust-linux-raw-sys0.11
rust-lru0.13
rust-manyhow
rust-manyhow-macros
rust-matchit0.8
rust-moxcms
rust-nix0.30
rust-nom_locate
rust-object0.36
rust-object0.37
rust-os_str_bytes6
rust-pastey
rust-phf_codegen0.11
rust-phf_generator0.11
rust-phf_macros0.11
rust-phf_shared0.11
rust-phf0.11
rust-pipewire-sys0.8
rust-pipewire0.8
rust-png0.17
rust-podman-sequoia
rust-proc-macro-utils
rust-purl
rust-pxfm
rust-pyo3_0.25
rust-pyo3_0.26
rust-pyo3-build-config0.25
rust-pyo3-build-config0.26
rust-pyo3-ffi0.25
rust-pyo3-ffi0.26
rust-pyo3-macros-backend0.25
rust-pyo3-macros-backend0.26
rust-pyo3-macros0.25
rust-pyo3-macros0.26
rust-quick-xml0.37
rust-quote-use
rust-quote-use-macros
rust-rand_chacha0.9
rust-rand_core0.9
rust-rand_distr0.5
rust-rand_isaac0.3
rust-rand_pcg0.9
rust-rand_xorshift0.4
rust-rand0.9
rust-rangemap
rust-reqsign
rust-reqsign-aws-v4
rust-reqsign-command-execute-tokio
rust-reqsign-core
rust-reqsign-file-read-tokio
rust-reqsign-google
rust-reqsign-http-send-reqwest
rust-reqwest-middleware0.4
rust-reqwest0.12
rust-retry-policies0.4
rust-rusqlite0.31
rust-rustflags
rust-ruzstd0.7
rust-serde_core
rust-serde_spanned0.6
rust-snapbox-macros0.4
rust-snapbox0.6
rust-socket2_0.5
rust-spin0.9
rust-term0.7
rust-termtree0.5
rust-tiff0.9
rust-tikv-jemallocator
rust-tokio-tungstenite0.23
rust-toml_datetime0.6
rust-toml_datetime0.7
rust-toml_edit0.22
rust-toml_edit0.23
rust-toml_edit0.24
rust-toml_parser
rust-toml_writer
rust-toml0.8
rust-toml0.9
rust-trycmd0.15
rust-tryfn0.2
rust-tungstenite0.23
rust-unit-prefix
rust-v_frame0.3
rust-web_atoms
rust-zerotrie
rust-zmij
rust-zune-core0.4
rust-zune-core0.4rust-zune-jpeg0.4
rust-zune-jpeg0.4
vipr

I've found more missing repos. I'll keep updating this comment, if I find anything more to keep the original one short: - cargo-rpm-macros - dns-root-data - f43-backgrounds - gap-srpm-macros - gnulib-l10n - llvm20 - mjpegtools - mozjs140 - ngtcp2 - nodejs24 - python-exceptiongroup - python-geographiclib - python-iplotx - python-sphinxcontrib-doxylink - rubygem-rack-session - rubygem-rackup - rust-aligned - rust-ambient-id - rust-annotate-snippets0.11 - rust-anstream0.6 - rust-anstyle-parse0.2 - rust-anstyle-svg0.1 - rust-argfile0.2 - rust-as-slice - rust-assert2 - rust-assert2-macros - rust-astral_async_http_range_reader - rust-astral_async_zip - rust-astral-pubgrub - rust-astral-reqwest-middleware - rust-astral-reqwest-retry - rust-astral-tl - rust-astral-version-ranges - rust-async-global-executor2 - rust-attribute-derive - rust-attribute-derive-macro - rust-av-scenechange - rust-axum-core0.4 - rust-axum-macros0.4 - rust-axum0.7 - rust-bindgen0.71 - rust-bitstream-io2 - rust-blobby0.3 - rust-block-buffer0.10 - rust-block-padding0.3 - rust-borrow-or-share - rust-camino-tempfile - rust-cargo_metadata0.19 - rust-cipher0.4 - rust-clang - rust-clap_lex0.7 - rust-collection_literals - rust-cpufeatures0.2 - rust-crypto-common0.1 - rust-cyclonedx-bom - rust-cyclonedx-bom-macros - rust-darling_core0.20 - rust-darling_macro0.20 - rust-darling0.20 - rust-datatest-stable - rust-dlwrap - rust-fax - rust-fax_derive - rust-find-msvc-tools - rust-fluent-uri - rust-foldhash0.1 - rust-get-size-derive2 - rust-get-size2 - rust-getrandom0.3 - rust-gif0.13 - rust-glycin-common - rust-hashbrown0.15 - rust-hashlink0.10 - rust-hickory-proto0.24 - rust-hickory-resolver0.24 - rust-hybrid-array - rust-inout0.1 - rust-interpolator - rust-jiter0.10 - rust-jiter0.11 - rust-kvm-bindings0.11 - rust-libspa-sys0.8 - rust-libspa0.8 - rust-libsqlite3-sys0.28 - rust-linux-raw-sys0.11 - rust-lru0.13 - rust-manyhow - rust-manyhow-macros - rust-matchit0.8 - rust-moxcms - rust-nix0.30 - rust-nom_locate - rust-object0.36 - rust-object0.37 - rust-os_str_bytes6 - rust-pastey - rust-phf_codegen0.11 - rust-phf_generator0.11 - rust-phf_macros0.11 - rust-phf_shared0.11 - rust-phf0.11 - rust-pipewire-sys0.8 - rust-pipewire0.8 - rust-png0.17 - rust-podman-sequoia - rust-proc-macro-utils - rust-purl - rust-pxfm - rust-pyo3_0.25 - rust-pyo3_0.26 - rust-pyo3-build-config0.25 - rust-pyo3-build-config0.26 - rust-pyo3-ffi0.25 - rust-pyo3-ffi0.26 - rust-pyo3-macros-backend0.25 - rust-pyo3-macros-backend0.26 - rust-pyo3-macros0.25 - rust-pyo3-macros0.26 - rust-quick-xml0.37 - rust-quote-use - rust-quote-use-macros - rust-rand_chacha0.9 - rust-rand_core0.9 - rust-rand_distr0.5 - rust-rand_isaac0.3 - rust-rand_pcg0.9 - rust-rand_xorshift0.4 - rust-rand0.9 - rust-rangemap - rust-reqsign - rust-reqsign-aws-v4 - rust-reqsign-command-execute-tokio - rust-reqsign-core - rust-reqsign-file-read-tokio - rust-reqsign-google - rust-reqsign-http-send-reqwest - rust-reqwest-middleware0.4 - rust-reqwest0.12 - rust-retry-policies0.4 - rust-rusqlite0.31 - rust-rustflags - rust-ruzstd0.7 - rust-serde_core - rust-serde_spanned0.6 - rust-snapbox-macros0.4 - rust-snapbox0.6 - rust-socket2_0.5 - rust-spin0.9 - rust-term0.7 - rust-termtree0.5 - rust-tiff0.9 - rust-tikv-jemallocator - rust-tokio-tungstenite0.23 - rust-toml_datetime0.6 - rust-toml_datetime0.7 - rust-toml_edit0.22 - rust-toml_edit0.23 - rust-toml_edit0.24 - rust-toml_parser - rust-toml_writer - rust-toml0.8 - rust-toml0.9 - rust-trycmd0.15 - rust-tryfn0.2 - rust-tungstenite0.23 - rust-unit-prefix - rust-v_frame0.3 - rust-web_atoms - rust-zerotrie - rust-zmij - rust-zune-core0.4 - rust-zune-core0.4rust-zune-jpeg0.4 - rust-zune-jpeg0.4 - vipr

pawelwms commented

2026-04-09 21:30:31 +00:00

Author

@kevin, if the code for updating the manifest is available somewhere publicly, I'm happy to take a look at it myself and see if I can come up with a fix. Do you know, where I could find it, though, or the best contact to reach out to?

kevin commented

2026-04-13 20:02:57 +00:00

Owner

Well, I don't recall all the details, but I thought the manifest was actually updated by the hook in each git repo.

You can see the ansible setup that deploys things at:

https://forge.fedoraproject.org/infra/ansible/src/branch/main/roles/grokmirror_mirror

if you see something there thats wrong, you can submit a pr there.
It's odd to me that it's just some repos, not everything, so I am not sure whats failing and haven't had time to investigate further.

Well, I don't recall all the details, but I thought the manifest was actually updated by the hook in each git repo. You can see the ansible setup that deploys things at: https://forge.fedoraproject.org/infra/ansible/src/branch/main/roles/grokmirror_mirror if you see something there thats wrong, you can submit a pr there. It's odd to me that it's just some repos, not everything, so I am not sure whats failing and haven't had time to investigate further.

👍 1

kevin was assigned by james

2026-07-30 15:47:37 +00:00

james commented

2026-07-30 15:49:00 +00:00

Member

We talked about this in this weeks infra. meeting, Kevin said it would need to be looked at with the move of backup01 to rhel10 ... and he's probably doing that, so he volunteered to be assigned to the ticket.

We talked about this in this weeks infra. meeting, Kevin said it would need to be looked at with the move of `backup01` to rhel10 ... and he's probably doing that, so he volunteered to be assigned to the ticket.

❤️ 1

kevin commented

2026-08-06 20:32:58 +00:00

Owner

Sorry for the delay here. I just reinstalled backup01 so this was a good time to look into this.

I've regnerated the manifest.
I've removed all the requests repos from this as they are not actually exposed there.

Can you recheck for any you still see missing now and I can look more closely at those...

Sorry for the delay here. I just reinstalled backup01 so this was a good time to look into this. I've regnerated the manifest. I've removed all the requests repos from this as they are not actually exposed there. Can you recheck for any you still see missing now and I can look more closely at those...

kevin added this to the Sprint ( 2026-07-27 to 2026-08-09) project

2026-08-06 20:33:05 +00:00

pawelwms commented

2026-08-06 21:50:15 +00:00

Author

Thanks for fixing this, @kevin. I scanned through Fedora 43+ dist-git repos and found only a few still missing:

Fedora 44 and Rawhide:
- python-tomllint
- rust-amq-protocol-tcp
Rawhide-only:

Thanks for fixing this, @kevin. I scanned through Fedora 43+ dist-git repos and found only a few still missing: - Fedora 44 and Rawhide: - [python-tomllint](https://src.fedoraproject.org/rpms/python-tomllint/branches) - [rust-amq-protocol-tcp](https://src.fedoraproject.org/rpms/rust-amq-protocol-tcp/branches?branchname=rawhide) - Rawhide-only: - [rust-blowfish0.9](https://src.fedoraproject.org/rpms/rust-blowfish0.9/branches?branchname=rawhide) - [rust-cbc0.1](https://src.fedoraproject.org/rpms/rust-cbc0.1/branches?branchname=rawhide) - [rust-ecb0.1](https://src.fedoraproject.org/rpms/rust-ecb0.1/branches?branchname=rawhide) - [rust-hkdf0.12](https://src.fedoraproject.org/rpms/rust-hkdf0.12/branches?branchname=rawhide) - [rust-num-bigint-dig0.8](https://src.fedoraproject.org/rpms/rust-num-bigint-dig0.8/branches?branchname=rawhide)

kevin commented

2026-08-06 22:04:28 +00:00

Owner

ok, I'll try and figure out why just those are missing... thanks!

kevin commented

2026-08-06 22:06:44 +00:00

Owner

Ah ha. All those are just newly added... so it's not adding them when they are added.

pawelwms commented

2026-08-06 22:22:50 +00:00

Author

Ah, I didn't even check how new they were. Could this be the reason why the ones spotted earlier were missing? They were also simply added after the last time the manifest was regenerated?

kevin commented

2026-08-06 23:00:34 +00:00

Owner

Quite possibly yes. The current setup just updates the existing manifest. I thought it would add those repos when they got commits, but it seems not somehow. ;(

kevin removed this from the Sprint ( 2026-07-27 to 2026-08-09) project

2026-08-10 14:12:05 +00:00

Rows
Columns

Missing repositories in grokmirror manifest #13170

Description of request