releng/tickets

Fork 2

Drop modularity from EPEL8. #11292

New issue

Open

opened 2023-02-16 03:43:08 +00:00 by dherrera · 21 comments

dherrera commented

2023-02-16 03:43:08 +00:00

Contributor

As it was approved by the EPEL Steering Committee, Modularity support is getting retired from EPEL 8.

To complete the retirement, we need some support from releng, specifically in tasks regarding archiving repositories, removing tags and targets, and things like removing epel modules from MBS.

A more comprehensive list of tasks can be found in the corresponding epel ticket

As it was [approved by the EPEL Steering Committee](https://lists.fedoraproject.org/archives/list/epel-announce@lists.fedoraproject.org/thread/GYX7BNBGIUJ53CGOCKWECWV3ZDI5DQUM/), Modularity support is getting retired from EPEL 8. To complete the retirement, we need some support from releng, specifically in tasks regarding archiving repositories, removing tags and targets, and things like removing epel modules from MBS. A more comprehensive list of tasks can be found [in the corresponding epel ticket](https://pagure.io/epel/issue/198)

phsmoura commented

2023-02-16 19:10:53 +00:00

Metadata Update from @phsmoura:

Issue tagged with: low-gain, low-trouble, ops

**Metadata Update from @phsmoura**: - Issue tagged with: low-gain, low-trouble, ops

humaton commented

2023-02-21 16:48:43 +00:00

Owner

Metadata Update from @humaton:

Issue assigned to humaton

**Metadata Update from @humaton**: - Issue assigned to humaton

humaton commented

2023-03-28 15:43:47 +00:00

Owner

Koji tags are removed and bodhi should not be pushing anything epel8-modular related.

humaton commented

2023-03-28 15:43:48 +00:00

Owner

Metadata Update from @humaton:

Issue close_status updated to: Fixed
Issue status updated to: Closed (was: Open)

**Metadata Update from @humaton**: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)

smooge commented

2023-03-28 16:09:50 +00:00

🍻

:beers:

carlwgeorge commented

2024-06-29 06:38:21 +00:00

Contributor

This was only partially completed. Remaining tasks:

archive the repo content (copy https://dl.fedoraproject.org/pub/epel/8/Modular/ to https://archive.fedoraproject.org/pub/archive/epel/8/Modular/)
update MirrorManager to send requests for epel-modular-8 to archive mirrors

Please re-open this ticket.

This was only partially completed. Remaining tasks: - archive the repo content (copy https://dl.fedoraproject.org/pub/epel/8/Modular/ to https://archive.fedoraproject.org/pub/archive/epel/8/Modular/) - update MirrorManager to send requests for `epel-modular-8` to archive mirrors Please re-open this ticket.

churchyard commented

2024-06-29 11:20:21 +00:00

Contributor

Metadata Update from @churchyard:

Issue status updated to: Open (was: Closed)

**Metadata Update from @churchyard**: - Issue status updated to: Open (was: Closed)

kevin commented

2024-07-23 23:55:15 +00:00

Owner

I syned the content over.

We need to sort out how to get mirrormanager to move now. (like we do for the other eol epel stuff)

I syned the content over. We need to sort out how to get mirrormanager to move now. (like we do for the other eol epel stuff)

jnsamyak commented

2024-07-31 14:02:24 +00:00

Owner

CC: @abompard, do you think you can handle the mirrormanager stuff along with the other epel retiring stuff?

abompard commented

2024-08-01 21:46:38 +00:00

Sure! It's a simple database update. It's just the epel-modular-8 repos, for all arches?

Sure! It's a simple database update. It's just the `epel-modular-8` repos, for all arches?

kevin commented

2024-08-02 01:37:53 +00:00

Owner

yeah, that and if there's a testing version of it I suppose...

carlwgeorge commented

2024-08-02 04:51:59 +00:00

Contributor

Should be all of these (grepped out of MM output when you ask for a repo that doesn't exist):

# repo=epel-modular-8&arch=aarch64
# repo=epel-modular-8&arch=ppc64le
# repo=epel-modular-8&arch=s390x
# repo=epel-modular-8&arch=x86_64
# repo=epel-modular-debug-8&arch=aarch64
# repo=epel-modular-debug-8&arch=ppc64le
# repo=epel-modular-debug-8&arch=s390x
# repo=epel-modular-debug-8&arch=x86_64
# repo=epel-modular-source-8&arch=source
# repo=testing-modular-debug-epel8&arch=aarch64
# repo=testing-modular-debug-epel8&arch=ppc64le
# repo=testing-modular-debug-epel8&arch=s390x
# repo=testing-modular-debug-epel8&arch=x86_64
# repo=testing-modular-epel8&arch=aarch64
# repo=testing-modular-epel8&arch=ppc64le
# repo=testing-modular-epel8&arch=s390x
# repo=testing-modular-epel8&arch=x86_64
# repo=testing-modular-source-epel8&arch=source

Should be all of these (grepped out of MM output when you ask for a repo that doesn't exist): ``` # repo=epel-modular-8&arch=aarch64 # repo=epel-modular-8&arch=ppc64le # repo=epel-modular-8&arch=s390x # repo=epel-modular-8&arch=x86_64 # repo=epel-modular-debug-8&arch=aarch64 # repo=epel-modular-debug-8&arch=ppc64le # repo=epel-modular-debug-8&arch=s390x # repo=epel-modular-debug-8&arch=x86_64 # repo=epel-modular-source-8&arch=source # repo=testing-modular-debug-epel8&arch=aarch64 # repo=testing-modular-debug-epel8&arch=ppc64le # repo=testing-modular-debug-epel8&arch=s390x # repo=testing-modular-debug-epel8&arch=x86_64 # repo=testing-modular-epel8&arch=aarch64 # repo=testing-modular-epel8&arch=ppc64le # repo=testing-modular-epel8&arch=s390x # repo=testing-modular-epel8&arch=x86_64 # repo=testing-modular-source-epel8&arch=source ```

jnsamyak commented

2024-08-27 16:54:01 +00:00

Owner

Hey @carlwgeorge!

Can you check and let us know if there's some work left for this? If not we can close this, if there is shoot so we can resolve and fix it.

Hey @carlwgeorge! Can you check and let us know if there's some work left for this? If not we can close this, if there is shoot so we can resolve and fix it.

carlwgeorge commented

2024-08-28 02:57:18 +00:00

Contributor

We still need to update MirrorManager to send requests for EPEL 8 Modular to archive mirrors. For example:

carl ~ 
❯ curl -s 'https://mirrors.fedoraproject.org/metalink?arch=x86_64&repo=testing-modular-epel8' | grep -m 1 url
    <url protocol="http" type="http" location="US" preference="100">http://mirrors.sonic.net/epel/testing/8/Modular/x86_64/repodata/repomd.xml</url>

Hey @abompard, can you perform the database update you mentioned to switch these to archive links? It needs to be done for the following repos on all architectures:

epel-modular-8
epel-modular-source-8
epel-modular-debug-8
testing-modular-epel8
testing-modular-source-epel8
testing-modular-debug-epel8

Once that is complete, we should be able to delete /pub/epel/8/Modular and /pub/epel/testing/8/Modular off the mirrors, then I believe we'll be done and the ticket can be closed.

We still need to update MirrorManager to send requests for EPEL 8 Modular to archive mirrors. For example: ``` carl ~ ❯ curl -s 'https://mirrors.fedoraproject.org/metalink?arch=x86_64&repo=testing-modular-epel8' | grep -m 1 url <url protocol="http" type="http" location="US" preference="100">http://mirrors.sonic.net/epel/testing/8/Modular/x86_64/repodata/repomd.xml</url> ``` Hey @abompard, can you perform the database update you mentioned to switch these to archive links? It needs to be done for the following repos on all architectures: - epel-modular-8 - epel-modular-source-8 - epel-modular-debug-8 - testing-modular-epel8 - testing-modular-source-epel8 - testing-modular-debug-epel8 Once that is complete, we should be able to delete `/pub/epel/8/Modular` and `/pub/epel/testing/8/Modular` off the mirrors, then I believe we'll be done and the ticket can be closed.

abompard commented

2024-08-28 07:48:08 +00:00

@carlwgeorge I've done it for the first 3, but the testing-* ones have apparently not been copied to the archive yet.

@carlwgeorge I've done it for the first 3, but the `testing-*` ones have apparently not been copied to the archive yet.

kevin commented

2024-08-28 21:21:34 +00:00

Owner

testing/8/Modular should now be copied over to archive.

abompard commented

2024-09-02 14:35:53 +00:00

It is copied but it's apparently not being scanned and loaded in the database. I thought it was because scan-primary-mirror.toml was lacking the appropriate section, so I've added:

[[repository_mapping]]
regex="pub/(archive/)?epel/testing/[\\d]+/Modular/.*"
prefix="testing-modular-epel"

[[repository_aliases]]
from="testing-modular-epel-debug-"
to="testing-modular-debug-epel"

but it's still not adding those repos to the DB. The scan is done by scan-primary-mirror, which is in Rust, and I don't know how to debug it or make it log more. Any help @adrian ? What info would you need to debug it?

The path does exist on the server:

$ ls /srv/pub/archive/epel/testing/8/Modular/
aarch64  ppc64le  s390x  source  SRPMS  state  x86_64

It is copied but it's apparently not being scanned and loaded in the database. I thought it was because `scan-primary-mirror.toml` was lacking the appropriate section, so I've added: ``` [[repository_mapping]] regex="pub/(archive/)?epel/testing/[\\d]+/Modular/.*" prefix="testing-modular-epel" [[repository_aliases]] from="testing-modular-epel-debug-" to="testing-modular-debug-epel" ``` but it's still not adding those repos to the DB. The scan is done by [scan-primary-mirror](https://github.com/adrianreber/scan-primary-mirror), which is in Rust, and I don't know how to debug it or make it log more. Any help @adrian ? What info would you need to debug it? The path does exist on the server: ``` $ ls /srv/pub/archive/epel/testing/8/Modular/ aarch64 ppc64le s390x source SRPMS state x86_64 ```

humaton commented

2025-01-15 10:36:03 +00:00

Owner

Metadata Update from @humaton:

Assignee reset

**Metadata Update from @humaton**: - Assignee reset

carlwgeorge commented

2025-07-14 20:25:35 +00:00

Contributor

Getting this resolved would allow us to drop /pub/epel/8/Modular and /pub/epel/testing/8/Modular from the mirrors, saving 6.5GB of space for every mirror.

Getting this resolved would allow us to drop `/pub/epel/8/Modular` and `/pub/epel/testing/8/Modular` from the mirrors, saving 6.5GB of space for every mirror.

jnsamyak commented

2025-09-30 12:04:37 +00:00

Owner

CC: @adrian any thoughts on how to fix it?

adrian commented

2025-09-30 13:14:22 +00:00

Looking at the database I see a lot of matching directories in the database:

mirrormanager2=> select count(id) from directory where name like '%archive%testing%/8/Modular%';
 count 
-------
   130
(1 row)

What is the error we are talking about. I guess one of the move-to-archive scripts is not working because of some data missing, right? What is the output of the move-to-archive script? Or how can I run the script to see the error for myself.

Looking at the database I see a lot of matching directories in the database: ``` mirrormanager2=> select count(id) from directory where name like '%archive%testing%/8/Modular%'; count ------- 130 (1 row) ``` What is the error we are talking about. I guess one of the move-to-archive scripts is not working because of some data missing, right? What is the output of the move-to-archive script? Or how can I run the script to see the error for myself.