carlwgeorge commented

Member

Describe what you would like us to do:

Old major versions of EPEL such as EPEL 6 are available in the Fedora archive, and have repos set up in MirrorManager to point users who request them to mirrors of the Fedora archive. It would be beneficial for some EPEL users, including users of RHEL EUS, to also get old minor versions the same way. While EPEL doesn't currently have minor versions, snapshots that emulate the minor versions (i.e. only contain packages that were compatible with a RHEL minor version) do already exist in the Fedora archive.

I would like for MirrorManager to respond with minor version paths in the mirrored archives if a user requests an EPEL minor version. For example:

if a user requests epel-8.8, they are returned a list of mirrors with the /pub/archive/epel/8.8/ path
if a user requests epel-9.2, they are returned a list of mirrors with the /pub/archive/epel/9.2/ path

The list of minor versions we have EPEL snapshots for can be viewed here. Ideally we would set up MirrorManager responses for each of these.

When do you need this to be done by? (YYYY/MM/DD)

No deadline, this is a nice to have that isn't currently blocking anything. I will note that the status quo has an load impact on the main archive.fpo server(s). Currently when users want to set this up, the most common course of action taken is directly configuring their systems to use a baseurl pointing at archive.fpo. Having the EPEL minor versions in MirrorManager will spread that load to the mirror network. It will also remove the need for users of RHEL EUS to manually configure their baseurl this way.

# Describe what you would like us to do: ---- Old major versions of EPEL such as EPEL 6 are available in the Fedora archive, and have repos set up in MirrorManager to point users who request them to mirrors of the Fedora archive. It would be beneficial for some EPEL users, including users of RHEL EUS, to also get old minor versions the same way. While EPEL doesn't currently have minor versions, snapshots that emulate the minor versions (i.e. only contain packages that were compatible with a RHEL minor version) do already exist in the Fedora archive. I would like for MirrorManager to respond with minor version paths in the mirrored archives if a user requests an EPEL minor version. For example: - if a user requests `epel-8.8`, they are returned a list of mirrors with the `/pub/archive/epel/8.8/` path - if a user requests `epel-9.2`, they are returned a list of mirrors with the `/pub/archive/epel/9.2/` path The list of minor versions we have EPEL snapshots for can be viewed [here](https://archive.fedoraproject.org/pub/archive/epel/). Ideally we would set up MirrorManager responses for each of these. # When do you need this to be done by? (YYYY/MM/DD) ---- No deadline, this is a nice to have that isn't currently blocking anything. I will note that the status quo has an load impact on the main archive.fpo server(s). Currently when users want to set this up, the most common course of action taken is directly configuring their systems to use a baseurl pointing at archive.fpo. Having the EPEL minor versions in MirrorManager will spread that load to the mirror network. It will also remove the need for users of RHEL EUS to manually configure their baseurl this way.

adrian commented

2024-06-03 17:58:29 +00:00

Metadata Update from @adrian:

Issue assigned to adrian

**Metadata Update from @adrian**: - Issue assigned to adrian

adrian commented

2024-06-03 17:59:03 +00:00

I will take a look at it.

carlwgeorge commented

2024-06-03 18:21:27 +00:00

Author

Member

It has come to my attention that this was suggested previously, and there were some mixed feelings about it. I'm going to stick it on the agenda for the next EPEL Steering Committee meeting (two days from now) to make sure the committee members have a chance to voice their concerns before we move forward. Please put this on hold until then.

It has come to my attention that this was [suggested previously](https://bugzilla.redhat.com/show_bug.cgi?id=1969500), and there were some mixed feelings about it. I'm going to stick it on the agenda for the next EPEL Steering Committee meeting (two days from now) to make sure the committee members have a chance to voice their concerns before we move forward. Please put this on hold until then.

phsmoura commented

2024-06-03 19:09:05 +00:00

Member

Metadata Update from @phsmoura:

Issue priority set to: Waiting on Assignee (was: Needs Review)
Issue tagged with: low-gain, low-trouble, ops

**Metadata Update from @phsmoura**: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: low-gain, low-trouble, ops

carlwgeorge commented

2024-06-06 03:14:01 +00:00

Author

Member

I brought this up at the committee meeting today. No one seemed opposed to it, but since I didn't file an issue until the day of the meeting, the consensus was to leave it open for a week to allow time for comments, and then vote on it next week.

https://pagure.io/epel/issue/278

I brought this up at the committee meeting today. No one seemed opposed to it, but since I didn't file an issue until the day of the meeting, the consensus was to leave it open for a week to allow time for comments, and then vote on it next week. https://pagure.io/epel/issue/278

carlwgeorge commented

2024-06-13 00:25:08 +00:00

Author

Member

Today the committee approved implementing this. We can proceed with adding the minor versions we have snapshots for (everything except 8.0, 8.6, and 9.0).

Today the committee approved implementing this. We can proceed with adding the minor versions we have snapshots for (everything except 8.0, 8.6, and 9.0). - [8.1](https://archive.fedoraproject.org/pub/archive/epel/8.1/) - [8.2](https://archive.fedoraproject.org/pub/archive/epel/8.2/) - [8.3](https://archive.fedoraproject.org/pub/archive/epel/8.3/) - [8.4](https://archive.fedoraproject.org/pub/archive/epel/8.4/) - [8.5](https://archive.fedoraproject.org/pub/archive/epel/8.5/) - [8.7](https://archive.fedoraproject.org/pub/archive/epel/8.7/) - [8.8](https://archive.fedoraproject.org/pub/archive/epel/8.8/) - [8.9](https://archive.fedoraproject.org/pub/archive/epel/8.9/) - [9.1](https://archive.fedoraproject.org/pub/archive/epel/9.1/) - [9.2](https://archive.fedoraproject.org/pub/archive/epel/9.2/) - [9.3](https://archive.fedoraproject.org/pub/archive/epel/9.3/)

adrian commented

2024-06-14 07:22:30 +00:00

@carlwgeorge

I created one repository as a test. Please try:

curl -s "https://mirrors.fedoraproject.org/metalink?repo=epel-8.8&arch=x86_64&country=global"

Following steps where necessary on the database level. First figure out the ID of the directory:

=> select id,name from directory where name like 'pub/archive/epel/8.8.2023-11-14/Everything/x86_64/repodata';
668521 | pub/archive/epel/8.8.2023-11-14/Everything/x86_64

then get the product ID:

=> select * from product;
 id |  name  | publiclist 
----+--------+------------
  1 | EPEL   | t

The create a new version:

=> insert into version (name, product_id, is_test, display, sortorder) values ('8.8', 1, false,false, 0);
=> select * from version where name = '8.8';
 934244 | 8.8             |          1 | f       | f       |              | t                  |          0 |

With all that information it should now be possible to create the repository:

=> insert into repository (name, prefix, category_id, version_id, arch_id, directory_id , disabled) values ('pub/archive/epel/8.8.2023-11-14/Everything/x86_64', 'epel-8.8', 4, 934244, 3, 668521, false);

@carlwgeorge I created one repository as a test. Please try: ``` curl -s "https://mirrors.fedoraproject.org/metalink?repo=epel-8.8&arch=x86_64&country=global" ``` Following steps where necessary on the database level. First figure out the ID of the directory: ``` => select id,name from directory where name like 'pub/archive/epel/8.8.2023-11-14/Everything/x86_64/repodata'; 668521 | pub/archive/epel/8.8.2023-11-14/Everything/x86_64 ``` then get the product ID: ``` => select * from product; id | name | publiclist ----+--------+------------ 1 | EPEL | t ``` The create a new version: ``` => insert into version (name, product_id, is_test, display, sortorder) values ('8.8', 1, false,false, 0); => select * from version where name = '8.8'; 934244 | 8.8 | 1 | f | f | | t | 0 | ``` With all that information it should now be possible to create the repository: ``` => insert into repository (name, prefix, category_id, version_id, arch_id, directory_id , disabled) values ('pub/archive/epel/8.8.2023-11-14/Everything/x86_64', 'epel-8.8', 4, 934244, 3, 668521, false); ```

carlwgeorge commented

2024-06-15 01:14:40 +00:00

Author

Member

I can confirm that curl works as expected. My only question is, should we be using the date directories like 8.8.2023-11-14 instead of just 8.8?

kevin commented

2024-06-17 17:43:23 +00:00

Owner

I would say no, the date ones are there to indicate when the archive was made, for mm, the minor release should be just fine...

abompard commented

2024-07-03 18:34:10 +00:00

Member

I've created the repos in Mirrormanager, they all seem to work except 8.3, 8.4, and 8.5. I'm going to investigate why, but probably tomorrow.

carlwgeorge commented

2024-07-15 18:24:29 +00:00

Author

Member

For posterity, here is the error I'm seeing with those three releases.

❯ curl 'https://mirrors.fedoraproject.org/metalink?repo=epel-8.4&arch=x86_64'
<?xml version="1.0" encoding="utf-8"?>
<metalink version="3.0" xmlns="http://www.metalinker.org/" type="dynamic" pubdate="Mon, 15 Jul 2024 17:36:43 GMT" generator="mirrormanager" xmlns:mm0="http://fedorahosted.org/mirrormanager">
<!-- pub/archive/epel/8.4.2021-11-30/Everything/x86_64/repodata/repomd.xml not found or has not metalink
-->
</metalink>

For posterity, here is the error I'm seeing with those three releases. ``` ❯ curl 'https://mirrors.fedoraproject.org/metalink?repo=epel-8.4&arch=x86_64' <?xml version="1.0" encoding="utf-8"?> <metalink version="3.0" xmlns="http://www.metalinker.org/" type="dynamic" pubdate="Mon, 15 Jul 2024 17:36:43 GMT" generator="mirrormanager" xmlns:mm0="http://fedorahosted.org/mirrormanager">  </metalink> ```

kevin commented

2024-08-19 18:22:26 +00:00

Owner

Any news here on those last archives?

carlwgeorge commented

2024-12-05 23:12:02 +00:00

Author

Member

We also need to add the recently archived 9.4 as well.

https://dl.fedoraproject.org/pub/archive/epel/9.4-2024-11-12/

This just came up over in the discussion forum.

https://discussion.fedoraproject.org/t/two-rhel-9-4-systems-one-produces-404-errors-for-epel-repositories/139130

We also need to add the recently archived 9.4 as well. https://dl.fedoraproject.org/pub/archive/epel/9.4-2024-11-12/ This just came up over in the discussion forum. https://discussion.fedoraproject.org/t/two-rhel-9-4-systems-one-produces-404-errors-for-epel-repositories/139130

abompard commented

2025-01-06 13:25:59 +00:00

Member

Hey! I looked back at this issue, and I think there are two issues:

the 9.4 directory is not in /srv/pub/archive/fullfiletimelist-archive
the pattern "pub/archive/epel/[\\d\\.-]+/Everything/.*" is not in the scan-primary-mirror.toml (that's something I can fix)

Also, those directories are currently in the EPEL category in MirrorManager. Should they be put in the Fedora Archive category?

Hey! I looked back at this issue, and I think there are two issues: - the 9.4 directory is not in `/srv/pub/archive/fullfiletimelist-archive` - the pattern `"pub/archive/epel/[\\d\\.-]+/Everything/.*"` is not in the `scan-primary-mirror.toml` (that's something I can fix) Also, those directories are currently in the `EPEL` category in MirrorManager. Should they be put in the `Fedora Archive` category?

kevin commented

2025-01-06 15:53:24 +00:00

Owner

Updated the archive fullfiletimelist... ( by running: time sudo -u ftpsync /usr/local/bin/update-fullfiletimelist -l /pub/fedora-secondary/update-fullfiletimelist.lock -t /pub archive on bodhi-backend01)

And yeah, they should probibly be in Fedora Archive.

Updated the archive fullfiletimelist... ( by running: time sudo -u ftpsync /usr/local/bin/update-fullfiletimelist -l /pub/fedora-secondary/update-fullfiletimelist.lock -t /pub archive on bodhi-backend01) And yeah, they should probibly be in Fedora Archive.

adrian commented

2025-01-06 15:57:57 +00:00

The EPEL 9.x repositories are created manually in the database. We do not try to detect anything in archive/ to not conflict with something.

The idea was that once the files have been detected and are added to the database the repository needs to be added manually using SQL. At least that is how I remember the discussion. It was a 9.x only thing that is why we thought there is no need to automate it. Especially as we do not want any MirrorManager repositories created from archive content.

The EPEL 9.x repositories are created manually in the database. We do not try to detect anything in `archive/` to not conflict with something. The idea was that once the files have been detected and are added to the database the repository needs to be added manually using SQL. At least that is how I remember the discussion. It was a 9.x only thing that is why we thought there is no need to automate it. Especially as we do not want any MirrorManager repositories created from archive content.

adrian commented

2025-01-07 07:23:34 +00:00

I tried to add it to the database:

=> select * from repository where prefix like 'epel-9.4';
    id    |                        name                        |  prefix  | category_id | version_id | arch_id | directory_id | disabled 
----------+----------------------------------------------------+----------+-------------+------------+---------+--------------+----------
 27710620 | pub/archive/epel/9.4-2024-11-12/Everything/x86_64  | epel-9.4 |           4 |     934261 |       3 |       834505 | f
 27710621 | pub/archive/epel/9.4-2024-11-12/Everything/s390x   | epel-9.4 |           4 |     934261 |       9 |       833413 | f
 27710622 | pub/archive/epel/9.4-2024-11-12/Everything/aarch64 | epel-9.4 |           4 |     934261 |      14 |       834481 | f
 27710623 | pub/archive/epel/9.4-2024-11-12/Everything/ppc64le | epel-9.4 |           4 |     934261 |      15 |       833011 | f
(4 rows)

I tried to add it to the database: ``` => select * from repository where prefix like 'epel-9.4'; id | name | prefix | category_id | version_id | arch_id | directory_id | disabled ----------+----------------------------------------------------+----------+-------------+------------+---------+--------------+---------- 27710620 | pub/archive/epel/9.4-2024-11-12/Everything/x86_64 | epel-9.4 | 4 | 934261 | 3 | 834505 | f 27710621 | pub/archive/epel/9.4-2024-11-12/Everything/s390x | epel-9.4 | 4 | 934261 | 9 | 833413 | f 27710622 | pub/archive/epel/9.4-2024-11-12/Everything/aarch64 | epel-9.4 | 4 | 934261 | 14 | 834481 | f 27710623 | pub/archive/epel/9.4-2024-11-12/Everything/ppc64le | epel-9.4 | 4 | 934261 | 15 | 833011 | f (4 rows) ```

abompard commented

2025-01-07 13:04:57 +00:00

Member

Thanks! I'll move them to the Fedora Archive category then.

Thanks! I'll move them to the `Fedora Archive` category then.

adrian commented

2025-01-07 13:08:25 +00:00

Thanks! I'll move them to the Fedora Archive category then.

Not necessary. They have been created in the archive category. This should be solved now for 9.4. We need to do it again once 9.5 is archived.

> Thanks! I'll move them to the `Fedora Archive` category then. Not necessary. They have been created in the archive category. This should be solved now for 9.4. We need to do it again once 9.5 is archived.

abompard commented

2025-01-07 13:12:31 +00:00

Member

Not sure I understand, in your SQL paste category_id 4 is EPEL, Fedora archive is category_id 12.

Not sure I understand, in your SQL paste `category_id` 4 is EPEL, Fedora archive is `category_id` 12.

adrian commented

2025-01-07 13:14:38 +00:00

Ah, maybe I selected the wrong id, thanks. Let me check.

adrian commented

2025-01-07 13:16:51 +00:00

You already changed it. Good. Thank you. I was just following my old example which seems to have used the wrong ID. I did not check it.

kevin commented

2025-02-18 00:39:32 +00:00

Owner

What is left to do here?

carlwgeorge commented

2025-02-18 02:13:57 +00:00

Author

Member

Everything seems to be in order except for 8.3, 8.4, and 8.5. Those still return the error I mentioned here.

Everything seems to be in order except for 8.3, 8.4, and 8.5. Those still return the error I mentioned [here](https://pagure.io/fedora-infrastructure/issue/11968#comment-919161).

kevin commented

2025-04-21 18:58:49 +00:00

Owner

Any news here? Or still those errors?

carlwgeorge commented

2025-05-14 06:33:45 +00:00

Author

Member

I created pub/archive/epel/9.5-2025-05-13 and pub/archive/epel/testing/9.5-2025-05-13 today. Is there a way I can add these to mirrormanager myself? I tried the create repository button in the admin panel, but it results in a 500 Internal Server Error.

I created [pub/archive/epel/9.5-2025-05-13](https://dl.fedoraproject.org/pub/archive/epel/9.5-2025-05-13/) and [pub/archive/epel/testing/9.5-2025-05-13](https://dl.fedoraproject.org/pub/archive/epel/testing/9.5-2025-05-13/) today. Is there a way I can add these to mirrormanager myself? I tried the create repository button in the admin panel, but it results in a 500 Internal Server Error.

kevin commented

2025-08-27 23:48:39 +00:00

Owner

Any news here?

carlwgeorge commented

2025-11-11 17:05:54 +00:00

Author

Member

The errors are still the same for 8.3, 8.4, and 8.5.

I created pub/archive/epel/9.6-2025-11-11 and pub/archive/epel/testing/9.6-2025-11-11 today, which like 9.5-2025-05-13 also need to be added.

The errors are still the same for 8.3, 8.4, and 8.5. I created [pub/archive/epel/9.6-2025-11-11](https://dl.fedoraproject.org/pub/archive/epel/9.6-2025-11-11/) and [pub/archive/epel/testing/9.6-2025-11-11](https://dl.fedoraproject.org/pub/archive/epel/testing/9.6-2025-11-11/) today, which like 9.5-2025-05-13 also need to be added.

zlopez commented

2026-03-12 17:52:08 +00:00

Owner

@carlwgeorge Do you see the same problem with EPEL 9 or EPEL 10 as well?

dherrera commented

2026-03-12 18:19:47 +00:00

Member

@zlopez only those 3 reproduce that behaviour

for version in {8..10}.{1..10}; do curl -s "https://mirrors.fedoraproject.org/metalink?repo=epel-$version&arch=x86_64" | grep 'not found'; done
<!-- pub/archive/epel/8.3.2021-05-20/Everything/x86_64/repodata/repomd.xml not found or has not metalink
<!-- pub/archive/epel/8.4.2021-11-30/Everything/x86_64/repodata/repomd.xml not found or has not metalink
<!-- pub/archive/epel/8.5.2022-05-10/Everything/x86_64/repodata/repomd.xml not found or has not metalink

@zlopez only those 3 reproduce that behaviour ```sh for version in {8..10}.{1..10}; do curl -s "https://mirrors.fedoraproject.org/metalink?repo=epel-$version&arch=x86_64" | grep 'not found'; done <!-- pub/archive/epel/8.3.2021-05-20/Everything/x86_64/repodata/repomd.xml not found or has not metalink <!-- pub/archive/epel/8.4.2021-11-30/Everything/x86_64/repodata/repomd.xml not found or has not metalink <!-- pub/archive/epel/8.5.2022-05-10/Everything/x86_64/repodata/repomd.xml not found or has not metalink ```

👍 1

carlwgeorge commented

2026-05-19 14:52:20 +00:00

Author

Member

I created pub/archive/epel/9.7-2026-05-19 and pub/archive/epel/testing/9.7-2026-05-19 today, which like 9.5-2025-05-13 and 9.6-2025-11-11 also need to be added.

I created [pub/archive/epel/9.7-2026-05-19](https://dl.fedoraproject.org/pub/archive/epel/9.7-2026-05-19/) and [pub/archive/epel/testing/9.7-2026-05-19](https://dl.fedoraproject.org/pub/archive/epel/testing/9.7-2026-05-19/) today, which like 9.5-2025-05-13 and 9.6-2025-11-11 also need to be added.

carlwgeorge referenced this issue from epel/releng

2026-05-19 14:52:56 +00:00

create EPEL 9.7 archive snapshot #66

Rows
Columns

EPEL minor version archive repos in MirrorManager #11968

Describe what you would like us to do:

When do you need this to be done by? (YYYY/MM/DD)