Restructuring Infrastructure and Release Engineering Repositories in Forgejo #12737
Labels
No labels
announcement
anubis
authentication
aws
backlog
blocked
bodhi
ci
cloud
communishift
copr
database
day-to-day
dc-move
deprecated
dev
discourse
dns
downloads
easyfix
epel
firmitas
forgejo_migration
Gain
High
Gain
Low
Gain
Medium
gitlab
greenwave
hardware
help wanted
high-trouble
koji
koschei
lists
low-trouble
medium-trouble
mirrorlists
monitoring
Needs investigation
odcs
OpenShift
ops
outage
packager_workflow_blocker
pagure
permissions
Priority
Needs Review
Priority
Next Meeting
Priority
🔥 URGENT 🔥
Priority
Waiting on Assignee
Priority
Waiting on External
Priority
Waiting on Reporter
rabbitmq
release-monitoring
releng
request-for-resources
s390x
security
SMTP
sprint-0
sprint-1
src.fp.o
staging
unfreeze
waiverdb
websites-general
wiki
Backlog Status
Needs Review
Backlog Status
Ready
chore
documentation
points
01
points
02
points
03
points
05
points
08
points
13
Priority
High
Priority
Low
Priority
Medium
Sprint Status
Blocked
Sprint Status
Done
Sprint Status
In Progress
Sprint Status
Review
Sprint Status
To Do
Technical Debt
Work Item
Bug
Work Item
Epic
Work Item
Spike
Work Item
Task
Work Item
User Story
No milestone
No project
No assignees
13 participants
Notifications
Due date
No due date set.
Depends on
#13104 Migration of https://pagure.io/fedora-infra/ansible.git to forge.fedoraproject.org
infra/tickets
#13106 Migration of https://pagure.io/fedora-infra/howtos.git to forge.fedoraproject.org
infra/tickets
#13107 Migration of https://pagure.io/fedora-infra/ipsilon-fedora.git to forge.fedoraproject.org
infra/tickets
#13108 Migration of https://pagure.io/fedora-infra/mailman-stack-maint.git to forge.fedoraproject.org
infra/tickets
#13109 Migration of https://pagure.io/fedora-infra/packager-sync-bz.git to forge.fedoraproject.org
infra/tickets
#13110 Migration of https://pagure.io/fedora-infra/review_stats.git to forge.fedoraproject.org
infra/tickets
#13111 Migration of https://pagure.io/fedora-infra/toddlers.git to forge.fedoraproject.org
infra/tickets
#13112 Migration of https://pagure.io/fedora-infra/w2fm-migration.git to forge.fedoraproject.org
infra/tickets
#13113 Migration of https://pagure.io/fedora-infra/zuul.git to forge.fedoraproject.org
infra/tickets
#13114 Migration of https://pagure.io/ci-resultsdb-listener to forge.fedoraproject.org
infra/tickets
#13117 Migration of https://pagure.io/fedora-infra/distgit-bugzilla-sync to forge.fedoraproject.org
infra/tickets
Reference
infra/tickets#12737
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
To create a more unified and organized environment within our new Forgejo instance, this proposal outlines a new structure for the Infrastructure and Release Engineering repositories. The goal is to improve clarity, streamline ticket tracking, and provide a centralized home for all application code, making it easier for new contributors to get involved.
Proposed Organizational Structure
I propose the creation of two distinct organizations:
Infrastructure: This organization will house repositories specific to infrastructure management. It will include a dedicated ticketing repository to replace the current tracker at https://pagure.io/fedora-infrastructure.
Release Engineering: This organization will contain repositories related to the release engineering process, such as failed-composes. It will also feature a dedicated ticketing repository to replace the one at https://pagure.io/releng.
These two organizations will be dedicated solely to their respective operational and engineering functions and will not be used for hosting application code.
Application Code Repository Consolidation
To centralize our development efforts, I propose creating a third organization:
Websites & Apps: All application code repositories currently located on pagure.io and GitLab will be migrated to this organization. This includes all applications actively developed and maintained by the CLE team.
By creating a unified home for all our applications, we can significantly lower the barrier to entry for new contributors. Each application repository will maintain its issue tracker, preserving the current workflow for development and maintenance.
Benefits
This restructuring will offer several key advantages:
Improved Organization: Separating infrastructure, release engineering, and application code into distinct organizations will create a more logical and manageable repository structure.
Enhanced Clarity: A clear and intuitive layout will make it easier for everyone, especially newcomers, to navigate our projects and understand where to find specific resources.
Centralized Development: Consolidating all application code will streamline development efforts and simplify the onboarding process for new team members.
Metadata Update from @james:
A few questions:
Is ansible 'an application' ? Would it be under infra as a sub org? or ?
When you say 'such as failed-composes' you mean... the tracker that that uses? where the code for the app that files the issues is under websites & apps? or ?
What about github.com/fedora-infra ? many/lots of our applications are there.
Websites & apps seems like it combines two kind of different areas, should we perhaps have seperate 'websites' and 'applications' orgs?
There's also a number of projects that IMHO we can just save off an archive of (for historical reasons) and not migrate.
Oh and a side note: I think no org should normally have 'fedora' in the name. Since they are on a fedoraproject.org domain.
This is where it might get a bit difficult, here is a loose list i made (doenst cover everything), but not sure lumping scripts and stuff in "Applications" is the right fit.
Infrastructure
ReleaseEngineering
Applications
Websites
I'd like to see us migrate the old pagure.io ones over, and then archive them straight away, so we have the history, but close them off (you can archive in forgejo, unlike pagure)
+100 agree to this
We may be able to retire this one. Perhaps we could retire it and put the ansible repo in it's place?
(That would perhaps allow for things like closing issues when a pr that addresses them is pushed, etc)
we still use howtos: https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/#_howtos
This is fuzzy to me. Yes, this is an application, yes we deploy and use it... but it's also used by a lot of other places. It's 'upstream' of us. We also don't really work on it, it's some other folks at Red Hat.
I guess we could ask them what they want to do... but being in our applications space implies that we are the ones who work on it right?
This might go under infra? it's to track packaging mailman for our use. Or perhaps it can be archived now?
Can we rename it poddlers as part of the move? :)
We use this. It's the thing that runs on batcave01 and listens for messages and syncs our local copy of the ansible repo.
This will of course have to be adjusted for forgejo.
This depends on the CI situation in forge.fp.o. Are we just adjusting zuul or using something else. If zuul this will need to stay around.
There's likely a ton of other websites stuff... fedora-web/* etc.
Should infra-docs-fpo be under websites? Or under infrastructure?
Oh, and all these need to be under releng IMHO:
https://pagure.io/fedora-comps
https://pagure.io/fedora-kickstarts/
https://pagure.io/fedora-kiwi-descriptions/
https://pagure.io/workstation-ostree-config
https://pagure.io/pungi-fedora
and probibly more. Would likely be good to look at branching docs and list everything thats touched there.
and...
https://pagure.io/pungi is in the same boat as koji. It's something we use a lot, but we don't maintain it...
Actually, I would prefer renaming our deployment in ansible to "toddlers". The "poddlers" name was a (very funny, yes yes) pun on the fact that the deployment will use multiple pods, but the app name is still toddlers and now that we don't need a separate name to run both in parallel, the toddlers deployment should be renamed back to toddlers I think.
ok, fair point on that.
I definitely agree with this.
In the same vein, I suggest we standardize on using simple lowercase organization names to have shorter/nicer/cleaner URLs. We can still set a longer name as the organization full name. We do this with our org on Codeberg.
fedoraFedora ProjectThat would work out as follows with the orgs that have been brought up so far.
Infrastructure➡️infraReleaseEngineering➡️relengApplications➡️appsWebsites➡️websitesI also noticed we already have a
Community-Linux-Engineeringorg, which could be be renamed tocle.Okay, so from what I am seeing on this thread, that a group layout like the following will suit most of the repos we are talking about.
infraInfrastructurerelengRelease EngineeringappsApplicationswebsitesWebsitescleCommunity Linux EngineeringepelEPELthe fullname for EPEL i left alone, as i feel it has transcended from an acronym to full word status, a-la-
SCUBA,RADAR, andLASERYeah, been thinking about this a bit, and we don't have many cases for this, but if they do want to move this to Fedora Forge (rather than codeberg or something else), do you think this is enough of an edge case for a specific "Koji" organization? Its not tied directly to a SIG or a group, but it is quite tied to Fedora, so it has a solid case to be included.
So, all those would be top level? or are we doing a flat tree anyhow?
I could make a case for epel being under sigs or under editions/subprojects if we are doing a tree...
as for pungi and koji... not sure.
@mikem and @lsedlar care to weigh in? would you prefer to move from pagure.io to forge.fedoraproject.org as a 'fedora' project or just move to codeberg/whatever as you choose to be more an independent upstream?
While I appreciate the sentiment, I don't mind spelling it out as Extra Packages for Enterprise Linux in the full name, as long as the regular name is just
epelfor URLs.Yay, time to revisit the identity crisis of "Is EPEL a SIG, an edition, an initiative, a subproject, or something else?" 😀
Currently we're working in two repo:
I could see this as the future layout:
Or if people don't like
epelbeing both a repo and an org in different contexts, we could do something like:I can bring this up at the next ESC meeting to see if folks have strong feelings about separating issues and docs or about having an
epelorg.I wonder if
siteswould be better thanwebsites? Are all future sites going to be web/HTTP(S) sites? We actually have an open ticket already requesting that agemini://site be created.forgejo is flat-tree only -- same concept as github orgs (gitlab has nested groups/orgs though)
IMO both options are fair for Pungi. Fedora is a big user, but there is at least another use case in Red Hat. Wherever it moves, the pagure.io repos for compose-utils and releng/python-multilib should like go with it. It might boil down to functionality. Is forge.fedoraproject.org going to support some kind of CI and are we okay running pull requests tests on it?
Another interesting use case is fedpkg and fedpkg-minimal. Those would make most sense for me to be fully Fedora projects (CC @onosek). And rpkg should probably remain next to fedpkg?
Hmm, I had previously assumed that the pagure to forgejo migration would continue to support independent projects. I see now that this is not the case.
While Fedora is a hugely important consumer for Koji, there are many others. Infra doesn't own the Koji upstream, nor does any other group or sig in Fedora.
I have a number of other Koji-adjacent repos (and mbs too) in pagure that are in a similar boat. If there is no longer going to be an offering for more independent repos here, then I guess I'll need to look elsewhere. I wish I'd realized this was the case earlier. I've been holding off on leaving pagure because I assumed I'd be moving to this.
Yeah, we decided there wasn't any reason for us to run a general source forge with forgejo when codeberg is right there and thats it's purpose...
I think this will be required for other projects, so I think thats going to be supported.
yes.
Yep.
pagure.io isn't going anywhere short term, there's lots of time. At least IMHO.
So, where are we here? I guess we need to create an exact mapping of what is on pagure.io that we want to move and where we want to move it to?
we did open a more specific ticket for infra and I think there's one for releng too. Is there any meta topics we wanted to still figure out here?
@mikem TBH, it is my optinion that koji and related projects are close enough to Fedora to warrant an organization(s) on fedora forge (if you want to move there).
Koji is a well-established project with close ties to Fedora, so belongs in the new forge. (my opinion)
I need to figure out how we'll handle sync2jira before we jump
Check with @zlopez and/or @humaton about that, they should know whats possible.
I did it for pagure, GitLab and GitHub, but forgejo is using webhooks and syncing to JIRA directly. @humaton should know more as he already set it up for some repositories.
So I want to create a tickets for each repository separately, so it's easier to work on them. Right now I have this in mind:
One question that comes to mind, do we want to migrate archived repos?
@zlopez wrote in #12737 (comment):
I plan to do this one as soon as we are unblocked in fedora-messaging. :)
+1
Sure, but as noted in the other ticket, we should migrate it and then just archive it.
That way we have it in case we need to look at something there after pagure.io is retired.
So, I would say move then immediately archive.
yes, this is being handled by releng already. Take it off our list here.
We do not use this at all that I know of anymore.
So, I guess migrate but archive?
We should remove this from our list. Where and when koji migrates is up to the upstream developers using that project.
Not sure, but perhaps?
+1
It was used to track getting new mailman3 into epel9.
I guess we should ask if it's still useful in the mailman matrix room?
Yeah, move then archive?
Well, it's more of an app I would say? We deploy it in our infra and it runs and gathers things and has search input from users
so I would say apps/ would be better?
+1
+1
Yeah, migrate then archive.
+1
yes, they are already tracking this. Remove from our list.
Yes, remove from our list and let releng track.
Probibly can be moved and archived (since everything is moved over to kiwi now)
No, releng. This is the main place for deliverables defining. Lets remove it from our list and let releng track it.
Yeah, remove from our list, let releng track.
Let upstream pungi developers decide when and where they would like to migrate to.
If they want to go to apps thats fine, but it should be up to them.
pagure doesn't have a archived state I didn't think?
But yeah, I would say for obsolete things we should migrate them and then archive on the forge side.
This will let us keep them in case we need history after pagure.io is retired.
So I want to create a tickets for each repository separately, so it's easier to work on them. Right now I have this in mind:
Following are the archived things on https://pagure.io/group/fedora-infra
We also have pagure and pagure-dist-git repositories in fedora-infra group, do we want to migrate them to forge.fedoraproject.org as well?
Note that this will require firmitas to be able to file forge tickets instead of pagure tickets right?
Looking at this one more, it's just patches we have against ipsilon, not the app, so I think perhaps it should just be in infra/
No. This should be under releng when it's ready to move and we use it for almost every deliverable now. ;)
Leave it off this list I would say for now and we can move it under releng when ready.
I think that would be kinda weird. I'd say leave them for now and before we do any pagure retirement we could move and arhive them to just have them around.
@kevin wrote in #12737 (comment):
Yes, it would need a changes in firmitas. So it's either move the functionality to zabbix or improve firmitas.
ipsilon-fedora -> infra/ipsilon-fedora
I will ignore it then.
Let skip them for now.
With this cleared, let me start creating tickets for the things we want to move.
Tickets created added as dependency for this one and added to next sprint.
fedora-kickstartsto forge.fedoraproject.org #13220fedora-kiwi-descriptionsto forge.fedoraproject.org #13221workstation-ostree-configto forge.fedoraproject.org #13222We (releng) now have dedicated tickets for
fedora-kickstarts[1],fedora-kiwi-descriptions[2], andworkstation-ostree-config[3] on the releng tracker.[1] releng/tickets#13220
[2] releng/tickets#13221
[3] releng/tickets#13222
Note that moving these repositories means I will need to be added to the releng organization, because I'm the primary maintainer of
fedora-kiwi-descriptionsand one of the maintainers offedora-comps.@kevin wrote in #12737 (comment):
I'm not a big fan of this decision, since it increases the friction for Fedora centric projects and weakens the gravity of contributors working within our spaces. But fortunately it's not a permanent decision and we can fix it long before pagure.io goes dark.
@zlopez @kevin
I have a question about the structure. We already have an archived repo under the releng org,
tag2distrepo[1]. It shows that it'sArchivedbut it still shows up and clutters the view which is undesirable as it's not easy to distinguish at first glance. Thoughts? Would it be possible to have something likereleng/archive/<project>or something like that?[1] https://forge.fedoraproject.org/releng
@ngompa - I added you to the releng team with admin rights in the members section. Can you log in and logout in the new forge to reflect?
Yup, I see it now, thanks! Now the only blocker for moving the kiwi descriptions is VM-based CI on the forge. The current options are not sufficient for what we need, that's why it uses Zuul CI (which gives us Fedora VMs).
@patrikp wrote in #12737 (comment):
I think this is on releng team to decide. I didn't tried to archive any repository yet, not sure if it's problematic that it shows up in repository list.
Yeah, it would be nice to not have them shown by default... @ryanlerch is that somehow possible?
fedora-compsto forge.fedoraproject.org #13225I wonder what to do about
https://pagure.io/fedora-pdr? Should we wait till private tickets are available? Or ask people to send mail toadmin@fedoraproject.org(we can probably have separate alias for this).I feel strongly that we should not move fedora-pdr until we have private issues.
email is not a good replacement in this case because we depend on authentication to confirm that the user actually is who they say they are. With email anyone could email from anywhere and we would have no way to know. Also, it would go unencrypted accross the network and potentitally be saved in clear somewhere.
That makes sense. Let's wait with this one till the private issues are possible.
So all the already opened tickets are now resolved. So only thing that remains from pagure is the
fedora-pdrrepository.@kevin Should I open a separate ticket for that and close this one? Or did I miss anything that should be moved as well?
Sure, lets do that I guess, but you can mark it as blocked (until we have private issues).
I'm sure there might be things we missed, but I can't see of/think of any right now... we can move them as we detect them.
The ticket is opened #13175