infra/tickets
24
10
Fork 1
Code Issues 81 Projects Wiki Activity Actions

rdu2-cc to rdu3 datacenter move #12818

New issue
Closed
opened 2025-09-25 13:57:06 +00:00 by kevin · 18 comments
kevin commented 2025-09-25 13:57:06 +00:00
Owner
Copy link

We are going to be moving machines/services from our rdu2 'community cage' datacenter to rdu3 (in a new vlan next to our rdu3 stuff)

This affects:

vmhost-x86-cc01.rdu-cc.fedoraproject.org:pagure-stg01.fedoraproject.org🏃1
vmhost-x86-cc01.rdu-cc.fedoraproject.org:pagure02.fedoraproject.org🏃1
vmhost-x86-cc01.rdu-cc.fedoraproject.org:proxy03.fedoraproject.org🏃1
vmhost-x86-cc01.rdu-cc.fedoraproject.org:smtp-auth-cc-rdu01.fedoraproject.org🏃1
vmhost-x86-cc03.rdu-cc.fedoraproject.org:download-cc-rdu01.fedoraproject.org🏃1
vmhost-x86-cc03.rdu-cc.fedoraproject.org:noc-cc01.rdu-cc.fedoraproject.org🏃1
vmhost-x86-cc03.rdu-cc.fedoraproject.org:proxy14.fedoraproject.org🏃1
vmhost-x86-cc03.rdu-cc.fedoraproject.org:smtp-mm-cc-rdu01.fedoraproject.org🏃1
vmhost-x86-cc06.rdu-cc.fedoraproject.org
vmhost-x86-cc05.rdu-cc.fedoraproject.org
storinator01
vmhost-x86-copr01.rdu-cc.fedoraproject.org
vmhost-x86-copr02.rdu-cc.fedoraproject.org
vmhost-x86-copr03.rdu-cc.fedoraproject.org
vmhost-x86-copr04.rdu-cc.fedoraproject.org
vmhost-p08-copr01.rdu-cc.fedoraproject.org
vmhost-p08-copr02.rdu-cc.fedoraproject.org
vmhost-p09-copr01.rdu-cc.fedoraproject.org

Exact scheduling is stil being determined, but looking like the move will happen in novenber.

These two old hosts will be replaced by a new server:
vmhost-x86-cc06.rdu-cc.fedoraproject.org
vmhost-x86-cc05.rdu-cc.fedoraproject.org

We are going to be moving machines/services from our rdu2 'community cage' datacenter to rdu3 (in a new vlan next to our rdu3 stuff) This affects: vmhost-x86-cc01.rdu-cc.fedoraproject.org:pagure-stg01.fedoraproject.org:running:1 vmhost-x86-cc01.rdu-cc.fedoraproject.org:pagure02.fedoraproject.org:running:1 vmhost-x86-cc01.rdu-cc.fedoraproject.org:proxy03.fedoraproject.org:running:1 vmhost-x86-cc01.rdu-cc.fedoraproject.org:smtp-auth-cc-rdu01.fedoraproject.org:running:1 vmhost-x86-cc03.rdu-cc.fedoraproject.org:download-cc-rdu01.fedoraproject.org:running:1 vmhost-x86-cc03.rdu-cc.fedoraproject.org:noc-cc01.rdu-cc.fedoraproject.org:running:1 vmhost-x86-cc03.rdu-cc.fedoraproject.org:proxy14.fedoraproject.org:running:1 vmhost-x86-cc03.rdu-cc.fedoraproject.org:smtp-mm-cc-rdu01.fedoraproject.org:running:1 vmhost-x86-cc06.rdu-cc.fedoraproject.org vmhost-x86-cc05.rdu-cc.fedoraproject.org storinator01 vmhost-x86-copr01.rdu-cc.fedoraproject.org vmhost-x86-copr02.rdu-cc.fedoraproject.org vmhost-x86-copr03.rdu-cc.fedoraproject.org vmhost-x86-copr04.rdu-cc.fedoraproject.org vmhost-p08-copr01.rdu-cc.fedoraproject.org vmhost-p08-copr02.rdu-cc.fedoraproject.org vmhost-p09-copr01.rdu-cc.fedoraproject.org Exact scheduling is stil being determined, but looking like the move will happen in novenber. These two old hosts will be replaced by a new server: vmhost-x86-cc06.rdu-cc.fedoraproject.org vmhost-x86-cc05.rdu-cc.fedoraproject.org
kevin commented 2025-09-25 13:57:32 +00:00
Author
Owner
Copy link

Issue tagged with: sprint-0

Issue tagged with: sprint-0
kevin commented 2025-09-29 21:03:33 +00:00
Author
Owner
Copy link

I'm writing up acls for the new rdu3 network these are going to, and I need some input from the copr team. ;)

@praiskup @frostyx

My long ago understanding of how copr hypervisors are used was that you ssh to the hypervisor with it's ipv4 address, then start / provision the build vm and then that vm comes up with a ipv6 address and you connect to it from backend and manage it via ipv6. Is that still the case? How do you assign the ipv6 addresses? Is there a pool or ?

For backups to the storinator, do you just ssh directly to it via ipv4 ? or some other path?

Once I get these acls to the networking folks and they get them setup, I can get those 3 power9's up and we can use them to test with... they are already there, just need reinstalled and setup on the right vlan.

I'm writing up acls for the new rdu3 network these are going to, and I need some input from the copr team. ;) @praiskup @frostyx My long ago understanding of how copr hypervisors are used was that you ssh to the hypervisor with it's ipv4 address, then start / provision the build vm and then that vm comes up with a ipv6 address and you connect to it from backend and manage it via ipv6. Is that still the case? How do you assign the ipv6 addresses? Is there a pool or ? For backups to the storinator, do you just ssh directly to it via ipv4 ? or some other path? Once I get these acls to the networking folks and they get them setup, I can get those 3 power9's up and we can use them to test with... they are already there, just need reinstalled and setup on the right vlan.
frostyx commented 2025-10-02 13:33:24 +00:00
Copy link

Is that still the case?

I think so, yes

How do you assign the ipv6 addresses? Is there a pool or ?

There is some range of reserved IP addresses that Fedora Infra told us they won't use for anything else.

I think this is our IPv6 prefix 2620:52:3:1:dead:beef:cafe:c, see pagure.io/fedora-infra/ansible@fdfb14225b/f/roles/copr/backend/templates/provision/libvirt-new (_394)

For backups to the storinator, do you just ssh directly to it via ipv4 ? or some other path?

We use rsnapshot over SSH. The relevant configuration seems to be

rsnapshot_push:
  server_host: storinator01.rdu-cc.fedoraproject.org
  backup_dir: /srv/nfs/copr-be
  cases:
    copr-be-copr-user:
      user: copr
      rsync_args: -H --relative /var/lib/copr/public_html --info=progress2
      command: rsnapshot_copr_backend
      timing_plan: copr_be
      # copr-backend's copr/resalloc pubkey
      ssh_pub_key: ...
> Is that still the case? I think so, yes > How do you assign the ipv6 addresses? Is there a pool or ? There is some range of reserved IP addresses that Fedora Infra told us they won't use for anything else. I think this is our IPv6 prefix `2620:52:3:1:dead:beef:cafe:c`, see https://pagure.io/fedora-infra/ansible/blob/fdfb14225bddc282fb182e57c08c51fcaa72b31e/f/roles/copr/backend/templates/provision/libvirt-new#_394 > For backups to the storinator, do you just ssh directly to it via ipv4 ? or some other path? We use `rsnapshot` over SSH. The relevant configuration seems to be ``` rsnapshot_push: server_host: storinator01.rdu-cc.fedoraproject.org backup_dir: /srv/nfs/copr-be cases: copr-be-copr-user: user: copr rsync_args: -H --relative /var/lib/copr/public_html --info=progress2 command: rsnapshot_copr_backend timing_plan: copr_be # copr-backend's copr/resalloc pubkey ssh_pub_key: ... ```
kevin commented 2025-10-03 17:17:13 +00:00
Author
Owner
Copy link

Thanks for the info!

Thanks for the info!
kevin commented 2025-10-10 17:55:55 +00:00
Author
Owner
Copy link

Status update:

'fedora-isolated' network has been setup.
I sent an initial set of network acls and such to networking and they have deployed them.
I'm moving forward with trying to bring the 3 power9's online there.

Those machines should allow us to see if things are setup as needed/working.

After that, we are expecting a new machine to replace several old rdu2-cc ones, once that arrives we will get it all provisioned.
Then, we hope to migrate pagure.io at least over to it and then the actual move of hardware/outage will happen.

Currently we are looking at early december for the hardware move.

Status update: 'fedora-isolated' network has been setup. I sent an initial set of network acls and such to networking and they have deployed them. I'm moving forward with trying to bring the 3 power9's online there. Those machines should allow us to see if things are setup as needed/working. After that, we are expecting a new machine to replace several old rdu2-cc ones, once that arrives we will get it all provisioned. Then, we hope to migrate pagure.io at least over to it and then the actual move of hardware/outage will happen. Currently we are looking at early december for the hardware move.
kevin commented 2025-10-14 18:27:13 +00:00
Author
Owner
Copy link

I mostly have the new copr power9's setup. Status update in the upstream issue: https://github.com/fedora-copr/copr/issues/3786#issuecomment-3403103822

I mostly have the new copr power9's setup. Status update in the upstream issue: https://github.com/fedora-copr/copr/issues/3786#issuecomment-3403103822
kevin commented 2025-10-22 21:52:48 +00:00
Author
Owner
Copy link

Almost done with those. Then need to wait for new machine to help with the move.

Almost done with those. Then need to wait for new machine to help with the move.
kevin commented 2025-12-03 00:28:44 +00:00
Author
Owner
Copy link

new machine didn't happen, so I repurposed one from the iad2 move.

I moved stg.pagure.io to it today.
Tomorrow we have an outage to move pagure.io to it.
Next monday is the main move of hardware/outage.

new machine didn't happen, so I repurposed one from the iad2 move. I moved stg.pagure.io to it today. Tomorrow we have an outage to move pagure.io to it. Next monday is the main move of hardware/outage.
kevin commented 2025-12-04 17:20:06 +00:00
Author
Owner
Copy link

pagure.io was moved yesterday.

On monday 2025-12-08 we need to:

at 12UTC:

  • disable proxy03 and proxy14 in dns
  • set 24h downtime? or 48h?
  • virsh shutdown any of the following vm's:

vmhost-x86-cc01.rdu-cc.fedoraproject.org:proxy03.fedoraproject.org🏃1
vmhost-x86-cc01.rdu-cc.fedoraproject.org:smtp-auth-cc-rdu01.fedoraproject.org🏃1
vmhost-x86-cc03.rdu-cc.fedoraproject.org:download-cc-rdu01.fedoraproject.org🏃1
vmhost-x86-cc03.rdu-cc.fedoraproject.org:noc-cc01.rdu-cc.fedoraproject.org🏃1
vmhost-x86-cc03.rdu-cc.fedoraproject.org:proxy14.fedoraproject.org🏃1
vmhost-x86-cc03.rdu-cc.fedoraproject.org:smtp-mm-cc-rdu01.fedoraproject.org🏃1

  • also 'virsh autostart --disable' them.
  • shutdown -h:

inventory/hardware:vmhost-x86-cc01.rdu-cc.fedoraproject.org
inventory/hardware:vmhost-x86-cc02.rdu-cc.fedoraproject.org
inventory/hardware:vmhost-x86-cc03.rdu-cc.fedoraproject.org
inventory/hardware:vmhost-x86-copr01.rdu-cc.fedoraproject.org
inventory/hardware:vmhost-x86-copr02.rdu-cc.fedoraproject.org
inventory/hardware:vmhost-x86-copr03.rdu-cc.fedoraproject.org
inventory/hardware:vmhost-x86-copr04.rdu-cc.fedoraproject.org
inventory/hardware:vmhost-p09-copr01.rdu-cc.fedoraproject.org
inventory/hardware:storinator01.rdu-cc.fedoraproject.org
and
retrace03.rdu-cc.fedoraproject.org

  • Mention in internal slack that everything is shutdown and ready to move.
pagure.io was moved yesterday. On monday 2025-12-08 we need to: at 12UTC: * disable proxy03 and proxy14 in dns * set 24h downtime? or 48h? * virsh shutdown any of the following vm's: vmhost-x86-cc01.rdu-cc.fedoraproject.org:proxy03.fedoraproject.org:running:1 vmhost-x86-cc01.rdu-cc.fedoraproject.org:smtp-auth-cc-rdu01.fedoraproject.org:running:1 vmhost-x86-cc03.rdu-cc.fedoraproject.org:download-cc-rdu01.fedoraproject.org:running:1 vmhost-x86-cc03.rdu-cc.fedoraproject.org:noc-cc01.rdu-cc.fedoraproject.org:running:1 vmhost-x86-cc03.rdu-cc.fedoraproject.org:proxy14.fedoraproject.org:running:1 vmhost-x86-cc03.rdu-cc.fedoraproject.org:smtp-mm-cc-rdu01.fedoraproject.org:running:1 * also 'virsh autostart --disable' them. * shutdown -h: inventory/hardware:vmhost-x86-cc01.rdu-cc.fedoraproject.org inventory/hardware:vmhost-x86-cc02.rdu-cc.fedoraproject.org inventory/hardware:vmhost-x86-cc03.rdu-cc.fedoraproject.org inventory/hardware:vmhost-x86-copr01.rdu-cc.fedoraproject.org inventory/hardware:vmhost-x86-copr02.rdu-cc.fedoraproject.org inventory/hardware:vmhost-x86-copr03.rdu-cc.fedoraproject.org inventory/hardware:vmhost-x86-copr04.rdu-cc.fedoraproject.org inventory/hardware:vmhost-p09-copr01.rdu-cc.fedoraproject.org inventory/hardware:storinator01.rdu-cc.fedoraproject.org and retrace03.rdu-cc.fedoraproject.org * Mention in internal slack that everything is shutdown and ready to move.
gwmngilfen commented 2025-12-08 12:45:46 +00:00
Member
Copy link

All hosts down and proxies should be disabled in DNS.

All hosts down and proxies should be disabled in DNS.
kevin commented 2025-12-12 00:03:32 +00:00
Author
Owner
Copy link

ok, things are moved but there's still work to clean things up. :)

  1. vmhost-x86-iso02 (formerly vmhost-x86-cc01), vmhost-x86-copr01/03 and vmhost-x86-iso03 (formerly vmhost-x86-cc02) all need 10G connections sorted out. Datacenter folks plan to work on that tomorrow morning.

  2. vmhost-x86-copr02 may need a service intervention. It doesn't see it's 10G card (which I swear exists) and it won't apply firmware updates. ;(

  3. storinator01 is moved, but I can't seem to login to the admin interface. ;( We may need to get someone to do a reset on it.

  4. smtp-auth-iso01 (formerly smtp-auth-cc-rdu01) is redeployed, but I need to get the saslauth passwords off the old one (which is on vmhost-x86-iso02). Or we need to reissue passwords to all the users of it.

  5. smtp-mm-iso01 (formerly smtp-mm-rdu01) needs to still be re deployed.

ok, things are moved but there's still work to clean things up. :) 1. vmhost-x86-iso02 (formerly vmhost-x86-cc01), vmhost-x86-copr01/03 and vmhost-x86-iso03 (formerly vmhost-x86-cc02) all need 10G connections sorted out. Datacenter folks plan to work on that tomorrow morning. 2. vmhost-x86-copr02 may need a service intervention. It doesn't see it's 10G card (which I swear exists) and it won't apply firmware updates. ;( 3. storinator01 is moved, but I can't seem to login to the admin interface. ;( We may need to get someone to do a reset on it. 4. smtp-auth-iso01 (formerly smtp-auth-cc-rdu01) is redeployed, but I need to get the saslauth passwords off the old one (which is on vmhost-x86-iso02). Or we need to reissue passwords to all the users of it. 5. smtp-mm-iso01 (formerly smtp-mm-rdu01) needs to still be re deployed.
kevin commented 2025-12-12 23:06:32 +00:00
Author
Owner
Copy link

Here's today's status:

  • vmhost-x86-iso02/03 are up and online. I didn't reinstall them, just reconfigured network and ran ansible over them. Just ran out of time for reinstalling.

  • vmhost-x86-copr01: DC folks have 10G connections to the machine, but it shows no link still. Will need more investigation.

  • vmhost-x86-copr02: lspci on the host shows a 10G card. DC ops confirm it's there and has fiber to it, but the OS does not see it at all. Might be worth trying to pxeboot and do a rhel10 install to see if it's a rhel8 driver issue somehow. (however, the drac also doesn't show it).

  • vmhost-x86-copr03/04: ready to be reinstalled.

  • storinator01: I can't get into mgmt interface, and there was no way DC ops could see to reset it. I guess we should see if we can work with them to boot a fedora live media and use ipmitool locally to reset the mgmt password.

  • smtp-auth-iso01 is up and I copied certs and saslauthdb from the old one. I don't have a easy/good way to test this tho...

  • smtp-mm-iso01 is done.

  • proxy03/14 are installed and setup (proxy03 I am still ansiblizing, but should be done soon).

So the outstanding things next week would be reinstalling copr machines (work with copr team), and working with DC ops/networking to figure out storinator and those copr machines with issues above.

Here's today's status: * vmhost-x86-iso02/03 are up and online. I didn't reinstall them, just reconfigured network and ran ansible over them. Just ran out of time for reinstalling. * vmhost-x86-copr01: DC folks have 10G connections to the machine, but it shows no link still. Will need more investigation. * vmhost-x86-copr02: lspci on the host shows a 10G card. DC ops confirm it's there and has fiber to it, but the OS does not see it at all. Might be worth trying to pxeboot and do a rhel10 install to see if it's a rhel8 driver issue somehow. (however, the drac also doesn't show it). * vmhost-x86-copr03/04: ready to be reinstalled. * storinator01: I can't get into mgmt interface, and there was no way DC ops could see to reset it. I guess we should see if we can work with them to boot a fedora live media and use ipmitool locally to reset the mgmt password. * smtp-auth-iso01 is up and I copied certs and saslauthdb from the old one. I don't have a easy/good way to test this tho... * smtp-mm-iso01 is done. * proxy03/14 are installed and setup (proxy03 I am still ansiblizing, but should be done soon). So the outstanding things next week would be reinstalling copr machines (work with copr team), and working with DC ops/networking to figure out storinator and those copr machines with issues above.
gwmngilfen commented 2025-12-17 16:49:24 +00:00
Member
Copy link

I've dug into the 10G links, and made no progress. The 01 links are down no matter what I try (even in the iDRAC), and the card doesn't even show up in the 02 iDRAC.

03/04 seem fine, I've reinstalled 03 with RHEL10 and working with @praiskup on setting that up.

storinator is waiting for onsite help with a livecd.

I've dug into the 10G links, and made no progress. The 01 links are down no matter what I try (even in the iDRAC), and the card doesn't even show up in the 02 iDRAC. 03/04 seem fine, I've reinstalled 03 with RHEL10 and working with @praiskup on setting that up. storinator is waiting for onsite help with a livecd.
gwmngilfen commented 2025-12-23 15:24:25 +00:00
Member
Copy link

Update before I wrap:

  • I worked with RDU3 to try the card from copr03 in 01 and 02. 01 came up, 02 did not - but now that the working card is back in 03 it also does not come up. Probably need to do more networking checks, and we need at least one new card.
  • copr04 is done, but Pavel says it didn't reboot. However it seems fine to me, need to recheck with Copr folks
  • Storinator01 had it's BC pw reset, but is getting high packet loss (40-50%) which makes the BMC near unusable.
Update before I wrap: - I worked with RDU3 to try the card from copr03 in 01 and 02. 01 came up, 02 did not - but now that the working card is back in 03 it also does not come up. Probably need to do more networking checks, and we need at least one new card. - copr04 is done, but Pavel says it didn't reboot. However it seems fine to me, need to recheck with Copr folks - Storinator01 had it's BC pw reset, but is getting high packet loss (40-50%) which makes the BMC near unusable.
kevin commented 2026-01-20 00:10:18 +00:00
Author
Owner
Copy link

So, current status:

  • 2 of the copr machines (01/02) their network cards don't work. So, we are engaging warentee work to replace / fix them.

  • The storinator turns out to be... not our storinator. ;( The wrong machine was shipped. Luckily the real machine is still in rdu2 and will move on feb 9th hopefully.

So, current status: * 2 of the copr machines (01/02) their network cards don't work. So, we are engaging warentee work to replace / fix them. * The storinator turns out to be... not our storinator. ;( The wrong machine was shipped. Luckily the real machine is still in rdu2 and will move on feb 9th hopefully.
gwmngilfen commented 2026-01-26 15:10:14 +00:00
Member
Copy link

Update:

  • Copr03 is back. Have at it @praiskup :)
  • Copr 01/02 are still down. Worked with MGalgoci on Fri, and the SFPs on copr02 have link, so I think it's really dead. Need to file tickets upstream
Update: - Copr03 is back. Have at it @praiskup :) - Copr 01/02 are still down. Worked with MGalgoci on Fri, and the SFPs on copr02 have link, so I think it's really dead. Need to file tickets upstream
kevin commented 2026-01-26 15:23:41 +00:00
Author
Owner
Copy link

I am going to close this ticket as the move is done.

Then file 2 new tickets to track:

  • copr x86 hosts issues
  • storinator
I am going to close this ticket as the move is done. Then file 2 new tickets to track: * copr x86 hosts issues * storinator
kevin closed this issue 2026-01-26 15:23:41 +00:00
kevin commented 2026-01-26 17:44:26 +00:00
Author
Owner
Copy link

#13083 ( vmhost-x86-01/02 network cards )

#13084 ( storinator, moving feb 9th)

#13085 ( copr power9 ipv6 issues )

https://forge.fedoraproject.org/infra/tickets/issues/13083 ( vmhost-x86-01/02 network cards ) https://forge.fedoraproject.org/infra/tickets/issues/13084 ( storinator, moving feb 9th) https://forge.fedoraproject.org/infra/tickets/issues/13085 ( copr power9 ipv6 issues )
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
announcement

Archived

  
anubis
Issues related to anubis

  
authentication
Authentication related issues

  
aws
Issues related to Amazon Web Services

  
backlog
Tasks that will take longer then ~4 hours to do

Archived

  
blocked
Issue is blocked

Archived

  
bodhi
Issues with Bodhi

  
ci
Continuous Integration

  
cloud
Items in FedoraInfraCloud or COmmunishift

  
communishift
issues/questions about os.edorainfracloud.org ( Community OpenShift)

  
copr
COPR system (run by other team)

  
database
Issues related to databases

  
day-to-day
These are small self contained tasks that are part of normal operations, but are async from sprint planning.

  
dc-move
Items related to datacenter move

Archived

  
deprecated
Tools which Infrastructure may not be able to fix

Archived

  
dev
Tasks requiring some dev work

  
discourse
outsourced tool

  
dns
Changes or problems with DNS

  
downloads
rsync,dl.fp.o,AWS,torrents

  
easyfix

Archived

  
epel
Issues relating to epel

Archived

  
firmitas

   
forgejo_migration
Request migration to the new forge

  
Gain
High
 task that gets us a lot / is important / makes many people happy

Archived

  
Gain
Low
 tasks that gets us little / isn't very important / makes only 1 person or few happy

Archived

  
Gain
Medium
 task that gets us some / is somewhat important / makes some people happy

Archived

  
gitlab

   
greenwave
Issues with greenwave

  
hardware
Problem is with Fedora Infrastructure Owned/Leased Hardware

  
help wanted

   
high-trouble
task that is complex/long/difficult

Archived

  
koji
Issues with the build system

  
koschei
Items needing Koschei team

  
lists
issues with lists

  
low-trouble
task that is easy/short

Archived

  
medium-trouble
task that is medium complex / difficult

Archived

  
mirrorlists
Way to tell systems where downloads are

  
monitoring

   
Needs investigation

   
odcs
On Demand Compose Service issues

Archived

  
OpenShift
Issues with OpenShift

  
ops
ops problem now...

Archived

  
outage

   
packager_workflow_blocker
Blocks the packager workflow

Archived

  
pagure
Issues with pagure.io

  
permissions
Items which need permissions set in outside tools

  
Priority
Needs Review
 Priority of Needs Review

Archived

  
Priority
Next Meeting
 Priority of Next Meeting

Archived

  
Priority
🔥 URGENT 🔥
 Priority of 🔥 URGENT 🔥

Archived

  
Priority
Waiting on Assignee
 Priority of Waiting on Assignee

Archived

  
Priority
Waiting on External
 Priority of Waiting on External

Archived

  
Priority
Waiting on Reporter
 Priority of Waiting on Reporter

Archived

  
rabbitmq
issues with the message broker

  
release-monitoring
release-monitoring.org (Anitya)

  
releng
For problems which should be in the releng ticket system

Archived

  
request-for-resources

   
s390x
The secondary architecture hardware and problems related with it

  
security

   
SMTP
Email issues outside of lists (DMARC, SPF, aliases)

  
sprint-0

Archived

  
sprint-1

Archived

  
src.fp.o
Issues related to src.fedoraproject.org

  
staging
Issues in staging

  
unfreeze
Patches to be applied after freeze is lifted

  
waiverdb
Issues with waiverdb

  
websites-general
General items with a website focus

  
wiki
Issues with the Wiki

  
Backlog Status
Needs Review
 Work items that have been created or updated but need review, refinement, or approval before they can be considered ready for sprint planning.

  
Backlog Status
Ready
 Work items that have been reviewed, refined, and are ready to be committed to a sprint. These items have clear requirements and can be worked on immediately when capacity is available.

  
chore
Work that needs to be done but doesn't add new functionality or fix bugs. Maintenance tasks that keep the system running smoothly and the codebase clean.

  
documentation
Work focused on creating, updating, or improving documentation for users, developers, or system administrators.

  
points
01
 A task that is simple, small, and has very little uncertainty. It should be easy enough for any team member to pick up and complete quickly, without a lot of discussion or research.

  
points
02
 This is a task that is slightly more complex or larger than a "1." It might involve two or three simple steps, but the outcome is still very clear and there's little uncertainty.

  
points
03
 Medium-sized task with some noticeable complexity. It might require a bit more thought or collaboration. The steps are well-defined, but it's not a trivial change.

  
points
05
 A significant piece of work. It has more complexity and a higher degree of uncertainty than smaller tasks. It might require some research, a design discussion, or collaboration across multiple teams.

  
points
08
 Complex task with a lot of uncertainty. It's often the largest size that a team will consider for a single sprint. It likely requires a detailed breakdown and multiple team members to complete.

  
points
13
 Any task that feels like it's a "13" or more is considered an Epic. These are too large and uncertain to be worked on in a single sprint and should be broken down.

  
Priority
High
 This is for work that is critical and needs to be addressed in the next sprint. It's often for urgent bugs, important features, or a major piece of technical debt that is blocking other work.

  
Priority
Low
 This is for work that is good to do but not urgent. It's a candidate for being worked on when there's extra time or when it rises in importance. It can stay in the backlog for a longer period.

  
Priority
Medium
 This is for important work that should be done soon, but isn't as urgent as a high-priority item. It will likely be considered for the next 1-2 sprints.

  
Sprint Status
Blocked
 Work items that cannot progress due to external dependencies, blockers, or issues that need resolution.

  
Sprint Status
Done
 Work items that have been completed, reviewed, and integrated. No further work is needed within the current sprint.

  
Sprint Status
In Progress
 Work items currently being worked on by a team member. Active development, research, or implementation is happening.

  
Sprint Status
Review
 Work items that are complete but need review, testing, or approval before being marked as done.

  
Sprint Status
To Do
 Work items that are committed to the sprint but haven't been started yet. These items are ready to be picked up by team members.

  
Technical Debt
Work that addresses shortcuts, workarounds, or suboptimal implementations that were done previously to meet deadlines. These items improve code quality, maintainability, or performance without adding new features.

  
Work Item
Bug
 Issues where the system is not working as intended or designed. Bugs represent deviations from expected behavior.

  
Work Item
Epic
 Large initiatives that are broken down into multiple smaller work items (user stories, tasks, bugs). Epics represent significant features or projects that span multiple sprints.

  
Work Item
Spike
 Time-boxed research, investigation, or proof-of-concept work to reduce uncertainty or explore solutions. Spikes help inform future development decisions.

  
Work Item
Task
 Used to break down larger user stories into smaller, manageable implementation pieces. They focus on the specific technical steps needed to deliver the user story's value.

  
Work Item
User Story
 User-facing features or functionality that can be framed from a user perspective, describing what someone wants to accomplish and why.

No labels
announcement
anubis
authentication
aws
backlog
blocked
bodhi
ci
cloud
communishift
copr
database
day-to-day
dc-move
deprecated
dev
discourse
dns
downloads
easyfix
epel
firmitas
forgejo_migration
Gain
High
Gain
Low
Gain
Medium
gitlab
greenwave
hardware
help wanted
high-trouble
koji
koschei
lists
low-trouble
medium-trouble
mirrorlists
monitoring
Needs investigation
odcs
OpenShift
ops
outage
packager_workflow_blocker
pagure
permissions
Priority
Needs Review
Priority
Next Meeting
Priority
🔥 URGENT 🔥
Priority
Waiting on Assignee
Priority
Waiting on External
Priority
Waiting on Reporter
rabbitmq
release-monitoring
releng
request-for-resources
s390x
security
SMTP
sprint-0
sprint-1
src.fp.o
staging
unfreeze
waiverdb
websites-general
wiki
Backlog Status
Needs Review
Backlog Status
Ready
chore
documentation
points
01
points
02
points
03
points
05
points
08
points
13
Priority
High
Priority
Low
Priority
Medium
Sprint Status
Blocked
Sprint Status
Done
Sprint Status
In Progress
Sprint Status
Review
Sprint Status
To Do
Technical Debt
Work Item
Bug
Work Item
Epic
Work Item
Spike
Work Item
Task
Work Item
User Story
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Sprint ( 2026-01-12 to 2026-01-25 )
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
infra/tickets#12818
No description provided.