Plan what to do about advanced storage testing

adamwill commented

2026-04-10 01:40:55 +00:00

Owner

Since lili is no longer available, we need someone else to own https://forge.fedoraproject.org/quality/fedora-release-autotest and the exotic storage hardware testing that was still(?) done using it. It was forked off fedora_openqa and hacked up to create jobs in Beaker, basically. So we need to find someone to initiate whoever the lucky winner is into the mysteries of Beaker. Would be nice if we can clean it up a bit and get result reporting done automatically, too.

adamwill added the

labels

2026-04-10 01:40:55 +00:00

adamwill added this to the Sprint 8 project

2026-04-10 01:40:55 +00:00

kparal commented

2026-04-10 10:33:44 +00:00

Owner

First I think we need to figure out the status of the tool. Was is really used? For what exactly? Was Lili's exotic hw testing really done through this?

Lili said the project is still active, and that's why we migrated it. But codebase was basically frozen for the last 4 years, and I'm quite skeptical that something that was used regularly didn't need any maintenance. Also, I can't find any actual test cases in that project, just their metadata descriptions. If this is used for say FCOE testing, where is the testcase code?

Once we figure this out, we should discuss whether we actually want to maintain this. And then designate the maintainer 🙂

(And for the almost-done F44 cycle, we should contact Server SIG and ask them to test that exotic hw in their team).

First I think we need to figure out the status of the tool. Was is really used? For what exactly? Was Lili's exotic hw testing really done through this? Lili said the project is still active, and that's why we migrated it. But codebase was basically frozen for the last 4 years, and I'm quite skeptical that something that was used regularly didn't need any maintenance. Also, I can't find any actual test cases in that project, just their metadata descriptions. If this is used for say FCOE testing, where is the testcase code? Once we figure this out, we should discuss whether we actually want to maintain this. And then designate the maintainer 🙂 (And for the almost-done F44 cycle, we should contact Server SIG and ask them to test that exotic hw in their team).

psklenar commented

2026-04-10 14:16:11 +00:00

Owner

I can look into this one.
Firstly I would like to check which hw were used and which hw are still available for us.

Some of the beaker jobs could be replaced by testing-farm which is now more supported.

I can look into this one. Firstly I would like to check which hw were used and which hw are still available for us. Some of the beaker jobs could be replaced by testing-farm which is now more supported.

psklenar self-assigned this

2026-04-10 14:26:59 +00:00

adamwill commented

2026-04-10 16:58:14 +00:00

Author

Owner

I think the 'tests' are really just kickstarts - after all, all we're establishing here is "does the installer work with this weird storage hardware?", so all we need to do is run a kickstart install on appropriately-configured hardware. Note that https://forge.fedoraproject.org/quality/fedora-release-autotest/src/branch/master/fedora_release_autotest/conf_test_cases.py has a bunch of ks_append values in it which are probably significant. I suspect Lili might have been doing post-install validation and reporting results manually, since I think beaker allows interactive access to the configured hardware.

My theory is that all this does, in the end, is generate appropriate kickstarts and beaker requests for the appropriate hardware and then plug Kickstart A into Beaker Hardware B.

As for the lack of maintenance - well, we haven't changed the tests in forever, and I don't think Beaker changes much, I think RH has it filed under "annoying old thing we haven't figured out how to get rid of yet". So I think it's kinda plausible this could keep working without much change.

I think the 'tests' are really just kickstarts - after all, all we're establishing here is "does the installer work with this weird storage hardware?", so all we need to do is run a kickstart install on appropriately-configured hardware. Note that https://forge.fedoraproject.org/quality/fedora-release-autotest/src/branch/master/fedora_release_autotest/conf_test_cases.py has a bunch of `ks_append` values in it which are probably significant. I suspect Lili might have been doing post-install validation and reporting results manually, since I think beaker allows interactive access to the configured hardware. My theory is that all this does, in the end, is generate appropriate kickstarts and beaker requests for the appropriate hardware and then plug Kickstart A into Beaker Hardware B. As for the lack of maintenance - well, we haven't changed the tests in forever, and I don't think Beaker changes much, I think RH has it filed under "annoying old thing we haven't figured out how to get rid of yet". So I think it's kinda plausible this could keep working without much change.

adamwill commented

2026-04-10 16:59:37 +00:00

Author

Owner

(And for the almost-done F44 cycle, we should contact Server SIG and ask them to test that exotic hw in their team).

We can ask if they want to, but let's be careful not to cast this as requiring them to do it - IIRC, all these exotic storage requirements have been around forever and didn't originate with the Server WG (I think they pre-date it).

If we can't figure this out quickly it might be reasonable for F44 to just not complete this testing and flag up that we were unable to do it due to the situation.

> (And for the almost-done F44 cycle, we should contact Server SIG and ask them to test that exotic hw in their team). We can ask if they *want* to, but let's be careful not to cast this as requiring them to do it - IIRC, all these exotic storage requirements have been around forever and didn't originate with the Server WG (I think they pre-date it). If we can't figure this out quickly it might be reasonable for F44 to just not complete this testing and flag up that we were unable to do it due to the situation.

adamwill commented

2026-04-28 18:29:29 +00:00

Author

Owner

So, update here: we now know how to run tests in beaker in theory (right @psklenar ?), but we don't have access to the specific hardware the matrix wants us to test on, due to...Red Hat stuff.

So, before Fedora 45 Beta, we have to I guess either:

Get that sorted out and set up / revive the testing process
Decide to drop the criteria requirements for affected hardware, and downgrade the table rows to Optional
Leave the requirements and test rows in but process-document (somehow) that they're currently "suspended" / "block-if-anyone-finds-a-failure" due to HW access issues

Thoughts?

So, update here: we now know how to run tests in beaker in theory (right @psklenar ?), but we don't have access to the specific hardware the matrix wants us to test on, due to...Red Hat stuff. So, before Fedora 45 Beta, we have to I guess either: 1. Get that sorted out and set up / revive the testing process 2. Decide to drop the criteria requirements for affected hardware, and downgrade the table rows to Optional 3. Leave the requirements and test rows in but process-document (somehow) that they're currently "suspended" / "block-if-anyone-finds-a-failure" due to HW access issues Thoughts?

adamwill commented

2026-04-28 18:30:02 +00:00

Author

Owner

Removing this from current sprint as we released F44 with a kinda waiver for this so it's no longer super urgent, we have a bit of time to think it through.

adamwill removed this from the Sprint 8 project

2026-04-28 18:30:07 +00:00

adamwill added

and removed

labels

2026-04-28 18:30:21 +00:00

adamwill added this to the Fedora 45 milestone

2026-04-28 18:30:25 +00:00

adamwill changed title from ~~Find new maintainer for fedora-release-autotest, onboard to mysteries of beaker etc.~~ to Plan what to do about advanced storage testing

2026-04-28 18:31:22 +00:00

kparal commented

2026-04-30 17:09:06 +00:00

Owner

#890 is related to this

psklenar commented

2026-05-05 10:05:58 +00:00

Owner

@adamwill wrote in #889 (comment):

So, update here: we now know how to run tests in beaker in theory (right @psklenar ?), but we don't have access to the

yes

specific hardware the matrix wants us to test on, due to...Red Hat stuff.

So, before Fedora 45 Beta, we have to I guess either:
1. Get that sorted out and set up / revive the testing process

2. Decide to drop the criteria requirements for affected hardware, and downgrade the table rows to Optional

I am going to ask 'server workgroup' about #890 what is really required. And then let's see what we can run with the current and free hw.

3. Leave the requirements and test rows in but process-document (somehow) that they're currently "suspended" / "block-if-anyone-finds-a-failure" due to HW access issues

Thoughts?

@adamwill wrote in https://forge.fedoraproject.org/quality/tickets/issues/889#issuecomment-676727: > So, update here: we now know how to run tests in beaker in theory (right @psklenar ?), but we don't have access to the yes > specific hardware the matrix wants us to test on, due to...Red Hat stuff. > > So, before Fedora 45 Beta, we have to I guess either: > > 1. Get that sorted out and set up / revive the testing process > > 2. Decide to drop the criteria requirements for affected hardware, and downgrade the table rows to Optional I am going to ask 'server workgroup' about [#890](https://forge.fedoraproject.org/quality/tickets/issues/890) what is really required. And then let's see what we can run with the current and free hw. > > 3. Leave the requirements and test rows in but process-document (somehow) that they're currently "suspended" / "block-if-anyone-finds-a-failure" due to HW access issues > > > Thoughts?

humaton added a new dependency

2026-08-11 15:56:05 +00:00

cle/tickets#4 - Fedora 45 release tracking

Rows
Columns

Plan what to do about advanced storage testing #889