operations/docs
operations/docs
4
2
Fork 11
Code Issues 3 Pull requests 1 Activity Actions

Add COVID-19 risk assessment

Browse source
This commit is contained in:
Ben Cotton 2020-03-20 16:49:54 -04:00
commit c17f1ddadb
2 changed files with 103 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
modules/ROOT/nav.adoc
View file

@ -1,4 +1,5 @@
* Changes
** xref:covid19.adoc[COVID-19 risk assessment]
** xref:changes_policy.adoc[Changes policy]
** xref:changes_guide.adoc[Change submission guidance]
*** xref:changes_guide.adoc#_what_are_the_change_process_deadline_dates_checkpoints[Deadlines]

102
modules/ROOT/pages/covid19.adoc Normal file
View file

@ -0,0 +1,102 @@
= COVID-19 risk assessment
:toc:
The primary concern of the Fedora leadership remains the health and safety of the Fedora community.
However, we are tracking the risk to Fedora release deliverables as the situation evolves.
This document is maintained by the Fedora Program Manager (Ben Cotton) with the Fedora Project Leader (Matthew Miller) as a backup.
== Status
.Project status
|===
|Project |Status
|Fedora 32 |On Track{set:cellbgcolor:green}
|Data center move{set:cellbgcolor:!}|At Risk{set:cellbgcolor:yellow}
|Fedora 33{set:cellbgcolor:!}|Not tracking yet
|===
=== Changelog
* 2020-03-20 - bcotton - Initial version
== Points of contact
.Points of contact
|===
|Area|Name|Email|Location/time zone
|Program Management|Ben Cotton|bcotton AT redhat DOT com|Indiana, US (Eastern)
|Fedora Project Leader|Matthew Miller|mattdm AT redhat DOT com|Massachusetts, US (Eastern)
|Infrastructure|Kevin Fenzi|kfenzi AT redhat DOT com|Oregon, US (Pacific)
|Release Engineering|Mohan Boddu|mboddu AT redhat DOT com|Massachusetts, US (Eastern)
|Data center move|Stephen Smoogen|ssmoogen AT redhat DOT com|North Carolina, US (Eastern)
|QA|Adam Williamson|awilliam AT redhat DOT com|British Columbia, CA (Pacific)
|===
== Fedora 32 release
**Key Dates:**
* 2020-04-07 - Final freeze begins
* 2020-04-21 - Fedora 32 final release preferred target date
* 2020-04-28 - Fedora 32 final release target date #1
[cols="2,1,1,4"]
.Fedora 32 risks
|===
|Risk area|Impact|Likelihood|Mitigation & Notes
|Release engineering staff|High|Low|Release Engineering knowledge is duplicated across multiple people with large geographic separation
|Infrastucture staff|High|Low|Infrastructure knowledge is duplicated across multiple people with large geographic separation
|QA test staff|High|Low|Testing staff (both Red Hat and non-Red Hat contributors) are widely distributed across the globe
|openQA test platform|Medium|Medium|Adam Williamson is the primary admin for openQA. However, other members of the team have knowledge and could keep the platform going if Adam is unavailable.
|QA test hardware|Medium|High|Some QA tests require physical access to hardware located in Red Hat offices (see table below). Some of these can be done with contributor’s personally-owned hardware and others may be skipped if Red Hat offices remain closed through the release.
|===
=== Test cases at risk
The table below represents test cases identified by the QA team which may be impacted by lack of access to Red Hat offices.
Where “use contributor-owned hardware” is listed in the mitigation section, someone has volunteered to run those tests.
For others, we will put out public calls for testing and may decide to exclude those tests for this release if office shutdown continues.
.Fedora 32 at-risk test cases
|===
|Test case|Mitigation
|Testcase_Boot_default_install|Use contributor-owned hardware. May require a longer period between release candidate compose and go/no-go decision
|Testcase_Anaconda_User_Interface_Basic_Video_Driver|Use contributor-owned hardware.
|Testcase_audio_basic|
|Testcase_desktop_menus|
|Testcase_desktop_automount|
|Testcase_Printing_New_Printer|Use contributor-owned hardware.
|===
== Data center move
**Key dates:**
* 2020-04-13 - Move 1: Essential hardware in IAD2 datacenter
* 2020-04-13 - Move 2: Communishift hardware to RDU-CC datacenter
* 2020-05-01 - CPE management Go/No-Go decision for move
* 2020-05-11 - Move 3: QA equipment to IAD2 datacenter
* 2020-05-20–2020-07-01: Reduced build capacity
* 2020-06-01 - Move 4: Remaining hardware to IAD2 datacenter
* 2020-06-15 - Hard deadline for hardware to be out of PHX2
* 2020-07-22 - Mass rebuild begins
[cols="2,1,1,4"]
.Data center move risks
|===
|Risk area|Impact|Likelihood|Mitigation & Notes
|Delayed delivery of Move 1 hardware to IAD2|High|Medium|Some delays have happened with delivery of new hardware. Team is exploring mitigation with alternative shelving and zip ties.
CPE team working with Red Hat IT and vendors on capacity plans. If it is delayed to Move 3 or 4, we will not have build infrastructure during the early part of the Fedora development cycle.
|Delayed delivery of Move 3 hardware to IAD2 after leaving PHX2|High|Medium|QA infrastructure remains offline. Impact is lessened unless the outage continues into late July. Otherwise, it may impact Fedora 33 schedule.
|Delayed delivery of Move 4 hardware to IAD2 after leaving PHX2|Medium|Medium|Reduced build capacity remains in place longer than expected. This may end up impacting the Fedora 33 mass rebuild window.
|Datacenter staff availability|High|Medium|Datacenters have implemented quarantine policies to mitigate the risk of staff infection. If staff become unavailable during racking/de-racking times, schedule will be delayed.
|Infrastructure staff availability|High|Medium|1.5 sysadmins are assigned to perform remote work. They are following the guidance of Red Hat and local health officials. If their capacity is reduced, we can try to find additional help, but reinforcements may lack the requisite knowledge.
|Hardware remains in PHX2 after 15 June|Medium|Medium|Hardware that remains in service will be removed by datacenter staff and stored for later shipment. Fedora will continue with reduced build capacity beyond the scheduled window, potentially impacting the mass rebuild.
|===