cle/docs
36
0
Fork 0
Code Issues 3 Pull requests Projects Releases Packages Wiki Activity Actions

Try another structure for the CPE doc

Browse source 
Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
This commit is contained in:
Pierre-YvesChibon 2019-07-26 17:37:13 +02:00
commit 1345c92229
8 changed files with 534 additions and 139 deletions
Download patch file Download diff file Expand all files Collapse all files

8
modules/cpe/nav.adoc
View file

@ -1,3 +1,7 @@
* xref:mission_statement.adoc[The CPE Team Mission]
* xref:working_with_us.adoc[Working with Community Platform Engineering]
* xref:working_with_us.adoc[Working with CPE]
** xref:day_to_day_fedora.adoc[Day to day in Fedora]
** xref:day_to_day_centos.adoc[Day to day in CentOS]
** xref:initiatives.adoc[Initiatives]
** xref:sle.adoc[SLE]
*** xref:sle_services.adoc[Services]
* xref:about.adoc[About the Team]

44
modules/cpe/pages/about.adoc Normal file
View file

@ -0,0 +1,44 @@
= About the team
In 2017 the Fedora Engineering team merged with the CentOS Engineering team
to form what is now called the Community Platform Engineering (CPE) team.
For the team members, the day to day work did not change much.
The members working on Fedora are still fully dedicated to work on the
Fedora Project, and those working on CentOS are still fully dedicated to
CentOS. On both projects its members are involved in infrastructure, release
engineering, and design. However, it brought the two infrastructures and
teams closer to each other, allowing for more collaboration between them.
There are 20 people on this consolidated team. The breakdown looks like this:
* In Fedora:
** 3 dedicated system administrators
** 5 dedicated developers
** 1 doing both development and system administration
** 1 doing both release engineering and system administration
** 1 person dedicated to Fedora CoreOS
** 2 release engineers
** 1 person dedicated to documentation
** 1 designer
* In CentOS:
** 1 system administrator
** 2 doing both development and system administration
** 1 dedicated to the build systems
* There is also one additional person working on projects internal to Red Hat
As you can see this team is both quite diverse as well as limited. For
example, we do not have dedicated database adminstrators or network engineers
even though most of our application is one or the other when not both.
For these reasons we need to be careful on how our limited resources are
spent. The team has therefore defined a xref:mission_statement.adoc[Mission
Statement] for itself, allowing to focus its work in certain areas.
We have also written a document explaining xref:working_with_us.adoc[How to
work with us] for small and large tasks. Following this document makes it
easier for us to help you.

8
modules/cpe/pages/day_to_day_centos.adoc Normal file
View file

@ -0,0 +1,8 @@
= Working with Community Platform Engineering in CentOS
This document explains how to efficiently work with the CPE team.
Your close attention to this document will help both you and us
do the work you are asking us to do.
`This document is a work in progress and should be updated soon`.

110
modules/cpe/pages/day_to_day_fedora.adoc Normal file
View file

@ -0,0 +1,110 @@
= Working with Community Platform Engineering in Fedora
This document explains how to efficiently work with the CPE team.
Your close attention to this document will help both you and us
do the work you are asking us to do.
== Our Workflow
0. Is your issue/problem related to security of an application or service we run?
-> send email to infra-security@fedoraproject.org
1. Is your issue/problem urgent? (A important service is down, you need a change asap)
or IS your issue/problem such that you cannot file a ticket (authentication, no account, ticketing system down)
-> on irc /join #fedora-admin and say '.oncall' and explain the issue to the oncall person.
-> If no answer or the oncall person asks you to: go to next step.
2. File a ticket in https://pagure.io/fedora-infrastructure/issues/ with as much information
as you think will be needed to handle your issue. This initial ticket will be made in the
'Needs review' state. Please note if there is a deadline or this issue blocks you.
-> your ticket will be reviewed within 48 hours. (usually sooner)
-> There is no need to ping team members or notify us about the newly filed ticket.
3. your ticket will be triaged by a team member and moved to a new state:
-> If it's moved to 'Waiting on asignee' it's waiting for a team member to start work on it.
-> If it's moved to 'Waiting on reporter' it means that you need to answer questions posed in the ticket before it can be worked on.
-> If the ticket is closed with 'initiative', See xref:initiatives.adoc[New Initiative Workflow]
-> If the ticket is otherwise closed, it will be with a explaination from a team member.
4. If you have an update to your issue/task or want to know when it might be worked on:
-> comment in the ticket adding that information or asking for timeframe.
5. When someone is available, your ticket will be assigned to someone to work on.
-> Watch for progress reports/ticket being marked done.
6. If the work is not fully completed as required, please re-open the ticket and indicate this.
-> go back to step 5 for additional work.
== The oncall role in our team
Weekly a team member will be designated “oncall”. You can find this
person on IRC by using '.oncall' in any of our various IRC channels.
When available, this person:
1. Accepts urgent work items for the team, for example a important
or high SLE service is down or causing issues. A ticket should
be filed by the reporter or the oncall to track this work in any case.
2. Shields other team members from distracting pings and less
urgent tickets, deciding when an issue is important enough to
interrupt another team member.
3. Triages incoming tickets for urgent items that need work outside
of normal triage process.
== Initiatives
All tasks involving new applications, major deployments, major development work
or the like will be asked to follow the xref:initiatives.adoc[New Initiative
Workflow]. It will then be scoped and prioritized from there.
== General Ticket Considerations
Please provide as much information as you can in your ticket to avoid
back and forth for information. If you know your issue is going to
cause a lot of discussion, start a mailing list thread for that.
Make sure your ticket:
* explains the problem or issue you are having, with URLs where
possible to the services or applications involved.
* tells us how important or urgent this is to you.
* includes any error messages or output you see.
It's up to you as ticket filer to follow your ticket, provide information
that is asked for and keep us aware of any urgency you may have, do not
simply file and forget your ticket.
Your ticket may take a while to process, depending on the work
that the team has and how important we think it is. If your ticket
is blocking you, make sure you note that in the ticket but keep in
mind that we may be working on tickets that are blocking more people.
== IRC
IRC is a great way to communicate, but please do not ping team members directly.
Instead, update your ticket with any new information you have and when the
team member(s) working on that issue have time/availability they may contact
you on IRC for more interactive debugging/testing.
== Direct emails
Email is also a great communication method, but if you mail work items or
information to one person directly, they cannot easily hand off the issue,
you must wait for them to have time to address the issue (when others could
perhaps have already solved it, etc). So, please avoid direct emails and instead
update tickets with any information you want to add.
== RFC 1149
link:https://tools.ietf.org/html/rfc1149[Pidgeons are too slow] for most work
items, instead update tickets.
Thank you for reading this entire document!

74
modules/cpe/pages/index.adoc
View file

@ -1,43 +1,53 @@
= The Community Platform Engineering Team
In 2017 the Fedora Engineering team merged with the CentOS Engineering team
to form what is now called the Community Platform Engineering (CPE) team.
For the team members, the day to day work did not change much.
The Community Platform Engineering Team (CPE) is a Red Hat team dedicated
to the Fedora and CentOS projects where they contribute to the infrastructure
and release engineering.
The members working on Fedora are still fully dedicated to work on the
Fedora Project, and those working on CentOS are still fully dedicated to
CentOS. On both projects its members are involved in infrastructure, release
engineering, and design. However, it brought the two infrastructures and
teams closer to each other, allowing for more collaboration between them.
There are 20 people on this consolidated team. The breakdown looks like this:
== Our Mission
* In Fedora:
Our mission rests on three axes:
** 3 dedicated system administrators
** 5 dedicated developers
** 1 doing both development and system administration
** 1 doing both release engineering and system administration
** 1 person dedicated to Fedora CoreOS
** 2 release engineers
** 1 person dedicated to documentation
** 1 designer
.Our Mission
[#img-mission]
[caption="Figure 1: "]
image::our_missions.jpg[Our Mission,500,500]
* In CentOS:
** 1 system administrator
** 2 doing both development and system administration
** 1 dedicated to the build systems
* **Infrastructure**: To provide power and ping to the different systems that
are used by our community.
* **Application maintenance**: For the applications that we maintain, we work
on bug fixes, new features and community development.
* **Initiatives**: For a limited time and scope we dedicate some of our
engineers and adminstrators to initiatives deemed important and beneficial to
the communities we are involved in.
* There is also one additional person working on projects internal to Red Hat
One of the key challenges this team faces is balancing these three axes.
For this reason, we worked on the following Mission Statement.
As you can see this team is both quite diverse as well as limited. For
example, we do not have dedicated database adminstrators or network engineers
even though most of our application is one or the other when not both.
== Our responsabilities
For these reasons we need to be careful on how our limited resources are
spent. The team has therefore defined a xref:mission_statement.adoc[Mission
Statement] for itself, allowing to focus its work in certain areas.
Our responsabilities are scoped in our mission statement:
We have also written a document explaining xref:working_with_us.adoc[How to
work with us] for small and large tasks. Following this document makes it
easier for us to help you.
=== Our Mission Statement
__The Community Platform Engineering Team is responsible for the
infrastructure and services that support developing, building, and
releasing of Fedora and CentOS platform artifacts and deliverables,
including:__
* __Hosting, automating, monitoring and maintaining infrastructure components__
* __Service monitoring and lifecycle management of services hosted within our infrastructure__
* __Feature development for infrastructure-related initiatives__
* __Tooling to enable all of the above__
=== Why a Mission Statement?
A mission statement means the team shares a common vision about what our
work is or should be. It gives us clear boundaries for our work and it
helps us having a critical view on our work (current or in the future): “Does
this fit our mission statement?”. With this in mind we can focus on bringing
more value to the community. In other words, we want to maximize our value and
impact in a targeted mission rather than having limited impact on a broad
spectrum.

146
modules/cpe/pages/sle.adoc Normal file
View file

@ -0,0 +1,146 @@
= Service Level Expectations
The CPE team does not have any agreement, or contract about the availability
of its different services. However, we do try our best to keep the services
running and as a result you can have some expectations as to what we will
do to this extent.
== Primary Business Hours
Fedora Infrastructure's primary business hours are aligned with the work
schedule of Red Hat Inc. Normal hours should be seen as during Monday
through Friday 1400 UTC to 2300 UTC with US national holidays and a 2 week
end of year closure affecting staffing and response times.
Services outside of primary business hours are done on call and depend on
the availability of staff.
== Roles and Responsibilities
=== Fedora Infrastructure to Community
* To have staff present and available in appropriate IRC channels to answer
questions during primary hours.
* To have particular staff on-call during off hours so that community
members can contact for Critical and Important services.
* Interact with community members with respect and courtesy.
* Work with community members to get accurate and thorough documentation of
incidents, problems, or feature requests.
* When possible resolve the problem as soon as acknowledged.
* Communicate clearly estimated resolution times.
* Move items which can not be resolved within a reasonable time to future
feature requests or close out.
=== Community Member to Fedora Infrastructure
* Give full and detailed reports of the problem or requested service.
* Provide clear and complete contact information and times when available.
* Leave alternative contacts who can also be available in case of vacation
or other emergencies.
* When contacted by Fedora IT, respond back within 5 business days
=== Fedora Infrastructure to Fedora Infrastructure
* Have a clear schedule of reachable hours.
* Set and take regular vacation time to be rested.
* Rotate through days on-call in IRC and tickets.
* If adding a new service, be available outside of normal business hours to
help debug problems.
* Follow procedures and checklists when adding or updating services.
* Help with regular audits of the documentation
== Definition of Service Priorities
The general design of service priorities is that of concentric circles where
items rely on services in their own circle or a circle below them.
1. Critical services are ones which Fedora Infrastructure will work on 24x7
with 52 week coverage if an unplanned outage occurs. Services will be
configured to be highly available with an estimated planned/unplanned
uptime of 95%. Response time should be within 1 hour.
2. Important services are ones which Fedora Infrastructure will work to be
available 24x7 with 50 week coverage. Response times during primary hours
should be 1 hour and outside of it should be 4 hours.
3. Normal services are ones which Fedora Infrastructure will work to be
available during primary work hours. Problems outside of these hours will
be looked at as people are available. The services may be available
outside of these but are of a lower priority than important services.
4. Low priority services are ones which are not critical or important for
the primary function of Fedora Infrastructure. They will be worked on and
looked at during primary business hours. Response times should be within
1 business day.
5. Third Party services are ones which Fedora Infrastructure has outsourced
tools and services to. Uptimes, service hours, and coverage are dictated
by the third party. Depending on the type of problem, Fedora Infrastructure
will act as an intermediary or in the case of tools like retrace and COPR
direct the user to talk with the service owners.
6. Deprecated services are ones which Fedora Infrastructure are no longer
putting resources into. This may be because the project has completed its
mission, the upstream software is dead, or the original reasons for the
product are available. Problems with these services will be looked at
during primary business hours. Responses may be mostly "Will Not Fix".
== Limitations on Support
* Some services that are associated with Fedora are provided by third
parties. Changes and outages which affect them are outside the control
of Fedora Infrastructure.
* Fedora Infrastructure will prioritize issues and requests that affect
multiple people or teams over a smaller group or individual.
* Fedora Infrastructure has limited budget and hours. Requests and features
will be prioritized to fit within those.
* Fedora Infrastructure is bound by the laws and regulations of the United
States of America. This means that certain requests, changes and problems
are outside the ability of members to deal with.
== Glossary
* **Planned outage**: A planned outage is one that is announced with enough
time to give most user's the ability to plan around it.
* **Unplanned outage**: An outage that occurs suddenly without proper
allowance for users to plan around it.
* **Scheduled outage**: An outage which has been scheduled to occur but may
not have been announced with enough time for users to plan around it.
* **High Availability**: Systems are available during specified operating
hours with any unplanned outages 'masked' by other tools.
* **Continuous Operations**: Systems are available 24 hours a day, 7 days a
week with no scheduled outages. Unplanned outages are possible during this time.
* **Continuous Availability**: Systems or applications are available 24x7
with no planned or unplanned outages. This is a combination of high
availability and continuous operations.
* **Level of availability**: | Percentage | Max outage time per day | | 90% | 144.0 minutes | | 95% | 72.0 minutes | | 99% | 14.4 minutes | | 99.9% | 1.4 minutes |
* **Commited Hours of Availability**: Hours that an organization will have
staff available to help deal with issues with systems, services, and
applications. Also known as "Regular Business Hours"
* **Outage Hours**: Total number of hours of outage considered normal for
calculating achieved availability.
* **Response Time**: The time between the users notification of the problem
and when the help desk will begin to work on that problem.
* **Resolution Update**: The frequency of updates to tickets
== Estimated Time of Resolution:
By priority Levels:
* **Emergency**: Problems which are site wide, and affect the core functions
of the project.
* **Urgent**: Problems which affect multiple functions and groups in the
project.
* **Normal**: Problems which affect a single user from performing needed
duties.
* **Low**: A request for service, instruction, information that has no
immediate impact on services.

142
modules/cpe/pages/sle_services.adoc Normal file
View file

@ -0,0 +1,142 @@
= Services SLE
Here is the list of our services per SLE level. For memory these levels are
presented in our xref:sle.adoc[SLE Documentation].
== Critical
* DNS
* Bastion ssh
* Authentication/authorization (FAS) https://admin.fedoraproject.org/accounts/
* Authentication/authorization (Ipsilon) https://id.fedoraproject.org
* Configuration Management (ansible)
* Source control (git)
* Backups
* DHCP/PXE
* IPA
== Important
* Koji https://koji.fedoraproject.org
* Bodhi https://bodhi.fedoraproject.org
* Downloads https://dl.fedoraproject.org/
* Fedora Souce code https://src.fedoraproject.org/
* MirrorManager https://admin.fedoraproject.org/mirrormanager
* Nagios https://nagios.fedoraproject.org
* HAProxy https://admin.fedoraproject.org/haproxy/proxy01
* Postgres databases
* Mysql databases
* Product Definition Center https://pdc.fedoraproject.org/
* Greenwave
* Fedmsg
* DataGrepper https://apps.fedoraproject.org/datagrepper/
* Email gateway bastion.fedoraproject.org
* Autosign
* Composer
* Buildhosts
* Docker registry
* Memcached
* Logging
* Basset
* PDC
* resultsdb
* certgetter
* mbs
* odcs
== Normal
* Pagure https://pagure.io
* CI https://ci.centos.org
* The Mailing Lists https://lists.fedoraproject.org
* Wiki https://fedoraproject.org/wiki/Fedora_Project_Wiki
* Documentation
* Notifications https://apps.fedoraproject.org/notifications
* Kerneltest https://apps.fedoraproject.org/kerneltest
* Fedora InfraCloud https://fedorainfracloud.org
* FMN
* FAF
* Beaker
* Freshmaker
* Data Analysis
== Low
* Ambassadors https://fedoraproject.org/membership-map/ambassadors.html
* Asknot http://whatcanidoforfedora.org/
* Badges https://badges.fedoraproject.org/
* Blocker Bugs https://qa.fedoraproject.org/blockerbugs/
* Easyfix https://fedoraproject.org/easyfix/
* Elections https://apps.fedoraproject.org/#Elections
* FedoCal https://apps.fedoraproject.org/#FedoCal
* Fedora Magazine https://fedoramagazine.org/
* Fedora People https://fedorapeople.org/
* GeoIP https://geoip.fedoraproject.org/
* github2fedmsg https://apps.fedoraproject.org/github2fedmsg
* Mdapi https://apps.fedoraproject.org/mdapi/
* Meetbot https://apps.fedoraproject.org/#Meetbot
* Nuancier https://apps.fedoraproject.org/#Nuancier
* Paste https://paste.fedoraproject.org
* Release Monitoring (anitya) https://release-monitoring.org/
* Review Status https://fedoraproject.org/PackageReviewStatus/
* The Planet https://fedoraplanet.org/
* Unbound
* Autocloud
* Bugyou
* Gobby
* Keys
* Koschei
* Loopabull
* Hotness
* OpenShift
* statscache
* Packages https://admin.fedoraproject.org/pkgdb/packages/
* bugzilla2fedmsg
* fed-image
* zanata2fedmsg
* secondary
* freemedia
* pager_server
* bz_review
* websites
== Websites
* fedora-web
* fedora-budget
* fedora-docs
* developer
* whatcanidoforfedora
* membership-map
* zanata
* review-stats
* fedora_owner_change
== Third Party
Outside of Fedora Infrastructure to fix.
* Network connectivity to PHX2/RDU2
* FreeNode IRC https://freenode.net
* Ask Fedora https://ask.fedoraproject.org/
* COPR https://copr.fedorainfracloud.org/
* Retrace https://retrace.fedoraproject.org
* Bugzilla https://bugzilla.redhat.com/
* Status https://status.fedoraproject.org
* Taskotron https://taskotron.fedoraproject.org/
* Openqa
* Taiga https://teams.fedoraproject.org/
== Deprecated
* Torrents https://torrent.fedoraproject.org
* Darkserver https://darkserver.fedoraproject.org/
* PkgDB https://admin.fedoraproject.org/pkgdb/
* Jenkins https://jenkins.fedorainfracloud.org/
* summershum
* Tagger https://apps.fedoraproject.org/tagger/
* Taiga https://taiga.fedorainfracloud.org/

139
modules/cpe/pages/working_with_us.adoc
View file

@ -1,110 +1,41 @@
= Working with Community Platform Engineering
This document explains the basic principles of working with the CPE team.
This document explains how to efficiently work with the CPE team.
Your close attention to this document will help both you and us
do the work you are asking us to do.
The general rules are:
== Our Workflow
* Do not ping a single person
By doing this you reduce the number of people having access to your request
drastically, meaning, if that person is busy with something else, your
request may take much longer to be processed.
* If it is not in a ticket, it did not happen
We need to have everything in tickets. This helps sharing knowledge within
the team but also tracking issues and potentially find the ones that are
recurring frequently and may thus be an indicator of something we need to
fix.
Even if you have discussed something on IRC or in a corridor with someone
in the team, you want to log the discussion and its outcome in a ticket so
we can get back to it in 2 months or 3 when we need to investigate why this
change was made.
* When you cannot open a ticket
There are a few cases where it is acceptable to send an email or to reach
directly to people, for example if you cannot access or authenticate to the
ticket tracking system :)
In this case, you do not want to send your email or ping a single person
(cf the first point above).
- In Fedora, you can use the email: `contact` @ fedoraproject.org
- In CentOS, you can contact the sysadmins on the `#centos-devel` channel on irc.freenode.net
0. Is your issue/problem related to security of an application or service we run?
-> send email to infra-security@fedoraproject.org
1. Is your issue/problem urgent? (A important service is down, you need a change asap)
or IS your issue/problem such that you cannot file a ticket (authentication, no account, ticketing system down)
-> on irc /join #fedora-admin and say '.oncall' and explain the issue to the oncall person.
-> If no answer or the oncall person asks you to: go to next step.
2. File a ticket in https://pagure.io/fedora-infrastructure/issues/ with as much information
as you think will be needed to handle your issue. This initial ticket will be made in the
'Needs review' state. Please note if there is a deadline or this issue blocks you.
-> your ticket will be reviewed within 48 hours. (usually sooner)
-> There is no need to ping team members or notify us about the newly filed ticket.
3. your ticket will be triaged by a team member and moved to a new state:
-> If it's moved to 'Waiting on asignee' it's waiting for a team member to start work on it.
-> If it's moved to 'Waiting on reporter' it means that you need to answer questions posed in the ticket before it can be worked on.
-> If the ticket is closed with 'initiative', See xref:initiatives.adoc[New Initiative Workflow]
-> If the ticket is otherwise closed, it will be with a explaination from a team member.
4. If you have an update to your issue/task or want to know when it might be worked on:
-> comment in the ticket adding that information or asking for timeframe.
5. When someone is available, your ticket will be assigned to someone to work on.
-> Watch for progress reports/ticket being marked done.
6. If the work is not fully completed as required, please re-open the ticket and indicate this.
-> go back to step 5 for additional work.
== The oncall role in our team
Weekly a team member will be designated “oncall”. You can find this
person on IRC by using '.oncall' in any of our various IRC channels.
When available, this person:
1. Accepts urgent work items for the team, for example a important
or high SLE service is down or causing issues. A ticket should
be filed by the reporter or the oncall to track this work in any case.
2. Shields other team members from distracting pings and less
urgent tickets, deciding when an issue is important enough to
interrupt another team member.
3. Triages incoming tickets for urgent items that need work outside
of normal triage process.
== Initiatives
All tasks involving new applications, major deployments, major development work
or the like will be asked to follow the xref:initiatives.adoc[New Initiative
Workflow]. It will then be scoped and prioritized from there.
== General Ticket Considerations
Please provide as much information as you can in your ticket to avoid
back and forth for information. If you know your issue is going to
cause a lot of discussion, start a mailing list thread for that.
Make sure your ticket:
* explains the problem or issue you are having, with URLs where
possible to the services or applications involved.
* tells us how important or urgent this is to you.
* includes any error messages or output you see.
It's up to you as ticket filer to follow your ticket, provide information
that is asked for and keep us aware of any urgency you may have, do not
simply file and forget your ticket.
Your ticket may take a while to process, depending on the work
that the team has and how important we think it is. If your ticket
is blocking you, make sure you note that in the ticket but keep in
mind that we may be working on tickets that are blocking more people.
== IRC
IRC is a great way to communicate, but please do not ping team members directly.
Instead, update your ticket with any new information you have and when the
team member(s) working on that issue have time/availability they may contact
you on IRC for more interactive debugging/testing.
== Direct emails
Email is also a great communication method, but if you mail work items or
information to one person directly, they cannot easily hand off the issue,
you must wait for them to have time to address the issue (when others could
perhaps have already solved it, etc). So, please avoid direct emails and instead
update tickets with any information you want to add.
== RFC 1149
link:https://tools.ietf.org/html/rfc1149[Pidgeons are too slow] for most work
items, instead update tickets.
Thank you for reading this entire document!
Both projects have their own, additional, processes for their day to day work,
you can find them there:
- xref:day_to_day_fedora.adoc[for Fedora]
- xref:day_to_day_centos.adoc[for CentOS]