Restructure the repo for consistency

Signed-off-by: Michael Winters <fedora@mwinters.net>

modules/sysadmin_sops/pages/pagure.adoc

@@ -0,0 +1,166 @@
+= Pagure Infrastructure SOP
+
+Pagure is a code hosting and management site.
+
+== Contents
+
+* <<_contact_information>>
+* <<_description>>
+* <<_when_unresponsive>>
+* <<_git_repo_locations>>
+* <<_services_and_what_they_do>>
+
+== Contact Information
+
+Owner::
+  Fedora Infrastructure Team
+Contact::
+  #fedora-apps
+Location::
+  OSUOSL
+Servers::
+  pagure01, pagure-stg01
+Purpose::
+  Source code and issue tracking
+
+== Description
+
+Pagure ( https://pagure.io/pagure ) is a source code management and
+issue tracking application. Its written in flask. It uses: celery,
+redis, postgresql, and pygit2.
+
+== When unresponsive
+
+Sometimes pagure will stop responding, even though it's still running.
+You can issue a 'systemctl reload httpd' and that will usually get it
+running again.
+
+== Git repo locations
+
+* Main repos are in `/srv/git/repositories/<projectname>`
+* issue/ticket repos are under
+`/srv/git/repositories/tickets/<projectname>`
+* Docs are under `/srv/git/repositories/docs/<projectname>`
+* Releases (not a git repo) are under `/var/www/releases/`
+
+== Services and what they do
+
+* `pagure` service is the main flask application, it runs from httpd wsgi.
+* `pagure_ci` service talks to jenkins or other CI for testing PR's
+* `pagure_ev` service talks to websockets and updates issues and comments
+live for users.
+* `pagure_loadjson` service takes issues loads from pagure-importer and
+processes them.
+* `pagure_logcom` service handles logging.
+* `pagure_milter` processes email actions.
+* `pagure_webhook` service processes webhooks to notify about changes.
+* `pagure worker` service updates git repos with changes.
+
+== Useful commands
+
+This section lists commands that can be useful to fix issues encountered
+every once in a while.
+
+* Recompile the gitolite configuration file
++
+....
+# sudo -u git HOME=/srv/git/ gitolite compile && sudo -u git HOME=/srv/git/ gitolite trigger POST_COMPILE
+....
+* Duplicated projects
+
+We have observed that every so often two different workers create a
+project in the database. This leads to pagure failing to give access to
+the project as it finds multiple projects with the same namespace/name
+where it expects these to be unique.
+
+The following two SQL commands allows finding out which projects are in
+this situation:
+
+....
+select user_id, name, namespace, is_fork from projects where is_fork =
+FALSE group by namespace, name, is_fork, user_id having count(user_id) >
+1;
+
+select user_id, name, namespace, is_fork from projects where is_fork =
+TRUE group by namespace, name, is_fork, user_id having count(user_id) >
+1;
+....
+
+This will return you the _namespace_/_name_ as well as the _user_id_ of the
+user who duplicated the projects in the database.
+
+You can then do:
+
+....
+select id, user_id, name, namespace, is_fork from projects where name =
+'<the name of the project>' order by user_id;
+....
+
+In that query you will see the project _id_, _user_id_, _name_ and _namespace_
+of the project. You will see in this one of the projects is listed twice
+with the same _user_id_ (the one returned in the previous query).
+
+From there, you will have to delete the duplicates (potentially the one
+with the highest project id).
+
+If the project remains un-accessible, check the apache logs, it could be
+that the git repositories have not been created. In that case, the
+simplest course of action is to delete all the duplicates and let the
+users re-create the projects as they wish.
+
+== Troubleshooting
+
+=== Pull request throws 500 when opened
+
+. Check the permissions on `pagure02`
++
+The repositories could be found on `/srv/git/repositories/` and inside each
+you can check `refs/pull/<PR_number>` directory.
+For checking the permissions you don't need any special permissions, but
+if you need to adjust anything you need to root permissions.
+
+. Close the PR using `requests` repository
++
+If there isn't anything wrong with permissions or the directory is empty,
+try to close the PR using `requests` repository.
+You need to have commit/admin permissions on the repository.
++
+.. Clone the `requests` repository
++
+You can find the clone link in `Other Git URLs` in `Clone` button
++
+.. Find the correct pull requests by hash
++
+Use `git log` for that
++
+.. Update the status of the pull request to `Closed`
+.. Push the change
+
+. Removing the PR from project
++
+This should be last resort when everything others fails.
+You need to have root access to `pagure02`.
++
+.. Log in to PostgreSQL db on `pagure02`
++
+....
+`sudo -u postgres psql`
+....
++
+.. Connect to pagure database
++
+....
+`\connect pagure`
+....
++
+.. Find the project_id by name
++
+....
+select id from projects where name='<name_of_project>' and is_fork=false;
+....
++
+.. Delete the desired pull request
++
+....
+delete from pull_requests where id=<PR_number_in_pagure> and project_id=536
+....