diff --git a/.ai_review/project.md b/.ai_review/project.md new file mode 100644 index 0000000000..c18c3f4e5c --- /dev/null +++ b/.ai_review/project.md @@ -0,0 +1,159 @@ +## Project Overview + +**Purpose:** Ansible automation for the entire Fedora Project infrastructure — managing hundreds of bare-metal hosts, VMs, and OpenShift-deployed applications across production and staging environments. +**Type:** Infrastructure as Code (Ansible) +**Domain:** Linux distribution infrastructure (build systems, package repositories, CI/CD, web services, identity management) +**Key Components:** `roles/` (134 reusable roles), `playbooks/` (groups, hosts, openshift-apps, manual), `inventory/` (host/group vars with constructed plugin) + +## Technology Stack + +### Versions (current as of 2026-03-05) +- **Ansible** — core automation engine; playbooks target Fedora and RHEL/CentOS hosts +- **yamllint** v1.35.1 — pre-commit hook for YAML validation +- **ansible-lint** — CI linting (skips: `yaml`, `role-name[path]`, `var-naming[no-role-prefix]`, `no-changed-when`, `ignore-errors`) +- **OpenShift 4** (OCP4) — hosts ~60 containerized applications via `openshift-apps/` playbooks + +### Target Platforms +- **Fedora** (current cycle: F43 stable, F44 branched, F45 rawhide) +- **RHEL/CentOS** 8+ (EPEL 10, current minor: 10.3) +- **OpenShift 4** cluster (rdu3 datacenter) + +### Dev Tools +- **Linting:** yamllint (pre-commit) + ansible-lint (CI) +- **CI:** Forgejo Actions on `quay.io/fedora/fedora:latest` — runs yamllint and ansible-lint on changed files only +- **Control host:** batcave01 (`/srv/web/infra/ansible` public, `/srv/private/ansible` private) + +## Resource Organization + +### Structure +``` +├── main.yml # Master playbook — imports all group/host playbooks +├── playbooks/ +│ ├── groups/ # 59 group playbooks (one per service group) +│ ├── hosts/ # 2 host-specific playbooks (FQDN.yml) +│ ├── openshift-apps/ # 60 OCP4 application deployments +│ ├── manual/ # 39 admin-run-only playbooks +│ └── include/ # Shared proxy/virt/cert playbook fragments +├── roles/ # 134 roles +│ ├── base/ # Applied to ALL hosts (packages, SSH, SELinux, nftables) +│ ├── openshift/ # OCP4 primitives (project, object, keytab, route, rollout) +│ └── openshift-apps/ # Per-app roles (templates, files for OCP4 deployments) +├── inventory/ +│ ├── inventory/ # Host definitions +│ ├── group_vars/ # 150 group variable files (incl. _stg variants) +│ ├── host_vars/ # Per-host overrides +│ └── zzz-inventory.config # Constructed inventory plugin (dynamic groups by datacenter, distro, vmhost) +├── vars/ +│ ├── global.yml # Global vars (paths, SSL config, base packages) +│ ├── all/ # Release cycle vars (Fedora versions, freeze states, EPEL) +│ ├── apps/ # Per-application vars (bodhi, mirrormanager, etc.) +│ ├── Fedora.yml # Distro-specific packages/services +│ └── RedHat.yml / CentOS.yml +├── tasks/ # Reusable task snippets (cloud, postfix, yumrepos, etc.) +├── handlers/ # restart_services.yml +├── library/ # Custom modules (delete_old_oci_images.py, virt_boot, etc.) +├── callback_plugins/ # fedora_messaging_callback.py, logdetail.py +├── files/ # Static files/templates organized by service +└── scripts/ # Admin utility scripts (auth-keys-from-fas, freezelist, etc.) +``` + +### Module/Role Structure + +**Standard role layout:** `tasks/main.yml`, `templates/`, `files/`, `handlers/main.yml`, `meta/main.yml` + +**OpenShift app pattern** — the dominant pattern for new services: +1. Playbook in `playbooks/openshift-apps/.yml` targets `os_control[0]:os_control_stg[0]` +2. Uses composable `openshift/*` roles (`project`, `object`, `keytab`, `secret-file`, `imagestream`, `route`, `rollout`) +3. App-specific templates in `roles/openshift-apps//templates/` +4. Staging vs production controlled by `env` variable and `env_suffix` + +**Group playbook pattern** — for traditional VM-based services: +1. Playbook in `playbooks/groups/.yml` with `hosts:` matching inventory group +2. Must include standard `vars_files` triplet (see Review Guidance) + +### Critical Resources + +- **`vars/all/`** — Fedora release cycle variables. Changed every ~6 months during branching/release. Incorrect values break builds, composes, and Bodhi across the entire infrastructure. +- **`roles/base/tasks/main.yml`** (700+ lines) — Applied to every managed host. Changes here have maximum blast radius. +- **`inventory/group_vars/`** — 150 files controlling per-group behavior. Many have `_stg` counterparts for staging. +- **`main.yml`** — Master playbook importing all groups. Nightly `--check --diff` cron runs all playbooks here. + +## Review Guidance + +### What Reviewers Must Know +- **All playbooks must be idempotent.** They can be run at any time by the nightly cron. The checked-in state must always be the desired state. +- **Standard vars_files triplet is required** in all group/host playbooks: + ```yaml + vars_files: + - /srv/web/infra/ansible/vars/global.yml + - "{{ private }}/vars.yml" + - /srv/web/infra/ansible/vars/{{ ansible_distribution }}.yml + ``` + Plus `include_vars` for `vars/all/` when release cycle vars are needed. +- **Hardcoded paths are standard** — paths like `/srv/web/infra/ansible/` and `/srv/private/ansible/` are the production layout on batcave01. Don't suggest making them relative or configurable. +- **Use `yml` not `yaml`** for Ansible files (per STYLEGUIDE). The exception is `vars/all/*.yaml` which uses `.yaml` historically. +- **Add `.j2` extension** to all Jinja2 templates. +- **YAML indentation is 2 spaces.** Line length is not enforced. +- **Prefer readable multi-line module args** over single-line `module: name=x arg=y` format. +- **Staging uses `_stg` suffixed group_vars** and `env_suffix` variable (empty string for prod, `.stg` for staging). +- **Tags `packages` and `config`** should be applied to relevant tasks. `build` and `rollout` tags use `never` to prevent accidental execution. +- **OpenShift apps target `os_control[0]:os_control_stg[0]`** — always the first control node. Don't suggest targeting all control nodes. + +### Do NOT Flag (Known False Positives) +- `ansible-lint` skip list includes `no-changed-when` and `ignore-errors` — these are intentionally suppressed project-wide +- `yaml` rule category is skipped in ansible-lint — yamllint handles YAML validation separately +- Hardcoded absolute paths in playbooks (e.g., `/srv/web/infra/ansible/...`) — this is the expected deployment layout +- `mock_modules` and `mock_roles` in `.ansible-lint` — used to pass syntax checks without all dependencies +- Octal values forbidden in yamllint — intentional policy to avoid ambiguous YAML parsing +- `when: env == "production"` / `when: env == "staging"` conditional duplication in openshift-apps — standard pattern for different scaling/config per environment + +### Common Pitfalls +- **Forgetting to update `vars/all/` during release transitions** — these variables control Fedora version numbers, freeze states, and EPEL branches. Multiple files must be updated together (e.g., branching requires changes to `FedoraBranched.yaml`, `00-FedoraCycleNumber.yaml`, `FedoraBranchedBodhi.yaml`, and `Frozen.yaml`). +- **Not testing with staging first** — staging group_vars (`*_stg`) should be updated before production. Changes that work in staging may still break production due to different scaling or secrets. +- **Breaking idempotency** — a task that makes changes on every run will generate noise in the nightly `--check --diff` report and mask real drift. +- **Wrong file extension for templates** — placing a `.yml` file in `templates/` instead of `.yml.j2` means Jinja2 expressions won't be rendered. +- **ansible-lint file/role misclassification** — the `.ansible-lint` `kinds` section maps `tasks/*.yml` and `vars/*.yml` explicitly. New directories with tasks may need similar mappings. + +## Internal & Proprietary + +- **`/srv/private/ansible/`** — Private vars (secrets, credentials) stored on batcave01. Referenced as `{{ private }}/vars.yml`. Never committed to this repo. +- **`callback_plugins/fedora_messaging_callback.py`** — Custom Ansible callback that publishes play results to Fedora's AMQP message bus. Don't suggest replacing with standard callback plugins. +- **`callback_plugins/logdetail.py`** — Custom detailed logging callback for the nightly check-diff runs. +- **`library/virt_boot`** / **`library/delete_old_oci_images.py`** — Custom Ansible modules for VM management and OCI image cleanup. Not upstream modules. +- **`scripts/auth-keys-from-fas`** — Fetches SSH authorized keys from Fedora Account System (FAS). Referenced by `auth_keys_from_fas` global variable. +- **Constructed inventory plugin** (`zzz-inventory.config`) — Dynamically creates groups by datacenter (`rdu3`), distro, vmhost, and virtualization role. The `zzz-` prefix ensures it loads last. + +--- + + +## Architecture & Design Decisions + +- **Mostly flat role directory with some nesting**: Most roles live directly under `roles/`, but several use subdirectory namespacing — `openshift/` (OCP4 primitives), `openshift-apps/` (per-app deployments), `awx/`, `openqa/`, `rabbit/`, among others. +- **OpenShift apps via Ansible, not Helm/Kustomize**: OCP4 applications are deployed through Ansible roles that template and apply OpenShift objects. This keeps all infrastructure in one tool and one repo. +- **Staging/production parity through group_vars**: Rather than separate inventories, staging hosts are in the same inventory with `_stg` group_vars files providing overrides. The `env` and `env_suffix` variables control behavior. +- **Release cycle managed through simple YAML vars**: Fedora's complex release lifecycle (rawhide, branched, stable, EOL) is encoded in `vars/all/` as a set of interdependent variables rather than a database or API. This is intentional — the variables are updated manually during each release milestone by the release engineering team. + +## Business Logic + +- **Fedora release cycle states**: Three main states — unbranched (rawhide only), branched (rawhide + pre-release), and post-release. Controlled by `FedoraBranched`, `FedoraCycleNumber`, `FedoraBranchedBodhi` (preenable/prebeta/postbeta), and freeze flags. These cascade through templates across the entire infrastructure. +- **EPEL minor version lifecycle**: EPEL 10+ has minor versions that move through states: `epel_minor` (built against CentOS), `epel_branched_minor` (branched, built against CentOS snapshot), `epel_z_minor` (built against RHEL). Up to three active minor versions at once. +- **Critical path applications**: forge, pagure, mirrormanager, bodhi, koji, dist-git, and ~20 others require two-reviewer PRs and coordinated downtime scheduling for risky changes. +- **Nightly check-diff**: All playbooks under `playbooks/{groups,hosts}` are run nightly with `--check --diff`. The ideal state is zero changes reported. + +## Domain-Specific Context + +- **batcave01** — The Ansible control host. All playbooks are run from here via `sudo -i ansible-playbook`. +- **env / env_suffix** — `env` is `"production"` or `"staging"`. `env_suffix` is `""` for prod, `".stg"` for staging. Used throughout to construct hostnames, queue names, and paths. +- **FAS (Fedora Account System)** — Identity provider for the Fedora community. SSH keys, group memberships, and permissions come from FAS/IPA. +- **Koji** — Fedora's build system. Build hosts (buildvm, buildhw) are managed here. Koji hub is VM-based (`playbooks/groups/koji-hub.yml` + `roles/koji_hub`), not on OpenShift. +- **Bodhi** — Fedora's update management system. Runs on OpenShift with complex RabbitMQ messaging integration. +- **dist-git / Pagure** — Package source repositories. The lookaside cache and git hosting are managed by separate roles. +- **RabbitMQ / fedora-messaging** — AMQP message bus connecting Fedora services. Certificates managed per-service via `openshift/secret-file` role. + +## Special Cases + +- **`playbooks/openshift-apps/` uses `gather_facts: false`** — OCP4 playbooks target the control node to run `oc` commands, not the apps themselves. Facts aren't needed and would slow execution. +- **`vars/all/*.yaml` uses `.yaml` extension** despite STYLEGUIDE mandating `.yml` — historical exception, don't "fix" this. +- **`ansible-lint` runs in offline mode** (`offline: true`) — dependencies aren't installed during linting. `mock_modules` and `mock_roles` paper over missing dependencies. +- **Some playbooks are excluded from ansible-lint** (e.g., `copr-db.yml`, `list-vms-per-host.yml`) due to known issues with hardcoded paths or unicode errors. +- **`linux-system-roles.network`** is mocked in ansible-lint — it's an external role not available during CI. diff --git a/.ansible-lint b/.ansible-lint index d50fc9a060..cc05ba098d 100644 --- a/.ansible-lint +++ b/.ansible-lint @@ -67,3 +67,4 @@ skip_list: - role-name[path] - var-naming[no-role-prefix] - no-changed-when + - ignore-errors diff --git a/.forgejo/workflows/ai-review.yaml b/.forgejo/workflows/ai-review.yaml new file mode 100644 index 0000000000..0b4ec71952 --- /dev/null +++ b/.forgejo/workflows/ai-review.yaml @@ -0,0 +1,15 @@ +--- +name: AI Code Review +on: + pull_request_target: + types: [labeled] + +jobs: + ai-review: + runs-on: infra-1 + if: forgejo.event.label.name == 'ai-review-please' + uses: quality/workflows/.forgejo/workflows/ai-review.yml@main + with: + pr: ${{ forgejo.event.pull_request.number }} + secrets: + GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }} diff --git a/.forgejo/workflows/ci.yaml b/.forgejo/workflows/ci.yaml index 7eec83112f..63673333da 100644 --- a/.forgejo/workflows/ci.yaml +++ b/.forgejo/workflows/ci.yaml @@ -39,7 +39,11 @@ jobs: steps: - name: Install testing tools run: | - dnf install -y ansible-lint nodejs git + dnf install -y ansible-lint nodejs git ansible + + - name: Install ansible collections + run: | + ansible-galaxy collection install community.zabbix - name: Checkout code uses: https://code.forgejo.org/actions/checkout@v6 diff --git a/.yamllint.yaml b/.yamllint.yaml index 45753282c7..04f74930a2 100644 --- a/.yamllint.yaml +++ b/.yamllint.yaml @@ -26,4 +26,5 @@ rules: ignore: - '*/templates/*' + - '*/openshift-apps/*/templates/*' ... diff --git a/files/httpd/fedorahosted-redirects.conf b/files/httpd/fedorahosted-redirects.conf index bc1eb6c299..bbdb3bed6a 100644 --- a/files/httpd/fedorahosted-redirects.conf +++ b/files/httpd/fedorahosted-redirects.conf @@ -39,7 +39,7 @@ RewriteRule ^/released/i/m/imapsync https://pagure.io/releases/imapsync [R=301] RewriteRule ^/released/imapsync https://pagure.io/releases/imapsync [R=301] RewriteRule ^/fedora-infrastructure/report https://forge.fedoraproject.org/infra/tickets [R=301] -RewriteRule ^/fedora-infrastructure/ticket/(.*) https://forge.fedoraproject.org/infra/tickets/$1 [R=301] +RewriteRule ^/fedora-infrastructure/ticket/(.*) https://forge.fedoraproject.org/infra/tickets/issues/$1 [R=301] RewriteRule ^/fedora-infrastructure https://forge.fedoraproject.org/infra/tickets [R=301] RewriteRule ^/fesco/report https://pagure.io/fesco/issues [R=301] diff --git a/files/scripts/ansilog-playbook.py b/files/scripts/ansilog-playbook.py new file mode 100755 index 0000000000..2a80f2776c --- /dev/null +++ b/files/scripts/ansilog-playbook.py @@ -0,0 +1,1483 @@ +#! /usr/bin/python3 + +# Create/view log files produced by ansible-plybook. + +# For examples, see the help command. + +import os +import sys + +import argparse +import gzip +import fnmatch +import json +import shutil +import subprocess +import time + +import glob + +# Use utf8 prefixes in diff, these need to be a "normal" width 1 character +conf_utf8 = True +_conf_utf8_warn = '⚠' # Rebooted +_conf_utf8_info = '⚐' # Rebooted and updated +_conf_utf8_okay = '➚' + +# Arrow seperator. Doesn't need to be a single character +conf_host_arrow_asci = '->' +conf_host_arrow_utf8 = '→' + +# Use ansi codes. None means auto, aka. look for a tty on stdout. +conf_ansi_terminal = None +conf_term_cmd = 'dim' +conf_term_user = '' +conf_term_warn = '' +conf_term_info = '' +conf_term_okay = '' +conf_term_highlight = 'bold,underline' +conf_term_keyword = 'underline' +conf_term_time = 'underline' +conf_term_title = 'italic' + +# Use _ instead of , for number seperator. +conf_num_sep_ = False + +# Make it easier to see different date's +conf_ui_date = True + +# Do we use a shorter duration by default (drop minutes/seconds) +conf_short_duration = True + +# Do we want a small osinfo in diff/list/etc. +conf_small_osinfo = True + +# Try to print OS/ver even nicer (when small) ... but includes spaces. +conf_align_osinfo_small = True + +# How many playbook runs to look at by default, _per_ playbook... +conf_play_max = 16 + +# How many hosts to show in list/etc. +conf_show_hosts_max = 4 + +# How old playbook runs to look at by default... +conf_play_duration = "2w" + +# Hosts that we'll show info. for, by default. info/host cmds. +conf_important_hosts = ["batcave*", "bastion01*", "noc*"] + +# Remove suffix noise in names. +conf_suffix_dns_replace = { + '.fedoraproject.org' : '..org', + '.fedorainfracloud.org' : '..org', +} +_suffix_dns_replace = {} + +# Some of our playbook names are big, so make them nicer. +conf_play_remap = { + 'generate-updates-uptimes-per-host-file' : 'Updates+uptimes', + 'communishift_send_email_notifications' : 'CS_emails', +} + +# Dir. where we put, and look for, the files... +conf_path = "/var/log/ansible/" + +conf_user_conf_path = "~/.config/ansilog-playbook/config.conf" + +# Now we can change the above conf_ variables, via. a conf file. +def _user_conf(): + ucp = os.path.expanduser(conf_user_conf_path) + if not os.path.exists(ucp): + return + + for line in open(ucp): + _user_conf_line(line) + +def _user_conf_line(line): + line = line.lstrip() + if not line: return + if line[0] == '#': return + + op = "+=" + x = line.split(op, 2) + if len(x) != 2: + op = ":=" + x = line.split(op, 2) + if len(x) != 2: + op = "=" + x = line.split(op, 2) + if len(x) != 2: + print(" Error: Configuration: ", line, file=sys.stderr) + return + key,val = x + + key = 'conf_' + key.strip().lower() + if key not in globals(): + print(" Warn: Configuration not found: ", key, file=sys.stderr) + return + + if False: pass + elif op == '=': + val = val.strip() + if False: pass + elif val.lower() in ("false", "no"): val = False + elif val.lower() in ("true", "yes"): val = True + elif val == '[]': val = [] + elif val == '{}': val = {} + elif val.isdigit(): val = int(val) + + if type(globals()[key]) != type(val): + print(" Error: Configuration ", key,'bad:',val, file=sys.stderr) + return + globals()[key] = val + elif op == '+=': + val = val.strip() + if type(globals()[key]) != type([]): + print(" Error: Configuration ", key, 'not []', file=sys.stderr) + return + globals()[key].append(val) + elif op == ':=': + if type(globals()[key]) != type({}): + print(" Error: Configuration ", key, 'not {}', file=sys.stderr) + return + + if '=' not in val: + print(" Error: Configuration bad :=", file=sys.stderr) + return + dkey, dval = val.split('=', 1) + globals()[key][dkey.strip()] = dval.strip() + else: + print(" Error: Configuration ", key,'bad op', file=sys.stderr) + return + + +# This is kind of fast and kind of small. No re, and no allocation. +# Sort as: 0, 00, 000, 01, 011, 1, 11, a01, a1, z01, z1, etc. +def natcmp(x, y): + """ Natural sort string comparison. + https://en.wikipedia.org/wiki/Natural_sort_order + Aka. vercmp() """ + + def _cmp_xy_mix(): # One is a digit, the other isn't. + if inum is not None: # 0/1 vs. x/. + return 1 + if x[i] > y[i]: + return 1 + else: + return -1 + + inum = None + check_zeros = False + for i in range(min(len(x), len(y))): + if x[i] in "0123456789" and y[i] not in "0123456789": + return _cmp_xy_mix() + if x[i] not in "0123456789" and y[i] in "0123456789": + return _cmp_xy_mix() + + if x[i] in "0123456789": # Both are digits... + if inum is None: + check_zeros = True + inum = 0 + + if check_zeros: # Leading zeros... (0 < 00 < 01 < 011 < 1 < 11) + if x[i] == '0' and y[i] == '0': + continue + elif x[i] == '0': + return -1 + elif y[i] == '0': + return 1 + else: + check_zeros = False + + # If we are already in a number, we only care about the length or + # the first digit that is different. + if inum != 0: + continue + + if x[i] == y[i]: + continue + + # Non-zero first digit, Eg. 7 < 9 + inum = int(x[i]) - int(y[i]) + continue + + # Both are not digits... + if inum is not None and inum != 0: + return inum + inum = None + + # Can be equal + if x[i] > y[i]: + return 1 + if x[i] < y[i]: + return -1 + + if len(x) > len(y): + if inum is not None and inum != 0 and x[i+1] not in "0123456789": + return inum + return 1 + if len(x) < len(y): + if inum is not None and inum != 0 and y[i+1] not in "0123456789": + return inum + return -1 + + if inum is None: # Same length, not in a num. + assert x == y + return 0 # So the strings are equal. + + return inum + + +class NatCmp(): + __slots__ = ['s',] + def __init__(self, s): + self.s = s + + def __str__(self): + return self.s + + def __eq__(self, other): + return self.s == other.s + + def __gt__(self, other): + ret = natcmp(self.s, other.s) + if ret > 0: + return True + return False + +# Given a list of strings, sort them using natcmp() +def nat_sorted(xs): + for ret in sorted(NatCmp(x) for x in xs): + yield ret.s + + +def _fnmatchi(path, pat): + """ Simple way to always use case insensitive filename matching. """ + return fnmatch.fnmatch(path.lower(), pat.lower()) + +# Have nice "plain" numbers... +def _ui_int(num): + if conf_num_sep_: + return "{:_}".format(int(num)) + return "{:,}".format(int(num)) + +# See: https://en.wikipedia.org/wiki/ANSI_escape_code#Select_Graphic_Rendition_parameters +# We merge the colours for 16 values so fg0-fgf bg0-bgf +ansi = {'bold' : '\033[1m', 'dim' : '\033[2m', 'italic' : '\033[3m', + 'underline' : '\033[4m', 'blink' : '\033[5m', 'reverse' :'\033[7m'} +ansi_stop = '\033[0m' +for i in range(7): + ansi['fg:' + str(i)] = '\033[3' + str(i) + 'm' + ansi['bg:' + str(i)] = '\033[4' + str(i) + 'm' +for i, j in ((0, '8'), (1, '9'), (2, 'a'), (3, 'b'), + (4, 'c'), (5, 'd'), (6, 'e'), (7, 'f')): + ansi['fg:' + j] = '\033[9' + str(i) + 'm' + ansi['bg:' + j] = '\033[10' + str(i) + 'm' +def _ui_t_align(text, align=None, olen=None): + if align is None or align == 0: + return text + if olen is None: + olen = len(text) + if abs(align) > olen: # "%*s", align, text + extra = abs(align) - olen + if align > 0: + text = " " * extra + text + else: + text = text + " " * extra + return text +def _ui_t_ansi(text, codes, align=0): + olen = len(text) + text = _ui_t_align(text, align) + if not conf_ansi_terminal or not codes or olen == 0: + return text + + esc = '' + for c in codes.split(','): + if c == 'reset': + esc = '' + if c not in ansi: # Ignore bad codes + continue + esc += ansi[c] + + if not esc: + return text + # Deal with leading/trailing spaces, mainly for underline. + olen = len(text) + text = text.lstrip() + prefix = olen - len(text) + text = text.rstrip() + suffix = (olen - prefix) - len(text) + return "%*s%s%s%s%*s" % (prefix, '', esc, text, ansi_stop, suffix, '') + +def _ui_t_cmd(text, align=0): + return _ui_t_ansi(text, conf_term_cmd, align=align) +def _ui_t_high(text, align=0): + return _ui_t_ansi(text, conf_term_highlight, align=align) +def _ui_t_key(text, align=0): + return _ui_t_ansi(text, conf_term_keyword, align=align) +def _ui_t_time(text, align=0): + return _ui_t_ansi(text, conf_term_time, align=align) +def _ui_t_title(text, align=0): + return _ui_t_ansi(text, conf_term_title, align=align) + +def _ui_t_user(text, align=0): + return _ui_t_ansi(text, conf_term_user, align=align) +def _ui_t_warn(text, align=0): + return _ui_t_ansi(text, conf_term_warn, align=align) +def _ui_t_info(text, align=0): + return _ui_t_ansi(text, conf_term_info, align=align) +def _ui_t_okay(text, align=0): + return _ui_t_ansi(text, conf_term_okay, align=align) + +# Make it easier to spot date differences +def _ui_date(d1, align=None, prev=None): + if not conf_ui_date: + return _ui_t_align(d1.date, align) + if prev is not None and d1.date == prev: + return _ui_t_align(" \" ", align) + # YYYY-MM-DD HH:MM + # 1234567890 23456 + if prev is None: + prev = _today + if conf_ansi_terminal and d1.date != prev: + for i in (15, 14, 12, 11, 9, 8, 7, 5): + if d1.date[:i] == prev[:i]: + ndate = d1.date[:i] + _ui_t_high(d1.date[i:]) + return _ui_t_align(ndate, align, len(d1.date)) + return _ui_t_align(d1.date, align) + +def _pre_cmd__setup(): + global conf_path + global plays + global _today + global _yesterday + global conf_ansi_terminal + + conf_path = os.path.expanduser(conf_path) + + if conf_path[0] != '/': + print(" Warning: Conf path isn't absolute", file=sys.stderr) + + plays = [] + # Eg. /2026/02/13/23.08.09/playbook-4067657.info + pat = "/????/??/??/??.??.??/playbook-*.info" + cpr = parse_duration(conf_play_duration) + # NOTE: Try to be a bit clever and only look at this years logs, if we can. + # Could expand this. + patyr = time.strftime("%Y", time.gmtime(time.time())) + if patyr == time.strftime("%Y", time.gmtime(time.time() - cpr)): + pat = "/" + patyr + "/??/??/??.??.??/playbook-*.info" + + for pbdir in glob.glob(conf_path + "/*"): + num = 0 + for pbi in reversed(glob.glob(pbdir + pat)): + pb = Playbook(pbi) + if not pb.hosts: + continue + if cpr > 0 and int(time.time() - pb.beg) > cpr: + break + plays.append(pb) + num += 1 + if conf_play_max > 0 and num >= conf_play_max: + break + # plays = list(sorted(plays)) + plays = list(reversed(sorted(plays, key=lambda x: x.beg))) + + _suffix_dns_replace.clear() + for x in conf_suffix_dns_replace: + _suffix_dns_replace[x] = False + if conf_ansi_terminal is None: + conf_ansi_terminal = sys.stdout.isatty() + + tm_today = int(time.time()) + _today = time.strftime("%Y-%m-%d", time.gmtime(tm_today)) + + tm_yesterday = int(time.time()) - (60*60*24) + _yesterday = time.strftime("%Y-%m-%d", time.gmtime(tm_yesterday)) + + +def _pre_cmd__verbose(args): + if args.verbose <= 0: + return + + if args.verbose >= 3: + globals()['conf_ui_date'] = False + if args.verbose >= 2: + globals()['conf_small_osinfo'] = False + globals()['conf_suffix_dns_replace'] = {} + globals()['conf_hist_show'] = 0 + globals()['conf_host_end_total_hostnum'] = 0 + globals()['conf_host_skip_eq'] = False + globals()['conf_info_machine_ids'] = True + globals()['conf_short_duration'] = False + globals()['conf_stat_4_hosts'] *= (args.verbose * 2) + +def _wild_eq(s1, s2): + """ Compare two strings, but allow '?' to mean anything. """ + if s1 == '?' or s2 == '?': + return True + return s1 == s2 + +class Task(): + """ Class for holding the Task data. """ + + __slots__ = ['datetime', 'json', 'name', 'num', 'status'] + + def __init__ (self, line): + data = line.split('\t', 4) + + self.datetime = data[0] + self.json = data[4] + self.name = data[3] + self.num = data[1] + self.status = data[2] + + def __str__(self): + return self.name + + def __eq__(self, other): + for key in self.__slots__: + if getattr(self, key) != getattr(other, key): + return False + return True + + def __gt__(self, other): + if self.num > other.num: + return True + if self.num != other.num: + return False + + if self.name > other.name: + return True + + return False + +def statuses_hosts(hosts): + ret = set() + for host in hosts: + for task in host.tasks: + ret.add(task.status.lower()) + return list(sorted(ret)) + +class Host(): + """ Class for holding the Host data from a line in the files. """ + + __slots__ = ['name', '_logfn', '_tasks', + "t_ok", "t_failures", "t_unreachable", + "t_changed", "t_skipped", "t_rescued", "t_ignored"] + + def __init__ (self, name, logfn, playbook_stats): + self.name = name + self._logfn = logfn + self._tasks = None + + for key in self.__slots__: + if key == 'name' or key[0] == '_': + continue + setattr(self, key, playbook_stats[key[2:]]) + + def __str__(self): + return self.name + + def __eq__(self, other): + for key in self.__slots__: + if getattr(self, key) != getattr(other, key): + return False + return True + + def __gt__(self, other): + ret = natcmp(self.name, other.name) + if ret > 0: + return True + if ret < 0: + return False + + for key in self.__slots__: + if key == 'name' or key[0] == '_': + continue + if getattr(self, key) > getattr(other, key): + return True + if getattr(self, key) != getattr(other, key): + return False + + return False + + # Pretend to be a dict... + def __getitem__(self, key): + if key not in self.__slots__: + raise KeyError() + return getattr(self, key) + + @property + def ofcs(self): + return "OK=%s Fail=%s Changed=%s" % (_ui_int(self.t_ok), + _ui_int(self.t_failures), + _ui_int(self.t_changed)) + + @property + def statuses(self): + return statuses_hosts([self]) + + @property + def changed(self): + fail = ' ' + if self.t_failures > 0: + fail = '!' + return "%s%s" % (fail, _ui_int(self.t_changed)) + + @property + def tasks(self): + tasks = [] + for line in gzip.open(self._logfn, 'rt'): + tasks.append(Task(line)) + return tasks + + +def _json_load_multi(fo, multi=0): + """ Read multiple json objects from a file. """ + content = fo.read() + + decoder = json.JSONDecoder() + pos = 0 + results = [] + + while pos < len(content) and (multi <= 0 or len(results) < multi): + # Skip leading whitespace/newlines + if content[pos].isspace(): + pos += 1 + continue + + # raw_decode returns the parsed object and the index where it ended + obj, pos = decoder.raw_decode(content, pos) + results.append(obj) + + return results + +def ofcs_hosts(hosts): + o, f, c = 0, 0, 0 + for host in hosts: + o += host.t_ok + f += host.t_failures + c += host.t_changed + return "OK=%s Fail=%s Changed=%s" % (_ui_int(o), _ui_int(f), _ui_int(c)) + +class Playbook(): + """ Class for holding the Playbook data from the log files. """ + + __slots__ = ['play', 'title', 'user', 'ro', + 'beg', 'end', + 'hosts'] + + def __init__ (self, path): + # print("JDBG:", "loading:", path) + + jdata = _json_load_multi(open(path)) + + hdr = jdata[0] + self.play = hdr['playbook'] + self.user = hdr['userid'] + self.beg = hdr['playbook_start'] + + inv = hdr['inventory'][0] + if inv.endswith('/inventory'): + inv = inv[:-len('/inventory')] + if self.play.startswith(inv): + self.play = self.play[len(inv):] + + # See if the playbook finished: + self.end = None + end = jdata[-1] + if 'playbook_end' in end: + self.end = end['playbook_end'] + + # See if we have stats. + self.hosts = [] + stats = jdata[-2] + + if 'stats' not in stats: + return + + hosts = stats['stats'] + for host in hosts: + logfn = os.path.dirname(path) + "/" + host + ".log.gz" + self.hosts.append(Host(host, logfn, hosts[host])) + self.hosts = list(sorted(self.hosts)) + + if not self.hosts: + return + + play = jdata[1] + self.title = play['play'] + self.ro = play.get('check', False) or play.get('diff', False) + xtra_plays = 0 + while xtra_plays < (len(jdata)-4): # FIXME: WTF to do with title? + xtra_plays += 1 + play = jdata[1+xtra_plays] + if self.ro: + self.ro = play['check'] or play['diff'] + + + def __str__(self): + return self.name + + def __eq__(self, other): + if self.play != other.play: + return False + if self.beg != other.beg: + return False + return True + + def __gt__(self, other): + ret = natcmp(self.play, other.play) + if ret > 0: + return True + if ret < 0: + return False + + if self.beg > other.beg: + return True + return False + + # Pretend to be a dict... + def __getitem__(self, key): + if key not in self.__slots__: + raise KeyError() + return getattr(self, key) + + @property + def name(self): + return os.path.basename(self.play) + + @property + def date(self): + return time.strftime("%Y-%m-%d %H:%M", time.gmtime(self.beg)) + + @property + def date_end(self): + if self.end is None: + return "" + return time.strftime("%Y-%m-%d %H:%M", time.gmtime(self.end)) + + @property + def ofcs(self): + return ofcs_hosts(self.hosts) + + @property + def statuses(self): + return statuses_hosts(self.hosts) + + @property + def changed(self): + f, c = 0, 0 + for host in self.hosts: + f += host.t_failures + c += host.t_changed + fail = ' ' + if f > 0: + fail = '!' + return "%s%s" % (fail, _ui_int(c)) + + +_tm_d = {'d' : 60*60*24, 'h' : 60*60, 'm' : 60, 's' : 1, + 'w' : 60*60*24*7, + 'q' : 60*60*24*7*13} +def parse_duration(seconds): + if seconds is None: + return None + if seconds.isdigit(): + return int(seconds) + + ret = 0 + for mark in ('w', 'd', 'h', 'm', 's'): + pos = seconds.find(mark) + if pos == -1: + continue + val = seconds[:pos] + seconds = seconds[pos+1:] + if not val.isdigit(): + # dbg("!isdigit", val) + return None + ret += _tm_d[mark]*int(val) + if seconds.isdigit(): + ret += int(seconds) + elif seconds != '': + # dbg("!empty", seconds) + return None + + return ret + + +def _add_dur(dur, ret, nummod, suffix, static=False): + mod = dur % nummod + dur = dur // nummod + if mod > 0 or (static and dur > 0): + ret.append(suffix) + if static and dur > 0: + ret.append("%0*d" % (len(str(nummod)), mod)) + else: + ret.append(str(mod)) + return dur + +def format_duration(seconds, short=False, static=False): + if seconds is None: + seconds = 0 + dur = int(seconds) + + ret = [] + dur = _add_dur(dur, ret, 60, "s", static=static) + dur = _add_dur(dur, ret, 60, "m", static=static) + if short: + if dur == 0 and not static: + return '<1h' + if dur == 0: + return '<01h' + ret = [] + dur = _add_dur(dur, ret, 24, "h", static=static) + dur = _add_dur(dur, ret, 7, "d", static=static) + if dur > 0: + ret.append("w") + ret.append(str(dur)) + return "".join(reversed(ret)) + +# Duration in UI for lists/etc. +def _ui_dur(dur, short=None): + if short is None: + short = conf_short_duration + return format_duration(dur, short=short, static=True) + +__get_login_user = None +# Get the username of the person who logged in, not root, cached. +def _get_login(): + global __get_login_user + if __get_login_user is None: + # Checked on 3.9.25 and it did the work everytime. + __get_login_user = os.getlogin() + return __get_login_user + +# Filter datas using name as a filename wildcard match. +def filter_name_datas(datas, names): + if not names: # Allow everything... + for data in datas: + yield data + return + + for data in datas: + found = False + for host in data.hosts: + for name in names: + if _fnmatchi(host.name, name): + found = True + break + if found: + break + if found: + yield data + +# Use this to only get then get only the specific hosts that matched... +def iter_name_play(play, names): + if not names: # Allow everything... + for host in play.hosts: + yield host + return + + for host in play.hosts: + for name in names: + if _fnmatchi(host.name, name): + yield host + break + +# Filter datas using name as a filename wildcard match. +def filter_play_datas(datas, names): + if not names: # Allow everything... + for data in datas: + yield data + return + + for data in datas: + nyml = data.name + if nyml.endswith(".yml"): # Should be true always? + nyml = nyml[:-len(".yml")] + found = False + for name in names: + if False: pass + elif _fnmatchi(data.name, name): + found = True + elif _fnmatchi(nyml, name): + found = True + elif _fnmatchi(_ui_play_name(data.name), name): + found = True + + if found: + break + + if found: + yield data + +# Filter datas using name as a filename wildcard match. +def filter_status_datas(datas, statuses): + if not statuses: # Allow everything... + for data in datas: + yield data + return + + for data in datas: + for status in statuses: + if status.lower() in data.statuses: + yield data + +# Filter datas user using name as a filename wildcard match. +def filter_user_datas(datas, names): + if not names: # Allow everything... + for data in datas: + yield data + return + + for data in datas: + found = False + for name in names: + if name == '.': + name = _get_login() + if _fnmatchi(data.user, name): + found = True + break + if found: + yield data + +# Filter datas date using name as a filename wildcard match. +def filter_date_datas(datas, dates): + if not dates: # Allow everything... + for data in datas: + yield data + return + + for data in datas: + found = False + for date in dates: + if _fnmatchi(data.date, date): + found = True + break + if found: + yield data + +# Filter datas using osname/vers/info as a filename wildcard match. +def filter_osname_datas(datas, names): + if not names: # Allow everything... + for data in datas: + yield data + return + + for data in datas: + for name in names: + if _fnmatchi(data.osinfo, name): + break + if _fnmatchi(data.osinfo_small, name): + break + if _fnmatchi(data.osname, name): + break + if _fnmatchi(data.osname_small, name): + break + if _fnmatchi(data.osvers, name): + break + off = data.osvers.find('.') + if off != -1: + vers = data.osvers[:off] + if _fnmatchi(vers, name): + break + + else: + continue + yield data + +# Sub. suffix of DNS names for UI +def _ui_name(name): + for suffix in conf_suffix_dns_replace: + if name.endswith(suffix): + _suffix_dns_replace[suffix] = True + return name[:-len(suffix)] + conf_suffix_dns_replace[suffix] + return name + +# Reset the usage after _max_update() +def _reset_ui_name(): + for suffix in sorted(_suffix_dns_replace): + _suffix_dns_replace[suffix] = False + +# Explain if we used any suffix subs. +def _explain_ui_name(): + done = False + pre = "* NOTE:" + for suffix in sorted(_suffix_dns_replace): + if _suffix_dns_replace[suffix]: + print("%s %12s = %s" % (pre,conf_suffix_dns_replace[suffix],suffix)) + pre = " :" + done = True + if done: + print(" : Use -vv to show full names.") + + +def _ui_play_name(name): + if name.endswith('.yml'): + name = name[:-len('.yml')] + name = conf_play_remap.get(name, name) + if conf_utf8: + if len(name) > 20: + name = name[:18] + '…' + else: + if len(name) > 20: + name = name[:17] + '...' + return name + +def _ui_hosts(data): + return _ui_int(len(data.hosts)) + +_max_len_name = 0 +_max_len_host = 0 # Host numbers +_max_len_ofcs = 0 # OK/Failed/Changed numbers +_max_len_date = 0 # YYYY-MM-DD HH:MM = 4+1+2+1+2 +1+2+1+2 +_max_terminal_width = shutil.get_terminal_size().columns +if _max_terminal_width < 20: + _max_terminal_width = 80 +# _max_terminal_width -= 14 +def _max_update(datas): + for data in datas: + _max_update_data(data) + +def _max_update_data(data): + global _max_len_name + global _max_len_host + global _max_len_ofcs + global _max_len_date + + name = _ui_play_name(data.name) + if len(name) > _max_len_name: + _max_len_name = len(name) + + for host in data.hosts: + hn = _ui_name(host.name) + if len(hn) > _max_len_host: + _max_len_host = len(hn) + + ofcs = host.changed + if len(ofcs) > _max_len_ofcs: + _max_len_ofcs = len(ofcs) + + if len(data.date) > _max_len_date: + _max_len_date = len(data.date) + +def _max_update_correct(prefix): + global _max_len_name + global _max_len_host + global _max_len_ofcs + global _max_len_date + mw = _max_terminal_width - len(prefix) + if _max_len_name + _max_len_host + _max_len_ofcs + _max_len_date < (mw-4): + _max_len_name += 1 + _max_len_host += 1 + _max_len_ofcs += 1 + _max_len_date += 1 + + while _max_len_name + _max_len_host + _max_len_ofcs + _max_len_date >= mw: + _max_len_host -= 1 + + + +# This is the real __main__ start ... +def _usage(short=False): + prog = "updates+uptime" + if sys.argv: + prog = os.path.basename(sys.argv[0]) + print("""\ + Usage: %s + Optional arguments: + --help, -h Show this help message and exit. + --verbose, -v Increase verbosity. + --conf CONF Specify configuration. + --ansi ANSI Use ansi terminal codes. + Cmds: +""" % (prog,), end='') + if short: + print("""\ + help + info [host*] [backup] [backup]... + json [host*] [task*] + list [host*]... +""", end='') + else: + # Also see: _cmd_help() below... + print("""\ + info [host*] [backup] [backup]... + = See the current state, in long form, can be filtered by name. + json [host*] [task*] + --play playbook* + = Get the JSON data from a task. + list [host*]... + --date date* + --play playbook* + = See the current state, can be filtered by name. +""", end='') + + +def inventory_hosts(): + # The "correct" way to do this is something like: + # ansible-inventory --list | jq -r '._meta.hostvars | keys[]' + # ...but that is _much_ slower, as it's loading a lot of data/facts which + # we ignore. + cmds = ["ansible", "all", "--list-host"] + p = subprocess.Popen(cmds, text=True, stdout=subprocess.PIPE) + header = p.stdout.readline() + if not header.strip().startswith("hosts ("): + return set() + + ret = set() + for line in p.stdout: + ret.add(line.strip()) + return ret + +def _pre_cmd__check_paths(): + # Below here are the query commands, stuff needs to exist at this point. + if not os.path.exists(conf_path): + print(" Error: No log file dir. Run a playbook?", file=sys.stderr) + sys.exit(4) + +def _date_suffix(dt): + suffix = '' + if dt == _today: + suffix = ' (today)' + if dt == _yesterday: + suffix = ' (yesterday)' + return suffix + +def _cli_match_host(args, data): + if args.hosts: + hosts = args.hosts[:] + print("Matching:", ", ".join(hosts)) + data = filter_name_datas(data, hosts) + data = list(data) + if not data: + print("Not host(s) matched:", ", ".join(hosts)) + sys.exit(2) + return data + +def __cmd_std_filters(args, data, hosts): + if args.play is not None: + data = list(filter_play_datas(data, [args.play])) + + if args.user is not None: + data = list(filter_user_datas(data, [args.user])) + + if args.date is not None: + data = list(filter_date_datas(data, [args.date])) + + data = list(filter_name_datas(data, hosts)) + + if args.status: + data = list(filter_status_datas(data, [args.status])) + + return data + +def _cmd_info(args): + hosts = [] + if args.host: + # print("JDBG:", args.host) + hosts = [args.host] + + tasks = [] + if args.task is not None: + tasks = [args.task] + + data = __cmd_std_filters(args, plays, hosts) + + for play in data: + print("=" * 70) + print("Play :", play.play) + print("Title:", play.title) + print("User :", play.user) + print("Beg :", play.date) + print("End :", play.date_end) + print("OFC :", ofcs_hosts(iter_name_play(play, hosts))) + done = False + for host in iter_name_play(play, hosts): + host_done = False + for task in host.tasks: + if args.status and args.status != task.status.lower(): + continue + if tasks and not _fnmatchi(task.name, tasks[0]): + continue + if not host_done: + print(' ', host.name, host.ofcs) + done = True + host_done = True + print(' ', task.num, "%s:" % task.status, task.name) + if args.json: + js = json.loads(task.json) + if not js or 'results' not in js: + continue + js = js['results'] + print(' ', json.dumps(js, indent=4, sort_keys=True)) + if done and args.one: + return + + +# If save=True, and we haven't output anything then save the line as we might +# not do anything. After something has gone out save=True does nothing. +_prnt_line_saved = [] +def _print_line_add(line): + global _prnt_line_saved + if _prnt_line_saved is None: + _prnt_line_saved = [] + _prnt_line_saved.append(line) +def _print_line_reset(): + global _prnt_line_saved + ret = _prnt_line_saved is not None + _prnt_line_saved = [] + return ret +def _print_play(prefix, data, hosts=[], high='', prev=None): + global _prnt_line_saved + if prev is not None: + prev = prev.date + done = False + hosts = list(iter_name_play(data, hosts)) + if len(hosts) > (conf_show_hosts_max+1): + hosts = hosts[:conf_show_hosts_max] + xtra = None + if len(hosts) < len(data.hosts): + if conf_utf8: + ellipsis = '…' + else: + ellipsis = '...' + xtra = " " + ellipsis + xtra += "%s more hosts" % (_ui_int(len(data.hosts) - len(hosts))) + xtra += " " + ellipsis + name = data.name + for host in hosts: + uiname = "%-*s" % (_max_len_name, _ui_play_name(name)) + uihost = "%-*s" % (_max_len_host, _ui_name(host.name)) + if high and not done: + uiname = _ui_t_ansi(uiname, high) + done = True + if high: + uinhost = _ui_t_ansi(uihost, high) + line = "%s%s %s %*s %s" % (prefix, + uiname, + uihost, + _max_len_ofcs, host.changed, + _ui_date(data, align=_max_len_date, prev=prev)) + prev = data.date + print(line) + name = "" + if xtra is not None: + print(xtra) + +def _print_plays(prefix, data, hosts=[], explain=True): + pd1 = None + for d1 in data: + _print_play(prefix, d1, hosts, prev=pd1) + pd1 = d1 + if explain: + _explain_ui_name() + +# -n variants match multiple things, but only allow looking at current data +def _cmd_list(args): + # FIXME: Ideally argparse would do this for us :( + hosts = [] + if hasattr(args, 'hosts'): + hosts = args.hosts[:] + + data = __cmd_std_filters(args, plays, hosts) + + _max_update(data) + _max_update_correct('') + print(_ui_t_title("Play", -_max_len_name), + _ui_t_title("Hosts", -_max_len_host), + _ui_t_title("Chg", _max_len_ofcs), + _ui_t_title("Date", _max_len_date)) + _print_plays('', data, hosts) + +def _cmd_json(args): + hosts = [] + if args.host is not None: + hosts = [args.host] + + tasks = [] + if args.task is not None: + tasks = [args.task] + + data = __cmd_std_filters(args, plays, hosts) + + done = False + for play in data: + for host in iter_name_play(play, hosts): + for task in host.tasks: + if args.status and args.status != task.status.lower(): + continue + if tasks and not _fnmatchi(task.name, tasks[0]): + continue + done = True + print(task.json) + if done and args.one: + return + +# CMDLINE validation: + +def _cmdline_arg_date(oval): + if len(oval) > len("YYYY-MM-DD HH:MM"): + raise argparse.ArgumentTypeError(f"{oval} is too big for a date") + val = oval.lower() + return val + +def _cmdline_arg_play(oval): + val = oval.lower() + return val + +def _cmdline_arg_user(oval): + val = oval.lower() + return val + +def _cmdline_arg_status(oval): + val = oval.lower() + if val not in ("changed", "failed", "ok", "skipped", "stats"): + raise argparse.ArgumentTypeError(f"{oval} is not an ansible task status") + return val + +def _cmdline_arg_ansi(oval): + val = oval.lower() + if val in ("true", "y", "yes", "on", "1", "always"): + return True + if val in ("false", "n", "no", "off", "0", "never"): + return False + if val in ("automatic", "?", "tty", "auto"): + return None + raise argparse.ArgumentTypeError(f"{oval} is not valid: always/never/auto") + +def _cmdline_arg_duration(oval): + if oval == "forever": + return 0 + val = parse_duration(oval) + if val is None: + raise argparse.ArgumentTypeError(f"{oval} is not a duration") + return val + +def _cmdline_arg_positive_integer(oval): + try: + val = int(oval) + except: + val = -1 + if val <= 0: + raise argparse.ArgumentTypeError(f"{oval} is not a positive integer") + return val + +_cmds_als = { + "information" : ["info"], + "json" : [], + "list" : [], + None : set(), +} +for c in _cmds_als: + if c is None: continue + _cmds_als[c] = set(_cmds_als[c]) + _cmds_als[None].update(_cmds_als[c]) + _cmds_als[None].add(c) + +def _cmd_help(args): + prog = "ansilog-playbook" + if sys.argv: + prog = os.path.basename(sys.argv[0]) + if not args.hcmd: + _usage() + if args.hcmd not in _cmds_als[None]: + print(" Unknown command:", args.hcmd) + _usage() + + def _eq_cmd(x): + return args.hcmd == x or args.hcmd in _cmds_als[x] + def _hlp_als(x): + if not _cmds_als[x]: + return '' + als = set() + als.add(x) + als.update(_cmds_als[x]) + als.remove(args.hcmd) + return f"""Aliases: {", ".join(sorted(als))}\n""" + + if False: pass + elif _eq_cmd("information"): + print(f"""\ + Usage: {prog} {args.hcmd} [host*] [task*] + {' '*len(prog)} {' '*len(args.hcmd)} --date date* + {' '*len(prog)} {' '*len(args.hcmd)} --json + {' '*len(prog)} {' '*len(args.hcmd)} --one -1 + {' '*len(prog)} {' '*len(args.hcmd)} --play playbook* + {' '*len(prog)} {' '*len(args.hcmd)} --status status + {' '*len(prog)} {' '*len(args.hcmd)} --user user* + + See the results of playbook(s), in long form, can be filtered by: + date, playbook name, hostname, task name, task status, user. + + --json pretty prints the JSON, if there are results. + + --one make it stop after the first playbook match. + + If you want to compare things by hand, use this. + + {_hlp_als("information")} + Eg. {prog} {args.hcmd} + {prog} {args.hcmd} 'batcave*' + {prog} {args.hcmd} --play check-etc -1 --user=. 'noc*' 'Report*' +""", end='') + + elif _eq_cmd("list"): + print(f"""\ + Usage: {prog} {args.hcmd} [host*] [host*]... + {' '*len(prog)} {' '*len(args.hcmd)} --date date* + {' '*len(prog)} {' '*len(args.hcmd)} --play playbook* + {' '*len(prog)} {' '*len(args.hcmd)} --status status + {' '*len(prog)} {' '*len(args.hcmd)} --user user* + + See the current state of the hosts/playbooks, can be filtered by: + date, playbook name, hostname, task status, user. + + {_hlp_als("list")} + Eg. {prog} {args.hcmd} + {prog} {args.hcmd} --conf=play_duration=13w 'batcave*' + {prog} {args.hcmd} 'batcave*' 'noc*' + {prog} {args.hcmd} --play check-etc --user=. '*stg*' '*test*' +""", end='') + + elif _eq_cmd("json"): + print(f"""\ + Usage: {prog} {args.hcmd} [host*] [task*] + {' '*len(prog)} {' '*len(args.hcmd)} --date date* + {' '*len(prog)} {' '*len(args.hcmd)} --one -1 + {' '*len(prog)} {' '*len(args.hcmd)} --play playbook* + {' '*len(prog)} {' '*len(args.hcmd)} --status status + {' '*len(prog)} {' '*len(args.hcmd)} --user user* + + See JSON for the task on the host. Note that the json is just printed raw +without filtering for every task (unlike "info --json"). + + {_hlp_als("json")} + Eg. {prog} {args.hcmd} 'batcave*' + {prog} {args.hcmd} --user=. --play check-etc --status=ok -1 \\* Report\\* \\ + | jq .results[].msg \\ + | sed 's/"UNKNOWN File: \\(- .*\)"/\\1/' \\ + | sort -V | uniq | less +""", end='') + +def _main(): + global conf_ansi_terminal + global conf_path + global cmd + + _user_conf() + + parser = argparse.ArgumentParser(add_help=False) + parser.add_argument('--verbose', '-v', action='count', default=0) + parser.add_argument("--conf", action='append', default=[]) + parser.add_argument("--ansi", type=_cmdline_arg_ansi, + help="Use ansi terminal codes") + parser.add_argument("--colour", type=_cmdline_arg_ansi, dest='ansi', + help=argparse.SUPPRESS) + parser.add_argument("--color", type=_cmdline_arg_ansi, dest='ansi', + help=argparse.SUPPRESS) + parser.add_argument('-h', '--help', action='store_true', + help='Show this help message') + + # We do this here so that `$0 -v blah -v` works. + margs, args = parser.parse_known_args() + + if margs.help: + _usage(short=True) + sys.exit(0) + + subparsers = parser.add_subparsers(dest="cmd") + + cmd = subparsers.add_parser("help") + cmd.add_argument("hcmd", nargs='?', help="cmd to get help for") + cmd.set_defaults(func=_cmd_help) + + def __defs(func): + cmd.set_defaults(func=func) + + # HIDDEN commands... + cmd = subparsers.add_parser("dur2secs", help=argparse.SUPPRESS) + cmd.add_argument("dur", type=_cmdline_arg_duration, help="duration") + __defs(func=lambda x: print("secs:", x.dur)) + + cmd = subparsers.add_parser("secs2dur", help=argparse.SUPPRESS) + cmd.add_argument("secs", type=int, help="seconds") + __defs(func=lambda x: print("dur:", _ui_dur(x.secs, short=False))) + + cmd = subparsers.add_parser("int2num", help=argparse.SUPPRESS) + cmd.add_argument("num", type=int, help="int") + __defs(func=lambda x: print("num:", _ui_int(x.num))) + + # -- Start of the real commands... + + # info command + als = _cmds_als["information"] + hlp = "show host information" + cmd = subparsers.add_parser("information", aliases=als, help=hlp) + cmd.add_argument("--date", type=_cmdline_arg_date, help="wildcard Date") + cmd.add_argument("--play", type=_cmdline_arg_play, help="wildcard Playbook") + cmd.add_argument("--status", type=_cmdline_arg_status, help="task status") + cmd.add_argument("--user", type=_cmdline_arg_user, help="task user") + cmd.add_argument("-1", "--one", action='store_true', default=False, help="latest play only") + cmd.add_argument("--json", action='store_true', default=False, help="also show JSON") + cmd.add_argument("host", nargs='?', help="wildcard hostname") + cmd.add_argument("task", nargs='?', help="wildcard taskname") + __defs(func=_cmd_info) + + # list cmd + cmd = subparsers.add_parser("list", help="list hosts") + cmd.add_argument("--date", type=_cmdline_arg_date, help="wildcard Date") + cmd.add_argument("--play", type=_cmdline_arg_play, help="wildcard Playbook") + cmd.add_argument("--user", type=_cmdline_arg_user, help="task user") + cmd.add_argument("--status", type=_cmdline_arg_status, help="task status") + cmd.add_argument("hosts", nargs='*', help="wildcard hostname(s)") + __defs(func=_cmd_list) + + # json command + cmd = subparsers.add_parser("json", help="get json") + cmd.add_argument("--date", type=_cmdline_arg_date, help="wildcard Date") + cmd.add_argument("--play", type=_cmdline_arg_play, help="wildcard Playbook") + cmd.add_argument("--status", type=_cmdline_arg_status, help="task status") + cmd.add_argument("--user", type=_cmdline_arg_user, help="task user") + cmd.add_argument("-1", "--one", action='store_true', default=False, help="latest play only") + cmd.add_argument("host", nargs='?', help="wildcard hostname") + cmd.add_argument("task", nargs='?', help="wildcard taskname") + __defs(func=_cmd_json) + + # Need to presetup for cmd line validation ... but conf can change + # so just validate format? And revalidate later? + # FIXME: We do no real validation on options, so skip doing the glob twice. + # _pre_cmd__setup() + + # Parse the above options/cmds + args = parser.parse_args(args) + + for line in margs.conf: + _user_conf_line(line) + + if margs.ansi is not None: + conf_ansi_terminal = margs.ansi + + # Setup based on the config. + _pre_cmd__setup() + + _pre_cmd__verbose(margs) + + _pre_cmd__check_paths() + + # Run the actual command. + if not hasattr(args, "func"): + cmd = "list" + args.date = None + args.play = None + args.hosts = [] + _cmd_list(args) + else: + cmd = args.cmd + args.func(args) + + +if __name__ == "__main__": + _main() diff --git a/files/scripts/updates-uptime-cmd.py b/files/scripts/updates-uptime-cmd.py index a641b99f63..752c710b95 100755 --- a/files/scripts/updates-uptime-cmd.py +++ b/files/scripts/updates-uptime-cmd.py @@ -1848,9 +1848,9 @@ def _cmd_host(args): def _cmdline_arg_ansi(oval): val = oval.lower() - if val in ("yes", "on", "1", "always"): + if val in ("true", "y", "yes", "on", "1", "always"): return True - if val in ("no", "off", "0", "never"): + if val in ("false", "n", "no", "off", "0", "never"): return False if val in ("automatic", "?", "tty", "auto"): return None diff --git a/inventory/group_vars/all b/inventory/group_vars/all index 42bf7ed998..ebdad85fb0 100644 --- a/inventory/group_vars/all +++ b/inventory/group_vars/all @@ -3,7 +3,7 @@ # BEGIN: Ansible roles_path variables # # Background/reference about external repos pulled in: -# https://forge.fedoraproject.org/infra/tickets/5476 +# https://forge.fedoraproject.org/infra/tickets/issues/5476 # # IPA settings additional_host_keytabs: [] @@ -333,6 +333,7 @@ zabbix_httpapi_use_ssl: true zabbix_httpapi_validate_certs: false zabbix_url_path: "" # If Zabbix WebUI runs on non-default (zabbix) path ,e.g. http:///zabbixeu # Zabbix agent vars +zabbix_agentd: /etc/zabbix_agentd.conf # Needed for zabbix_sender in other roles zabbix_host: "zabbix01{{ env_suffix }}.{{ datacenter }}.fedoraproject.org" zabbix_tls_psk_identity: "Fedora" zabbix_tls_psk: "{{ zabbix_tls_prod_psk }}" # in ansible-private repo diff --git a/inventory/group_vars/batcave b/inventory/group_vars/batcave index 51a5358481..eb86f03654 100644 --- a/inventory/group_vars/batcave +++ b/inventory/group_vars/batcave @@ -24,6 +24,7 @@ ipa_client_shell_groups: - sysadmin-datanommer - sysadmin-debuginfod - sysadmin-epel + - sysadmin-gpu - sysadmin-koschei - sysadmin-libravatar - sysadmin-messaging diff --git a/inventory/group_vars/copr_aws b/inventory/group_vars/copr_aws index 8761c9be87..4e26f94b89 100644 --- a/inventory/group_vars/copr_aws +++ b/inventory/group_vars/copr_aws @@ -46,6 +46,14 @@ builders: ppc64le: [15, 5, 15] p09_hypervisor_04: ppc64le: [15, 5, 15] + hp_p09_hypervisor_01: + ppc64le: [1, 1, 1] + hp_p09_hypervisor_02: + ppc64le: [1, 1, 1] + hp_p09_hypervisor_03: + ppc64le: [1, 1, 1] + hp_p09_hypervisor_04: + ppc64le: [1, 1, 1] x86_hypervisor_01: x86_64: [20, 4, 20] x86_hypervisor_02: @@ -208,7 +216,7 @@ aws_cloudfront_distribution: E2PUZIRCXCOXTG nrpe_client_uid: 500 rsnapshot_push: - server_host: storinator01.rdu-cc.fedoraproject.org + server_host: storinator01.fedoraproject.org backup_dir: /srv/nfs/copr-be cases: copr-be-copr-user: @@ -221,7 +229,7 @@ rsnapshot_push: deployment_type: prod -pulp_content_url: "https://console.redhat.com/api/pulp-content/public-copr/" +pulp_content_url: "https://packages.redhat.com/api/pulp-content/public-copr/" zabbix_host: 38.145.32.41 zabbix_macros: diff --git a/inventory/group_vars/copr_dev_aws b/inventory/group_vars/copr_dev_aws index 8e5e822e6a..85b399a001 100644 --- a/inventory/group_vars/copr_dev_aws +++ b/inventory/group_vars/copr_dev_aws @@ -45,6 +45,14 @@ builders: ppc64le: [1, 1, 1] p09_hypervisor_04: ppc64le: [1, 1, 1] + hp_p09_hypervisor_01: + ppc64le: [1, 1, 1] + hp_p09_hypervisor_02: + ppc64le: [1, 1, 1] + hp_p09_hypervisor_03: + ppc64le: [1, 1, 1] + hp_p09_hypervisor_04: + ppc64le: [1, 1, 1] x86_hypervisor_01: x86_64: [2, 1, 1] x86_hypervisor_02: @@ -184,6 +192,6 @@ aws_cloudfront_distribution: EX55ITR8LVMOH nrpe_client_uid: 500 -pulp_content_url: "https://console.redhat.com/api/pulp-content/public-copr-stage/" +pulp_content_url: "https://packages.redhat.com/api/pulp-content/public-copr-stage/" zabbix_host: 38.145.32.42 diff --git a/inventory/group_vars/download b/inventory/group_vars/download index 2e0512579b..3f896606ca 100644 --- a/inventory/group_vars/download +++ b/inventory/group_vars/download @@ -106,6 +106,7 @@ dl_tier1: - zaphod.gtlib.gatech.edu # 128.61.111.12 - sv.mirrors.kernel.org - dfw.mirrors.kernel.org + - mirror.raiolanetworks.com # 91.132.103.246 / 2a12:d282:102:f6::1 ipa_host_group: download ipa_host_group_desc: Download servers nagios_Check_Services: diff --git a/inventory/group_vars/gpu b/inventory/group_vars/gpu new file mode 100644 index 0000000000..3cfc305420 --- /dev/null +++ b/inventory/group_vars/gpu @@ -0,0 +1,13 @@ +--- +primary_auth_source: ipa +freezes: false +ipa_client_shell_groups: + - sysadmin-gpu +ipa_client_sudo_groups: + - sysadmin-gpu +ipa_host_group: gpu +ipa_host_group_desc: gpu servers + +notes: | + gpu machine for testing fedora with gpus + diff --git a/inventory/group_vars/pkgs b/inventory/group_vars/pkgs index f8fe5efdf9..9a326c9999 100644 --- a/inventory/group_vars/pkgs +++ b/inventory/group_vars/pkgs @@ -48,3 +48,4 @@ wsgi_procs: 20 wsgi_threads: 5 zabbix_macros: 'VFS.FS.FSTYPE.MATCHES': '^(btrfs|ext2|ext3|ext4|reiser|xfs|ffs|ufs|jfs|jfs2|vxfs|hfs|apfs|refs|ntfs|fat32|zfs|nfs)$' +postfix_group: pkgs diff --git a/inventory/group_vars/pkgs_stg b/inventory/group_vars/pkgs_stg index ca39f590a2..b68917129d 100644 --- a/inventory/group_vars/pkgs_stg +++ b/inventory/group_vars/pkgs_stg @@ -43,3 +43,4 @@ wsgi_procs: 4 wsgi_threads: 4 zabbix_macros: 'VFS.FS.FSTYPE.MATCHES': '^(btrfs|ext2|ext3|ext4|reiser|xfs|ffs|ufs|jfs|jfs2|vxfs|hfs|apfs|refs|ntfs|fat32|zfs|nfs)$' +postfix_group: pkgs diff --git a/inventory/host_vars/backup01.rdu3.fedoraproject.org b/inventory/host_vars/backup01.rdu3.fedoraproject.org index 9ee50a9a9f..d565321a0c 100644 --- a/inventory/host_vars/backup01.rdu3.fedoraproject.org +++ b/inventory/host_vars/backup01.rdu3.fedoraproject.org @@ -62,3 +62,13 @@ network_connections: nrpe_procs_crit: 1400 nrpe_procs_warn: 1200 weblate_backup_topdir: /fedora_backups/misc/weblate +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_purchase: "2012-12-31" + hardware: PowerEdge R650 + location: RDU3 + oob_ip: 10.3.160.099 + serialno_a: 717DRT3 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/buildhw-x86-01.rdu3.fedoraproject.org b/inventory/host_vars/buildhw-x86-01.rdu3.fedoraproject.org index ebb9c1e3cb..24cd5201c6 100644 --- a/inventory/host_vars/buildhw-x86-01.rdu3.fedoraproject.org +++ b/inventory/host_vars/buildhw-x86-01.rdu3.fedoraproject.org @@ -50,3 +50,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2023-05-27" + date_hw_purchase: "2016-05-26" + hardware: PowerEdge FC430 + location: RDU3 + oob_ip: 10.3.160.012 + serialno_a: 2TFSHB2 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/buildhw-x86-02.rdu3.fedoraproject.org b/inventory/host_vars/buildhw-x86-02.rdu3.fedoraproject.org index 5a2eb8e830..575bf02b4e 100644 --- a/inventory/host_vars/buildhw-x86-02.rdu3.fedoraproject.org +++ b/inventory/host_vars/buildhw-x86-02.rdu3.fedoraproject.org @@ -51,3 +51,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2023-05-27" + date_hw_purchase: "2016-05-26" + hardware: PowerEdge FC430 + location: RDU3 + oob_ip: 10.3.160.013 + serialno_a: 2TFTHB2 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/buildhw-x86-03.rdu3.fedoraproject.org b/inventory/host_vars/buildhw-x86-03.rdu3.fedoraproject.org index a9bf3306c0..245c3863b3 100644 --- a/inventory/host_vars/buildhw-x86-03.rdu3.fedoraproject.org +++ b/inventory/host_vars/buildhw-x86-03.rdu3.fedoraproject.org @@ -50,3 +50,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2023-05-27" + date_hw_purchase: "2016-05-26" + hardware: PowerEdge FC430 + location: RDU3 + oob_ip: 10.3.160.014 + serialno_a: 2TGRHB2 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/buildhw-x86-04.rdu3.fedoraproject.org b/inventory/host_vars/buildhw-x86-04.rdu3.fedoraproject.org index 6a496ac3f6..37be3daa79 100644 --- a/inventory/host_vars/buildhw-x86-04.rdu3.fedoraproject.org +++ b/inventory/host_vars/buildhw-x86-04.rdu3.fedoraproject.org @@ -52,3 +52,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2023-05-27" + date_hw_purchase: "2016-05-26" + hardware: PowerEdge FC430 + location: RDU3 + oob_ip: 10.3.160.015 + serialno_a: 2TGSHB2 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/buildhw-x86-08.rdu3.fedoraproject.org b/inventory/host_vars/buildhw-x86-08.rdu3.fedoraproject.org index fa0fd06e73..745efaa0ef 100644 --- a/inventory/host_vars/buildhw-x86-08.rdu3.fedoraproject.org +++ b/inventory/host_vars/buildhw-x86-08.rdu3.fedoraproject.org @@ -52,3 +52,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2023-05-27" + date_hw_purchase: "2016-05-26" + hardware: PowerEdge FC430 + location: RDU3 + oob_ip: 10.3.160.019 + serialno_a: 2THTHB2 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/buildhw-x86-09.rdu3.fedoraproject.org b/inventory/host_vars/buildhw-x86-09.rdu3.fedoraproject.org index 958ee99bfa..f39e67c351 100644 --- a/inventory/host_vars/buildhw-x86-09.rdu3.fedoraproject.org +++ b/inventory/host_vars/buildhw-x86-09.rdu3.fedoraproject.org @@ -52,3 +52,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2023-05-27" + date_hw_purchase: "2016-05-26" + hardware: PowerEdge FC430 + location: RDU3 + oob_ip: 10.3.160.021 + serialno_a: 2TBTHB2 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/buildhw-x86-10.rdu3.fedoraproject.org b/inventory/host_vars/buildhw-x86-10.rdu3.fedoraproject.org index 08d58f417a..2157b23141 100644 --- a/inventory/host_vars/buildhw-x86-10.rdu3.fedoraproject.org +++ b/inventory/host_vars/buildhw-x86-10.rdu3.fedoraproject.org @@ -52,3 +52,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2023-05-27" + date_hw_purchase: "2016-05-26" + hardware: PowerEdge FC430 + location: RDU3 + oob_ip: 10.3.160.022 + serialno_a: 2TCRHB2 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/buildhw-x86-12.rdu3.fedoraproject.org b/inventory/host_vars/buildhw-x86-12.rdu3.fedoraproject.org index 6f7ad00f39..98227ee5b0 100644 --- a/inventory/host_vars/buildhw-x86-12.rdu3.fedoraproject.org +++ b/inventory/host_vars/buildhw-x86-12.rdu3.fedoraproject.org @@ -52,3 +52,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2023-05-27" + date_hw_purchase: "2016-05-26" + hardware: PowerEdge FC430 + location: RDU3 + oob_ip: 10.3.160.024 + serialno_a: 2TFRHB2 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/buildhw-x86-13.rdu3.fedoraproject.org b/inventory/host_vars/buildhw-x86-13.rdu3.fedoraproject.org index 3acd0e7d22..4ca9e01a78 100644 --- a/inventory/host_vars/buildhw-x86-13.rdu3.fedoraproject.org +++ b/inventory/host_vars/buildhw-x86-13.rdu3.fedoraproject.org @@ -52,3 +52,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2023-05-27" + date_hw_purchase: "2016-05-26" + hardware: PowerEdge FC430 + location: RDU3 + oob_ip: 10.3.160.025 + serialno_a: 2TCSHB2 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/buildvm-x86-riscv01.rdu3.fedoraproject.org b/inventory/host_vars/buildvm-x86-riscv01.rdu3.fedoraproject.org index 0ce28c1df0..d233c155ab 100644 --- a/inventory/host_vars/buildvm-x86-riscv01.rdu3.fedoraproject.org +++ b/inventory/host_vars/buildvm-x86-riscv01.rdu3.fedoraproject.org @@ -5,3 +5,13 @@ eth0_ipv4_ip: 10.16.172.23 ipa_server: ipa01.rdu3.fedoraproject.org resolvconf: "resolv.conf/rdu3" vmhost: bvmhost-x86-riscv01.rdu3.fedoraproject.org +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2028-10-12" + date_hw_purchase: "2024-01-31" + hardware: PowerEdge R450 + location: RDU3 + serialno_a: D922FZ3 + type: Build_Vmserver + vendor: Dell diff --git a/inventory/host_vars/bvmhost-a64-01.rdu3.fedoraproject.org b/inventory/host_vars/bvmhost-a64-01.rdu3.fedoraproject.org index 6f2cecf956..1bd8573f54 100644 --- a/inventory/host_vars/bvmhost-a64-01.rdu3.fedoraproject.org +++ b/inventory/host_vars/bvmhost-a64-01.rdu3.fedoraproject.org @@ -59,3 +59,13 @@ network_connections: mtu: 1500 nrpe_procs_crit: 4500 nrpe_procs_warn: 4000 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2028-07-20" + date_hw_purchase: "2024-01-31" + hardware: Mt Snow + location: RDU3 + serialno_a: 231001325 + type: Build_Vmserver + vendor: Ampere/Gigabyte diff --git a/inventory/host_vars/bvmhost-a64-01.stg.rdu3.fedoraproject.org b/inventory/host_vars/bvmhost-a64-01.stg.rdu3.fedoraproject.org index 190fe476ea..728d69c195 100644 --- a/inventory/host_vars/bvmhost-a64-01.stg.rdu3.fedoraproject.org +++ b/inventory/host_vars/bvmhost-a64-01.stg.rdu3.fedoraproject.org @@ -57,3 +57,13 @@ nrpe_procs_crit: 4500 nrpe_procs_warn: 4000 zabbix_macros: NET.IF.IFNAME.NOT_MATCHES: "^vnet.*" +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2028-07-20" + date_hw_purchase: "2024-01-30" + hardware: Mt Snow + location: RDU3 + serialno_a: 231001329 + type: Build_Vmserver + vendor: Ampere/Gigabyte diff --git a/inventory/host_vars/bvmhost-a64-02.rdu3.fedoraproject.org b/inventory/host_vars/bvmhost-a64-02.rdu3.fedoraproject.org index 90e2da80c6..4df1845a35 100644 --- a/inventory/host_vars/bvmhost-a64-02.rdu3.fedoraproject.org +++ b/inventory/host_vars/bvmhost-a64-02.rdu3.fedoraproject.org @@ -59,3 +59,13 @@ network_connections: mtu: 1500 nrpe_procs_crit: 4500 nrpe_procs_warn: 4000 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2028-07-20" + date_hw_purchase: "2024-01-31" + hardware: Mt Snow + location: RDU3 + serialno_a: 231001328 + type: Build_Vmserver + vendor: Ampere/Gigabyte diff --git a/inventory/host_vars/bvmhost-a64-03.rdu3.fedoraproject.org b/inventory/host_vars/bvmhost-a64-03.rdu3.fedoraproject.org index 64bc9ee96f..53a4ce6d41 100644 --- a/inventory/host_vars/bvmhost-a64-03.rdu3.fedoraproject.org +++ b/inventory/host_vars/bvmhost-a64-03.rdu3.fedoraproject.org @@ -59,3 +59,13 @@ network_connections: mtu: 1500 nrpe_procs_crit: 4500 nrpe_procs_warn: 4000 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2028-07-20" + date_hw_purchase: "2024-01-31" + hardware: Mt Snow + location: RDU3 + serialno_a: 231001327 + type: Build_Vmserver + vendor: Ampere/Gigabyte diff --git a/inventory/host_vars/bvmhost-a64-04.rdu3.fedoraproject.org b/inventory/host_vars/bvmhost-a64-04.rdu3.fedoraproject.org index c24e86396a..cd47d13e77 100644 --- a/inventory/host_vars/bvmhost-a64-04.rdu3.fedoraproject.org +++ b/inventory/host_vars/bvmhost-a64-04.rdu3.fedoraproject.org @@ -59,3 +59,13 @@ network_connections: mtu: 1500 nrpe_procs_crit: 4500 nrpe_procs_warn: 4000 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2028-07-20" + date_hw_purchase: "2024-01-31" + hardware: Mt Snow + location: RDU3 + serialno_a: 231001326 + type: Build_Vmserver + vendor: Ampere/Gigabyte diff --git a/inventory/host_vars/bvmhost-p09-05.rdu3.fedoraproject.org b/inventory/host_vars/bvmhost-p09-05.rdu3.fedoraproject.org index 47e60a218f..58def58c44 100644 --- a/inventory/host_vars/bvmhost-p09-05.rdu3.fedoraproject.org +++ b/inventory/host_vars/bvmhost-p09-05.rdu3.fedoraproject.org @@ -60,3 +60,14 @@ nrpe_procs_crit: 13000 nrpe_procs_warn: 12000 zabbix_macros: MEMORY.AVAILABLE.MIN: 0 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2027-01-31" + date_hw_purchase: "2021-11-19" + hardware: Power 9 9183-22x + location: RDU3 + oob_ip: 10.3.160.62 + serialno_a: 780483A + type: Build_vmServer + vendor: IBM diff --git a/inventory/host_vars/bvmhost-s390x-01.s390.fedoraproject.org b/inventory/host_vars/bvmhost-s390x-01.s390.fedoraproject.org index 5200012b4b..a6f0da3360 100644 --- a/inventory/host_vars/bvmhost-s390x-01.s390.fedoraproject.org +++ b/inventory/host_vars/bvmhost-s390x-01.s390.fedoraproject.org @@ -3,4 +3,4 @@ dns1: 10.16.163.33 dns2: 10.16.163.34 dns_search1: rdu3.fedoraproject.org nbde: false -zabbix_ping_only: true # no agent access on this host +zabbix_ping_only: false diff --git a/inventory/host_vars/bvmhost-s390x-01.stg.s390.fedoraproject.org b/inventory/host_vars/bvmhost-s390x-01.stg.s390.fedoraproject.org index 603cfa4685..83ca7076e1 100644 --- a/inventory/host_vars/bvmhost-s390x-01.stg.s390.fedoraproject.org +++ b/inventory/host_vars/bvmhost-s390x-01.stg.s390.fedoraproject.org @@ -4,4 +4,4 @@ dns1: 10.16.163.33 dns2: 10.16.163.34 dns_search1: rdu3.fedoraproject.org nbde: false -zabbix_ping_only: true # no agent access on this host +zabbix_ping_only: false diff --git a/inventory/host_vars/bvmhost-x86-01.stg.rdu3.fedoraproject.org b/inventory/host_vars/bvmhost-x86-01.stg.rdu3.fedoraproject.org index 1f8ecdc14b..f2238fa940 100644 --- a/inventory/host_vars/bvmhost-x86-01.stg.rdu3.fedoraproject.org +++ b/inventory/host_vars/bvmhost-x86-01.stg.rdu3.fedoraproject.org @@ -58,3 +58,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2025-01-01" + date_hw_purchase: "2018-04-12" + hardware: PowerEdge R630 + location: RDU3 + oob_ip: 10.3.160191 + serialno_a: CLTKMN2 + type: #Build vm hosts - Other + vendor: Dell diff --git a/inventory/host_vars/bvmhost-x86-02.stg.rdu3.fedoraproject.org b/inventory/host_vars/bvmhost-x86-02.stg.rdu3.fedoraproject.org index bbdf45d758..aace137063 100644 --- a/inventory/host_vars/bvmhost-x86-02.stg.rdu3.fedoraproject.org +++ b/inventory/host_vars/bvmhost-x86-02.stg.rdu3.fedoraproject.org @@ -58,3 +58,13 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_purchase: "2022-12-31" + hardware: PowerEdge R650 + location: RDU3 + oob_ip: 10.3.160.192 + serialno_a: 3ZGTRT3 + type: Staging_Vmserver + vendor: Dell diff --git a/inventory/host_vars/bvmhost-x86-03.stg.rdu3.fedoraproject.org b/inventory/host_vars/bvmhost-x86-03.stg.rdu3.fedoraproject.org index 1872cc6ae5..8a1729374d 100644 --- a/inventory/host_vars/bvmhost-x86-03.stg.rdu3.fedoraproject.org +++ b/inventory/host_vars/bvmhost-x86-03.stg.rdu3.fedoraproject.org @@ -58,3 +58,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2025-01-01" + date_hw_purchase: "2018-04-12" + hardware: PowerEdge R630 + location: RDU3 + oob_ip: 10.3.160.193 + serialno_a: CLTDMN2 + type: #Build vm hosts - Other + vendor: Dell diff --git a/inventory/host_vars/gpu01.rdu3.fedoraproject.org b/inventory/host_vars/gpu01.rdu3.fedoraproject.org new file mode 100644 index 0000000000..987445d994 --- /dev/null +++ b/inventory/host_vars/gpu01.rdu3.fedoraproject.org @@ -0,0 +1,54 @@ +--- +datacenter: rdu3 +dns1: 10.16.163.33 +br0_ipv4_ip: 10.16.179.35 +br0_ipv4_gw: 10.16.179.254 +br0_ipv4_nm: 24 +has_ipv4: yes +has_ipv6: no +mac0: c4:cb:e1:ed:b1:6e +mac1: c4:cb:e1:ed:b1:6f +mac2: 30:3e:a7:2a:65:28 +mac3: 30:3e:a7:2a:65:29 +mac4: 30:3e:a7:2a:65:2a +mac5: 30:3e:a7:2a:65:2b +network_connections: + # Bridge profile + - name: br0 + state: up + type: bridge + mtu: 1500 + autoconnect: yes + ip: + address: + - "{{ br0_ipv4_ip }}/{{ br0_ipv4_nm }}" + dhcp4: no + dns: + - "{{ dns1 }}" + - "{{ dns2 }}" + dns_search: + - "{{ dns_search1 }}" + - "{{ dns_search2 }}" + gateway4: "{{ br0_ipv4_gw }}" + # Bond profile + - name: bond0 + type: bond + interface_name: bond0 + mtu: 1500 + controller: br0 + bond: + mode: 802.3ad + # Port profile for the 1st Ethernet device + - name: bond0-port1 + mac: "{{ mac2 }}" + type: ethernet + controller: bond0 + state: up + mtu: 1500 + # Port profile for the 2nd Ethernet device + - name: bond0-port2 + mac: "{{ mac3 }}" + type: ethernet + controller: bond0 + state: up + mtu: 1500 diff --git a/inventory/host_vars/ibiblio02.fedoraproject.org b/inventory/host_vars/ibiblio02.fedoraproject.org index 65bec6fd51..60f98e8da7 100644 --- a/inventory/host_vars/ibiblio02.fedoraproject.org +++ b/inventory/host_vars/ibiblio02.fedoraproject.org @@ -63,3 +63,13 @@ nrpe_procs_warn: 1250 postfix_group: vpn vpn: true notes: "vhost at ibiblio" +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_purchase: "2022-12-20" + hardware: PowerEdge R750 + location: RDU3 + serialno_a: 2KC66X3 + type: Web_Vmserver + vendor: Dell + diff --git a/inventory/host_vars/ibiblio05.fedoraproject.org b/inventory/host_vars/ibiblio05.fedoraproject.org index 0af1500c6e..7edc8e945f 100644 --- a/inventory/host_vars/ibiblio05.fedoraproject.org +++ b/inventory/host_vars/ibiblio05.fedoraproject.org @@ -66,3 +66,12 @@ vpn: true zabbix_macros: VFS.DEV.READ.AWAIT.WARN: 100 VFS.DEV.WRITE.AWAIT.WARN: 100 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_purchase: "2015-06-29" + hardware: PowerEdge R630 + location: RDU3 + serialno_a: 4T05S52 + type: Web_Vmserver + vendor: Dell diff --git a/inventory/host_vars/kernel02.rdu3.fedoraproject.org b/inventory/host_vars/kernel02.rdu3.fedoraproject.org index 3fc14d8792..75184ae689 100644 --- a/inventory/host_vars/kernel02.rdu3.fedoraproject.org +++ b/inventory/host_vars/kernel02.rdu3.fedoraproject.org @@ -48,3 +48,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2029-02-10" + date_hw_purchase: "2024-04-15" + hardware: PowerEdge R650 + location: RDU3 + oob_ip: 10.3.160.066 + serialno_a: F76HC14 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/logdetective01.fedorainfracloud.org b/inventory/host_vars/logdetective01.fedorainfracloud.org index 51d085aaf0..4afd0beb7b 100644 --- a/inventory/host_vars/logdetective01.fedorainfracloud.org +++ b/inventory/host_vars/logdetective01.fedorainfracloud.org @@ -20,5 +20,5 @@ root_auth_users: msuchy frostyx praiskup nikromen ttomecek jpodivin sgallagh mma nrpe_client_uid: 500 tcp_ports: [ - 22, 80, 443, + 22, 80, 443, 8090, ] diff --git a/inventory/host_vars/openqa-x86-worker01.rdu3.fedoraproject.org b/inventory/host_vars/openqa-x86-worker01.rdu3.fedoraproject.org index dd4bf34b3d..85fd624cf6 100644 --- a/inventory/host_vars/openqa-x86-worker01.rdu3.fedoraproject.org +++ b/inventory/host_vars/openqa-x86-worker01.rdu3.fedoraproject.org @@ -77,3 +77,14 @@ sudoers: "{{ private }}/files/sudo/qavirt-sudoers" # $ENV{QEMUPORT} = ($options{instance}) * 10 + 20002; # so for worker 1 it's 20012, for worker 2 it's 20022, etc etc tcp_ports: ['20013', '20023', '20033', '20043', '20053', '20063', '20073', '20083', '20093', '20103', '20113', '20123', '20133', '20143', '20153', '20163', '20173', '20183', '20193', '20203', '20213', '20223', '20233', '20243', '20253', '20263', '20273', '20283', '20293', '20303', '20313', '20323', '20333', '20343', '20353', '20363', '20373', '20383', '20393', '20403', '20413', '20423', '20433', '20443', '20453', '20463', '20473', '20483', '20493', '20503', '20513', '20523', '20533', '20543', '20553', '20563', '20573', '20583', '20593', '20603'] +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2025-01-01" + date_hw_purchase: "2019-12-20" + hardware: PowerEdge R640 + location: RDU3 + oob_ip: 10.3.160.137 + serialno_a: 1483H13 + type: QA_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/openqa-x86-worker02.rdu3.fedoraproject.org b/inventory/host_vars/openqa-x86-worker02.rdu3.fedoraproject.org index fe8f656ab6..4cd0d19be6 100644 --- a/inventory/host_vars/openqa-x86-worker02.rdu3.fedoraproject.org +++ b/inventory/host_vars/openqa-x86-worker02.rdu3.fedoraproject.org @@ -77,3 +77,14 @@ sudoers: "{{ private }}/files/sudo/qavirt-sudoers" # $ENV{QEMUPORT} = ($options{instance}) * 10 + 20002; # so for worker 1 it's 20012, for worker 2 it's 20022, etc etc tcp_ports: ['20013', '20023', '20033', '20043', '20053', '20063', '20073', '20083', '20093', '20103', '20113', '20123', '20133', '20143', '20153', '20163', '20173', '20183', '20193', '20203', '20213', '20223', '20233', '20243', '20253', '20263', '20273', '20283', '20293', '20303', '20313', '20323', '20333', '20343', '20353', '20363', '20373', '20383', '20393', '20403', '20413', '20423', '20433', '20443', '20453', '20463', '20473', '20483', '20493', '20503', '20513', '20523', '20533', '20543', '20553', '20563', '20573', '20583', '20593', '20603'] +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2025-01-01" + date_hw_purchase: "2019-09-22" + hardware: PowerEdge R640 + location: RDU3 + oob_ip: 10.3.160.138 + serialno_a: 8693MR2 + type: QA_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/openqa-x86-worker03.rdu3.fedoraproject.org b/inventory/host_vars/openqa-x86-worker03.rdu3.fedoraproject.org index cc36d2c626..9de0fdef29 100644 --- a/inventory/host_vars/openqa-x86-worker03.rdu3.fedoraproject.org +++ b/inventory/host_vars/openqa-x86-worker03.rdu3.fedoraproject.org @@ -77,3 +77,12 @@ sudoers: "{{ private }}/files/sudo/qavirt-sudoers" # $ENV{QEMUPORT} = ($options{instance}) * 10 + 20002; # so for worker 1 it's 20012, for worker 2 it's 20022, etc etc tcp_ports: ['20013', '20023', '20033', '20043', '20053', '20063', '20073', '20083', '20093', '20103', '20113', '20123', '20133', '20143', '20153', '20163', '20173', '20183', '20193', '20203', '20213', '20223', '20233', '20243', '20253', '20263', '20273', '20283', '20293', '20303', '20313', '20323', '20333', '20343', '20353', '20363', '20373', '20383', '20393', '20403', '20413', '20423', '20433', '20443', '20453', '20463', '20473', '20483', '20493', '20503', '20513', '20523', '20533', '20543', '20553', '20563', '20573', '20583', '20593', '20603'] +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_purchase: "2021-01-13" + hardware: PowerEdge R630 + location: RDU3 + serialno_a: CLTGMN2 + type: QA_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/openqa-x86-worker04.rdu3.fedoraproject.org b/inventory/host_vars/openqa-x86-worker04.rdu3.fedoraproject.org index 48764b8c37..2e8246d34a 100644 --- a/inventory/host_vars/openqa-x86-worker04.rdu3.fedoraproject.org +++ b/inventory/host_vars/openqa-x86-worker04.rdu3.fedoraproject.org @@ -77,3 +77,14 @@ sudoers: "{{ private }}/files/sudo/qavirt-sudoers" # $ENV{QEMUPORT} = ($options{instance}) * 10 + 20002; # so for worker 1 it's 20012, for worker 2 it's 20022, etc etc tcp_ports: ['20013', '20023', '20033', '20043', '20053', '20063', '20073', '20083', '20093', '20103', '20113', '20123', '20133', '20143', '20153', '20163', '20173', '20183', '20193', '20203', '20213', '20223', '20233', '20243', '20253', '20263', '20273', '20283', '20293', '20303', '20313', '20323', '20333', '20343', '20353', '20363', '20373', '20383', '20393', '20403', '20413', '20423', '20433', '20443', '20453', '20463', '20473', '20483', '20493', '20503', '20513', '20523', '20533', '20543', '20553', '20563', '20573', '20583', '20593', '20603'] +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2025-01-01" + date_hw_purchase: "2018-09-22" + hardware: PowerEdge R640 + location: RDU3 + oob_ip: 10.3.160.053 + serialno_a: 8692MR2 + type: QA_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/openqa-x86-worker05.rdu3.fedoraproject.org b/inventory/host_vars/openqa-x86-worker05.rdu3.fedoraproject.org index 9db7148c2c..0cb92a4ef8 100644 --- a/inventory/host_vars/openqa-x86-worker05.rdu3.fedoraproject.org +++ b/inventory/host_vars/openqa-x86-worker05.rdu3.fedoraproject.org @@ -77,3 +77,11 @@ sudoers: "{{ private }}/files/sudo/qavirt-sudoers" # $ENV{QEMUPORT} = ($options{instance}) * 10 + 20002; # so for worker 1 it's 20012, for worker 2 it's 20022, etc etc tcp_ports: ['20013', '20023', '20033', '20043', '20053', '20063', '20073', '20083', '20093', '20103', '20113', '20123', '20133', '20143', '20153', '20163', '20173', '20183', '20193', '20203', '20213', '20223', '20233', '20243', '20253', '20263', '20273', '20283', '20293', '20303', '20313', '20323', '20333', '20343', '20353', '20363', '20373', '20383', '20393', '20403', '20413', '20423', '20433', '20443', '20453', '20463', '20473', '20483', '20493', '20503', '20513', '20523', '20533', '20543', '20553', '20563', '20573', '20583', '20593', '20603'] +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + hardware: PowerEdge R630 + location: RDU3 + serialno_a: CLTHMN2 + type: QA_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/osuosl02.fedoraproject.org b/inventory/host_vars/osuosl02.fedoraproject.org index f6b26e058c..6599c25a1e 100644 --- a/inventory/host_vars/osuosl02.fedoraproject.org +++ b/inventory/host_vars/osuosl02.fedoraproject.org @@ -31,3 +31,11 @@ network_connections: nrpe_procs_crit: 2500 nrpe_procs_warn: 2000 virthost: true +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_purchase: "2024-05-01" + hardware: PowerEdge R650 + location: RDU3 + serialno_a: 2RH5X04 + vendor: Dell diff --git a/inventory/host_vars/sign-vault01.rdu3.fedoraproject.org b/inventory/host_vars/sign-vault01.rdu3.fedoraproject.org index 39248ac614..1833df4350 100644 --- a/inventory/host_vars/sign-vault01.rdu3.fedoraproject.org +++ b/inventory/host_vars/sign-vault01.rdu3.fedoraproject.org @@ -52,3 +52,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2025-01-01" + date_hw_purchase: "2018-04-18" + hardware: PowerEdge R430 + location: RDU3 + oob_ip: 10.3.160.132 + serialno_a: 1VHTMN2 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/sign-vault02.rdu3.fedoraproject.org b/inventory/host_vars/sign-vault02.rdu3.fedoraproject.org index 811ff411c7..3eb6b2a9ff 100644 --- a/inventory/host_vars/sign-vault02.rdu3.fedoraproject.org +++ b/inventory/host_vars/sign-vault02.rdu3.fedoraproject.org @@ -8,7 +8,7 @@ dns2: 10.16.163.34 dns_search1: "rdu3.fedoraproject.org" dns_search2: "fedoraproject.org" has_ipv4: yes -mac0: c4:cb:e1:e1:56:1c +mac0: c4:cb:e1:e1:56:1c mac1: c4:cb:e1:e1:56:1d mac2: c4:70:bd:c8:cf:cc mac3: c4:70:bd:c8:cf:cd @@ -52,3 +52,13 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2028-10-12" + date_hw_purchase: "2024-01-31" + hardware: PowerEdge R450 + location: RDU3 + serialno_a: G922FZ3 + type: Prod_Dedicated_HW + vendor: Dell diff --git a/inventory/host_vars/vmhost-p09-copr01.rdu3.fedoraproject.org b/inventory/host_vars/vmhost-p09-copr01.rdu3.fedoraproject.org index b259531754..b91c505c99 100644 --- a/inventory/host_vars/vmhost-p09-copr01.rdu3.fedoraproject.org +++ b/inventory/host_vars/vmhost-p09-copr01.rdu3.fedoraproject.org @@ -19,8 +19,9 @@ mac3: "b8:ce:f6:c6:00:c7" mac4: "b8:ce:f6:c6:00:d0" mac5: "b8:ce:f6:c6:00:d1" libvirt_host: "[{{ br0_ipv6_ip }}]" -libvirt_pool: vmhost_p09_01 -libvirt_pool_order_id: 4 +libvirt_pools: + - name: vmhost_p09_01 + - name: vmhost_p_p09_01 libvirt_arch: ppc64le network_connections: # Bridge profile @@ -65,3 +66,13 @@ network_connections: state: up mtu: 1500 zabbix_host: zabbix01.vpn.fedoraproject.org +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2027-01-31" + date_hw_purchase: "2021-09-29" + hardware: Power 9 9183-22X + location: RDU3 + serialno_a: 780484A + type: #bvmhost-p09-01.stg.rdu3.fedoraproject.org - Other + vendor: IBM diff --git a/inventory/host_vars/vmhost-p09-copr02.rdu3.fedoraproject.org b/inventory/host_vars/vmhost-p09-copr02.rdu3.fedoraproject.org index db9ea516b5..f467fdbceb 100644 --- a/inventory/host_vars/vmhost-p09-copr02.rdu3.fedoraproject.org +++ b/inventory/host_vars/vmhost-p09-copr02.rdu3.fedoraproject.org @@ -22,8 +22,9 @@ mac7: ac:1f:6b:a5:4e:f5 mac8: ac:1f:6b:a5:4e:f6 mac9: ac:1f:6b:a5:4e:f7 libvirt_host: "[{{ br0_ipv6_ip }}]" -libvirt_pool: vmhost_p09_02 -libvirt_pool_order_id: 0 +libvirt_pools: + - name: vmhost_p09_02 + - name: vmhost_p_p09_02 libvirt_arch: ppc64le network_connections: # Bridge profile diff --git a/inventory/host_vars/vmhost-p09-copr03.rdu3.fedoraproject.org b/inventory/host_vars/vmhost-p09-copr03.rdu3.fedoraproject.org index 4f7121d8b6..2024f6bf78 100644 --- a/inventory/host_vars/vmhost-p09-copr03.rdu3.fedoraproject.org +++ b/inventory/host_vars/vmhost-p09-copr03.rdu3.fedoraproject.org @@ -22,8 +22,9 @@ mac7: ac:1f:6b:8a:9a:31 mac8: ac:1f:6b:8a:9a:32 mac9: ac:1f:6b:8a:9a:33 libvirt_host: "[{{ br0_ipv6_ip }}]" -libvirt_pool: vmhost_p09_03 -libvirt_pool_order_id: 1 +libvirt_pools: + - name: vmhost_p09_03 + - name: vmhost_p_p09_03 libvirt_arch: ppc64le network_connections: # Bridge profile diff --git a/inventory/host_vars/vmhost-p09-copr04.rdu3.fedoraproject.org b/inventory/host_vars/vmhost-p09-copr04.rdu3.fedoraproject.org index fe80863b88..f9d014ccb1 100644 --- a/inventory/host_vars/vmhost-p09-copr04.rdu3.fedoraproject.org +++ b/inventory/host_vars/vmhost-p09-copr04.rdu3.fedoraproject.org @@ -22,8 +22,9 @@ mac7: ac:1f:6b:a4:e3:b1 mac8: ac:1f:6b:a4:e3:b2 mac9: ac:1f:6b:a4:e3:b3 libvirt_host: "[{{ br0_ipv6_ip }}]" -libvirt_pool: vmhost_p09_04 -libvirt_pool_order_id: 2 +libvirt_pools: + - name: vmhost_p09_04 + - name: vmhost_p_p09_04 libvirt_arch: ppc64le network_connections: # Bridge profile diff --git a/inventory/host_vars/vmhost-x86-01.stg.rdu3.fedoraproject.org b/inventory/host_vars/vmhost-x86-01.stg.rdu3.fedoraproject.org index 6a2ec57698..168662203e 100644 --- a/inventory/host_vars/vmhost-x86-01.stg.rdu3.fedoraproject.org +++ b/inventory/host_vars/vmhost-x86-01.stg.rdu3.fedoraproject.org @@ -58,3 +58,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2029-02-17" + date_hw_purchase: "2024-04-15" + hardware: PowerEdge R650 + location: RDU3 + oob_ip: 10.3.160.057 + serialno_a: B61SW04 + type: Staging_Vmserver + vendor: Dell diff --git a/inventory/host_vars/vmhost-x86-02.stg.rdu3.fedoraproject.org b/inventory/host_vars/vmhost-x86-02.stg.rdu3.fedoraproject.org index 2ab13e2b6f..b663972f98 100644 --- a/inventory/host_vars/vmhost-x86-02.stg.rdu3.fedoraproject.org +++ b/inventory/host_vars/vmhost-x86-02.stg.rdu3.fedoraproject.org @@ -58,3 +58,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2023-04-29" + date_hw_purchase: "2016-04-28" + hardware: PowerEdge R630 + location: RDU3 + oob_ip: 10.3.160.033 + serialno_a: 8FVCGB2 + type: Staging_Vmserver + vendor: Dell diff --git a/inventory/host_vars/vmhost-x86-05.stg.rdu3.fedoraproject.org b/inventory/host_vars/vmhost-x86-05.stg.rdu3.fedoraproject.org index 8100f2edd0..5c4b726f28 100644 --- a/inventory/host_vars/vmhost-x86-05.stg.rdu3.fedoraproject.org +++ b/inventory/host_vars/vmhost-x86-05.stg.rdu3.fedoraproject.org @@ -58,3 +58,14 @@ network_connections: controller: bond0 state: up mtu: 1500 +# This is used to populate the inventory fields, only specific keys are allowed, see +# https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory +zabbix_inventory: + date_hw_expiry: "2025-01-01" + date_hw_purchase: "2017-02-20" + hardware: PowerEdge R630 + location: RDU3 + oob_ip: 10.3.160.036 + serialno_a: 8D3NCH2 + type: Staging_Vmserver + vendor: Dell diff --git a/inventory/host_vars/vmhost-x86-copr01.rdu3.fedoraproject.org b/inventory/host_vars/vmhost-x86-copr01.rdu3.fedoraproject.org index 50e76b5346..0ef30ea3ea 100644 --- a/inventory/host_vars/vmhost-x86-copr01.rdu3.fedoraproject.org +++ b/inventory/host_vars/vmhost-x86-copr01.rdu3.fedoraproject.org @@ -16,12 +16,12 @@ mac2: 84:16:0c:bc:3a:60 mac3: b4:96:91:63:3b:e8 mac4: 84:16:0c:bc:3a:60 mac5: b4:96:91:63:3b:e9 -mac6: b4:96:91:63:3b:ea +mac6: b4:96:91:63:3b:ea mac7: b4:96:91:63:3b:eb mac8: f4:02:70:d3:15:95 libvirt_host: "[{{ br0_ipv6_ip }}]" -libvirt_pool: vmhost_x86_01 -libvirt_pool_order_id: 7 +libvirt_pools: + - name: vmhost_x86_01 libvirt_arch: x86_64 network_connections: # Bridge profile @@ -69,9 +69,12 @@ zabbix_host: zabbix01.vpn.fedoraproject.org # This is used to populate the inventory fields, only specific keys are allowed, see # https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory zabbix_inventory: + date_hw_expiry: "2026-12-31" + date_hw_purchase: "2020-01-10" hardware: PowerEdge R6525 location: RDU3 oob_ip: 10.16.160.23 serialno_a: BM4LS13 site_rack: W34 U18 + type: #autosign01.rdu3.fedoraproject.org vendor: Dell diff --git a/inventory/host_vars/vmhost-x86-copr02.rdu3.fedoraproject.org b/inventory/host_vars/vmhost-x86-copr02.rdu3.fedoraproject.org index 49e3560f72..854d6ce8a1 100644 --- a/inventory/host_vars/vmhost-x86-copr02.rdu3.fedoraproject.org +++ b/inventory/host_vars/vmhost-x86-copr02.rdu3.fedoraproject.org @@ -19,8 +19,8 @@ mac5: 84:16:0c:bc:24:e0 mac6: b4:96:91:63:3b:9e mac7: b4:96:91:63:3b:9f libvirt_host: "[{{ br0_ipv6_ip }}]" -libvirt_pool: vmhost_x86_02 -libvirt_pool_order_id: 8 +libvirt_pools: + - name: vmhost_x86_02 libvirt_arch: x86_64 network_connections: # Bridge profile @@ -68,9 +68,12 @@ zabbix_host: zabbix01.vpn.fedoraproject.org # This is used to populate the inventory fields, only specific keys are allowed, see # https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory zabbix_inventory: + date_hw_expiry: "2026-12-31" + date_hw_purchase: "2020-01-10" hardware: PowerEdge R6525 location: RDU3 oob_ip: 10.16.160.24 serialno_a: BM4MS13 site_rack: W34 U19 + type: #autosign01.rdu3.fedoraproject.org vendor: Dell diff --git a/inventory/host_vars/vmhost-x86-copr03.rdu3.fedoraproject.org b/inventory/host_vars/vmhost-x86-copr03.rdu3.fedoraproject.org index 994869794f..006e5a6bfb 100644 --- a/inventory/host_vars/vmhost-x86-copr03.rdu3.fedoraproject.org +++ b/inventory/host_vars/vmhost-x86-copr03.rdu3.fedoraproject.org @@ -19,8 +19,8 @@ mac5: "b4:96:91:63:3b:51" mac6: "b4:96:91:63:3b:52" mac7: "b4:96:91:63:3b:53" libvirt_host: "[{{ br0_ipv6_ip }}]" -libvirt_pool: vmhost_x86_03 -libvirt_pool_order_id: 9 +libvirt_pools: + - name: vmhost_x86_03 libvirt_arch: x86_64 network_connections: # Bridge profile @@ -68,9 +68,12 @@ zabbix_host: zabbix01.vpn.fedoraproject.org # This is used to populate the inventory fields, only specific keys are allowed, see # https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory zabbix_inventory: + date_hw_expiry: "2026-12-31" + date_hw_purchase: "2020-01-10" hardware: PowerEdge R6525 location: RDU3 oob_ip: 10.16.160.25 serialno_a: BM4KS13 site_rack: W34 U20 + type: #autosign01.rdu3.fedoraproject.org vendor: Dell diff --git a/inventory/host_vars/vmhost-x86-copr04.rdu3.fedoraproject.org b/inventory/host_vars/vmhost-x86-copr04.rdu3.fedoraproject.org index 10c1471940..a685227e2f 100644 --- a/inventory/host_vars/vmhost-x86-copr04.rdu3.fedoraproject.org +++ b/inventory/host_vars/vmhost-x86-copr04.rdu3.fedoraproject.org @@ -19,8 +19,8 @@ mac5: "b4:96:91:63:3a:a1" mac6: "b4:96:91:63:3a:a2" mac7: "b4:96:91:63:3a:a3" libvirt_host: "[{{ br0_ipv6_ip }}]" -libvirt_pool: vmhost_x86_04 -libvirt_pool_order_id: 3 +libvirt_pools: + - name: vmhost_x86_04 libvirt_arch: x86_64 network_connections: # Bridge profile @@ -68,9 +68,12 @@ zabbix_host: zabbix01.vpn.fedoraproject.org # This is used to populate the inventory fields, only specific keys are allowed, see # https://www.zabbix.com/documentation/current/en/manual/api/reference/host/object#host-inventory zabbix_inventory: + date_hw_expiry: "2026-12-31" + date_hw_purchase: "2020-01-10" hardware: PowerEdge R6525 location: RDU3 oob_ip: 10.16.160.26 serialno_a: BM3RS13 site_rack: BB34 U18 + type: #autosign01.rdu3.fedoraproject.org vendor: Dell diff --git a/inventory/inventory b/inventory/inventory index 57d83019c5..b5f4b46ed1 100644 --- a/inventory/inventory +++ b/inventory/inventory @@ -249,6 +249,9 @@ download_ibiblio download_iso_rdu3 download_rdu3 +[gpu] +gpu01.rdu3.fedoraproject.org + [kernel_qa] kernel02.rdu3.fedoraproject.org diff --git a/playbooks/check-etc.yml b/playbooks/check-etc.yml index 899b06d924..506a6b8ae0 100644 --- a/playbooks/check-etc.yml +++ b/playbooks/check-etc.yml @@ -8,18 +8,37 @@ vars: # Add files here for things that store data in /etc/ known_prefixes: + - /etc/builder-keytabs + - /etc/containers/systemd/ipatuura.container - /etc/dnf/modules.d + - /etc/dirsrv/slapd-FEDORAPROJECT-ORG + - /etc/dirsrv/slapd-STG-FEDORAPROJECT-ORG + - /etc/gconf/gconf.xml.defaults + - /etc/ipa-tuura + - /etc/ipatuura-container-config + - /etc/java/java-11-openjdk # rpm:*.aarch64/lib/security/blacklisted.certs + - /etc/letsencrypt/accounts + - /etc/letsencrypt/archive + - /etc/letsencrypt/csr + - /etc/letsencrypt/keys + - /etc/letsencrypt/live # ? + - /etc/letsencrypt/renewal # ? - /etc/lvm/archive - /etc/lvm/backup - /etc/lvm/devices/backup - - /etc/dirsrv/slapd-STG-FEDORAPROJECT-ORG - /etc/libvirt/storage - /etc/libvirt/qemu + - /etc/mailman3/__pycache__ + - /etc/mock/koji - /etc/NetworkManager/system-connections + - /etc/nagios # Getting rid of nagios and there are 666 files. + - /etc/nrpe.d # Getting rid of nagios and there are 666 files. + - /etc/openshift_apps # FIXME: ? + - /etc/openvpn/server/ccd # ? + - /etc/openvpn/server/ccd.bad + - /etc/pki/ca-trust/extracted/pem/directory-hash # ? lots of files. + - /etc/pki/pki-tomcat/ca/archives - /etc/udev/rules.d - - /etc/ipatuura-container-config - - /etc/containers/systemd/ipatuura.container - - /etc/ipa-tuura # Add files here that you know are safe but aren't from an RPM known_filenames: @@ -30,21 +49,118 @@ # Email aliases files: - /etc/aliases.db - /etc/aliases.lmdb + - /etc/aliases.static # Ansible files: - /etc/ansible/facts.d/install_date.fact + # Ansubis files: + - /etc/anubis/policies.yaml + + # AWstats files: + - /etc/awstats/awstats.log01.rdu3.fedoraproject.org.conf + + # Bodhi files: + - /etc/bodhi/celeryconfig.py + - /etc/bodhi/createrepo_c.ini + - /etc/bodhi/logging.yaml + - /etc/bodhi/pungi_general.conf + - /etc/bodhi/pungi_multilib.conf + # Dracut files: + - /etc/dracut.conf.d/local.conf - /etc/dracut.conf.d/nbde_client.conf + - /etc/dracut.conf.d/sgdisk.conf + - /etc/dracut.conf.d/xen.conf # CollectD files: + - /etc/collectd.d/bind.conf + - /etc/collectd.d/fmn.conf + - /etc/collectd.d/memcached.conf - /etc/collectd.d/network.conf - /etc/collectd.d/nfs.conf + - /etc/collectd.d/postgres.conf + - /etc/collectd.d/unixsock.conf + - /etc/collectd.d/vfive-upgrade.conf # Our crond conf files: + - /etc/cron.daily/cleanup-stage-users - /etc/cron.daily/data-only-backup.sh - /etc/cron.daily/freshclam + - /etc/cron.daily/grab-daily-logs + - /etc/cron.daily/rotatelogs-cleanup. + - /etc/cron.daily/sync-http-logs-and-merge.sh + # prod, roughly + - /etc/cron.d/ansible-make-git-checkout-seed + - /etc/cron.d/bodhi-automated-pushes + - /etc/cron.d/branched + - /etc/cron.d/bz-review-report.cron + - /etc/cron.d/cgit-clean-lock.cron + - /etc/cron.d/check-broken-planet.cron + - /etc/cron.d/cloud-image-stat.cron + - /etc/cron.d/cloud-updates + - /etc/cron.d/compress-log.cron + - /etc/cron.d/condense-mirrorlogs.cron + - /etc/cron.d/container-updates + - /etc/cron.d/countme-update.cron + - /etc/cron.d/cron-backup-anitya-public + - /etc/cron.d/cron-backup-database-anitya + - /etc/cron.d/cron-backup-database-blockerbugs + - /etc/cron.d/cron-backup-database-bodhi2 + - /etc/cron.d/cron-backup-database-datanommer2 + - /etc/cron.d/cron-backup-database-elections + - /etc/cron.d/cron-backup-database-fedocal + - /etc/cron.d/cron-backup-database-fpo-mediawiki + - /etc/cron.d/cron-backup-database-hyperkitty + - /etc/cron.d/cron-backup-database-ipsilon + - /etc/cron.d/cron-backup-database-kerneltest + - /etc/cron.d/cron-backup-database-koji + - /etc/cron.d/cron-backup-database-koschei + - /etc/cron.d/cron-backup-database-mailman + - /etc/cron.d/cron-backup-database-mirrormanager2 + - /etc/cron.d/cron-backup-database-notifications + - /etc/cron.d/cron-backup-database-openqa + - /etc/cron.d/cron-backup-database-openqa-stg + - /etc/cron.d/cron-backup-database-pagure + - /etc/cron.d/cron-backup-database-postgres + - /etc/cron.d/cron-backup-database-resultsdb + - /etc/cron.d/cron-backup-database-tahrir + - /etc/cron.d/cron-backup-database-testdays + - /etc/cron.d/cron-backup-database-testdays_resultsdb + - /etc/cron.d/cron-backup-database-transtats + - /etc/cron.d/cron-backup-database-waiverdb + - /etc/cron.d/cron-backup-database-webhook2fedmsg + - /etc/cron.d/cron-backup-database-zezere + - /etc/cron.d/cron-docs-translation-update + - /etc/cron.d/cron-weblate-backup + - /etc/cron.d/directory-sizes-update + - /etc/cron.d/download-sync + - /etc/cron.d/eln + - /etc/cron.d/f42-bootc + - /etc/cron.d/f43-compose + - /etc/cron.d/f44-compose + - /etc/cron.d/fasjson-aliases + - /etc/cron.d/grokfsck.cron + - /etc/cron.d/grokmirror.cron + - /etc/cron.d/koji-directory-cleanup + - /etc/cron.d/koji-gc + - /etc/cron.d/koji-prune-signed-copies + - /etc/cron.d/koji-sidetag-cleanup + - /etc/cron.d/linuxsystemroles-logs-clean + - /etc/cron.d/make-people-git + - /etc/cron.d/make-people-page.cron + - /etc/cron.d/package-owner-aliases + - /etc/cron.d/rawhide + - /etc/cron.d/rawhide-compose + - /etc/cron.d/run-rdiff-backups + - /etc/cron.d/sig_policy + - /etc/cron.d/torrent-hash.cron + - /etc/cron.d/torrent-web-generate + - /etc/cron.d/updates-sync + - /etc/cron.d/update-fullfiletimelist + - /etc/cron.d/update-koji-owner + # stg, roughly - /etc/cron.d/ansible-check-update-hooks - /etc/cron.d/ansible-clamscan - /etc/cron.d/budget-sync @@ -76,6 +192,11 @@ - /etc/cron.d/sync-mirrors - /etc/cron.d/sync-start + - /etc/cron.d/cron-backup-database-fas2 + - /etc/cron.d/sa-update + + - /etc/cron.weekly/ftbfs.cron + # dconf files: - /etc/dconf/db/distro - /etc/dconf/db/local @@ -84,19 +205,34 @@ # dnf plugin files: - /etc/dnf/libdnf5-plugins/actions.d/mod_wsgi.actions + # docker! files: + - /etc/docker/certs.d/registry.fedoraproject.org/client.cert + - /etc/docker/certs.d/registry.fedoraproject.org/client.key + - /etc/docker/certs.d/registry.stg.fedoraproject.org/client.cert + - /etc/docker/certs.d/registry.stg.fedoraproject.org/client.key + - /etc/fedora-gather-easyfix/template.html # fedora-messaging files: - /etc/fedora-messaging/config.toml + - /etc/fedora-messaging/faf/ca.crt # These should be under /etc/pki? + - /etc/fedora-messaging/faf/faf.crt + - /etc/fedora-messaging/faf/faf.key - /etc/fedora-messaging/git-hooks-messaging.toml + - /etc/fedora-messaging/koji_sync_listener.toml - /etc/fedora-messaging/ursabot.toml + - /etc/fedora-messaging/zodbot.toml + + # ftbfs files: + - /etc/ftbfs.cfg # gitconfig files: - /etc/gitconfig - /etc/gssproxy/10-ipa.conf - # haproxy files: + # HAproxy files: + - /etc/haproxy/503.http - /etc/haproxy/ipa.pem - /etc/haproxy/ocp-stg.pem - /etc/haproxy/ocp-stg-rdu3.pem @@ -899,27 +1035,26 @@ - /etc/httpd/conf.d/zabbix.stg.fedoraproject.org/securityheaders.conf - /etc/httpd/conf.d/zabbix.stg.fedoraproject.org/zabbix.conf + - /etc/httpd/conf.d/compose.conf - /etc/httpd/conf.d/fp.conf + - /etc/httpd/conf.d/freemedia-app.conf + - /etc/httpd/conf.d/geoip-city-wsgi.conf + - /etc/httpd/conf.d/ipa-tuura.conf - /etc/httpd/conf.d/infrastructure.fedoraproject.org.conf + - /etc/httpd/conf.d/kojiweb-stg.conf + - /etc/httpd/conf.d/mailmanweb.conf - /etc/httpd/conf.d/meetbot.conf - /etc/httpd/conf.d/nss.conf + - /etc/httpd/conf.d/pager-app.conf - /etc/httpd/conf.d/referer-override.conf + - /etc/httpd/conf.d/rel-eng.conf + - /etc/httpd/conf.d/repo.conf - /etc/httpd/conf.d/testproxy.conf + - /etc/httpd/conf.d/wsgi.conf - /etc/httpd/conf/cacert.pem - /etc/httpd/conf/pkgs.fedoraproject.org_key_and_cert.pem - - /etc/httpd/conf.d/freemedia-app.conf - - /etc/httpd/conf.d/geoip-city-wsgi.conf - - /etc/httpd/conf.d/pager-app.conf - - /etc/httpd/conf.d/wsgi.conf - - - /etc/httpd/conf.d/kojiweb-stg.conf - - /etc/httpd/conf.d/rel-eng.conf - - /etc/httpd/conf.d/repo.conf - - - /etc/httpd/conf.d/ipa-tuura.conf - - /etc/httpd/ticketkey_staging.tkey # IPA files: @@ -928,89 +1063,120 @@ - /etc/ipa/custodia/custodia.conf - /etc/ipa/custodia/server.keys + # ipsilon files: + - /etc/ipsilon/root/configuration.conf + - /etc/ipsilon/root/idp.conf + - /etc/ipsilon/root/install_changes + - /etc/ipsilon/root/ipsilon.conf + - /etc/ipsilon/root/openidc.key + - /etc/ipsilon/root/openidc.static.cfg + - /etc/ipsilon/root/saml2/idp.crt + - /etc/ipsilon/root/saml2/idp.key + - /etc/ipsilon/root/saml2/metadata.xml + # Kerberos keytab files: - /etc/dirsrv/ds.keytab - - /etc/httpd.keytab + - /etc/httpd.keytab # Move to below? + - /etc/httpd/conf/http.keytab - /etc/kojid/kojid.keytab + - /etc/koji-hub/koji-hub.keytab - /etc/krb5.keytab + - /etc/krb5.releng.keytab - /etc/pkgs.keytab + - /etc/krb5.HTTP_admin.fedoraproject.org.keytab + - /etc/krb5.HTTP_id.fedoraproject.org.keytab + - /etc/krb5.HTTP_id.fedoraproject.org.keytab.combined + - /etc/krb5.HTTP_koji.fedoraproject.org.keytab + - /etc/krb5.HTTP_nagios.fedoraproject.org.keytab + - /etc/krb5.HTTP_nagios-external.fedoraproject.org.keytab + - /etc/krb5.HTTP_riscv-koji.fedoraproject.org.keytab + - /etc/krb5.bodhi_bodhi.fedoraproject.org.keytab + - /etc/krb5.compose_koji.fedoraproject.org.keytab + - /etc/krb5.compose_riscv-koji.fedoraproject.org.keytab + - /etc/krb5.kojira_koji.fedoraproject.org.keytab + - /etc/krb5.kojira_koji.stg.fedoraproject.org.keytab + - /etc/krb5.kojira_riscv-koji.fedoraproject.org.keytab + - /etc/krb5.koji-gc_koji.fedoraproject.org.keytab + - /etc/krb5.koji-gc_koji.stg.fedoraproject.org.keytab + - /etc/krb5.koji-gc_riscv-koji.fedoraproject.org.keytab + - /etc/krb5.mash_koji.fedoraproject.org.keytab + - /etc/krb5.mash_koji.stg.fedoraproject.org.keytab + - /etc/krb5.monitoring_ipa01.rdu3.fedoraproject.org.keytab + - /etc/krb5.stage-users_ipa01.rdu3.fedoraproject.org.keytab + - /etc/krb5.zodbot_value01.rdu3.fedoraproject.org.keytab - /etc/krb5.HTTP_id.stg.fedoraproject.org.keytab - /etc/krb5.HTTP_id.stg.fedoraproject.org.keytab.combined + - /etc/krb5.HTTP_ipatuura01.stg.rdu3.fedoraproject.org.keytab + - /etc/krb5.HTTP_koji.stg.fedoraproject.org.keytab + - /etc/krb5.bodhi_bodhi.stg.fedoraproject.org.keytab + - /etc/krb5.compose_compose-x86-01.stg.rdu3.fedoraproject.org.keytab + - /etc/krb5.compose_koji.stg.fedoraproject.org.keytab - /etc/krb5.monitoring_ipa01.stg.rdu3.fedoraproject.org.keytab - /etc/krb5.stage-users_ipa01.stg.rdu3.fedoraproject.org.keytab - /etc/krb5.ursabot_value01.stg.rdu3.fedoraproject.org.keytab # Kerberos files: - /etc/krb5.conf.d/freeipa + - /etc/krb5.conf.d/freeipa-realm - /etc/krb5.conf.d/freeipa-server # Kernel/init files: - /etc/kernel/cmdline + - /etc/modprobe.d/blacklist-nouveau.conf + - /etc/modprobe.d/disable-cdc_ether.conf + - /etc/modprobe.d/i40e.conf + - /etc/modprobe.d/kvm_intel.conf + - /etc/modules-load.d/nf_conntrack.conf # KOJI files: - - /etc/kojid/plugins/flatpak.conf - /etc/kojira/extras_cacert.pem - /etc/kojira/kojira_cert_key.pem - /etc/koji-hub/gssapi.keytab - /etc/koji-osbuild/builder.conf + - /etc/koji.conf.d/bodhi.conf + - /etc/koji.conf.d/compose.conf + - /etc/kojid/plugins/flatpak.conf - # FIXME: ? + # FIXME: Seem sus bad name files: - /etc/pki/tls/certs/extras_cacert.pem - /etc/pki/tls/certs/extras_upload_cacert.pem - /etc/pki/tls/certs/localhost.crt - /etc/pki/tls/certs/upload_cacert.pem - /etc/pki/tls/private/localhost.key - # logrotate files: + - /etc/logrotate.d/bittorrent + - /etc/logrotate.d/merged-rsyslog - /etc/logrotate.d/mirrormanager + - /etc/logrotate.d/rsync-fedora - /etc/logrotate.d/rsyslog + - /etc/logrotate.d/spamassassin + - /etc/logrotate.d/syslog # LVM files: - /etc/lvm/devices/system.devices + # Spam files: + - /etc/mail/spamassassin/sa-update-keys/pubring.kbx + - /etc/mail/spamassassin/sa-update-keys/trustdb.gpg + + # Mailman files: + - /etc/mailman3/django_fedora_nosignup.py + - /etc/mailman3/gunicorn.conf.py + - /etc/mailman3/initial-data.json + - /etc/mailman3/settings_admin.py + - /etc/mailman3/urls.py + # MDADM files: - /etc/mdadm.conf - - /etc/modprobe.d/kvm_intel.conf + # Named files: + - /etc/named/zones.conf # NF Tables files: - /etc/nftables/fedora-infra-ipv4.nft - /etc/nftables/fedora-infra-ipv6.nft - # Nagios files: - - /etc/nrpe.d/check_basset.cfg - - /etc/nrpe.d/check_celery_redis_queue.cfg - - /etc/nrpe.d/check_countme.cfg - - /etc/nrpe.d/check_cron.cfg - - /etc/nrpe.d/check_datanommer_history.cfg - - /etc/nrpe.d/check_disk.cfg - - /etc/nrpe.d/check_fedmsg_composer_proc.cfg - - /etc/nrpe.d/check_fedmsg_consumers.cfg - - /etc/nrpe.d/check_fedmsg_gateway_proc.cfg - - /etc/nrpe.d/check_fedmsg_hub_proc.cfg - - /etc/nrpe.d/check_fedmsg_irc_proc.cfg - - /etc/nrpe.d/check_fedmsg_relay_proc.cfg - - /etc/nrpe.d/check_fmn.cfg - - /etc/nrpe.d/check_happroxy_conns.cfg - - /etc/nrpe.d/check_ipa.cfg - - /etc/nrpe.d/check_lock.cfg - - /etc/nrpe.d/check_lock_file_age.cfg - - /etc/nrpe.d/check_memcache.cfg - - /etc/nrpe.d/check_mirrorlist_cache.cfg - - /etc/nrpe.d/check_mirrorlist_docker_proxy.cfg - - /etc/nrpe.d/check_postfix_queue.cfg - - /etc/nrpe.d/check_postfix_redhat.cfg - - /etc/nrpe.d/check_proxies.cfg - - /etc/nrpe.d/check_raid.cfg - - /etc/nrpe.d/check_readonly_fs.cfg - - /etc/nrpe.d/check_redis_proc.cfg - - /etc/nrpe.d/check_rsyslogd_proc.cfg - - /etc/nrpe.d/check_swap.cfg - - /etc/nrpe.d/check_testcloud.cfg - - /etc/nrpe.d/check_websites_buildtime.cfg - - /etc/nrpe.d/check_varnish_proc.cfg - # NVME files: - /etc/nvme/hostid - /etc/nvme/hostnqn @@ -1022,24 +1188,32 @@ # OpenVPN files: - /etc/openvpn/client/ca.crt + - /etc/openvpn/client/client.crt + - /etc/openvpn/client/client.key - /etc/openvpn/client/openvpn.conf - /etc/openvpn/fix-routes.sh - /etc/openvpn/server/ca.crt + # PAM files: + - /etc/pam.d/mock + # Pagure files: - /etc/pagure/client_secrets.json - /etc/pagure/pagure_hook.cfg - /etc/pagure/pagure_plugins.cfg # PKI files: - - /etc/pki/ca-trust/extracted/pem/directory-hash/STG.FEDORAPROJECT.ORG_IPA_CA.pem - - /etc/pki/fedora-messaging/cacert.pem - - /etc/pki/fedora-messaging/mediawiki.stg-cert.pem - - /etc/pki/fedora-messaging/mediawiki.stg-key.pem + - /etc/pki/fedora-messaging/bodhi-cert.pem + - /etc/pki/fedora-messaging/bodhi-key.pem - /etc/pki/fedora-messaging/ca.crt + - /etc/pki/fedora-messaging/cacert.pem - /etc/pki/fedora-messaging/ipa.stg.crt - /etc/pki/fedora-messaging/ipa.stg.key + - /etc/pki/fedora-messaging/mediawiki.stg-cert.pem + - /etc/pki/fedora-messaging/mediawiki.stg-key.pem - /etc/pki/fedora-messaging/rabbitmq-ca.crt + - /etc/pki/fedora-messaging/rabbitmq-pungi.crt + - /etc/pki/fedora-messaging/rabbitmq-pungi.key - /etc/pki/fedora-messaging/ursabot.crt - /etc/pki/fedora-messaging/ursabot.key @@ -1269,13 +1443,21 @@ - /etc/pki/tls/certs/wildcard-2025.id.fedoraproject.org.intermediate.cert - /etc/pki/tls/certs/wildcard-2025.id.stg.fedoraproject.org.cert - /etc/pki/tls/certs/wildcard-2025.id.stg.fedoraproject.org.intermediate.cert + - /etc/pki/tls/certs/wildcard-2025.fedorapeople.org.cert + - /etc/pki/tls/certs/wildcard-2025.fedorapeople.org.intermediate.cert - /etc/pki/tls/certs/wildcard-2025.stg.fedoraproject.org.cert - /etc/pki/tls/certs/wildcard-2025.stg.fedoraproject.org.intermediate.cert - /etc/pki/tls/certs/wildcard-2026.stg.fedoraproject.org.cert - /etc/pki/tls/certs/wildcard-2026.stg.fedoraproject.org.intermediate.cert + - /etc/pki/tls/private/br.fedoracommunity.org.key - /etc/pki/tls/private/coreos.stg.fedoraproject.org.key - /etc/pki/tls/private/epel.io.key - /etc/pki/tls/private/fedora.im.key + - /etc/pki/tls/private/fedoracommunity.org.key + - /etc/pki/tls/private/fedorahosted.org.key + - /etc/pki/tls/private/fedoramagazine.org.key + - /etc/pki/tls/private/flocktofedora.org.key + - /etc/pki/tls/private/fpaste.org.key - /etc/pki/tls/private/fedoraloveskde.org.key - /etc/pki/tls/private/fedoraplanet.org.key - /etc/pki/tls/private/getfedora.org.key @@ -1296,6 +1478,7 @@ - /etc/pki/tls/private/wildcard-2025.apps.ocp-rdu3.fedoraproject.org.key - /etc/pki/tls/private/wildcard-2025.apps.ocp-rdu3.stg.fedoraproject.org.key - /etc/pki/tls/private/wildcard-2025.apps.ocp.stg.fedoraproject.org.key + - /etc/pki/tls/private/wildcard-2025.fedorapeople.org.key - /etc/pki/tls/private/wildcard-2025.fedoraproject.org.key - /etc/pki/tls/private/wildcard-2025.id.fedoraproject.org.key - /etc/pki/tls/private/wildcard-2025.id.stg.fedoraproject.org.key @@ -1308,15 +1491,38 @@ - /etc/pki/rabbitmq/kojicert/koji.ca - /etc/pki/rabbitmq/kojicert/koji.crt - /etc/pki/rabbitmq/kojicert/koji.key + - /etc/pki/rabbitmq/mailman/mailman.ca + - /etc/pki/rabbitmq/mailman/mailman.crt + - /etc/pki/rabbitmq/mailman/mailman.key - /etc/pki/rabbitmq/pagurecert/src.fp.o.ca - /etc/pki/rabbitmq/pagurecert/src.fp.o.crt - /etc/pki/rabbitmq/pagurecert/src.fp.o.key + - /etc/pki/releng # ? + - /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Messaging + - /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-9 + # Profile files: + - /etc/profile.d/cudapath.sh + - /etc/profile.d/externalcaches.sh + - /etc/profile.d/history_off.sh + - /etc/profile.d/models.sh + - /etc/profile.d/setprodps1.sh - /etc/profile.d/setprodiad2ps1.sh # FIXME ? - /etc/profile.d/setprodrdu3ps1.sh - /etc/profile.d/setstgps1.sh + # RabbitMQ files: + - /etc/rabbitmq/ca.crt + - /etc/rabbitmq/enabled_plugins + - /etc/rabbitmq/inter_node_tls.config + - /etc/rabbitmq/nodecert.combined.pem + - /etc/rabbitmq/nodecert/node.crt + - /etc/rabbitmq/nodecert/node.key + - /etc/rabbitmq/pubsub_federation/client_cert.pem + - /etc/rabbitmq/pubsub_federation/client_key.pem + - /etc/rabbitmq/rabbitmq-env.conf + # Resolve files: - /etc/resolv.conf @@ -1326,10 +1532,17 @@ - /etc/rsyslog.d/rsyslog-audit.conf - /etc/rsyslog.d/rsyslog-disablerate.conf - /etc/rsyslog.d/rsyslog-imjournal-limits.conf + - /etc/rsyslog.d/rsyslog-limits.conf - /etc/rsyslog.d/rsyslog-log01.conf # SSH files: - /etc/ssh/sshd_config.d/04-ipa.conf + - /etc/ssh/sshd_config.d/50-cloud-init.conf + - /etc/ssh/ssh_config.d/04-ipa.conf + - /etc/ssh/ssh_config.d/99-x11.conf + - /etc/ssh/ssh_host_dsa_key + - /etc/ssh/ssh_host_dsa_key.pub + - /etc/ssh/ssh_host_dsa_key-cert.pub - /etc/ssh/ssh_host_ecdsa_key - /etc/ssh/ssh_host_ecdsa_key.pub - /etc/ssh/ssh_host_ed25519_key @@ -1345,13 +1558,26 @@ - /etc/sssd/conf.d/fedora-nss-ignore.conf # sudoers files: + - /etc/sudoers.d/01-sysadmin-main + - /etc/sudoers.d/90-cloud-init-users + - /etc/sudoers.d/arm-packager-sudoers + - /etc/sudoers.d/arm-retrace-sudoers + - /etc/sudoers.d/default - /etc/sudoers.d/norequiretty + - /etc/sudoers.d/pkgs01_rdu3_fedoraproject_org-sudoers # Sysconfig files: - /etc/sysconfig/anaconda - /etc/sysconfig/authconfig + - /etc/sysconfig/bittorrent + - /etc/sysconfig/bootloader + - /etc/sysconfig/firstboot + - /etc/sysconfig/global-update-applied # FIXME: ? + - /etc/sysconfig/freshclam + - /etc/sysconfig/ipv4-br-aggregated.zone + - /etc/sysconfig/ipv6-br.zone - /etc/sysconfig/ipv4-cu-aggregated.zone - /etc/sysconfig/ipv4-ir-aggregated.zone - /etc/sysconfig/ipv4-kp-aggregated.zone @@ -1374,37 +1600,95 @@ - /etc/sysconfig/sshd-permitrootlogin + # Sysctl.d files: + - /etc/sysctl.d/10-tcp-socket-buffers.conf + # SystemD files: + - /etc/systemd/system/anubis.service + - /etc/systemd/system/bodhi-celery.service - /etc/systemd/system/btrfs-balance.timer.d/schedule.conf + - /etc/systemd/system/collectd.service.d/timeout.conf + - /etc/systemd/system/debuginfod.service.d/override.conf + - /etc/systemd/system/dirsrv@FEDORAPROJECT-ORG.service.d/ipa-env.conf - /etc/systemd/system/dirsrv@STG-FEDORAPROJECT-ORG.service.d/ipa-env.conf - /etc/systemd/system/dnf-automatic.timer.d/weekdays.conf - /etc/systemd/system/dnf5-automatic.timer.d/weekdays.conf - /etc/systemd/system/dnf-automatic-install.timer.d/weekdays.conf + - /etc/systemd/system/fm-consumer@.service.d/local.conf - /etc/systemd/system/git@.service + - /etc/systemd/system/haproxy.service.d/postvpn.conf + - /etc/systemd/system/httpd.service - /etc/systemd/system/httpd.service.d/env.conf - /etc/systemd/system/httpd.service.d/httpdoverride.conf - /etc/systemd/system/httpd.service.d/ipa.conf + - /etc/systemd/system/httpd.service.d/override.conf + - /etc/systemd/system/hyperkitty.target + - /etc/systemd/system/hyperkitty-daily.service + - /etc/systemd/system/hyperkitty-daily.timer + - /etc/systemd/system/hyperkitty-hourly.service + - /etc/systemd/system/hyperkitty-hourly.timer + - /etc/systemd/system/hyperkitty-minutely.service + - /etc/systemd/system/hyperkitty-minutely.timer + - /etc/systemd/system/hyperkitty-monthly.service + - /etc/systemd/system/hyperkitty-monthly.timer + - /etc/systemd/system/hyperkitty-quarter_hourly.service + - /etc/systemd/system/hyperkitty-quarter_hourly.timer + - /etc/systemd/system/hyperkitty-weekly.service + - /etc/systemd/system/hyperkitty-weekly.timer + - /etc/systemd/system/hyperkitty-yearly.service + - /etc/systemd/system/hyperkitty-yearly.timer - /etc/systemd/system/kojid.service + - /etc/systemd/system/machine-.scope.d/80-infra.conf + - /etc/systemd/system/mailman3.service.d/mailman3.conf + - /etc/systemd/system/mailmanweb.service - /etc/systemd/system/mirrorlist1.service - /etc/systemd/system/mirrorlist2.service - /etc/systemd/system/pagure_ev.service + - /etc/systemd/system/pagure_fast_worker.service - /etc/systemd/system/pagure_logcom.service + - /etc/systemd/system/pagure_medium_worker.service + - /etc/systemd/system/pagure_mirror.service + - /etc/systemd/system/pagure_slow_worker.service - /etc/systemd/system/pagure_webhook.service - /etc/systemd/system/pagure_worker.service - /etc/systemd/system/pki-tomcatd@pki-tomcat.service.d/ipa.conf + - /etc/systemd/system/rabbitmq-server.service.d/override.conf + - /etc/systemd/system/rsyslog.service.d/limits.conf + - /etc/systemd/system/send-rabbitmq-queue.service + - /etc/systemd/system/send-rabbitmq-queue.timer - /etc/systemd/system/ursabot.service - /etc/systemd/system/varnish.service - + - /etc/systemd/system/varnish.service.d/postvpn.conf + - /etc/systemd/system/varnish.service.d/restart-on-fail.conf + - /etc/systemd/system/webui-qcluster.service + - /etc/systemd/system/webui-warm-up-cache.service + - /etc/systemd/system/zodbot.service # tmpfiles files: - /etc/tmpfiles.d/dirsrv-STG-FEDORAPROJECT-ORG.conf + # Varnish files: + - /etc/varnish/secret + # YUM files: + - /etc/yum.repos.d/_copr:copr.fedorainfracloud.org:dvraaij:ada.repo + - /etc/yum.repos.d/_copr:copr.fedorainfracloud.org:group_abrt:faf-el8.repo + - /etc/yum.repos.d/_copr:copr.fedorainfracloud.org:group_abrt:faf-el8-devel.repo + - /etc/yum.repos.d/_copr:copr.fedorainfracloud.org:group_osbuild:osbuild.repo + - /etc/yum.repos.d/_copr:copr.fedorainfracloud.org:hobbes1069:testing.repo + - /etc/yum.repos.d/_copr:copr.fedorainfracloud.org:packit:abrt-retrace-server-434.repo + - /etc/yum.repos.d/_copr:copr.fedorainfracloud.org:psloboda:mariadb10.11-rebase-to-10.11.13.repo + - /etc/yum.repos.d/centos9s-rabbitmq38.repo + - /etc/yum.repos.d/cuda-fedora41.repo + - /etc/yum.repos.d/epel.repo # FIXME ? - /etc/yum.repos.d/epel8.repo - /etc/yum.repos.d/epel9.repo - /etc/yum.repos.d/epel10.repo + - /etc/yum.repos.d/group_abrt-faf-el8-epel-8.repo - /etc/yum.repos.d/infra-tags.repo - /etc/yum.repos.d/infra-tags-stg.repo + - /etc/yum.repos.d/redhat.repo + - /etc/yum.repos.d/rhel-infra-tags.repo - /etc/yum.repos.d/rhel8.repo - /etc/yum.repos.d/rhel9.repo - /etc/yum.repos.d/rhel10.repo @@ -1415,14 +1699,28 @@ - /etc/zabbix/zabbix_agentd.d/interface-alias.conf - /etc/zabbix/zabbix_agentd.d/ipa-backup.conf - /etc/zabbix/zabbix_agentd.d/postfix.conf + - /etc/zabbix/zabbix_agentd.d/rabbitmq.conf - /etc/zabbix/zabbix_agentd.d/raid.conf - /etc/zabbix/zabbix_agentd.conf + # FIXME: Why this and the above? + - /etc/zabbix_agentd.conf + # FIXME: SELinux BS goes here? + - /etc/zabbix/zabbix_agentd.d/zabbix_rabbitmq.mod + - /etc/zabbix/zabbix_agentd.d/zabbix_rabbitmq.pp + - /etc/zabbix/zabbix_agentd.d/zabbix_rabbitmq.te - # Are these known though? - - /etc/firewalld/zones/public.xml - - /etc/modules-load.d/nf_conntrack.conf - - /etc/system_identification - - /etc/varnish/secret + # Misc files: + # FIXME: Are these actually known/good though? + - /etc/crio/crio.conf + - /etc/firewalld/zones/public.xml # everything but s390x has this. + - /etc/grub.d/00_tuned # from 2024 on pkgs01.stg.rdu3 + - /etc/iscsi/initiatorname.iscsi + - /etc/motd_fedora + - /etc/stunnel/stunnel.conf + - /etc/sync-http-logs.yaml + - /etc/system_identification # only bvmhost-s390x-01.stg.s390 + - /etc/systemd/dont-synthesize-nobody # empty file on memcached02.stg.rdu3 + - /etc/xinetd.d/rsync tasks: diff --git a/playbooks/groups/gpu.yml b/playbooks/groups/gpu.yml new file mode 100644 index 0000000000..51c200dd0c --- /dev/null +++ b/playbooks/groups/gpu.yml @@ -0,0 +1,27 @@ +--- +- import_playbook: "/srv/web/infra/ansible/playbooks/include/happy_birthday.yml" + vars: + myhosts: "gpu" + +- name: make gpu host on raw hw + hosts: gpu + remote_user: root + gather_facts: true + + vars_files: + - /srv/web/infra/ansible/vars/global.yml + - "/srv/private/ansible/vars.yml" + - /srv/web/infra/ansible/vars/{{ ansible_distribution }}.yml + + pre_tasks: + - import_tasks: "{{ tasks_path }}/yumrepos.yml" + + roles: + - base + - hosts + - openvpn/client + - ipa/client + - sudo + + handlers: + - import_tasks: "{{ handlers_path }}/restart_services.yml" diff --git a/playbooks/groups/nfs-servers.yml b/playbooks/groups/nfs-servers.yml index 57f6b478f3..1cfb4a56a5 100644 --- a/playbooks/groups/nfs-servers.yml +++ b/playbooks/groups/nfs-servers.yml @@ -30,7 +30,7 @@ ## This should be in a different playbook. - name: Deal with drive items on storinator01 - hosts: storinator01.rdu-cc.fedoraproject.org + hosts: storinator01.rdu3.fedoraproject.org user: root gather_facts: true vars_files: @@ -72,7 +72,7 @@ - copr - name: Deal with NFS - hosts: storinator01.rdu-cc.fedoraproject.org + hosts: storinator01.rdu3.fedoraproject.org user: root gather_facts: true vars_files: diff --git a/playbooks/include/proxies-reverseproxy.yml b/playbooks/include/proxies-reverseproxy.yml index de9aef4e52..afefd8cffb 100644 --- a/playbooks/include/proxies-reverseproxy.yml +++ b/playbooks/include/proxies-reverseproxy.yml @@ -13,7 +13,7 @@ - import_tasks: "{{ handlers_path }}/restart_services.yml" vars: - - varnish_url: http://localhost:6081 + varnish_url: http://localhost:6081 pre_tasks: @@ -661,6 +661,7 @@ website: koji.fedoraproject.org destname: koji keephost: true + proxyopts: "keepalive=on ttl=10" balancer_name: koji balancer_members: - "koji01.{{ datacenter }}.fedoraproject.org" @@ -674,6 +675,7 @@ website: koji.fedoraproject.org destname: koji keephost: true + proxyopts: "keepalive=on ttl=10" balancer_name: koji balancer_members: - "koji01.stg.{{ datacenter }}.fedoraproject.org" @@ -896,20 +898,6 @@ keephost: true tags: discourse2fedmsg -# - role: httpd/reverseproxy -# website: ipsilon-project.org -# destname: ipsilon-website -# balancer_name: apps-ocp -# balancer_members: "{{ (env == 'staging')|ternary(ocp_nodes_rdu3_stg, ocp_nodes) }}" -# targettype: openshift -# ocp4: "{{ (env == 'production') | bool }}" -# ocp4_rdu3: "{{ (env == 'staging') | bool }}" -# # When prod has moved to rdu3: -# #ocp4: false -# #ocp4_rdu3: true -# keephost: true -# tags: ipsilon-website - - role: httpd/reverseproxy website: awx.fedoraproject.org destname: awx diff --git a/playbooks/include/proxies-websites.yml b/playbooks/include/proxies-websites.yml index 40a7c40ee0..e6ceb68d97 100644 --- a/playbooks/include/proxies-websites.yml +++ b/playbooks/include/proxies-websites.yml @@ -1210,18 +1210,6 @@ tags: - fedora.im -# - role: httpd/website -# site_name: ipsilon-project.org -# cert_name: ipsilon-project.org -# server_aliases: -# - ipsilon-project.org -# - www.ipsilon-project.org -# ssl: true -# sslonly: true -# certbot: true -# tags: -# - ipsilon-website - - role: httpd/website site_name: directory.fedoraproject.org ssl: true diff --git a/playbooks/manual/epel-minor-release/branch-distgit-packages.yml b/playbooks/manual/epel-minor-release/branch-distgit-packages.yml index 6da0554037..57edfad144 100644 --- a/playbooks/manual/epel-minor-release/branch-distgit-packages.yml +++ b/playbooks/manual/epel-minor-release/branch-distgit-packages.yml @@ -2,15 +2,15 @@ # creates EPEL branches from a target branch. # # Running the following line will do a dry-run of the process on the staging server. -# $ ansible-playbook \ +# $ sudo rbac-playbook \ # -l pkgs_stg \ -# playbooks/manual/epel-minor-release/branch-disgit-packages.yml +# manual/epel-minor-release/branch-distgit-packages.yml # # You can also use it locally to test how it works # $ ansible-playbook \ # -l localhost \ # -e checkout_path=/tmp/rpms \ -# playbooks/manual/epel-minor-release/branch-disgit-packages.yml +# playbooks/manual/epel-minor-release/branch-distgit-packages.yml # # Expected extra-vars: # checkout_path diff --git a/playbooks/manual/epel-minor-release/prepare-bodhi-repos.yml b/playbooks/manual/epel-minor-release/prepare-bodhi-repos.yml index 5b815485c7..907dd2e767 100644 --- a/playbooks/manual/epel-minor-release/prepare-bodhi-repos.yml +++ b/playbooks/manual/epel-minor-release/prepare-bodhi-repos.yml @@ -19,22 +19,51 @@ ansible.builtin.stat: path: /mnt/koji/compose/updates/epel{{ epel_major }}.{{ epel_branched_minor }} register: compose_stat + - name: Create compose symlink become: yes ansible.builtin.file: src: "{{ compose_stat.stat.lnk_target }}" dest: "/mnt/koji/compose/updates/epel{{ epel_major }}.{{ epel_minor }}" state: link + follow: false + owner: apache + group: apache + - name: Determine current testing compose ansible.builtin.stat: path: /mnt/koji/compose/updates/epel{{ epel_major }}.{{ epel_branched_minor }}-testing register: compose_testing_stat + - name: Create testing compose symlink become: yes ansible.builtin.file: src: "{{ compose_testing_stat.stat.lnk_target }}" dest: "/mnt/koji/compose/updates/epel{{ epel_major }}.{{ epel_minor }}-testing" state: link + follow: false + owner: apache + group: apache + + - name: Set major-only compose symlink + become: yes + ansible.builtin.file: + src: epel{{ epel_major }}.{{ epel_minor }} + dest: /mnt/koji/compose/updates/epel{{ epel_major }} + state: link + follow: false + owner: apache + group: apache + + - name: Set major-only testing compose symlink + become: yes + ansible.builtin.file: + src: epel{{ epel_major }}.{{ epel_minor }}-testing + dest: /mnt/koji/compose/updates/epel{{ epel_major }}-testing + state: link + follow: false + owner: apache + group: apache - name: Create repos for packages block: diff --git a/playbooks/manual/ocp4-postinstall-setup.yml b/playbooks/manual/ocp4-postinstall-setup.yml index 272cf302b2..81f11a183d 100644 --- a/playbooks/manual/ocp4-postinstall-setup.yml +++ b/playbooks/manual/ocp4-postinstall-setup.yml @@ -18,3 +18,5 @@ - gwmngilfen - nphilipp - zlopez + cluster_readonly_appowners: + - smoliicek diff --git a/playbooks/openshift-apps/ipsilon-website.yml b/playbooks/openshift-apps/ipsilon-website.yml deleted file mode 100644 index 1b94e6fa16..0000000000 --- a/playbooks/openshift-apps/ipsilon-website.yml +++ /dev/null @@ -1,58 +0,0 @@ ---- -- name: Make the app be real - hosts: os_control_stg[0]:os_control[0] - user: root - gather_facts: false - - vars_files: - - /srv/web/infra/ansible/vars/global.yml - - "/srv/private/ansible/vars.yml" - - /srv/web/infra/ansible/vars/{{ ansible_distribution }}.yml - - vars: - - roles: - - role: openshift/project - project_app: ipsilon-website - project_description: "ipsilon-project.org" - project_appowners: - - abompard - tags: - - apply-appowners - - - role: openshift/imagestream - imagestream_app: ipsilon-website - imagestream_imagename: ipsilon-website - - - role: openshift/object - object_app: ipsilon-website - object_template: buildconfig.yml.j2 - object_objectname: buildconfig.yml - - - role: openshift/object - object_app: ipsilon-website - object_file: service.yml - object_objectname: service.yml - - - role: openshift/route - route_app: ipsilon-website - route_name: web-internal - route_host: "ipsilon-website.apps.ocp{{ env_suffix }}.fedoraproject.org" - route_serviceport: web - route_servicename: web - route_annotations: - haproxy.router.openshift.io/timeout: 5m - - - role: openshift/route - route_app: ipsilon-website - route_name: web - route_host: "ipsilon-project.org" - route_serviceport: web - route_servicename: web - route_annotations: - haproxy.router.openshift.io/timeout: 5m - - - role: openshift/object - object_app: ipsilon-website - object_template: deploymentconfig.yml.j2 - object_objectname: deploymentconfig.yml diff --git a/playbooks/openshift-apps/resultsdb-ci-listener.yml b/playbooks/openshift-apps/resultsdb-ci-listener.yml index b828abb365..24404c3e50 100644 --- a/playbooks/openshift-apps/resultsdb-ci-listener.yml +++ b/playbooks/openshift-apps/resultsdb-ci-listener.yml @@ -29,7 +29,7 @@ user_sent_topics: ^$ # The openshift/project role breaks if the project already exists: - # https://forge.fedoraproject.org/infra/tickets/6404 + # https://forge.fedoraproject.org/infra/tickets/issues/6404 - role: openshift/project project_app: resultsdb-ci-listener project_description: resultsdb-ci-listener diff --git a/playbooks/openshift-apps/resultsdb.yml b/playbooks/openshift-apps/resultsdb.yml index c02bd86df2..60a01f1eb2 100644 --- a/playbooks/openshift-apps/resultsdb.yml +++ b/playbooks/openshift-apps/resultsdb.yml @@ -44,7 +44,7 @@ user_sent_topics: ^org\.fedoraproject\.{{ env_short }}\.resultsdb\..* # The openshift/project role breaks if the project already exists: - # https://forge.fedoraproject.org/infra/tickets/6404 + # https://forge.fedoraproject.org/infra/tickets/issues/6404 - role: openshift/project project_app: resultsdb project_description: resultsdb diff --git a/playbooks/openshift-apps/waiverdb.yml b/playbooks/openshift-apps/waiverdb.yml index d68935f38a..7ea818db13 100644 --- a/playbooks/openshift-apps/waiverdb.yml +++ b/playbooks/openshift-apps/waiverdb.yml @@ -46,7 +46,7 @@ user_sent_topics: ^org\.fedoraproject\.{{ env_short }}\.waiverdb\..* # The openshift/project role breaks if the project already exists: - # https://forge.fedoraproject.org/infra/tickets/6404 + # https://forge.fedoraproject.org/infra/tickets/issues/6404 - role: openshift/project project_app: waiverdb project_description: waiverdb diff --git a/roles/ansible-server/files/requirements.yml b/roles/ansible-server/files/requirements.yml index 2086032e25..675c844fcf 100644 --- a/roles/ansible-server/files/requirements.yml +++ b/roles/ansible-server/files/requirements.yml @@ -1,7 +1,7 @@ --- roles: # Needed for copr-pulp playbooks - # https://forge.fedoraproject.org/infra/tickets/11396 + # https://forge.fedoraproject.org/infra/tickets/issues/11396 - name: geerlingguy.postgresql version: 3.5.0 diff --git a/roles/apps-fp-o/files/apps.yaml b/roles/apps-fp-o/files/apps.yaml index 058ee14dbe..af6762e528 100644 --- a/roles/apps-fp-o/files/apps.yaml +++ b/roles/apps-fp-o/files/apps.yaml @@ -104,7 +104,7 @@ children: url: https://fedoramagazine.org docs_url: https://codex.wordpress.org/ # We don't have a SOP for the magazine yet. - # https://forge.fedoraproject.org/infra/tickets/5149 + # https://forge.fedoraproject.org/infra/tickets/issues/5149 # sops: # - put the url here description: > @@ -129,7 +129,7 @@ children: # TODO - add the docs_url. I asked pete travis for info on this # docs_url: put the url here # TODO - add a sop. - # https://forge.fedoraproject.org/infra/tickets/5150 + # https://forge.fedoraproject.org/infra/tickets/issues/5150 # sops: # - add the sop url here. description: > @@ -152,7 +152,7 @@ children: bugs_url: https://github.com/abrt/retrace-server/issues docs_url: https://abrt.readthedocs.org/en/latest/howitworks.html#faf # TODO - write SOPs for this - # https://forge.fedoraproject.org/infra/tickets/5151 + # https://forge.fedoraproject.org/infra/tickets/issues/5151 # sops: # - url goes here # - and another one goes here @@ -190,7 +190,7 @@ children: package_url: > https://bugzilla.redhat.com/buglist.cgi?component=Package%20Review&query_format=advanced&short_desc_type=allwordssubstr&short_desc={package} # TODO - write the SOP for this - # https://forge.fedoraproject.org/infra/tickets/5152 + # https://forge.fedoraproject.org/infra/tickets/issues/5152 # sops: # - url goes here description: > @@ -243,7 +243,7 @@ children: bugs_url: https://github.com/fedora-infra/asknot-ng/issues docs_url: https://github.com/fedora-infra/asknot-ng/blob/develop/README.md # TODO - write SOP for asknot-ng - # https://forge.fedoraproject.org/infra/tickets/5154 + # https://forge.fedoraproject.org/infra/tickets/issues/5154 # sops: # - url goes here status_mappings: [] @@ -425,7 +425,7 @@ children: bugs_url: https://github.com/fedora-infra/anitya/issues docs_url: https://fedoraproject.org/wiki/Upstream_release_monitoring # TODO - write sops for anitya and the-new-hotness - # https://forge.fedoraproject.org/infra/tickets/5157 + # https://forge.fedoraproject.org/infra/tickets/issues/5157 # sops: # - https://infrastructure.fedoraproject.org/infra/docs/anitya.rst # - https://infrastructure.fedoraproject.org/infra/docs/hotness.rst @@ -479,7 +479,7 @@ children: bugs_url: https://github.com/fedora-infra/geoip-city-wsgi/issues docs_url: https://github.com/fedora-infra/geoip-city-wsgi/blob/master/geoip-city.wsgi # TODO - write a sop for this thing - # https://forge.fedoraproject.org/infra/tickets/5159 + # https://forge.fedoraproject.org/infra/tickets/issues/5159 # sops: # - https://infrastructure.fedoraproject.org/infra/docs/geoip.rst description: > diff --git a/roles/base/files/postfix/main.cf/main.cf b/roles/base/files/postfix/main.cf/main.cf index ce902a578e..9f193df3df 100644 --- a/roles/base/files/postfix/main.cf/main.cf +++ b/roles/base/files/postfix/main.cf/main.cf @@ -410,7 +410,7 @@ masquerade_exceptions = root apache # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -421,7 +421,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) diff --git a/roles/base/files/postfix/main.cf/main.cf.copr b/roles/base/files/postfix/main.cf/main.cf.copr index 06c58d6033..50a99e72ff 100644 --- a/roles/base/files/postfix/main.cf/main.cf.copr +++ b/roles/base/files/postfix/main.cf/main.cf.copr @@ -387,7 +387,7 @@ relayhost = bastion.fedoraproject.org # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -398,7 +398,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) diff --git a/roles/base/files/postfix/main.cf/main.cf.copr_smtp_auth_relay b/roles/base/files/postfix/main.cf/main.cf.copr_smtp_auth_relay index 9fe9b817a8..36085624a2 100644 --- a/roles/base/files/postfix/main.cf/main.cf.copr_smtp_auth_relay +++ b/roles/base/files/postfix/main.cf/main.cf.copr_smtp_auth_relay @@ -321,7 +321,7 @@ relayhost = smtp-auth-cc-rdu01.fedoraproject.org smtp_use_tls = yes smtp_sasl_auth_enable = yes smtp_sasl_security_options = -smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd +smtp_sasl_password_maps = lmdb:/etc/postfix/sasl_passwd # REJECTING UNKNOWN RELAY USERS @@ -394,7 +394,7 @@ smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -405,7 +405,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) diff --git a/roles/base/files/postfix/main.cf/main.cf.gateway b/roles/base/files/postfix/main.cf/main.cf.gateway index 5d8ee82ed2..04b9df6ff0 100644 --- a/roles/base/files/postfix/main.cf/main.cf.gateway +++ b/roles/base/files/postfix/main.cf/main.cf.gateway @@ -419,7 +419,7 @@ relay_domains = $mydestination # #alias_maps = dbm:/etc/aliases #alias_maps = hash:/etc/aliases -alias_maps = hash:/etc/aliases, hash:/etc/postfix/package-owner, hash:/etc/postfix/package-maintainers +alias_maps = lmdb:/etc/aliases, hash:/etc/postfix/package-owner, hash:/etc/postfix/package-maintainers #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -430,7 +430,7 @@ alias_maps = hash:/etc/aliases, hash:/etc/postfix/package-owner, hash:/etc/postf # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) @@ -736,7 +736,7 @@ smtpd_tls_eecdh_grade = ultra # smtp TLS Client smtp_tls_fingerprint_digest=sha1 smtp_tls_note_starttls_offer = yes -smtp_tls_policy_maps = hash:/etc/postfix/tls_policy +smtp_tls_policy_maps = lmdb:/etc/postfix/tls_policy smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtp_tls_mandatory_ciphers = high smtp_tls_mandatory_exclude_ciphers= aNULL, MD5, RC4 @@ -749,7 +749,7 @@ smtp_tls_connection_reuse = no smtp_connection_cache_destinations = mx1.redhat.com,gmail.com,google.com,scrye.com,redhat.com smtp_tls_session_cache_database = btree:/var/lib/postfix/smtp_scache smtp_tls_session_cache_timeout = 3600s -smtp_tls_policy_maps = hash:/etc/postfix/tls_policy +smtp_tls_policy_maps = lmdb:/etc/postfix/tls_policy ## End smtp_tls ## General TLS tls_random_source = dev:/dev/urandom @@ -766,7 +766,7 @@ non_smtpd_milters = $smtpd_milters smtpd_sender_restrictions = regexp:/etc/postfix/sender_access meta_directory = /etc/postfix shlib_directory = /usr/lib64/postfix -transport_maps = hash:/etc/postfix/transport +transport_maps = lmdb:/etc/postfix/transport local_header_rewrite_clients = static:all message_size_limit = 20971520 @@ -779,7 +779,7 @@ smtpd_relay_restrictions = permit_mynetworks, reject_unauth_destination # here we send emails _from_ redhat.com addresses back out the redhat.com mx # This avoids us sending them and causing SPF failures. # It depends on them allowing us to relay email out. -sender_dependent_relayhost_maps = hash:/etc/postfix/bysender +sender_dependent_relayhost_maps = lmdb:/etc/postfix/bysender # RHEL postfix disables RFC3030 CHUNKING by default for security reasons # http://www.postfix.org/BDAT_README.html diff --git a/roles/base/files/postfix/main.cf/main.cf.kojibuilder b/roles/base/files/postfix/main.cf/main.cf.kojibuilder index fd4c283488..b13de7648a 100644 --- a/roles/base/files/postfix/main.cf/main.cf.kojibuilder +++ b/roles/base/files/postfix/main.cf/main.cf.kojibuilder @@ -386,7 +386,7 @@ masquerade_exceptions = root apache # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -397,7 +397,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) diff --git a/roles/base/files/postfix/main.cf/main.cf.mailman b/roles/base/files/postfix/main.cf/main.cf.mailman index 7558da1d19..b455ca4c90 100644 --- a/roles/base/files/postfix/main.cf/main.cf.mailman +++ b/roles/base/files/postfix/main.cf/main.cf.mailman @@ -388,7 +388,7 @@ masquerade_exceptions = root apache # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -399,7 +399,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) @@ -691,9 +691,9 @@ message_size_limit = 20971520 # Mailman, see MTA.rst owner_request_special = no -transport_maps = hash:/var/lib/mailman3/data/postfix_lmtp -local_recipient_maps = hash:/var/lib/mailman3/data/postfix_lmtp -relay_domains = hash:/var/lib/mailman3/data/postfix_domains +transport_maps = lmdb:/var/lib/mailman3/data/postfix_lmtp +local_recipient_maps = lmdb:/var/lib/mailman3/data/postfix_lmtp +relay_domains = lmdb:/var/lib/mailman3/data/postfix_domains smtpd_sender_restrictions = check_sender_access regexp:/etc/postfix/sender_access diff --git a/roles/base/files/postfix/main.cf/main.cf.mailman01.stg.rdu3.fedoraproject.org b/roles/base/files/postfix/main.cf/main.cf.mailman01.stg.rdu3.fedoraproject.org index ecd01cadc5..942b2e89c6 100644 --- a/roles/base/files/postfix/main.cf/main.cf.mailman01.stg.rdu3.fedoraproject.org +++ b/roles/base/files/postfix/main.cf/main.cf.mailman01.stg.rdu3.fedoraproject.org @@ -388,7 +388,7 @@ masquerade_exceptions = root apache # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -399,7 +399,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) @@ -691,8 +691,8 @@ message_size_limit = 20971520 # Mailman, see MTA.rst owner_request_special = no -transport_maps = hash:/var/lib/mailman3/data/postfix_lmtp -local_recipient_maps = hash:/var/lib/mailman3/data/postfix_lmtp -relay_domains = hash:/var/lib/mailman3/data/postfix_domains +transport_maps = lmdb:/var/lib/mailman3/data/postfix_lmtp +local_recipient_maps = lmdb:/var/lib/mailman3/data/postfix_lmtp +relay_domains = lmdb:/var/lib/mailman3/data/postfix_domains smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination diff --git a/roles/base/files/postfix/main.cf/main.cf.noc02.fedoraproject.org b/roles/base/files/postfix/main.cf/main.cf.noc02.fedoraproject.org index a0680ed174..ebcbc2df18 100644 --- a/roles/base/files/postfix/main.cf/main.cf.noc02.fedoraproject.org +++ b/roles/base/files/postfix/main.cf/main.cf.noc02.fedoraproject.org @@ -395,7 +395,7 @@ unknown_local_recipient_reject_code = 550 # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -406,7 +406,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) @@ -691,7 +691,7 @@ inet_protocols = ipv4 # mapping so we know where to go for .redhat.com mail -transport_maps = hash:/etc/postfix/transport +transport_maps = lmdb:/etc/postfix/transport local_header_rewrite_clients = static:all diff --git a/roles/base/files/postfix/main.cf/main.cf.norelay b/roles/base/files/postfix/main.cf/main.cf.norelay index 85b6947567..40a6cdb952 100644 --- a/roles/base/files/postfix/main.cf/main.cf.norelay +++ b/roles/base/files/postfix/main.cf/main.cf.norelay @@ -386,7 +386,7 @@ masquerade_exceptions = root apache # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -397,7 +397,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) diff --git a/roles/base/files/postfix/main.cf/main.cf.pkgs b/roles/base/files/postfix/main.cf/main.cf.pkgs new file mode 100644 index 0000000000..ce902a578e --- /dev/null +++ b/roles/base/files/postfix/main.cf/main.cf.pkgs @@ -0,0 +1,718 @@ +# Global Postfix configuration file. This file lists only a subset +# of all parameters. For the syntax, and for a complete parameter +# list, see the postconf(5) manual page (command: "man 5 postconf"). +# +# For common configuration examples, see BASIC_CONFIGURATION_README +# and STANDARD_CONFIGURATION_README. To find these documents, use +# the command "postconf html_directory readme_directory", or go to +# http://www.postfix.org/BASIC_CONFIGURATION_README.html etc. +# +# For best results, change no more than 2-3 parameters at a time, +# and test if Postfix still works after every change. + +# COMPATIBILITY +# +# The compatibility_level determines what default settings Postfix +# will use for main.cf and master.cf settings. These defaults will +# change over time. +# +# To avoid breaking things, Postfix will use backwards-compatible +# default settings and log where it uses those old backwards-compatible +# default settings, until the system administrator has determined +# if any backwards-compatible default settings need to be made +# permanent in main.cf or master.cf. +# +# When this review is complete, update the compatibility_level setting +# below as recommended in the RELEASE_NOTES file. +# +# The level below is what should be used with new (not upgrade) installs. +# +compatibility_level = 2 + +# SOFT BOUNCE +# +# The soft_bounce parameter provides a limited safety net for +# testing. When soft_bounce is enabled, mail will remain queued that +# would otherwise bounce. This parameter disables locally-generated +# bounces, and prevents the SMTP server from rejecting mail permanently +# (by changing 5xx replies into 4xx replies). However, soft_bounce +# is no cure for address rewriting mistakes or mail routing mistakes. +# +#soft_bounce = no + +# LOCAL PATHNAME INFORMATION +# +# The queue_directory specifies the location of the Postfix queue. +# This is also the root directory of Postfix daemons that run chrooted. +# See the files in examples/chroot-setup for setting up Postfix chroot +# environments on different UNIX systems. +# +queue_directory = /var/spool/postfix + +# The command_directory parameter specifies the location of all +# postXXX commands. +# +command_directory = /usr/sbin + +# The daemon_directory parameter specifies the location of all Postfix +# daemon programs (i.e. programs listed in the master.cf file). This +# directory must be owned by root. +# +daemon_directory = /usr/libexec/postfix + +# The data_directory parameter specifies the location of Postfix-writable +# data files (caches, random numbers). This directory must be owned +# by the mail_owner account (see below). +# +data_directory = /var/lib/postfix + +# QUEUE AND PROCESS OWNERSHIP +# +# The mail_owner parameter specifies the owner of the Postfix queue +# and of most Postfix daemon processes. Specify the name of a user +# account THAT DOES NOT SHARE ITS USER OR GROUP ID WITH OTHER ACCOUNTS +# AND THAT OWNS NO OTHER FILES OR PROCESSES ON THE SYSTEM. In +# particular, don't specify nobody or daemon. PLEASE USE A DEDICATED +# USER. +# +mail_owner = postfix + +# The default_privs parameter specifies the default rights used by +# the local delivery agent for delivery to external file or command. +# These rights are used in the absence of a recipient user context. +# DO NOT SPECIFY A PRIVILEGED USER OR THE POSTFIX OWNER. +# +#default_privs = nobody + +# INTERNET HOST AND DOMAIN NAMES +# +# The myhostname parameter specifies the internet hostname of this +# mail system. The default is to use the fully-qualified domain name +# from gethostname(). $myhostname is used as a default value for many +# other configuration parameters. +# +#myhostname = host.domain.tld +#myhostname = virtual.domain.tld + +# The mydomain parameter specifies the local internet domain name. +# The default is to use $myhostname minus the first component. +# $mydomain is used as a default value for many other configuration +# parameters. +# +#mydomain = domain.tld + +# SENDING MAIL +# +# The myorigin parameter specifies the domain that locally-posted +# mail appears to come from. The default is to append $myhostname, +# which is fine for small sites. If you run a domain with multiple +# machines, you should (1) change this to $mydomain and (2) set up +# a domain-wide alias database that aliases each user to +# user@that.users.mailhost. +# +# For the sake of consistency between sender and recipient addresses, +# myorigin also specifies the default domain name that is appended +# to recipient addresses that have no @domain part. +# +#myorigin = $myhostname +#myorigin = $mydomain + +mydomain = fedoraproject.org +myorigin = fedoraproject.org + +# RECEIVING MAIL + +# The inet_interfaces parameter specifies the network interface +# addresses that this mail system receives mail on. By default, +# the software claims all active interfaces on the machine. The +# parameter also controls delivery of mail to user@[ip.address]. +# +# See also the proxy_interfaces parameter, for network addresses that +# are forwarded to us via a proxy or network address translator. +# +# Note: you need to stop/start Postfix when this parameter changes. +# +#inet_interfaces = all +#inet_interfaces = $myhostname +#inet_interfaces = $myhostname, localhost +inet_interfaces = all + +# The proxy_interfaces parameter specifies the network interface +# addresses that this mail system receives mail on by way of a +# proxy or network address translation unit. This setting extends +# the address list specified with the inet_interfaces parameter. +# +# You must specify your proxy/NAT addresses when your system is a +# backup MX host for other domains, otherwise mail delivery loops +# will happen when the primary MX host is down. +# +#proxy_interfaces = +#proxy_interfaces = 1.2.3.4 + +# The mydestination parameter specifies the list of domains that this +# machine considers itself the final destination for. +# +# These domains are routed to the delivery agent specified with the +# local_transport parameter setting. By default, that is the UNIX +# compatible delivery agent that lookups all recipients in /etc/passwd +# and /etc/aliases or their equivalent. +# +# The default is $myhostname + localhost.$mydomain. On a mail domain +# gateway, you should also include $mydomain. +# +# Do not specify the names of virtual domains - those domains are +# specified elsewhere (see VIRTUAL_README). +# +# Do not specify the names of domains that this machine is backup MX +# host for. Specify those names via the relay_domains settings for +# the SMTP server, or use permit_mx_backup if you are lazy (see +# STANDARD_CONFIGURATION_README). +# +# The local machine is always the final destination for mail addressed +# to user@[the.net.work.address] of an interface that the mail system +# receives mail on (see the inet_interfaces parameter). +# +# Specify a list of host or domain names, /file/name or type:table +# patterns, separated by commas and/or whitespace. A /file/name +# pattern is replaced by its contents; a type:table is matched when +# a name matches a lookup key (the right-hand side is ignored). +# Continue long lines by starting the next line with whitespace. +# +# See also below, section "REJECTING MAIL FOR UNKNOWN LOCAL USERS". +# +mydestination = $myhostname, localhost.$mydomain, fedora.redhat.com, localhost +#mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain +#mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain, +# mail.$mydomain, www.$mydomain, ftp.$mydomain + +# REJECTING MAIL FOR UNKNOWN LOCAL USERS +# +# The local_recipient_maps parameter specifies optional lookup tables +# with all names or addresses of users that are local with respect +# to $mydestination, $inet_interfaces or $proxy_interfaces. +# +# If this parameter is defined, then the SMTP server will reject +# mail for unknown local users. This parameter is defined by default. +# +# To turn off local recipient checking in the SMTP server, specify +# local_recipient_maps = (i.e. empty). +# +# The default setting assumes that you use the default Postfix local +# delivery agent for local delivery. You need to update the +# local_recipient_maps setting if: +# +# - You define $mydestination domain recipients in files other than +# /etc/passwd, /etc/aliases, or the $virtual_alias_maps files. +# For example, you define $mydestination domain recipients in +# the $virtual_mailbox_maps files. +# +# - You redefine the local delivery agent in master.cf. +# +# - You redefine the "local_transport" setting in main.cf. +# +# - You use the "luser_relay", "mailbox_transport", or "fallback_transport" +# feature of the Postfix local delivery agent (see local(8)). +# +# Details are described in the LOCAL_RECIPIENT_README file. +# +# Beware: if the Postfix SMTP server runs chrooted, you probably have +# to access the passwd file via the proxymap service, in order to +# overcome chroot restrictions. The alternative, having a copy of +# the system passwd file in the chroot jail is just not practical. +# +# The right-hand side of the lookup tables is conveniently ignored. +# In the left-hand side, specify a bare username, an @domain.tld +# wild-card, or specify a user@domain.tld address. +# +#local_recipient_maps = unix:passwd.byname $alias_maps +#local_recipient_maps = proxy:unix:passwd.byname $alias_maps +#local_recipient_maps = + +# The unknown_local_recipient_reject_code specifies the SMTP server +# response code when a recipient domain matches $mydestination or +# ${proxy,inet}_interfaces, while $local_recipient_maps is non-empty +# and the recipient address or address local-part is not found. +# +# The default setting is 550 (reject mail) but it is safer to start +# with 450 (try again later) until you are certain that your +# local_recipient_maps settings are OK. +# +unknown_local_recipient_reject_code = 550 + +# TRUST AND RELAY CONTROL + +# The mynetworks parameter specifies the list of "trusted" SMTP +# clients that have more privileges than "strangers". +# +# In particular, "trusted" SMTP clients are allowed to relay mail +# through Postfix. See the smtpd_recipient_restrictions parameter +# in postconf(5). +# +# You can specify the list of "trusted" network addresses by hand +# or you can let Postfix do it for you (which is the default). +# +# By default (mynetworks_style = subnet), Postfix "trusts" SMTP +# clients in the same IP subnetworks as the local machine. +# On Linux, this does works correctly only with interfaces specified +# with the "ifconfig" command. +# +# Specify "mynetworks_style = class" when Postfix should "trust" SMTP +# clients in the same IP class A/B/C networks as the local machine. +# Don't do this with a dialup site - it would cause Postfix to "trust" +# your entire provider's network. Instead, specify an explicit +# mynetworks list by hand, as described below. +# +# Specify "mynetworks_style = host" when Postfix should "trust" +# only the local machine. +# +#mynetworks_style = class +#mynetworks_style = subnet +#mynetworks_style = host + +# Alternatively, you can specify the mynetworks list by hand, in +# which case Postfix ignores the mynetworks_style setting. +# +# Specify an explicit list of network/netmask patterns, where the +# mask specifies the number of bits in the network part of a host +# address. +# +# You can also specify the absolute pathname of a pattern file instead +# of listing the patterns here. Specify type:table for table-based lookups +# (the value on the table right-hand side is not used). +# +#mynetworks = 168.100.189.0/28, 127.0.0.0/8 +#mynetworks = $config_directory/mynetworks +#mynetworks = hash:/etc/postfix/network_table + + +# The relay_domains parameter restricts what destinations this system will +# relay mail to. See the smtpd_recipient_restrictions description in +# postconf(5) for detailed information. +# +# By default, Postfix relays mail +# - from "trusted" clients (IP address matches $mynetworks) to any destination, +# - from "untrusted" clients to destinations that match $relay_domains or +# subdomains thereof, except addresses with sender-specified routing. +# The default relay_domains value is $mydestination. +# +# In addition to the above, the Postfix SMTP server by default accepts mail +# that Postfix is final destination for: +# - destinations that match $inet_interfaces or $proxy_interfaces, +# - destinations that match $mydestination +# - destinations that match $virtual_alias_domains, +# - destinations that match $virtual_mailbox_domains. +# These destinations do not need to be listed in $relay_domains. +# +# Specify a list of hosts or domains, /file/name patterns or type:name +# lookup tables, separated by commas and/or whitespace. Continue +# long lines by starting the next line with whitespace. A file name +# is replaced by its contents; a type:name table is matched when a +# (parent) domain appears as lookup key. +# +# NOTE: Postfix will not automatically forward mail for domains that +# list this system as their primary or backup MX host. See the +# permit_mx_backup restriction description in postconf(5). +# +relay_domains = $mydestination + + + +# INTERNET OR INTRANET + +# The relayhost parameter specifies the default host to send mail to +# when no entry is matched in the optional transport(5) table. When +# no relayhost is given, mail is routed directly to the destination. +# +# On an intranet, specify the organizational domain name. If your +# internal DNS uses no MX records, specify the name of the intranet +# gateway host instead. +# +# In the case of SMTP, specify a domain, host, host:port, [host]:port, +# [address] or [address]:port; the form [host] turns off MX lookups. +# +# If you're connected via UUCP, see also the default_transport parameter. +# +#relayhost = $mydomain +#relayhost = [gateway.my.domain] +#relayhost = [mailserver.isp.tld] +#relayhost = uucphost +#relayhost = [an.ip.add.ress] +relayhost = bastion + + +# REJECTING UNKNOWN RELAY USERS +# +# The relay_recipient_maps parameter specifies optional lookup tables +# with all addresses in the domains that match $relay_domains. +# +# If this parameter is defined, then the SMTP server will reject +# mail for unknown relay users. This feature is off by default. +# +# The right-hand side of the lookup tables is conveniently ignored. +# In the left-hand side, specify an @domain.tld wild-card, or specify +# a user@domain.tld address. +# +#relay_recipient_maps = hash:/etc/postfix/relay_recipients + +# INPUT RATE CONTROL +# +# The in_flow_delay configuration parameter implements mail input +# flow control. This feature is turned on by default, although it +# still needs further development (it's disabled on SCO UNIX due +# to an SCO bug). +# +# A Postfix process will pause for $in_flow_delay seconds before +# accepting a new message, when the message arrival rate exceeds the +# message delivery rate. With the default 100 SMTP server process +# limit, this limits the mail inflow to 100 messages a second more +# than the number of messages delivered per second. +# +# Specify 0 to disable the feature. Valid delays are 0..10. +# +#in_flow_delay = 1s + +# ADDRESS REWRITING +# +# The ADDRESS_REWRITING_README document gives information about +# address masquerading or other forms of address rewriting including +# username->Firstname.Lastname mapping. + +masquerade_domains = redhat.com +masquerade_exceptions = root apache + +# ADDRESS REDIRECTION (VIRTUAL DOMAIN) +# +# The VIRTUAL_README document gives information about the many forms +# of domain hosting that Postfix supports. + +# "USER HAS MOVED" BOUNCE MESSAGES +# +# See the discussion in the ADDRESS_REWRITING_README document. + +# TRANSPORT MAP +# +# See the discussion in the ADDRESS_REWRITING_README document. + +# ALIAS DATABASE +# +# The alias_maps parameter specifies the list of alias databases used +# by the local delivery agent. The default list is system dependent. +# +# On systems with NIS, the default is to search the local alias +# database, then the NIS alias database. See aliases(5) for syntax +# details. +# +# If you change the alias database, run "postalias /etc/aliases" (or +# wherever your system stores the mail alias file), or simply run +# "newaliases" to build the necessary DBM or DB file. +# +# It will take a minute or so before changes become visible. Use +# "postfix reload" to eliminate the delay. +# +#alias_maps = dbm:/etc/aliases +alias_maps = hash:/etc/aliases +#alias_maps = hash:/etc/aliases, nis:mail.aliases +#alias_maps = netinfo:/aliases + +# The alias_database parameter specifies the alias database(s) that +# are built with "newaliases" or "sendmail -bi". This is a separate +# configuration parameter, because alias_maps (see above) may specify +# tables that are not necessarily all under control by Postfix. +# +#alias_database = dbm:/etc/aliases +#alias_database = dbm:/etc/mail/aliases +alias_database = hash:/etc/aliases +#alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases + +# ADDRESS EXTENSIONS (e.g., user+foo) +# +# The recipient_delimiter parameter specifies the separator between +# user names and address extensions (user+foo). See canonical(5), +# local(8), relocated(5) and virtual(5) for the effects this has on +# aliases, canonical, virtual, relocated and .forward file lookups. +# Basically, the software tries user+foo and .forward+foo before +# trying user and .forward. +# +recipient_delimiter = + + +# DELIVERY TO MAILBOX +# +# The home_mailbox parameter specifies the optional pathname of a +# mailbox file relative to a user's home directory. The default +# mailbox file is /var/spool/mail/user or /var/mail/user. Specify +# "Maildir/" for qmail-style delivery (the / is required). +# +#home_mailbox = Mailbox +#home_mailbox = Maildir/ + +# The mail_spool_directory parameter specifies the directory where +# UNIX-style mailboxes are kept. The default setting depends on the +# system type. +# +#mail_spool_directory = /var/mail +#mail_spool_directory = /var/spool/mail + +# The mailbox_command parameter specifies the optional external +# command to use instead of mailbox delivery. The command is run as +# the recipient with proper HOME, SHELL and LOGNAME environment settings. +# Exception: delivery for root is done as $default_user. +# +# Other environment variables of interest: USER (recipient username), +# EXTENSION (address extension), DOMAIN (domain part of address), +# and LOCAL (the address localpart). +# +# Unlike other Postfix configuration parameters, the mailbox_command +# parameter is not subjected to $parameter substitutions. This is to +# make it easier to specify shell syntax (see example below). +# +# Avoid shell meta characters because they will force Postfix to run +# an expensive shell process. Procmail alone is expensive enough. +# +# IF YOU USE THIS TO DELIVER MAIL SYSTEM-WIDE, YOU MUST SET UP AN +# ALIAS THAT FORWARDS MAIL FOR ROOT TO A REAL USER. +# +#mailbox_command = /usr/bin/procmail +#mailbox_command = /some/where/procmail -a "$EXTENSION" + +# The mailbox_transport specifies the optional transport in master.cf +# to use after processing aliases and .forward files. This parameter +# has precedence over the mailbox_command, fallback_transport and +# luser_relay parameters. +# +# Specify a string of the form transport:nexthop, where transport is +# the name of a mail delivery transport defined in master.cf. The +# :nexthop part is optional. For more details see the sample transport +# configuration file. +# +# NOTE: if you use this feature for accounts not in the UNIX password +# file, then you must update the "local_recipient_maps" setting in +# the main.cf file, otherwise the SMTP server will reject mail for +# non-UNIX accounts with "User unknown in local recipient table". +# +# Cyrus IMAP over LMTP. Specify ``lmtpunix cmd="lmtpd" +# listen="/var/imap/socket/lmtp" prefork=0'' in cyrus.conf. +#mailbox_transport = lmtp:unix:/var/lib/imap/socket/lmtp + +# If using the cyrus-imapd IMAP server deliver local mail to the IMAP +# server using LMTP (Local Mail Transport Protocol), this is prefered +# over the older cyrus deliver program by setting the +# mailbox_transport as below: +# +# mailbox_transport = lmtp:unix:/var/lib/imap/socket/lmtp +# +# The efficiency of LMTP delivery for cyrus-imapd can be enhanced via +# these settings. +# +# local_destination_recipient_limit = 300 +# local_destination_concurrency_limit = 5 +# +# Of course you should adjust these settings as appropriate for the +# capacity of the hardware you are using. The recipient limit setting +# can be used to take advantage of the single instance message store +# capability of Cyrus. The concurrency limit can be used to control +# how many simultaneous LMTP sessions will be permitted to the Cyrus +# message store. +# +# Cyrus IMAP via command line. Uncomment the "cyrus...pipe" and +# subsequent line in master.cf. +#mailbox_transport = cyrus + +# The fallback_transport specifies the optional transport in master.cf +# to use for recipients that are not found in the UNIX passwd database. +# This parameter has precedence over the luser_relay parameter. +# +# Specify a string of the form transport:nexthop, where transport is +# the name of a mail delivery transport defined in master.cf. The +# :nexthop part is optional. For more details see the sample transport +# configuration file. +# +# NOTE: if you use this feature for accounts not in the UNIX password +# file, then you must update the "local_recipient_maps" setting in +# the main.cf file, otherwise the SMTP server will reject mail for +# non-UNIX accounts with "User unknown in local recipient table". +# +#fallback_transport = lmtp:unix:/var/lib/imap/socket/lmtp +#fallback_transport = + +# The luser_relay parameter specifies an optional destination address +# for unknown recipients. By default, mail for unknown@$mydestination, +# unknown@[$inet_interfaces] or unknown@[$proxy_interfaces] is returned +# as undeliverable. +# +# The following expansions are done on luser_relay: $user (recipient +# username), $shell (recipient shell), $home (recipient home directory), +# $recipient (full recipient address), $extension (recipient address +# extension), $domain (recipient domain), $local (entire recipient +# localpart), $recipient_delimiter. Specify ${name?value} or +# ${name:value} to expand value only when $name does (does not) exist. +# +# luser_relay works only for the default Postfix local delivery agent. +# +# NOTE: if you use this feature for accounts not in the UNIX password +# file, then you must specify "local_recipient_maps =" (i.e. empty) in +# the main.cf file, otherwise the SMTP server will reject mail for +# non-UNIX accounts with "User unknown in local recipient table". +# +#luser_relay = $user@other.host +#luser_relay = $local@other.host +#luser_relay = admin+$local + +# JUNK MAIL CONTROLS +# +# The controls listed here are only a very small subset. The file +# SMTPD_ACCESS_README provides an overview. + +# The header_checks parameter specifies an optional table with patterns +# that each logical message header is matched against, including +# headers that span multiple physical lines. +# +# By default, these patterns also apply to MIME headers and to the +# headers of attached messages. With older Postfix versions, MIME and +# attached message headers were treated as body text. +# +# For details, see "man header_checks". +# +header_checks = regexp:/etc/postfix/header_checks + +# FAST ETRN SERVICE +# +# Postfix maintains per-destination logfiles with information about +# deferred mail, so that mail can be flushed quickly with the SMTP +# "ETRN domain.tld" command, or by executing "sendmail -qRdomain.tld". +# See the ETRN_README document for a detailed description. +# +# The fast_flush_domains parameter controls what destinations are +# eligible for this service. By default, they are all domains that +# this server is willing to relay mail to. +# +#fast_flush_domains = $relay_domains + +# SHOW SOFTWARE VERSION OR NOT +# +# The smtpd_banner parameter specifies the text that follows the 220 +# code in the SMTP server's greeting banner. Some people like to see +# the mail version advertised. By default, Postfix shows no version. +# +# You MUST specify $myhostname at the start of the text. That is an +# RFC requirement. Postfix itself does not care. +# +#smtpd_banner = $myhostname ESMTP $mail_name +#smtpd_banner = $myhostname ESMTP $mail_name ($mail_version) + +# PARALLEL DELIVERY TO THE SAME DESTINATION +# +# How many parallel deliveries to the same user or domain? With local +# delivery, it does not make sense to do massively parallel delivery +# to the same user, because mailbox updates must happen sequentially, +# and expensive pipelines in .forward files can cause disasters when +# too many are run at the same time. With SMTP deliveries, 10 +# simultaneous connections to the same domain could be sufficient to +# raise eyebrows. +# +# Each message delivery transport has its XXX_destination_concurrency_limit +# parameter. The default is $default_destination_concurrency_limit for +# most delivery transports. For the local delivery agent the default is 2. + +#local_destination_concurrency_limit = 2 +#default_destination_concurrency_limit = 20 + +# DEBUGGING CONTROL +# +# The debug_peer_level parameter specifies the increment in verbose +# logging level when an SMTP client or server host name or address +# matches a pattern in the debug_peer_list parameter. +# +debug_peer_level = 2 + +# The debug_peer_list parameter specifies an optional list of domain +# or network patterns, /file/name patterns or type:name tables. When +# an SMTP client or server host name or address matches a pattern, +# increase the verbose logging level by the amount specified in the +# debug_peer_level parameter. +# +#debug_peer_list = 127.0.0.1 +#debug_peer_list = some.domain + +# The debugger_command specifies the external command that is executed +# when a Postfix daemon program is run with the -D option. +# +# Use "command .. & sleep 5" so that the debugger can attach before +# the process marches on. If you use an X-based debugger, be sure to +# set up your XAUTHORITY environment variable before starting Postfix. +# +debugger_command = + PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin + ddd $daemon_directory/$process_name $process_id & sleep 5 + +# If you can't use X, use this to capture the call stack when a +# daemon crashes. The result is in a file in the configuration +# directory, and is named after the process name and the process ID. +# +# debugger_command = +# PATH=/bin:/usr/bin:/usr/local/bin; export PATH; (echo cont; +# echo where) | gdb $daemon_directory/$process_name $process_id 2>&1 +# >$config_directory/$process_name.$process_id.log & sleep 5 +# +# Another possibility is to run gdb under a detached screen session. +# To attach to the screen session, su root and run "screen -r +# " where uniquely matches one of the detached +# sessions (from "screen -list"). +# +# debugger_command = +# PATH=/bin:/usr/bin:/sbin:/usr/sbin; export PATH; screen +# -dmS $process_name gdb $daemon_directory/$process_name +# $process_id & sleep 1 + +# INSTALL-TIME CONFIGURATION INFORMATION +# +# The following parameters are used when installing a new Postfix version. +# +# sendmail_path: The full pathname of the Postfix sendmail command. +# This is the Sendmail-compatible mail posting interface. +# +sendmail_path = /usr/sbin/sendmail.postfix + +# newaliases_path: The full pathname of the Postfix newaliases command. +# This is the Sendmail-compatible command to build alias databases. +# +newaliases_path = /usr/bin/newaliases.postfix + +# mailq_path: The full pathname of the Postfix mailq command. This +# is the Sendmail-compatible mail queue listing command. +# +mailq_path = /usr/bin/mailq.postfix + +# setgid_group: The group for mail submission and queue management +# commands. This must be a group name with a numerical group ID that +# is not shared with other accounts, not even with the Postfix account. +# +setgid_group = postdrop + +# html_directory: The location of the Postfix HTML documentation. +# +html_directory = no + +# manpage_directory: The location of the Postfix on-line manual pages. +# +manpage_directory = /usr/share/man + +# sample_directory: The location of the Postfix sample configuration files. +# This parameter is obsolete as of Postfix 2.1. +# +sample_directory = /usr/share/doc/postfix/samples + +# readme_directory: The location of the Postfix README files. +# +readme_directory = /usr/share/doc/postfix/README_FILES + +# add this to new postfix to get it to add proper message-id and other +# headers to outgoing emails via the gateway. + + +message_size_limit = 20971520 +#inet_protocols = ipv4 + +# This has to be set in newer postfix 3.3.0 or later or it will refuse to +# send any email. ;( +smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination +smtpd_relay_restrictions = permit_mynetworks, reject_unauth_destination diff --git a/roles/base/files/postfix/main.cf/main.cf.rdu b/roles/base/files/postfix/main.cf/main.cf.rdu index f3c260f736..39f134ba42 100644 --- a/roles/base/files/postfix/main.cf/main.cf.rdu +++ b/roles/base/files/postfix/main.cf/main.cf.rdu @@ -386,7 +386,7 @@ masquerade_exceptions = root apache # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -397,7 +397,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) diff --git a/roles/base/files/postfix/main.cf/main.cf.rdu-cc b/roles/base/files/postfix/main.cf/main.cf.rdu-cc index 59b2bec6ff..473ea086d9 100644 --- a/roles/base/files/postfix/main.cf/main.cf.rdu-cc +++ b/roles/base/files/postfix/main.cf/main.cf.rdu-cc @@ -386,7 +386,7 @@ masquerade_exceptions = root apache # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -397,7 +397,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) diff --git a/roles/base/files/postfix/main.cf/main.cf.rdu3 b/roles/base/files/postfix/main.cf/main.cf.rdu3 index d17f0e798d..a741537708 100644 --- a/roles/base/files/postfix/main.cf/main.cf.rdu3 +++ b/roles/base/files/postfix/main.cf/main.cf.rdu3 @@ -385,7 +385,7 @@ masquerade_exceptions = root apache # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -396,7 +396,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) diff --git a/roles/base/files/postfix/main.cf/main.cf.releng b/roles/base/files/postfix/main.cf/main.cf.releng index 13602bee9e..2dce174950 100644 --- a/roles/base/files/postfix/main.cf/main.cf.releng +++ b/roles/base/files/postfix/main.cf/main.cf.releng @@ -386,7 +386,7 @@ masquerade_exceptions = root apache # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -397,7 +397,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) diff --git a/roles/base/files/postfix/main.cf/main.cf.sign b/roles/base/files/postfix/main.cf/main.cf.sign index f3c260f736..39f134ba42 100644 --- a/roles/base/files/postfix/main.cf/main.cf.sign +++ b/roles/base/files/postfix/main.cf/main.cf.sign @@ -386,7 +386,7 @@ masquerade_exceptions = root apache # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -397,7 +397,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) diff --git a/roles/base/files/postfix/main.cf/main.cf.smtp-auth b/roles/base/files/postfix/main.cf/main.cf.smtp-auth index 0153eab82d..eb6f20ed25 100644 --- a/roles/base/files/postfix/main.cf/main.cf.smtp-auth +++ b/roles/base/files/postfix/main.cf/main.cf.smtp-auth @@ -397,8 +397,8 @@ relayhost = [192.168.0.1] # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -409,7 +409,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) @@ -696,7 +696,7 @@ inet_protocols = all # mapping so we know where to go for .redhat.com mail -transport_maps = hash:/etc/postfix/transport +transport_maps = lmdb:/etc/postfix/transport message_size_limit = 20971520 @@ -716,7 +716,7 @@ smtpd_tls_cert_file = /etc/pki/tls/certs/smtpd.crt smtpd_tls_key_file = /etc/pki/tls/private/smtpd.key smtpd_tls_CAfile = /etc/pki/tls/certs/ca.crt smtpd_tls_session_cache_timeout = 3600s -smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache +smtpd_tls_session_cache_database = lmdb:${queue_directory}/smtpd_scache smtpd_tls_received_header = yes smtpd_tls_ask_ccert = yes smtpd_tls_received_header = yes @@ -728,7 +728,7 @@ tls_eecdh_ultra_curve = secp384r1 #TLS Client smtp_tls_fingerprint_digest=sha1 smtp_tls_note_starttls_offer = yes -smtp_tls_policy_maps = hash:/etc/postfix/tls_policy +smtp_tls_policy_maps = lmdb:/etc/postfix/tls_policy smtp_tls_security_level = may smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtp_tls_mandatory_ciphers = high diff --git a/roles/base/files/postfix/main.cf/main.cf.smtp-mm b/roles/base/files/postfix/main.cf/main.cf.smtp-mm index e6b04f2651..b49fff8ae1 100644 --- a/roles/base/files/postfix/main.cf/main.cf.smtp-mm +++ b/roles/base/files/postfix/main.cf/main.cf.smtp-mm @@ -397,8 +397,8 @@ relayhost = # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -409,7 +409,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) @@ -696,7 +696,7 @@ inet_protocols = all # mapping so we know where to go for .redhat.com mail -transport_maps = hash:/etc/postfix/transport +transport_maps = lmdb:/etc/postfix/transport message_size_limit = 20971520 @@ -728,7 +728,7 @@ tls_eecdh_ultra_curve = secp384r1 smtp_use_tls = yes smtp_tls_fingerprint_digest=sha1 smtp_tls_note_starttls_offer = yes -smtp_tls_policy_maps = hash:/etc/postfix/tls_policy +smtp_tls_policy_maps = lmdb:/etc/postfix/tls_policy smtp_tls_security_level = may smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtp_tls_mandatory_ciphers = high diff --git a/roles/base/files/postfix/main.cf/main.cf.staging b/roles/base/files/postfix/main.cf/main.cf.staging index 2ed9792e80..ab1ca40bb9 100644 --- a/roles/base/files/postfix/main.cf/main.cf.staging +++ b/roles/base/files/postfix/main.cf/main.cf.staging @@ -386,7 +386,7 @@ masquerade_exceptions = root apache # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -397,7 +397,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) diff --git a/roles/base/files/postfix/main.cf/main.cf.vpn b/roles/base/files/postfix/main.cf/main.cf.vpn index 4ba4fc9b85..3115f2ac3a 100644 --- a/roles/base/files/postfix/main.cf/main.cf.vpn +++ b/roles/base/files/postfix/main.cf/main.cf.vpn @@ -386,7 +386,7 @@ masquerade_exceptions = root apache # "postfix reload" to eliminate the delay. # #alias_maps = dbm:/etc/aliases -alias_maps = hash:/etc/aliases +alias_maps = lmdb:/etc/aliases #alias_maps = hash:/etc/aliases, nis:mail.aliases #alias_maps = netinfo:/aliases @@ -397,7 +397,7 @@ alias_maps = hash:/etc/aliases # #alias_database = dbm:/etc/aliases #alias_database = dbm:/etc/mail/aliases -alias_database = hash:/etc/aliases +alias_database = lmdb:/etc/aliases #alias_database = hash:/etc/aliases, hash:/opt/majordomo/aliases # ADDRESS EXTENSIONS (e.g., user+foo) diff --git a/roles/base/files/postfix/master.cf/master.cf.smtp-auth-cc-rdu01.fedoraproject.org b/roles/base/files/postfix/master.cf/master.cf.smtp-auth-iso01.fedoraproject.org similarity index 100% rename from roles/base/files/postfix/master.cf/master.cf.smtp-auth-cc-rdu01.fedoraproject.org rename to roles/base/files/postfix/master.cf/master.cf.smtp-auth-iso01.fedoraproject.org diff --git a/roles/base/handlers/main.yml b/roles/base/handlers/main.yml index e9a48d524b..767e432dd2 100644 --- a/roles/base/handlers/main.yml +++ b/roles/base/handlers/main.yml @@ -7,30 +7,50 @@ - "{{ if_uuid.stdout_lines }}" - name: Restart iptables - service: name=iptables state=restarted + ansible.builtin.service: + name: iptables + state: restarted - name: Restart nftables - service: name=nftables state=restarted + ansible.builtin.service: + name: nftables + state: restarted - name: Restart ip6tables - service: name=ip6tables state=restarted + ansible.builtin.service: + name: ip6tables + state: restarted - name: Restart NetworkManager - service: name=NetworkManager state=restarted + ansible.builtin.service: + name: NetworkManager + state: restarted - name: Reload NetworkManager-connections ansible.builtin.command: nmcli c reload - name: Restart postfix - service: name=postfix state=restarted + ansible.builtin.service: + name: postfix + state: restarted - name: Restart rsyslog - service: name=rsyslog state=restarted + ansible.builtin.service: + name: rsyslog + state: restarted - name: Restart watchdog - service: name=watchdog state=restarted + ansible.builtin.service: + name: watchdog + state: restarted - name: Reload libvirtd - service: name=libvirtd state=reloaded + ansible.builtin.service: + name: libvirtd + state: reloaded ignore_errors: true when: ansible_virtualization_role == 'host' + +- name: Create lmdb files + ansible.builtin.include_tasks: lmdb.yml + listen: "Create lmdb files" diff --git a/roles/base/tasks/crypto-policies.yml b/roles/base/tasks/crypto-policies.yml index 497535c716..ae61a2c2e5 100644 --- a/roles/base/tasks/crypto-policies.yml +++ b/roles/base/tasks/crypto-policies.yml @@ -35,7 +35,7 @@ - crypto-policies - base/crypto-policies -# see https://forge.fedoraproject.org/infra/tickets/12321 +# see https://forge.fedoraproject.org/infra/tickets/issues/12321 # This is needed to get SAML2 auth working with bugzilla.redhat.com - name: Set crypto-policy on ipsilon servers to FEDORA40 command: "update-crypto-policies --set DEFAULT" diff --git a/roles/base/tasks/lmdb.yml b/roles/base/tasks/lmdb.yml new file mode 100644 index 0000000000..787adeee0a --- /dev/null +++ b/roles/base/tasks/lmdb.yml @@ -0,0 +1,17 @@ +--- +- name: Read the config file + ansible.builtin.slurp: + src: /etc/postfix/main.cf + register: maincf + +- name: Find all the lmdb files + ansible.builtin.set_fact: + lmdb_files: "{{ (maincf.content | b64decode) | regex_findall('lmdb:([^,\\s]+)') }}" + +- name: Convert to lmdb + ansible.builtin.command: "{{ (item == '/etc/aliases') | ternary('postalias', 'postmap') }} lmdb:{{ item }}" + args: + creates: "{{ item }}" + loop: "{{ lmdb_files }}" + register: result + failed_when: result.rc != 0 diff --git a/roles/base/tasks/postfix-monitoring.yml b/roles/base/tasks/postfix-monitoring.yml index 7819032bd8..ed63172e90 100644 --- a/roles/base/tasks/postfix-monitoring.yml +++ b/roles/base/tasks/postfix-monitoring.yml @@ -33,7 +33,7 @@ vars: selinux_module_dir: /usr/local/share/zabbix selinux_module_name: zabbix_sendmail - when: selinux_zabbix_file.changed + when: selinux_zabbix_file.changed # noqa: no-handler tags: - selinux - postfix @@ -42,7 +42,7 @@ # On a fresh install, the zabbix user won't exist yet # so don't try to set it here. The agent role will fix it later - name: Ensure Zabbix drop-in directory - file: + ansible.builtin.file: path: /etc/zabbix/zabbix_agentd.d state: directory mode: '0755' diff --git a/roles/base/tasks/postfix.yml b/roles/base/tasks/postfix.yml index ff5aaf6570..156064cf18 100644 --- a/roles/base/tasks/postfix.yml +++ b/roles/base/tasks/postfix.yml @@ -7,8 +7,20 @@ tags: - postfix +- name: Make sure postfix-lmdb is installed + ansible.builtin.package: + state: present + name: postfix-lmdb + when: not (ansible_distribution == "RedHat" and ansible_distribution_major_version <= "8") + tags: + - postfix + - base + - name: /etc/postfix/main.cf - ansible.builtin.copy: src={{ item }} dest=/etc/postfix/main.cf + ansible.builtin.copy: + src: "{{ item }}" + dest: /etc/postfix/main.cf + mode: "0644" with_first_found: - "{{ postfix_maincf }}" - "postfix/main.cf/main.cf.{{ ansible_fqdn }}" @@ -19,14 +31,31 @@ - "postfix/main.cf/main.cf" notify: - Restart postfix + - Create lmdb files tags: - postfix - config - base - smtp_auth_relay +- name: Enable SELinux for lmdb + ansible.posix.seboolean: + name: domain_can_mmap_files + state: true + persistent: true + when: not (ansible_distribution == "RedHat" and ansible_distribution_major_version <= "8") + notify: + - Restart postfix + tags: + - postfix + - base + - selinux + - name: Install /etc/postfix/master.cf file - ansible.builtin.copy: src={{ item }} dest=/etc/postfix/master.cf mode=0644 + ansible.builtin.copy: + src: "{{ item }}" + dest: /etc/postfix/master.cf + mode: "0644" with_first_found: - "postfix/master.cf/master.cf.{{ inventory_hostname }}" - "postfix/master.cf/master.cf.{{ host_group }}" @@ -41,7 +70,10 @@ - base - name: Deploy sender_access file - ansible.builtin.copy: src="{{private}}/files/smtpd/sender_access.{{postfix_group}}" dest="/etc/postfix/sender_access" + ansible.builtin.copy: + src: "{{ private }}/files/smtpd/sender_access.{{ postfix_group }}" + dest: "/etc/postfix/sender_access" + mode: "0644" when: postfix_group == "smtp-mm" or postfix_group == "mailman" or postfix_group == "gateway" notify: - Restart postfix @@ -51,11 +83,12 @@ - base - name: Work around s390 privatedevices bug - ini_file: + community.general.ini_file: path: /usr/lib/systemd/system/postfix.service section: Service option: PrivateDevices value: false + mode: "0644" notify: - Reload systemd when: inventory_hostname.startswith(('buildvm-s390x')) @@ -65,13 +98,19 @@ - base - name: Enable postfix to start - service: name=postfix state=started enabled=true + ansible.builtin.service: + name: postfix + state: started + enabled: true tags: - service - base - name: Install /etc/postfix/transport file - ansible.builtin.copy: src="postfix/{{ postfix_transport_filename }}" dest=/etc/postfix/transport + ansible.builtin.copy: + src: "postfix/{{ postfix_transport_filename }}" + dest: /etc/postfix/transport + mode: "0644" when: inventory_hostname.startswith(('smtp-mm','bastion','noc02')) and env != 'staging' notify: - Rebuild postfix transport @@ -82,7 +121,10 @@ - config - name: Install /etc/postfix/bysender file - ansible.builtin.copy: src="postfix/bysender" dest=/etc/postfix/bysender + ansible.builtin.copy: + src: "postfix/bysender" + dest: /etc/postfix/bysender + mode: "0644" when: inventory_hostname.startswith(('bastion')) and env != 'staging' notify: - Rebuild postfix bysender @@ -93,7 +135,10 @@ - config - name: Create /etc/postfix/tls_policy - ansible.builtin.copy: src="postfix/tls_policy" dest=/etc/postfix/tls_policy + ansible.builtin.copy: + src: "postfix/tls_policy" + dest: /etc/postfix/tls_policy + mode: "0644" when: inventory_hostname.startswith(('bastion','smtp-mm','pagure')) and env != 'staging' notify: - Rebuild postfix tls_policy @@ -106,11 +151,11 @@ # This cert is a digicert one, renew it there. - name: Install /etc/pki/tls/private/gateway-chain.pem ansible.builtin.copy: - src="{{private}}/files/smtpd/gateway-chain.pem" - dest=/etc/pki/tls/private/gateway-chain.pem - owner=root - group=root - mode=0600 + src: "{{ private }}/files/smtpd/gateway-chain.pem" + dest: /etc/pki/tls/private/gateway-chain.pem + owner: root + group: root + mode: "0600" when: inventory_hostname.startswith(('bastion','smtp-mm')) and env != 'staging' notify: - Restart postfix diff --git a/roles/batcave/files/centos-10-sync b/roles/batcave/files/centos-10-sync index ff4e35ceb6..cae3402450 100644 --- a/roles/batcave/files/centos-10-sync +++ b/roles/batcave/files/centos-10-sync @@ -22,5 +22,5 @@ ${RSYNC} ${RS_OPT} ${RS_DEADLY} ${CENT_EXCLUDES} ${SERVER}::${RSYNC_MOD} ${RSYNC # changes, we need to comment this out to have a snapshot of CentOS 10 that # resembles RHEL 10.x for the epel10.x-build tag to use temporarily until the # actual RHEL 10.x is released. -# https://forge.fedoraproject.org/infra/tickets/12394 +# https://forge.fedoraproject.org/infra/tickets/issues/12394 #${RSYNC} ${RS_OPT} ${RS_DEADLY} --link-dest=${RSYNC_DESTDIR} ${RSYNC_DESTDIR} ${RSYNC_SNAPDIR} diff --git a/roles/cgit/base/files/cgitrc.people b/roles/cgit/base/files/cgitrc.people index b0b13c3e1e..1f53e1e504 100644 --- a/roles/cgit/base/files/cgitrc.people +++ b/roles/cgit/base/files/cgitrc.people @@ -42,7 +42,8 @@ root-desc=Fedora people public git repos #root-readme=/var/www/html/about.html # Allow download of tar.gz, tar.bz2 and zip-files -snapshots=tar.gz tar.xz zip +#snapshots=tar.gz tar.xz zip +snapshots= ## ## List of common mimetypes diff --git a/roles/copr/backend/files/provision/libvirt-delete b/roles/copr/backend/files/provision/libvirt-delete index 7657a9b589..31ae9188c0 100755 --- a/roles/copr/backend/files/provision/libvirt-delete +++ b/roles/copr/backend/files/provision/libvirt-delete @@ -54,7 +54,7 @@ def _main(): logging.basicConfig(level=logging.INFO) args = _get_parser().parse_args() pool_id = os.getenv("RESALLOC_POOL_ID") - connection = get_hv_identification_from_pool_id(pool_id)[1] + connection, *_ = get_hv_identification_from_pool_id(pool_id) conn = repeat(libvirt.open, (connection,)) try: domain = repeat(conn.lookupByName, (args.domainname,)) diff --git a/roles/copr/backend/files/provision/libvirt-list b/roles/copr/backend/files/provision/libvirt-list index e89b656917..fade9bf88e 100755 --- a/roles/copr/backend/files/provision/libvirt-list +++ b/roles/copr/backend/files/provision/libvirt-list @@ -28,8 +28,9 @@ def _main(): sys.stderr.write("Specify pool ID by --pool or $RESALLOC_POOL_ID\n") sys.exit(1) - connection = args.connection if args.connection else \ - get_hv_identification_from_pool_id(pool_id)[1] + connection = args.connection + if not connection: + connection, *_ = get_hv_identification_from_pool_id(pool_id) try: conn = libvirt.openReadOnly(connection) diff --git a/roles/copr/backend/templates/provision/helpers.py.j2 b/roles/copr/backend/templates/provision/helpers.py.j2 index f2b8d09977..0af96e9d9c 100644 --- a/roles/copr/backend/templates/provision/helpers.py.j2 +++ b/roles/copr/backend/templates/provision/helpers.py.j2 @@ -1,15 +1,24 @@ def get_hv_identification_from_pool_id(pool_id): - """ Get unique ID of the hypervisor """ + """ + Get unique ID of the hypervisor + # rdu3 https://github.com/fedora-copr/copr/issues/3786#issuecomment-3412337856 + # starting with offset 0x100, that's reserved for hypervisor machines + # we have 16^3 - 16^2 IPs = 3840, dedicate the last 256 to staging. + """ {% for host in groups["copr_hypervisor"] %} {% set hostinfo = hostvars[host] %} - if pool_id.startswith("{{ hostinfo['libvirt_pool'] }}"): +{% for pool in hostinfo["libvirt_pools"] %} + if pool_id.startswith("{{ pool["name"] }}"): return ( - {{ hostinfo["libvirt_pool_order_id"] }}, "qemu+ssh://copr@{{ hostinfo['libvirt_host'] }}/system", "{{ hostinfo['libvirt_arch'] }}", + # devel uses 0x900+ + {% if devel is defined and devel %}"2620:52:6:1161:dead:beef:cafe:c900"{% else %}"2620:52:6:1161:dead:beef:cafe:c100"{% endif %}, + "2620:52:6:1161::1", ) +{% endfor %} {% endfor %} raise Exception("can't convert pool_id to hv ID") diff --git a/roles/copr/backend/templates/provision/libvirt-new b/roles/copr/backend/templates/provision/libvirt-new index 3ccbe56e01..de99ba803e 100755 --- a/roles/copr/backend/templates/provision/libvirt-new +++ b/roles/copr/backend/templates/provision/libvirt-new @@ -16,6 +16,8 @@ import shutil import shlex import time import argparse +import ipaddress + from helpers import get_hv_identification_from_pool_id @@ -51,7 +53,7 @@ class LibvirtSpawner: def __init__(self, resalloc_pool_id, log, args): self.args = args self.vm_name = args.name - host_id, self.connection, self.arch = get_hv_identification_from_pool_id( + self.connection, self.arch, *_ = get_hv_identification_from_pool_id( resalloc_pool_id) self.config_files.append(ConfigFile( "resalloc-vars.sh", @@ -382,46 +384,20 @@ def get_arg_parser(): parser.add_argument('--ram-size', metavar='MB', default=4096) parser.add_argument('--name') parser.add_argument('--resalloc-pool-id') - parser.add_argument('--resalloc-id-in-pool') + parser.add_argument('--resalloc-ip-offset') return parser -def get_fedora_ipv6_address(pool_id, id_in_pool, dev, log): +def get_fedora_ipv6_address(pool_id, ip_offset, log): """ Statically assign IPv6 + Gateway based on id_in_pool. """ - - hv_id, _, _ = get_hv_identification_from_pool_id(pool_id) - hv_id = int(hv_id) - - # rdu-cc - # gateway = "2620:52:3:1:ffff:ffff:ffff:fffe" - # base = "2620:52:3:1:dead:beef:cafe:c" - - # rdu3 https://github.com/fedora-copr/copr/issues/3786#issuecomment-3412337856 - gateway = "2620:52:6:1161::1" - base = "2620:52:6:1161:dead:beef:cafe:c" - - # The initial 256 addresses (:c0XX) is reserved for hypervisor machines - # itself (not really in rdu3, per copr issue/3786 at least, but it doesn't - # hurt to wait those in practice so we keep the pattern). - start = 0x100 - - # each hypervisor has block of 64 IPs - block = 64 - - log.info("Hypervisor ID: %s", hv_id) - - # give 48 IPs to each hv (32 prod, some dev), currently 4*48=192 ips - offset = hv_id * block - if not dev: - # give the first 8 addresses to Copr dev instance - offset += 8 - - addr_number = start + offset + int(id_in_pool) - addr_number = "{0:#05x}".format(addr_number).replace("0x", "") - log.info("Using an IPv6 ending with ':c%s'", addr_number) - return base + addr_number, gateway + _, _, ipv6_start_string, gw = get_hv_identification_from_pool_id(pool_id) + log.info("Getting IP in %s + 64 range", ipv6_start_string) + ip_start = ipaddress.IPv6Address(ipv6_start_string) + ip = ip_start + int(ip_offset) + log.info("IP %s", ip) + return str(ip), gw def _main(): @@ -440,16 +416,12 @@ def _main(): _arange_default("name", "RESALLOC_NAME") _arange_default("resalloc_pool_id", "RESALLOC_POOL_ID") - _arange_default("resalloc_id_in_pool", "RESALLOC_ID_IN_POOL") - - devel = True - if "prod" in args.name: - devel = False - + _arange_default("resalloc_ip_offset", + "RESALLOC_NAMED_COUNTER_FEDORA_IPV6_POOL") ip6_a, ip6_g = get_fedora_ipv6_address(args.resalloc_pool_id, - args.resalloc_id_in_pool, - devel, log) + args.resalloc_ip_offset, + log) spawner = LibvirtSpawner(args.resalloc_pool_id, log, args) spawner.add_nat_network() diff --git a/roles/copr/backend/templates/pulp-cli.toml b/roles/copr/backend/templates/pulp-cli.toml index 94194d9549..c39f3788bc 100644 --- a/roles/copr/backend/templates/pulp-cli.toml +++ b/roles/copr/backend/templates/pulp-cli.toml @@ -1,15 +1,20 @@ [cli] -base_url = "https://mtls.internal.console.redhat.com" +base_url = "https://packages.redhat.com" api_root = "/api/pulp/" -username = "" -password = "" +{% if devel %} +username = "{{ copr_dev_pulp_username }}" +password = "{{ copr_dev_pulp_password }}" +{% else %} +username = "{{ copr_prod_pulp_username }}" +password = "{{ copr_prod_pulp_password }}" +{% endif %} {% if env == "production" %} domain = "public-copr" {% else %} domain = "public-copr-stage" {% endif %} -cert = "/home/copr/.config/pulp/copr-pulp.crt" -key = "/home/copr/.config/pulp/copr-pulp.key" +cert = "" +key = "" verify_ssl = true format = "json" dry_run = false diff --git a/roles/copr/backend/templates/resalloc/pools.yaml.expand.sh b/roles/copr/backend/templates/resalloc/pools.yaml.expand.sh index 4bec454a86..4d4c97b01e 100755 --- a/roles/copr/backend/templates/resalloc/pools.yaml.expand.sh +++ b/roles/copr/backend/templates/resalloc/pools.yaml.expand.sh @@ -12,21 +12,20 @@ outdir=/tmp/pools_debugging sourcedir=$(dirname "$0") gitroot=$(cd "$sourcedir" && git rev-parse --show-toplevel) -pools=$gitroot/roles/copr/backend/templates/resalloc/pools.yaml.j2 if ! test -d "$outdir"; then - mkdir -p "$outdir" + mkdir -p "$outdir/python" (cd "$outdir" && git init .) fi +instances="devel production" -for i in devel production; do - if test $i = production; then +for instance in $instances; do + mkdir -p "$outdir/$instance" + if test "$instance" = production; then vars=$gitroot/inventory/group_vars/copr_aws - file=pools.prod.yaml else vars=$gitroot/inventory/group_vars/copr_dev_aws - file=pools.dev.yaml fi cat > $pbook < + A pool that provides one high-perf ppc64le machine on p09 {{ hv }} machine + in the Fedora Community Cage. These machines have POWER9 processors and + are located in RDU (N Carolina). Thank you Fedora Infrastructure team for + maintaining the hypervisors. + {% endif %} {% endfor %} diff --git a/roles/copr/certbot/tasks/letsencrypt.yml b/roles/copr/certbot/tasks/letsencrypt.yml index c1efefeaf2..70f70ac6b3 100644 --- a/roles/copr/certbot/tasks/letsencrypt.yml +++ b/roles/copr/certbot/tasks/letsencrypt.yml @@ -1,7 +1,7 @@ --- - set_fact: le_source_path: /etc/letsencrypt - # https://forge.fedoraproject.org/infra/tickets/10524 + # https://forge.fedoraproject.org/infra/tickets/issues/10524 le_backup_path: /srv/certbot-certs tags: - certbot @@ -111,7 +111,7 @@ # several occasions and then 'lighttpd' user needs to have the access. See the # following issues: # https://pagure.io/copr/copr/issue/2001 Resolves: -# https://forge.fedoraproject.org/infra/tickets/10391 +# https://forge.fedoraproject.org/infra/tickets/issues/10391 - name: Allow lighttpd to step into certbots directories acl: path: "{{ item }}" diff --git a/roles/copr/frontend/templates/copr.conf b/roles/copr/frontend/templates/copr.conf index 7715eb1251..c3796cd2d6 100644 --- a/roles/copr/frontend/templates/copr.conf +++ b/roles/copr/frontend/templates/copr.conf @@ -280,6 +280,10 @@ EXTRA_BUILDCHROOT_TAGS = [{ # powerful builders for RISC-V team - specific packages "pattern": "@forge-riscv-members/.*/.*riscv64/(kernel|gcc|llvm|clang).*", "tags": ["on_demand_powerful"], +},{ + # https://github.com/fedora-copr/copr/issues/4199 + "pattern": "eseiker/asahi-el-kernel/.*(x86_64|aarch64)/kernel", + "tags": ["on_demand_powerful"], }] {% endif %} diff --git a/roles/copr/keygen/files/backup_keyring.sh b/roles/copr/keygen/files/backup_keyring.sh index cb327d3152..8cf7d53636 100644 --- a/roles/copr/keygen/files/backup_keyring.sh +++ b/roles/copr/keygen/files/backup_keyring.sh @@ -7,7 +7,7 @@ # - it means we should always have at most a 24h-old backup # - the root gpg keychain should have PUBLIC key with `user name` # copr-keygen-backup-key, per -# https://forge.fedoraproject.org/infra/tickets/8904 +# https://forge.fedoraproject.org/infra/tickets/issues/8904 # - fixed in https://github.com/fedora-copr/copr/issues/3532 PATH_TO_KEYRING_DIR="/var/lib/copr-keygen" diff --git a/roles/dhcp_server/files/dhcpd.conf.noc01.rdu3.fedoraproject.org b/roles/dhcp_server/files/dhcpd.conf.noc01.rdu3.fedoraproject.org index 044e639ceb..cc153d7c68 100644 --- a/roles/dhcp_server/files/dhcpd.conf.noc01.rdu3.fedoraproject.org +++ b/roles/dhcp_server/files/dhcpd.conf.noc01.rdu3.fedoraproject.org @@ -1686,3 +1686,12 @@ host retrace03.rdu3.fedoraproject.org { option subnet-mask 255.255.255.0; option host-name "retrace03.rdu3.fedoraproject.org"; } + +host gpu01.rdu3.fedoraproject.org { + hardware ethernet 30:3e:a7:2a:65:28; + fixed-address 10.16.179.35; + next-server 10.16.163.10; + option routers 10.16.179.254; + option subnet-mask 255.255.255.0; + option host-name "gpu01.rdu3.fedoraproject.org"; +} diff --git a/roles/distgit/files/pagure_transfer_group.py b/roles/distgit/files/pagure_transfer_group.py new file mode 100644 index 0000000000..273a8b7ff3 --- /dev/null +++ b/roles/distgit/files/pagure_transfer_group.py @@ -0,0 +1,62 @@ +# -*- coding: utf-8 -*- +from __future__ import print_function, unicode_literals, absolute_import +import argparse +import logging +import os +import pagure.config +import pagure.exceptions +import pagure.lib.model_base +import pagure.lib.query + +def parse_arguments(): + """ Set-up the argument parsing. """ + parser = argparse.ArgumentParser() + parser.add_argument("group_name", help="Name of the group") + parser.add_argument( + "new_creator", + help="Name of the user that will be the new creator of the group", + ) + return parser.parse_args() + +def transfer_group(session, args): + # Validate user + if not args.new_creator: + raise pagure.exceptions.PagureException( + "An username must be provided to associate with the group" + ) + user = pagure.lib.query.get_user(session, args.new_creator) + # Validate group + group = pagure.lib.query.search_groups(session, group_name=args.group_name) + if not group: + raise pagure.exceptions.PagureException( + "The group %r does not exist" % args.group_name + ) + + if group.user_id == user.id: + raise pagure.exceptions.PagureException( + "The group %r was already created by user %r" % (args.group_name, args.new_creator) + ) + + previous_creator = group.creator.username + group.user_id = user.id + session.commit() + print("Group %r was transfered from %r to %r." % (args.group_name, previous_creator, args.new_creator)) + +def main(): + # Parse the arguments + args = parse_arguments() + if "PAGURE_CONFIG" not in os.environ and os.path.exists( + "/etc/pagure/pagure.cfg" + ): + print("Using configuration file `/etc/pagure/pagure.cfg`") + os.environ["PAGURE_CONFIG"] = "/etc/pagure/pagure.cfg" + _config = pagure.config.reload_config() + session = pagure.lib.model_base.create_session(_config["DB_URL"]) + try: + transfer_group(session, args) + finally: + session.remove() + +if __name__ == "__main__": + main() + diff --git a/roles/distgit/pagure/tasks/main.yml b/roles/distgit/pagure/tasks/main.yml index 0153f52c5f..5fed6f4fb5 100644 --- a/roles/distgit/pagure/tasks/main.yml +++ b/roles/distgit/pagure/tasks/main.yml @@ -83,7 +83,7 @@ - pagure - hotfix -# Fix for https://forge.fedoraproject.org/infra/tickets/11957 +# Fix for https://forge.fedoraproject.org/infra/tickets/issues/11957 - name: Set ACL for newly created files in /var/log/pagure ansible.posix.acl: path: /var/log/pagure diff --git a/roles/distgit/pagure/templates/pagure.cfg b/roles/distgit/pagure/templates/pagure.cfg index a213bedb9e..2cf1fce12f 100644 --- a/roles/distgit/pagure/templates/pagure.cfg +++ b/roles/distgit/pagure/templates/pagure.cfg @@ -39,7 +39,7 @@ DB_URL = 'postgresql://{{ distgit_pagure_db_user }}:{{ distgit_pagure_db_pass }} {% endif %} # Something breaks the database connections after a while, recycle them sooner -# https://forge.fedoraproject.org/infra/tickets/12622 +# https://forge.fedoraproject.org/infra/tickets/issues/12622 DB_POOL_RECYCLE = 300 ### FAS groups of pagure admins diff --git a/roles/distgit/pagure/templates/z_pagure.conf b/roles/distgit/pagure/templates/z_pagure.conf index 34ab6583ad..6a4482b77a 100644 --- a/roles/distgit/pagure/templates/z_pagure.conf +++ b/roles/distgit/pagure/templates/z_pagure.conf @@ -5,6 +5,7 @@ WSGIPythonOptimize 1 WSGIPassAuthorization On WSGIDaemonProcess pagureproc user=pagure group=packager maximum-requests=1000 display-name=pagure processes={{ wsgi_procs }} threads={{ wsgi_threads }} inactivity-timeout=300 WSGIProcessGroup pagureproc +WSGIApplicationGroup %{GLOBAL} WSGIScriptAlias / /var/www/pagure.wsgi Protocols h2 http/1.1 @@ -35,7 +36,7 @@ MaxConnectionsPerChild 1000 # # Redirect files viewed in master to the main branch -# https://forge.fedoraproject.org/infra/tickets/9620 +# https://forge.fedoraproject.org/infra/tickets/issues/9620 RedirectMatch 302 (.*)/rpms/(.*)blob/master/(.*) $1/rpms/$2/blob/rawhide/$3 RedirectMatch 302 (.*)/container/(.*)blob/master/(.*) $1/container/$2/blob/rawhide/$3 RedirectMatch 302 (.*)/flatpaks/(.*)blob/master/(.*) $1/flatpaks/$2/blob/stable/$3 diff --git a/roles/distgit/tasks/main.yml b/roles/distgit/tasks/main.yml index 5d3094600e..1c1f78375b 100644 --- a/roles/distgit/tasks/main.yml +++ b/roles/distgit/tasks/main.yml @@ -306,6 +306,19 @@ - distgit - mass-branching +- name: Deploy the pagure group transfer script + ansible.builtin.copy: + src: pagure_transfer_group.py + dest: /usr/local/bin/pagure_transfer_group.py + owner: root + group: root + mode: '0755' + when: env == 'staging' + tags: + - config + - distgit + - scripts + # -- Lookaside Cache ------------------------------------- # This is the annex to Dist Git, where we host source tarballs. - name: Install the Lookaside Cache httpd configs @@ -485,7 +498,7 @@ - grokmirror - pkgs -# https://forge.fedoraproject.org/infra/tickets/12428 +# https://forge.fedoraproject.org/infra/tickets/issues/12428 - name: Hotfix for links to accounts.fpo ansible.posix.patch: src: files/0001-Fix-link-to-accounts.fpo-for-staging-for-adding-user.patch diff --git a/roles/distgit/templates/lookaside.conf b/roles/distgit/templates/lookaside.conf index de5b441c14..b0d1c28b1c 100644 --- a/roles/distgit/templates/lookaside.conf +++ b/roles/distgit/templates/lookaside.conf @@ -3,5 +3,16 @@ Alias /lookaside /srv/cache/lookaside Options Indexes FollowSymLinks AllowOverride None Require all granted - +{% if env == 'staging' %} + # Disable mod_mime's automatic Content-Encoding for archives + RemoveEncoding .gz .tgz .bz2 .xz .zst .lzma .crate + + # Force all lookaside files to be served as generic binary data. + # This prevents mod_mime_magic from sniffing .crate files and adding x-gzip encoding. + ForceType application/octet-stream + + # Explicitly remove Content-Encoding just to be absolutely sure + Header unset Content-Encoding +{% endif %} + diff --git a/roles/dns/files/zones.conf b/roles/dns/files/zones.conf index bd73f6d9b9..e87c30703a 100644 --- a/roles/dns/files/zones.conf +++ b/roles/dns/files/zones.conf @@ -222,6 +222,11 @@ zone "fedora.pe" { file "/var/named/master/built/fedora.pe"; }; +zone "fedora.pt" { + type master; + file "/var/named/master/built/fedora.pt"; +}; + zone "fedora.tk" { type master; file "/var/named/master/built/fedora.tk"; diff --git a/roles/fasjson/files/aliases.static b/roles/fasjson/files/aliases.static index 19e8c66ce2..0f611a5459 100644 --- a/roles/fasjson/files/aliases.static +++ b/roles/fasjson/files/aliases.static @@ -208,7 +208,7 @@ fudcon-emea: flock-staff # fudcon-na: fudcon-latam: flock-staff -# some flock aliases disabled for now per https://forge.fedoraproject.org/infra/tickets/10411 +# some flock aliases disabled for now per https://forge.fedoraproject.org/infra/tickets/issues/10411 # flock #flockpress: fca,fpl #flockinfo: fca,fpl @@ -217,10 +217,10 @@ flock-staff: flock-team-members@fedoraproject.org #flock-access: flock-admin # # flock/sponsor alias -# https://forge.fedoraproject.org/infra/tickets/10591 +# https://forge.fedoraproject.org/infra/tickets/issues/10591 sponsors: flock-team-sponsors@fedoraproject.org -# swag customer email alias https://forge.fedoraproject.org/infra/tickets/10794 +# swag customer email alias https://forge.fedoraproject.org/infra/tickets/issues/10794 swag-info: fca,npazmino@redhat.com # News @@ -407,10 +407,10 @@ containerbuild: cverna # FTI bugzilla script - https://pagure.io/releng/issue/11169 fti-bugs: churchyard -# copr team - https://forge.fedoraproject.org/infra/tickets/10351 +# copr team - https://forge.fedoraproject.org/infra/tickets/issues/10351 coprteam: copr-team@redhat.com -# fedora-review-bot - https://forge.fedoraproject.org/infra/tickets/11232 +# fedora-review-bot - https://forge.fedoraproject.org/infra/tickets/issues/11232 fedora-review-bot: jkadlcik@redhat.com # packaging-reports - this account is used to send via out smtp servers various packaging reports @@ -418,4 +418,3 @@ fedora-review-bot: jkadlcik@redhat.com packaging-reports: maxwell@gtmx.me #### The rest of this file is automatically generated - edit using the accounts system! - diff --git a/roles/fedora-docs/proxy/files/docs-rsync b/roles/fedora-docs/proxy/files/docs-rsync index 2e032076b9..b44885856e 100755 --- a/roles/fedora-docs/proxy/files/docs-rsync +++ b/roles/fedora-docs/proxy/files/docs-rsync @@ -1,6 +1,6 @@ #!/bin/sh -# See https://forge.fedoraproject.org/infra/tickets/7130 +# See https://forge.fedoraproject.org/infra/tickets/issues/7130 # Basically we have an OLD (old 7 years ago, as of 2025-10) set of content # and a new set of content. We want to serve the new content, and if nothing @@ -11,7 +11,7 @@ # 2. rsync new to docs.fedoraproject.org # 3. rsync docs.fedoraproject.org to docs-combined -# BUT see https://forge.fedoraproject.org/infra/tickets/12848 +# BUT see https://forge.fedoraproject.org/infra/tickets/issues/12848 # -a is the std. copy everything "archive" mode. # -H means try to copy hardlinks as hardlinks diff --git a/roles/fedora-docs/proxy/files/docs-rsync.stg b/roles/fedora-docs/proxy/files/docs-rsync.stg index 2e032076b9..b44885856e 100755 --- a/roles/fedora-docs/proxy/files/docs-rsync.stg +++ b/roles/fedora-docs/proxy/files/docs-rsync.stg @@ -1,6 +1,6 @@ #!/bin/sh -# See https://forge.fedoraproject.org/infra/tickets/7130 +# See https://forge.fedoraproject.org/infra/tickets/issues/7130 # Basically we have an OLD (old 7 years ago, as of 2025-10) set of content # and a new set of content. We want to serve the new content, and if nothing @@ -11,7 +11,7 @@ # 2. rsync new to docs.fedoraproject.org # 3. rsync docs.fedoraproject.org to docs-combined -# BUT see https://forge.fedoraproject.org/infra/tickets/12848 +# BUT see https://forge.fedoraproject.org/infra/tickets/issues/12848 # -a is the std. copy everything "archive" mode. # -H means try to copy hardlinks as hardlinks diff --git a/roles/fedoraloveskde/build/files/syncfedoraloveskde.sh b/roles/fedoraloveskde/build/files/syncfedoraloveskde.sh index 5b9c667215..0834fddfa1 100644 --- a/roles/fedoraloveskde/build/files/syncfedoraloveskde.sh +++ b/roles/fedoraloveskde/build/files/syncfedoraloveskde.sh @@ -2,7 +2,7 @@ if [ ! -d /srv/web/fedoraloveskde.org/.git ] then - /usr/bin/git clone -q https://pagure.io/fedora-kde/fedoraloveskde.org /srv/web/fedoraloveskde.org + /usr/bin/git clone -q https://forge.fedoraproject.org/kde/fedoraloveskde.org /srv/web/fedoraloveskde.org fi cd /srv/web/fedoraloveskde.org diff --git a/roles/fedoraloveskde/build/files/syncfedoraloveskde.stg.sh b/roles/fedoraloveskde/build/files/syncfedoraloveskde.stg.sh index abe804407f..438448c8e7 100644 --- a/roles/fedoraloveskde/build/files/syncfedoraloveskde.stg.sh +++ b/roles/fedoraloveskde/build/files/syncfedoraloveskde.stg.sh @@ -2,7 +2,7 @@ if [ ! -d /srv/web/fedoraloveskde.org/.git ] then - /usr/bin/git clone -q https://pagure.io/fedora-kde/fedoraloveskde.org /srv/web/fedoraloveskde.org + /usr/bin/git clone -q https://forge.fedoraproject.org/kde/fedoraloveskde.org /srv/web/fedoraloveskde.org fi cd /srv/web/fedoraloveskde.org diff --git a/roles/github2fedmsg/tasks/main.yml b/roles/github2fedmsg/tasks/main.yml index 2cb7e94b4b..09763f4889 100644 --- a/roles/github2fedmsg/tasks/main.yml +++ b/roles/github2fedmsg/tasks/main.yml @@ -67,7 +67,7 @@ notify: - Restart apache - # Fix for https://forge.fedoraproject.org/infra/tickets/11776 + # Fix for https://forge.fedoraproject.org/infra/tickets/issues/11776 - name: Hotfix - Fix the KeyError when looking for user in github event ansible.posix.patch: src: 11776.patch diff --git a/roles/haproxy/templates/haproxy.cfg b/roles/haproxy/templates/haproxy.cfg index 9befc67a8c..3537daf0e7 100644 --- a/roles/haproxy/templates/haproxy.cfg +++ b/roles/haproxy/templates/haproxy.cfg @@ -237,23 +237,6 @@ backend oci-candidate-registry-backend balance hdr(appserver) server oci-candidate-registry01 oci-candidate-registry01:5000 check inter 10s rise 1 fall 2 -{% if datacenter == 'rdu3' %} - -# Only enable this on rdu3 proxies -frontend src-frontend - bind 0.0.0.0:10057 - default_backend src-backend - -backend src-backend - balance hdr(appserver) -{% if env == "staging" or datacenter == 'rdu3' %} - server pkgs01 pkgs01:80 check inter 10s rise 1 fall 2 -{% endif %} - option httpchk GET / - retries 5 - retry-on all-retryable-errors - -{% endif %} # This is an endpoint using only ipa01. This is used for API access, since sessions # are not synchronized. frontend ipa01-frontend diff --git a/roles/hosts/gpu01.rdu3.fedoraproject.org-hosts b/roles/hosts/gpu01.rdu3.fedoraproject.org-hosts new file mode 100644 index 0000000000..fe6a760640 --- /dev/null +++ b/roles/hosts/gpu01.rdu3.fedoraproject.org-hosts @@ -0,0 +1,7 @@ +127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 +::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 + +# Map canonical names of IPA servers to their VPN IP addresses +192.168.1.156 ipa01.rdu3.fedoraproject.org +192.168.1.157 ipa02.rdu3.fedoraproject.org +192.168.1.162 ipa03.rdu3.fedoraproject.org diff --git a/roles/httpd/proxy/templates/httpd.conf.j2 b/roles/httpd/proxy/templates/httpd.conf.j2 index f0c2cdf7f8..188fdb286c 100644 --- a/roles/httpd/proxy/templates/httpd.conf.j2 +++ b/roles/httpd/proxy/templates/httpd.conf.j2 @@ -87,7 +87,7 @@ KeepAlive On # during a persistent connection. Set to 0 to allow an unlimited amount. # We recommend you leave this number high, for maximum performance. # -MaxKeepAliveRequests 500 +MaxKeepAliveRequests 0 # # KeepAliveTimeout: Number of seconds to wait for the next request from the diff --git a/roles/ipsilon/files/openid_banner.patch b/roles/ipsilon/files/openid_banner.patch index 5fc03fdb96..0fe9df268b 100644 --- a/roles/ipsilon/files/openid_banner.patch +++ b/roles/ipsilon/files/openid_banner.patch @@ -7,7 +7,7 @@ +
+

+ You are using OpenID to authenticate. This authentication method will go away on 1st May 2026. Please consider migrating to OpenID Connect. -+ For more info look at https://forge.fedoraproject.org/infra/tickets/10241 ++ For more info look at https://forge.fedoraproject.org/infra/tickets/issues/10241 +

+
diff --git a/roles/kickstarts/tasks/main.yml b/roles/kickstarts/tasks/main.yml index 26309e0bf2..1d799748ed 100644 --- a/roles/kickstarts/tasks/main.yml +++ b/roles/kickstarts/tasks/main.yml @@ -12,6 +12,7 @@ - hardware-rhel-9-06disk - hardware-rhel-10-08disk - hardware-rhel-10-nodisk + - hardware-fedora-01disk-nvme - hardware-fedora-04disk-power10 - hardware-fedora-06disk - hardware-fedora-06disk-nvme diff --git a/roles/kickstarts/templates/hardware-fedora-01disk-nvme.j2 b/roles/kickstarts/templates/hardware-fedora-01disk-nvme.j2 new file mode 100644 index 0000000000..f672b106d8 --- /dev/null +++ b/roles/kickstarts/templates/hardware-fedora-01disk-nvme.j2 @@ -0,0 +1,79 @@ +# +## This kickstart is for Dell systems with 8 disks. It will build either a virthost or cloud. +## + +# Use network installation +vnc --password "{{ kickstart_vnc_password }}" +# Use network install +# metalink here should give internal mirrors in rdu3 and external ones outside +url --metalink "https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=$basearch" +repo --name=updates --metalink "https://mirrors.fedoraproject.org/metalink?repo=updates-released-f$releasever&arch=$basearch" + +# Firewall configuration +firewall --disabled +firstboot --disable +ignoredisk --only-use=nvme0n1 +# Keyboard layouts +# old format: keyboard us +# new format: +keyboard --vckeymap=us --xlayouts='' +# System language +lang en_US.UTF-8 + +# Network information +# Reboot after installation +reboot +# Root password +rootpw --iscrypted "{{ kickstart_initial_password_encrypted }}" +# SELinux configuration +selinux --enforcing +# System services +services --disabled="firewalld,kdump" --enabled="postfix,chronyd" +# Do not configure the X Window System +skipx +# System timezone +timezone UTC --utc +# System bootloader configuration +bootloader --location=mbr --boot-drive=nvme0n1 --append="net.ifnames=0" --driveorder=nvme0n1 +zerombr +clearpart --drives=nvme0n1 --all --initlabel + +# Disk partitioning information +reqpart --add-boot +part btrfs.007 --size=2000 --fstype=btrfs --grow --ondisk=nvme0n1 +btrfs none --label=fedora btrfs.007 +btrfs / --subvol --name=root LABEL=fedora + +%packages +-geolite2-city +-iwl*firmware +-subscription-manager +bash-completion +bind-utils +clevis* +cronie-noanacron +crontabs +dhclient +grubby +iptables-services +nfs-utils +nmap-ncat +openssh-clients +openssh-server +patch +postfix +rsync +screen +strace +s-nail +tmux +traceroute +vim-enhanced +zsh +%end + +%post --nochroot --log=/mnt/sysimage/root/post.output --erroronfail +mkdir /mnt/sysimage/root/tmp +chroot /mnt/sysimage /usr/bin/curl https://infrastructure.fedoraproject.org/rhel/ks/post/fedora-post.sh -o /root/tmp/fedora-post.sh +chroot /mnt/sysimage sh /root/tmp/fedora-post.sh +%end diff --git a/roles/koji_builder/tasks/main.yml b/roles/koji_builder/tasks/main.yml index 466296ae8c..a7ca34ae7e 100644 --- a/roles/koji_builder/tasks/main.yml +++ b/roles/koji_builder/tasks/main.yml @@ -143,7 +143,7 @@ # edk2-arm is dead as of F42 ... # From: https://src.fedoraproject.org/rpms/edk2/ # commit 53fa21dafcef14a7d264a89c2977cdbef58df770 -# edk2-aarch64 says: UEFI firmware for aarch64 virtual machines +# edk2-aarch64 says: UEFI firmware for aarch64 virtual machines - name: Install arm UEFI firmware package (aarch64 only) ansible.builtin.package: name=edk2-aarch64 state=present tags: @@ -289,7 +289,7 @@ - koji_builder when: env == "staging" -# https://forge.fedoraproject.org/infra/tickets/6636 +# https://forge.fedoraproject.org/infra/tickets/issues/6636 - name: Install libkcapi to get increased sockets on armv7 dnf: name=libkcapi enablerepo=updates-testing state=present tags: diff --git a/roles/koji_hub/templates/kojihub.conf.j2 b/roles/koji_hub/templates/kojihub.conf.j2 index a3a69b5455..7a28811667 100644 --- a/roles/koji_hub/templates/kojihub.conf.j2 +++ b/roles/koji_hub/templates/kojihub.conf.j2 @@ -3,6 +3,7 @@ # KeepAlive On +MaxKeepAliveRequests 0 Alias /kojihub /usr/share/koji-hub/kojiapp.py diff --git a/roles/koji_hub/templates/kojiweb.conf.j2 b/roles/koji_hub/templates/kojiweb.conf.j2 index 2c881ad5fe..1686fe0027 100644 --- a/roles/koji_hub/templates/kojiweb.conf.j2 +++ b/roles/koji_hub/templates/kojiweb.conf.j2 @@ -2,7 +2,7 @@ RewriteEngine On RewriteRule ^/$ /koji/ [R,L] KeepAlive On KeepAliveTimeout 16 -MaxKeepAliveRequests 500 +MaxKeepAliveRequests 0 ServerLimit 2000 MaxRequestWorkers 2000 diff --git a/roles/kojipkgs/templates/kojipkgs.conf b/roles/kojipkgs/templates/kojipkgs.conf index 9a7f8fe03a..96d70b50a7 100644 --- a/roles/kojipkgs/templates/kojipkgs.conf +++ b/roles/kojipkgs/templates/kojipkgs.conf @@ -17,6 +17,12 @@ Alias /ostree /mnt/fedora_koji/koji/ostree Require all granted + + Options -Indexes +FollowSymLinks + FileETag None + Require all granted + + Alias /atomic-cd /mnt/fedora_koji/koji/atomic-cd diff --git a/roles/mailman3/files/enable_dmarc_mitigation.py b/roles/mailman3/files/enable_dmarc_mitigation.py index f94405b726..e23a949cdf 100644 --- a/roles/mailman3/files/enable_dmarc_mitigation.py +++ b/roles/mailman3/files/enable_dmarc_mitigation.py @@ -7,7 +7,7 @@ For more info about DMARC mitigation in mailman3 see https://docs.mailman3.org/projects/mailman/en/latest/src/mailman/handlers/docs/dmarc-mitigations.html For more info why Fedora is doing this see -https://forge.fedoraproject.org/infra/tickets/11427 +https://forge.fedoraproject.org/infra/tickets/issues/11427 The script will set dmarc_mitigate related columns in `mailman` table to preferred values. diff --git a/roles/mailman3/files/top.html b/roles/mailman3/files/top.html index 8ca196eb19..8236e65018 100644 --- a/roles/mailman3/files/top.html +++ b/roles/mailman3/files/top.html @@ -1,10 +1,10 @@ - + diff --git a/roles/mailman3/tasks/main.yml b/roles/mailman3/tasks/main.yml index 33abb037a0..384a25e9e1 100644 --- a/roles/mailman3/tasks/main.yml +++ b/roles/mailman3/tasks/main.yml @@ -294,7 +294,7 @@ - Initialize authentication providers # Add the Nest banner -# See issue https://forge.fedoraproject.org/infra/tickets/10103 +# See issue https://forge.fedoraproject.org/infra/tickets/issues/10103 - name: Replace the top.html template in hyperkitty/postorius to change banner ansible.builtin.copy: src: top.html @@ -424,7 +424,7 @@ - Restart mailmanweb # Fix for mailman -# https://forge.fedoraproject.org/infra/tickets/12011 +# https://forge.fedoraproject.org/infra/tickets/issues/12011 # This should no longer be needed after update to mailman 3.3.10 - name: Apply hotfix for orphan issue ansible.posix.patch: diff --git a/roles/nagios_client/files/scripts/check_rabbitmq_size b/roles/nagios_client/files/scripts/check_rabbitmq_size deleted file mode 100644 index 9231ed239b..0000000000 --- a/roles/nagios_client/files/scripts/check_rabbitmq_size +++ /dev/null @@ -1,26 +0,0 @@ -#!/bin/python3 -import sys -import requests - -url = 'http://localhost:15672/api/queues/%%2f/%s' % (sys.argv[1]) - -r = requests.get(url, auth=('guest', 'guest')).json() -consumers = r['consumers'] -messages = r['messages'] - -msg = 'Messages in queue: %i (%i consumers)' % (messages, consumers) - -if consumers < 1: - print ('CRITICAL: No consumers: %s' % msg) - sys.exit(2) - -if messages > int(sys.argv[2]): - print ('WARNING: %s' % msg) - sys.exit(1) - -if messages > int(sys.argv[3]): - print ('CRITICAL: %s' % msg) - sys.exit(1) - -print ('OK: %s' % msg) -sys.exit(0) diff --git a/roles/nagios_client/tasks/main.yml b/roles/nagios_client/tasks/main.yml index 07d5fe30ca..3e761b1ee1 100644 --- a/roles/nagios_client/tasks/main.yml +++ b/roles/nagios_client/tasks/main.yml @@ -70,7 +70,6 @@ - check_raid.py - check_lock - check_fcomm_queue - - check_rabbitmq_size - check_datanommer_timesince.py - check_memcache_connect - check_readonly_fs @@ -352,38 +351,6 @@ name: /etc/openvpn/server/crl.pem state: present -- name: Install nrpe config for the RabbitMQ checks - ansible.builtin.template: - src: "rabbitmq_args.ini.j2" - dest: "/etc/nrpe.d/rabbitmq_args.ini" - owner: root - group: nrpe - mode: "0640" - when: inventory_hostname.startswith('rabbitmq') - tags: - - nagios_client - -- name: Install nrpe checks for the RabbitMQ cluster - ansible.builtin.template: - src: "{{ item }}.j2" - dest: "/etc/nrpe.d/{{ item }}" - owner: root - group: root - mode: "0644" - with_items: - - check_rabbitmq_server.cfg - - check_rabbitmq_watermark.cfg - - check_rabbitmq_cluster.cfg - - check_rabbitmq_connections.cfg - - check_rabbitmq_overview.cfg - - check_rabbitmq_exchange.cfg - - check_rabbitmq_queue.cfg - when: inventory_hostname.startswith('rabbitmq') - notify: - - Restart nrpe - tags: - - nagios_client - - name: Nrpe service start ansible.builtin.service: name: nrpe diff --git a/roles/nagios_client/templates/check_rabbitmq_cluster.cfg.j2 b/roles/nagios_client/templates/check_rabbitmq_cluster.cfg.j2 deleted file mode 100644 index 110e865bef..0000000000 --- a/roles/nagios_client/templates/check_rabbitmq_cluster.cfg.j2 +++ /dev/null @@ -1 +0,0 @@ -command[check_rabbitmq_cluster]=/usr/lib64/nagios/plugins-rabbitmq/check_rabbitmq_cluster --extra-opts=common@/etc/nrpe.d/rabbitmq_args.ini -n rabbit@rabbitmq01.rdu3.fedoraproject.org,rabbit@rabbitmq02.rdu3.fedoraproject.org,rabbit@rabbitmq03.rdu3.fedoraproject.org diff --git a/roles/nagios_client/templates/check_rabbitmq_connections.cfg.j2 b/roles/nagios_client/templates/check_rabbitmq_connections.cfg.j2 deleted file mode 100644 index 8b30a7ff35..0000000000 --- a/roles/nagios_client/templates/check_rabbitmq_connections.cfg.j2 +++ /dev/null @@ -1 +0,0 @@ -command[check_rabbitmq_connections]=/usr/lib64/nagios/plugins-rabbitmq/check_rabbitmq_connections --extra-opts=common@/etc/nrpe.d/rabbitmq_args.ini \ No newline at end of file diff --git a/roles/nagios_client/templates/check_rabbitmq_exchange.cfg.j2 b/roles/nagios_client/templates/check_rabbitmq_exchange.cfg.j2 deleted file mode 100644 index c5a729e42e..0000000000 --- a/roles/nagios_client/templates/check_rabbitmq_exchange.cfg.j2 +++ /dev/null @@ -1,2 +0,0 @@ -command[check_rabbitmq_exchange_pubsub]=/usr/lib64/nagios/plugins-rabbitmq/check_rabbitmq_exchange --extra-opts=common@/etc/nrpe.d/rabbitmq_args.ini --vhost /pubsub --exchange amq.topic --period 600 -command[check_rabbitmq_exchange_public_pubsub]=/usr/lib64/nagios/plugins-rabbitmq/check_rabbitmq_exchange --extra-opts=common@/etc/nrpe.d/rabbitmq_args.ini --vhost /public_pubsub --exchange amq.topic --period 600 \ No newline at end of file diff --git a/roles/nagios_client/templates/check_rabbitmq_overview.cfg.j2 b/roles/nagios_client/templates/check_rabbitmq_overview.cfg.j2 deleted file mode 100644 index 060fb2fab5..0000000000 --- a/roles/nagios_client/templates/check_rabbitmq_overview.cfg.j2 +++ /dev/null @@ -1 +0,0 @@ -command[check_rabbitmq_overview]=/usr/lib64/nagios/plugins-rabbitmq/check_rabbitmq_overview --extra-opts=common@/etc/nrpe.d/rabbitmq_args.ini \ No newline at end of file diff --git a/roles/nagios_client/templates/check_rabbitmq_queue.cfg.j2 b/roles/nagios_client/templates/check_rabbitmq_queue.cfg.j2 deleted file mode 100644 index d9c9367ce1..0000000000 --- a/roles/nagios_client/templates/check_rabbitmq_queue.cfg.j2 +++ /dev/null @@ -1,2 +0,0 @@ -command[check_rabbitmq_queue_pubsub]=/usr/lib64/nagios/plugins-rabbitmq/check_rabbitmq_queue --extra-opts=common@/etc/nrpe.d/rabbitmq_args.ini --vhost /pubsub -command[check_rabbitmq_queue_public_pubsub]=/usr/lib64/nagios/plugins-rabbitmq/check_rabbitmq_queue --extra-opts=common@/etc/nrpe.d/rabbitmq_args.ini --vhost /public_pubsub \ No newline at end of file diff --git a/roles/nagios_client/templates/check_rabbitmq_server.cfg.j2 b/roles/nagios_client/templates/check_rabbitmq_server.cfg.j2 deleted file mode 100644 index bc8fb20534..0000000000 --- a/roles/nagios_client/templates/check_rabbitmq_server.cfg.j2 +++ /dev/null @@ -1 +0,0 @@ -command[check_rabbitmq_server]=/usr/lib64/nagios/plugins-rabbitmq/check_rabbitmq_server --extra-opts=common@/etc/nrpe.d/rabbitmq_args.ini --node={{ inventory_hostname }} \ No newline at end of file diff --git a/roles/nagios_client/templates/check_rabbitmq_watermark.cfg.j2 b/roles/nagios_client/templates/check_rabbitmq_watermark.cfg.j2 deleted file mode 100644 index 3dc965b8a0..0000000000 --- a/roles/nagios_client/templates/check_rabbitmq_watermark.cfg.j2 +++ /dev/null @@ -1 +0,0 @@ -command[check_rabbitmq_watermark]=/usr/lib64/nagios/plugins-rabbitmq/check_rabbitmq_watermark --extra-opts=common@/etc/nrpe.d/rabbitmq_args.ini --vhost /pubsub --node={{ inventory_hostname }} \ No newline at end of file diff --git a/roles/nagios_client/templates/rabbitmq_args.ini.j2 b/roles/nagios_client/templates/rabbitmq_args.ini.j2 deleted file mode 100644 index ee3078d701..0000000000 --- a/roles/nagios_client/templates/rabbitmq_args.ini.j2 +++ /dev/null @@ -1,4 +0,0 @@ -[common] -hostname = localhost -username = nagios-monitoring -password = {{ (env == 'production')|ternary(rabbitmq_monitoring_password_production, rabbitmq_monitoring_password_staging) }} \ No newline at end of file diff --git a/roles/nagios_server/files/nagios/services/rdu3_internal/rabbitmq.cfg b/roles/nagios_server/files/nagios/services/rdu3_internal/rabbitmq.cfg deleted file mode 100644 index 378a6f0d53..0000000000 --- a/roles/nagios_server/files/nagios/services/rdu3_internal/rabbitmq.cfg +++ /dev/null @@ -1,96 +0,0 @@ -# RabbitMQ processes (for each host) - -define service { - host_name rabbitmq01.rdu3.fedoraproject.org - service_description Check bus server processes - check_command check_by_nrpe!check_rabbitmq_server - use defaulttemplate -} - -define service { - host_name rabbitmq02.rdu3.fedoraproject.org - service_description Check bus server processes - check_command check_by_nrpe!check_rabbitmq_server - use defaulttemplate -} - -define service { - host_name rabbitmq03.rdu3.fedoraproject.org - service_description Check bus server processes - check_command check_by_nrpe!check_rabbitmq_server - use defaulttemplate -} - -# RabbitMQ alarms (for each host) - -define service { - host_name rabbitmq01.rdu3.fedoraproject.org - service_description Check bus server alarms - check_command check_by_nrpe!check_rabbitmq_watermark - use defaulttemplate -} - -define service { - host_name rabbitmq02.rdu3.fedoraproject.org - service_description Check bus server alarms - check_command check_by_nrpe!check_rabbitmq_watermark - use defaulttemplate -} - -define service { - host_name rabbitmq03.rdu3.fedoraproject.org - service_description Check bus server alarms - check_command check_by_nrpe!check_rabbitmq_watermark - use defaulttemplate -} - -# The following results are cluster-wide, no need to run them on each cluster member - -define service { - host_name rabbitmq01.rdu3.fedoraproject.org - service_description Check bus cluster - check_command check_by_nrpe!check_rabbitmq_cluster - use defaulttemplate -} - -define service { - host_name rabbitmq01.rdu3.fedoraproject.org - service_description Check bus cluster connections - check_command check_by_nrpe!check_rabbitmq_connections - use defaulttemplate -} - -define service { - host_name rabbitmq01.rdu3.fedoraproject.org - service_description Check bus cluster overview - check_command check_by_nrpe!check_rabbitmq_overview - use defaulttemplate -} - -define service { - host_name rabbitmq01.rdu3.fedoraproject.org - service_description Check bus exchanges in /pubsub - check_command check_by_nrpe!check_rabbitmq_exchange_pubsub - use defaulttemplate -} - -define service { - host_name rabbitmq01.rdu3.fedoraproject.org - service_description Check bus exchanges in /public_pubsub - check_command check_by_nrpe!check_rabbitmq_exchange_public_pubsub - use defaulttemplate -} - -define service { - host_name rabbitmq01.rdu3.fedoraproject.org - service_description Check bus queues in /pubsub - check_command check_by_nrpe!check_rabbitmq_queue_pubsub - use defaulttemplate -} - -define service { - host_name rabbitmq01.rdu3.fedoraproject.org - service_description Check bus queues in /public_pubsub - check_command check_by_nrpe!check_rabbitmq_queue_public_pubsub - use defaulttemplate -} diff --git a/roles/nagios_server/tasks/main.yml b/roles/nagios_server/tasks/main.yml index 7a5f0c9d93..6ba4042740 100644 --- a/roles/nagios_server/tasks/main.yml +++ b/roles/nagios_server/tasks/main.yml @@ -213,7 +213,6 @@ - mailman.cfg - nrpe.cfg - pgsql.cfg - - rabbitmq.cfg tags: - nagios_config - nagios_server diff --git a/roles/openshift-apps/badges/templates/tahrir.cfg.py b/roles/openshift-apps/badges/templates/tahrir.cfg.py index 6d9efb75fd..dd11d0c112 100644 --- a/roles/openshift-apps/badges/templates/tahrir.cfg.py +++ b/roles/openshift-apps/badges/templates/tahrir.cfg.py @@ -47,7 +47,7 @@ TAHRIR_USE_OPENID_EMAIL = False # Cache CACHE = { # disabled, trying to see if this fix this bug -# https://forge.fedoraproject.org/infra/tickets/8689 +# https://forge.fedoraproject.org/infra/tickets/issues/8689 "backend": "dogpile.cache.null", "expiration_time": 100, "arguments": { diff --git a/roles/openshift-apps/docsbuilding/templates/buildconfig.yml.j2 b/roles/openshift-apps/docsbuilding/templates/buildconfig.yml.j2 index 349a6efce2..8d004c021b 100644 --- a/roles/openshift-apps/docsbuilding/templates/buildconfig.yml.j2 +++ b/roles/openshift-apps/docsbuilding/templates/buildconfig.yml.j2 @@ -11,7 +11,7 @@ spec: type: Docker source: git: - uri: "https://gitlab.com/fedora/docs/docs-website/docs-fp-o.git" + uri: "https://forge.fedoraproject.org/docs/docs-fp-o.git" ref: "{{ env_short }}" contextDir: "build-scripts" output: diff --git a/roles/openshift-apps/forgejo/default/main.yml b/roles/openshift-apps/forgejo/default/main.yml index 3f18708404..8d038113f9 100644 --- a/roles/openshift-apps/forgejo/default/main.yml +++ b/roles/openshift-apps/forgejo/default/main.yml @@ -2,3 +2,7 @@ forgejo_namespace: "forgejo" forgejo_project_description: "Forgejo Gitforge" forgejo_application_name: "{{ forgejo_namespace }}" + +forgejo_runnerhost_packages: + - podman + - git diff --git a/roles/openshift-apps/forgejo/tasks/create-forgejo-runner-config-secret.yaml b/roles/openshift-apps/forgejo/tasks/create-forgejo-runner-config-secret.yaml new file mode 100644 index 0000000000..606dc74418 --- /dev/null +++ b/roles/openshift-apps/forgejo/tasks/create-forgejo-runner-config-secret.yaml @@ -0,0 +1,17 @@ +--- +# generate the templates for project to be created +- name: Copy the templates to the host + ansible.builtin.template: + src: "forgejo-runners-secret.yaml" + dest: "/root/ocp4/openshift-apps/forgejo/forgejo-runners-secret.yaml" + mode: "0770" + vars: + forgejo-runners-vars.yaml: + "{{ lookup('file', + '/root/ocp4/openshift-apps/forgejo/forgejo-runners-vars.yaml') + }}" + +- name: Deploy the forgejo-runners-secret.yaml config + kubernetes.core.k8s: + state: present + src: "/root/ocp4/openshift-apps/forgejo/forgejo-runners-secret.yaml" diff --git a/roles/openshift-apps/forgejo/tasks/main.yaml b/roles/openshift-apps/forgejo/tasks/main.yaml index f1ebb65e1b..68c1157457 100644 --- a/roles/openshift-apps/forgejo/tasks/main.yaml +++ b/roles/openshift-apps/forgejo/tasks/main.yaml @@ -57,6 +57,7 @@ # - include_tasks: create-custom-theme-configmap.yaml - include_tasks: deploy-valkey.yaml - include_tasks: call-helm.yaml -- include_tasks: create-forgejo-runnerhost-vm.yaml - include_tasks: copy-forgejo-runner-vars.yaml +- include_tasks: create-forgejo-runner-config-secret.yaml +- include_tasks: create-forgejo-runnerhost-vm.yaml - include_tasks: register-runners.yaml diff --git a/roles/openshift-apps/forgejo/templates/forgejo-runnerhost-vm.yaml.j2 b/roles/openshift-apps/forgejo/templates/forgejo-runnerhost-vm.yaml.j2 index 3252cad848..9cf0199c39 100644 --- a/roles/openshift-apps/forgejo/templates/forgejo-runnerhost-vm.yaml.j2 +++ b/roles/openshift-apps/forgejo/templates/forgejo-runnerhost-vm.yaml.j2 @@ -57,4 +57,11 @@ spec: expire: false password: "{{ (env == 'production') | ternary(forgejo_runnerhostvm_password, forgejo_stg_runnerhostvm_password) }}" user: "{{ (env == 'production') | ternary(forgejo_runnerhostvm_user, forgejo_stg_runnerhostvm_user) }}" +{% if forgejo_runnerhost_packages | default([]) | length > 0 %} + package_update: true + packages: +{% for pkg in forgejo_runnerhost_packages %} + - {{ pkg }} +{% endfor %} +{% endif %} name: cloudinitdisk diff --git a/roles/openshift-apps/forgejo/templates/forgejo-runners-secret.yaml.j2 b/roles/openshift-apps/forgejo/templates/forgejo-runners-secret.yaml.j2 new file mode 100644 index 0000000000..298e19e6d4 --- /dev/null +++ b/roles/openshift-apps/forgejo/templates/forgejo-runners-secret.yaml.j2 @@ -0,0 +1,9 @@ +--- +apiVersion: v1 +kind: Secret +metadata: + name: "forgejo-runners-config" + namespace: "forgejo" +data: + forgejo-runners-vars.yaml: + "{{ forgejo_runners_config_file | b64encode }}" diff --git a/roles/openshift-apps/forgejo/templates/values.yaml.j2 b/roles/openshift-apps/forgejo/templates/values.yaml.j2 index a9c51a49ab..cda0e4cd7b 100644 --- a/roles/openshift-apps/forgejo/templates/values.yaml.j2 +++ b/roles/openshift-apps/forgejo/templates/values.yaml.j2 @@ -526,7 +526,21 @@ gitea: "forge-neuro-owners":{"neuro":["Owners"]}, "forge-neuro-members":{"neuro":["Members"]}, "forge-kde-owners":{"kde":["Owners"]}, - "forge-kde-members":{"kde":["Members"]} + "forge-kde-members":{"kde":["Members"]}, + "forge-security-owners":{"security":["Owners"]}, + "forge-security-members":{"security":["Members"]}, + "forge-coc-owners":{"coc":["Owners"]}, + "forge-coc-members":{"coc":["Members"]}, + "forge-btrfs-owners":{"btrfs":["Owners"]}, + "forge-btrfs-members":{"btrfs":["Members"]}, + "forge-miracle-owners":{"miracle":["Owners"]}, + "forge-miracle-members":{"miracle":["Members"]}, + "forge-personalsystems-owners":{"personalsystems":["Owners"]}, + "forge-personalsystems-members":{"personalsystems":["Members"]}, + "forge-games-owners":{"games":["Owners"]}, + "forge-games-members":{"games":["Members"]}, + "forge-flatpak-owners":{"flatpak":["Owners"]}, + "forge-flatpak-members":{"flatpak":["Members"]} }' {% endif %} # - name: 'OAuth 1' diff --git a/roles/openshift-apps/greenwave/templates/fedora.yaml.j2 b/roles/openshift-apps/greenwave/templates/fedora.yaml.j2 index cf26a2ef58..5024dedebf 100644 --- a/roles/openshift-apps/greenwave/templates/fedora.yaml.j2 +++ b/roles/openshift-apps/greenwave/templates/fedora.yaml.j2 @@ -195,9 +195,9 @@ rules: - !PassingTestCaseRule {test_case_name: update.podman_client, scenario: "fedora.updates-container.aarch64.aarch64"} --- !Policy -id: "bodhiupdate_bodhipush_openqa_netinst" -# the "build a netinst and test an install from it" tests are gating -# for the core groups, plus the anaconda and compose groups +id: "bodhiupdate_bodhipush_openqa_everything_netinst" +# the "build an everything netinst and test an install from it" tests +# are gating for the core groups, plus the anaconda and compose groups {{ product_versions() }} decision_contexts: - bodhi_update_push_testing_critpath @@ -219,6 +219,34 @@ rules: - !PassingTestCaseRule {test_case_name: update.installer_build, scenario: "fedora.updates-everything-boot-iso.aarch64.aarch64"} - !PassingTestCaseRule {test_case_name: update.installer_build, scenario: "fedora.updates-everything-boot-iso.x86_64.64bit"} +--- !Policy +id: "bodhiupdate_bodhipush_openqa_server_netinst" +# the "build a server netinst and test an install from it" tests +# are gating for the core groups, plus the anaconda and compose groups, +# and the server group +{{ product_versions() }} +decision_contexts: + - bodhi_update_push_testing_critpath + - bodhi_update_push_testing_core_critpath + - bodhi_update_push_testing_critical-path-anaconda_critpath + - bodhi_update_push_testing_critical-path-base_critpath + - bodhi_update_push_testing_critical-path-compose_critpath + - bodhi_update_push_testing_critical-path-server_critpath + - bodhi_update_push_stable_critpath + - bodhi_update_push_stable_core_critpath + - bodhi_update_push_stable_critical-path-anaconda_critpath + - bodhi_update_push_stable_critical-path-base_critpath + - bodhi_update_push_stable_critical-path-compose_critpath + - bodhi_update_push_stable_critical-path-server_critpath +subject_type: bodhi_update +rules: +# This list needs to stay synced with openQA if tests are added or renamed. + - !PassingTestCaseRule {test_case_name: update.install_default_update_netinst, scenario: "fedora.updates-server-boot-iso.aarch64.aarch64"} + - !PassingTestCaseRule {test_case_name: update.install_default_update_netinst, scenario: "fedora.updates-server-boot-iso.x86_64.64bit"} + - !PassingTestCaseRule {test_case_name: update.install_default_update_netinst, scenario: "fedora.updates-server-boot-iso.x86_64.bios"} + - !PassingTestCaseRule {test_case_name: update.installer_build, scenario: "fedora.updates-server-boot-iso.aarch64.aarch64"} + - !PassingTestCaseRule {test_case_name: update.installer_build, scenario: "fedora.updates-server-boot-iso.x86_64.64bit"} + --- !Policy id: "bodhiupdate_bodhipush_openqa_apps" # these tests are gating for updates in the apps critpath group: as of diff --git a/roles/openshift-apps/ipsilon-website/files/service.yml b/roles/openshift-apps/ipsilon-website/files/service.yml deleted file mode 100644 index cd3876c0a5..0000000000 --- a/roles/openshift-apps/ipsilon-website/files/service.yml +++ /dev/null @@ -1,15 +0,0 @@ ---- -apiVersion: v1 -kind: Service -metadata: - name: web - labels: - app: ipsilon-website -spec: - ports: - - name: web - port: 8080 - targetPort: 8080 - selector: - app: ipsilon-website - deploymentconfig: web diff --git a/roles/openshift-apps/ipsilon-website/templates/buildconfig.yml.j2 b/roles/openshift-apps/ipsilon-website/templates/buildconfig.yml.j2 deleted file mode 100644 index 4aa9514f1b..0000000000 --- a/roles/openshift-apps/ipsilon-website/templates/buildconfig.yml.j2 +++ /dev/null @@ -1,33 +0,0 @@ ---- -apiVersion: build.openshift.io/v1 -kind: BuildConfig -metadata: - name: web - labels: - app: ipsilon-website - build: ipsilon-website -spec: - runPolicy: Serial - source: - type: Git - git: - uri: https://pagure.io/ipsilon-website.git - ref: master - contextDir: / - strategy: - type: Docker - output: - to: - kind: ImageStreamTag - name: ipsilon-website:latest - triggers: - - type: ConfigChange - - type: ImageChange - - type: "Generic" - generic: - secretReference: -{% if env == "staging" %} - name: "{{ ipsilon_website_stg_webhook_secret }}" -{% else %} - name: "{{ ipsilon_website_webhook_secret }}" -{% endif %} diff --git a/roles/openshift-apps/ipsilon-website/templates/deploymentconfig.yml.j2 b/roles/openshift-apps/ipsilon-website/templates/deploymentconfig.yml.j2 deleted file mode 100644 index b4435d13e6..0000000000 --- a/roles/openshift-apps/ipsilon-website/templates/deploymentconfig.yml.j2 +++ /dev/null @@ -1,59 +0,0 @@ ---- -apiVersion: apps.openshift.io/v1 -kind: DeploymentConfig -metadata: - name: web - labels: - app: ipsilon-website -spec: - replicas: 1 - selector: - app: ipsilon-website - deploymentconfig: web - strategy: - type: Rolling - activeDeadlineSeconds: 21600 - rollingParams: - intervalSeconds: 1 - maxSurge: 25% - maxUnavailable: 25% - timeoutSeconds: 600 - updatePeriodSeconds: 1 - template: - metadata: - creationTimestamp: null - labels: - app: ipsilon-website - deploymentconfig: web - spec: - containers: - - name: ipsilon-website - imagePullPolicy: Always - ports: - - containerPort: 8080 - # protocol: TCP - readinessProbe: - timeoutSeconds: 5 - initialDelaySeconds: 30 - httpGet: - path: / - port: 8080 - livenessProbe: - timeoutSeconds: 5 - initialDelaySeconds: 30 - httpGet: - path: / - port: 8080 - # resources: {} - # terminationMessagePath: /dev/termination-log - # terminationMessagePolicy: File - triggers: - - type: ConfigChange - - type: ImageChange - imageChangeParams: - automatic: true - containerNames: - - ipsilon-website - from: - kind: ImageStreamTag - name: ipsilon-website:latest diff --git a/roles/openshift-apps/noggin/templates/after-navbar.html b/roles/openshift-apps/noggin/templates/after-navbar.html index c6f8774178..80142c6637 100644 --- a/roles/openshift-apps/noggin/templates/after-navbar.html +++ b/roles/openshift-apps/noggin/templates/after-navbar.html @@ -1,11 +1,11 @@ {% raw %} - + diff --git a/roles/openshift-apps/poddlers/templates/buildconfig.yml.j2 b/roles/openshift-apps/poddlers/templates/buildconfig.yml.j2 index e6509fd531..a0340b4453 100644 --- a/roles/openshift-apps/poddlers/templates/buildconfig.yml.j2 +++ b/roles/openshift-apps/poddlers/templates/buildconfig.yml.j2 @@ -41,7 +41,7 @@ spec: source: type: Git git: - uri: https://pagure.io/fedora-infra/toddlers.git + uri: https://forge.fedoraproject.org/apps/toddlers.git {% if env == 'staging' %} ref: "staging" {% else %} diff --git a/roles/openshift-apps/poddlers/templates/fedora-messaging.toml b/roles/openshift-apps/poddlers/templates/fedora-messaging.toml index 3a8ffed0e7..33ab21905a 100644 --- a/roles/openshift-apps/poddlers/templates/fedora-messaging.toml +++ b/roles/openshift-apps/poddlers/templates/fedora-messaging.toml @@ -24,7 +24,7 @@ certfile = "/etc/pki/rabbitmq/cert/toddlers.crt" [client_properties] app = "toddlers-{{ toddler.name }}" -app_url = "https://pagure.io/fedora-infra/toddlers" +app_url = "https://forge.fedoraproject.org/apps/toddlers.git" [queues."toddlers{{ env_suffix }}-{{ toddler.name }}"] durable = true diff --git a/roles/openshift-apps/resultsdb/vars/main.yml b/roles/openshift-apps/resultsdb/vars/main.yml index 7d5f785e75..812a324ece 100644 --- a/roles/openshift-apps/resultsdb/vars/main.yml +++ b/roles/openshift-apps/resultsdb/vars/main.yml @@ -9,7 +9,7 @@ resultsdb_publish: true resultsdb_mod_wsgi_dir: '${MOD_WSGI_MODULES_DIRECTORY}' # For CRASHED see: https://pagure.io/task-abicheck/issue/19 -# For QUEUED and RUNNING see: https://forge.fedoraproject.org/infra/tickets/8989 +# For QUEUED and RUNNING see: https://forge.fedoraproject.org/infra/tickets/issues/8989 resultsdb_additional_result_outcomes: ['CRASHED', 'QUEUED', 'RUNNING'] # frontend vars diff --git a/roles/openshift/cluster/handlers/main.yaml b/roles/openshift/cluster/handlers/main.yaml new file mode 100644 index 0000000000..636076db42 --- /dev/null +++ b/roles/openshift/cluster/handlers/main.yaml @@ -0,0 +1,7 @@ +--- +- name: Apply changes to openshift + ansible.builtin.command: "oc apply --validate=strict -f {{ cluster_filepath }}/{{ item }}" + loop: "{{ base_templates + (stg_templates if env == 'staging' else []) }}" + changed_when: true + tags: + - create-resources diff --git a/roles/openshift/cluster/tasks/main.yaml b/roles/openshift/cluster/tasks/main.yaml index 76f2ce413b..c24717402e 100644 --- a/roles/openshift/cluster/tasks/main.yaml +++ b/roles/openshift/cluster/tasks/main.yaml @@ -10,25 +10,26 @@ tags: - create-resources +- name: Set template lists + ansible.builtin.set_fact: + base_templates: + - sysadmin-openshift-group.yml + - sysadmin-openshift-rolebinding.yml + - webhooks-clusterrolebinding.yml + - forward-logs-to-log01.yml + - sysadmin-openshift-readonly-rolebinding.yml + - sysadmin-openshift-readonly-role.yml + - sysadmin-openshift-readonly-group.yml + stg_templates: [] + # generate the templates for project to be created -- name: Copy the templates +- name: Copy templates ansible.builtin.template: src: "{{ item }}.j2" dest: "{{ cluster_filepath }}/{{ item }}" mode: "0640" - with_items: - - sysadmin-openshift-group.yml - - sysadmin-openshift-rolebinding.yml - - webhooks-clusterrolebinding.yml - - forward-logs-to-log01.yml + with_items: "{{ base_templates + (stg_templates if env == 'staging' else []) }}" register: cluster_template_result - tags: - - create-resources - -# apply created openshift resources -- name: Oc apply resources - ansible.builtin.command: "oc apply --validate=strict -f {{ item.dest }}" - with_items: "{{ cluster_template_result.results }}" - when: item.changed + notify: Apply changes to openshift tags: - create-resources diff --git a/roles/openshift/cluster/templates/sysadmin-openshift-readonly-group.yml.j2 b/roles/openshift/cluster/templates/sysadmin-openshift-readonly-group.yml.j2 new file mode 100644 index 0000000000..78a30dd1b4 --- /dev/null +++ b/roles/openshift/cluster/templates/sysadmin-openshift-readonly-group.yml.j2 @@ -0,0 +1,9 @@ +--- +kind: Group +apiVersion: user.openshift.io/v1 +metadata: + name: "sysadmin-openshift-readonly" +users: +{% for item in cluster_readonly_appowners %} +- "{{ item }}" +{% endfor %} diff --git a/roles/openshift/cluster/templates/sysadmin-openshift-readonly-role.yml.j2 b/roles/openshift/cluster/templates/sysadmin-openshift-readonly-role.yml.j2 new file mode 100644 index 0000000000..74e60a815c --- /dev/null +++ b/roles/openshift/cluster/templates/sysadmin-openshift-readonly-role.yml.j2 @@ -0,0 +1,22 @@ +--- +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: "sysadmin-openshift-readonly" +rules: +- apiGroups: + - "" + resources: + - endpoints + - persistentvolumeclaims + - persistentvolumeclaims/status + - pods + - replicationcontrollers + - replicationcontrollers/scale + - serviceaccounts + - services + - services/status + verbs: + - get + - list + - watch diff --git a/roles/openshift/cluster/templates/sysadmin-openshift-readonly-rolebinding.yml.j2 b/roles/openshift/cluster/templates/sysadmin-openshift-readonly-rolebinding.yml.j2 new file mode 100644 index 0000000000..6a769d6ab8 --- /dev/null +++ b/roles/openshift/cluster/templates/sysadmin-openshift-readonly-rolebinding.yml.j2 @@ -0,0 +1,13 @@ +--- +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: "sysadmin-openshift-readonly" +subjects: + - kind: Group + apiGroup: rbac.authorization.k8s.io + name: "sysadmin-openshift-readonly" +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: sysadmin-openshift-readonly diff --git a/roles/openvpn/client/files/zabbix/template-openvpn.yml b/roles/openvpn/client/files/zabbix/template-openvpn.yml new file mode 100644 index 0000000000..a9cb1770b2 --- /dev/null +++ b/roles/openvpn/client/files/zabbix/template-openvpn.yml @@ -0,0 +1,47 @@ +zabbix_export: + version: '7.0' + template_groups: + - uuid: a333cbd6a3ad44baaa4eee4b0c0b1bec + name: Fedora + templates: + - uuid: 01a1d2dc827f4e1cbf803d3948237c1b + template: 'OpenVPN Client' + name: 'OpenVPN Client' + description: 'Uses zabbix-sender on the host to report the number of days left on the OpenVPN client certificate' + groups: + - name: Fedora + items: + - uuid: 93da423abd37459e9d5478e183048bd9 + name: 'Time left on OpenVPN Cert' + type: TRAP + key: openvpn.cert_date + delay: '0' + trends: '0' + triggers: + - uuid: d62edd1dd4a74cada65855c2e133afe6 + expression: 'last(/OpenVPN Client/openvpn.cert_date)<1' + name: 'OpenVPN Client cert expired' + priority: HIGH + tags: + - tag: service + value: openvpn + - uuid: b5fc3fd5fae346f8b247f4323838bbe9 + expression: 'last(/OpenVPN Client/openvpn.cert_date)<7' + name: 'OpenVPN Client cert has less than 7 days left' + priority: AVERAGE + dependencies: + - name: 'OpenVPN Client cert expired' + expression: 'last(/OpenVPN Client/openvpn.cert_date)<1' + tags: + - tag: service + value: openvpn + - uuid: 45e56e7945ec41789976dd29d7ec9e53 + expression: 'last(/OpenVPN Client/openvpn.cert_date)<30' + name: 'OpenVPN Client cert has less than 30 days left' + priority: WARNING + dependencies: + - name: 'OpenVPN Client cert has less than 7 days left' + expression: 'last(/OpenVPN Client/openvpn.cert_date)<7' + tags: + - tag: service + value: openvpn diff --git a/roles/openvpn/client/tasks/main.yml b/roles/openvpn/client/tasks/main.yml index c157ad196f..0ceb7c9db8 100644 --- a/roles/openvpn/client/tasks/main.yml +++ b/roles/openvpn/client/tasks/main.yml @@ -65,3 +65,42 @@ tags: - service - openvpn + +# Zabbix monitoring of the OpenVPN client + +# OpenVPN certs have restrcited permissions, so we use a zabbix-sender cronjob +# for this, which runs as root once per day +- name: Setup OpenVPN monitoring cron job + ansible.builtin.template: + src: zabbix/cron-openvpn-cert.j2 + dest: /etc/cron.daily/zabbix-openvpn-cert + owner: root + group: root + mode: "0755" + tags: + - openvpn + - zabbix_agent + +- name: Zabbix API Block + vars: + ansible_zabbix_auth_key: "{{ zabbix_auth_key }}" + ansible_network_os: "{{ zabbix_network_os }}" + ansible_connection: "{{ zabbix_connection }}" + ansible_httpapi_port: "{{ zabbix_httpapi_port }}" + ansible_httpapi_use_ssl: "{{ zabbix_httpapi_use_ssl }}" + ansible_httpapi_validate_certs: "{{ zabbix_httpapi_validate_certs }}" + ansible_host: "{{ zabbix_server }}" + ansible_zabbix_url_path: "{{ zabbix_url_path }}" + tags: + - openvpn + - zabbix_api + block: + - name: Import OpenVPN template file + community.zabbix.zabbix_template: + template_yaml: "{{ lookup('file', 'zabbix/template-openvpn.yml') }}" + state: present + - name: Add self to OpenVPN template in Zabbix + community.zabbix.zabbix_host: + host_name: "{{ inventory_hostname }}" + link_templates: OpenVPN Client + force: false diff --git a/roles/openvpn/client/templates/zabbix/cron-openvpn-cert.j2 b/roles/openvpn/client/templates/zabbix/cron-openvpn-cert.j2 new file mode 100644 index 0000000000..2225214f6c --- /dev/null +++ b/roles/openvpn/client/templates/zabbix/cron-openvpn-cert.j2 @@ -0,0 +1,20 @@ +#!/usr/bin/bash +# A script to report days-left on the openvpn cert to zabbix + +# Configuration +CERT_FILE="/etc/openvpn/client/client.crt" + +# Get certificate expiry +EXPIRY=$(openssl x509 -in "$CERT_FILE" -noout -enddate | cut -d= -f2) + +if [ -z "$EXPIRY" ]; then + echo "problem reading $CERT_FILE" + exit 1 +fi + +# Convert to epoch and calculate days remaining +EXPIRY_EPOCH=$(date -d "$EXPIRY" +%s) +NOW_EPOCH=$(date +%s) +DAYS_LEFT=$(( ($EXPIRY_EPOCH - $NOW_EPOCH) / 86400 )) + +/usr/bin/zabbix_sender -c {{ zabbix_agentd }} -k openvpn.cert_date -o "$DAYS_LEFT" > /dev/null diff --git a/roles/pagure/tasks/main.yml b/roles/pagure/tasks/main.yml index e5e49ae15c..6812711c8d 100644 --- a/roles/pagure/tasks/main.yml +++ b/roles/pagure/tasks/main.yml @@ -116,7 +116,7 @@ tags: - pagure -# https://forge.fedoraproject.org/infra/tickets/12428 +# https://forge.fedoraproject.org/infra/tickets/issues/12428 - name: Hotfix for links to accounts.fpo ansible.posix.patch: src: files/hotfix/0001-Fix-link-to-accounts.fpo-for-staging-for-adding-user.patch @@ -155,7 +155,7 @@ - fix_log - hotfix -# Fix for https://forge.fedoraproject.org/infra/tickets/11957 +# Fix for https://forge.fedoraproject.org/infra/tickets/issues/11957 - name: Set ACL for newly created files in /var/log/pagure ansible.posix.acl: path: /var/log/pagure diff --git a/roles/pagure/templates/0_pagure.conf b/roles/pagure/templates/0_pagure.conf index 5142988675..ed42fb9298 100644 --- a/roles/pagure/templates/0_pagure.conf +++ b/roles/pagure/templates/0_pagure.conf @@ -321,7 +321,7 @@ Listen 3923 http {% endif %} # redirects for sssd docs. - # https://forge.fedoraproject.org/infra/tickets/8995 + # https://forge.fedoraproject.org/infra/tickets/issues/8995 RedirectMatch ^/SSSD.sssd$ https://sssd.io RedirectMatch ^/SSSD.sssd/$ https://sssd.io RedirectMatch ^/SSSD.sssd/index.html https://sssd.io/index.html @@ -378,4 +378,3 @@ Listen 3923 http - diff --git a/roles/people/tasks/main.yml b/roles/people/tasks/main.yml index 7d2834968b..49560ee16c 100644 --- a/roles/people/tasks/main.yml +++ b/roles/people/tasks/main.yml @@ -13,7 +13,7 @@ - nano - tree # requested in ticket 5148 - mod_ssl - - tar # requested in https://forge.fedoraproject.org/infra/tickets/12638 + - tar # requested in https://forge.fedoraproject.org/infra/tickets/issues/12638 tags: - packages - people diff --git a/roles/people/templates/people.conf b/roles/people/templates/people.conf index ee9a8a1237..36a4d35880 100644 --- a/roles/people/templates/people.conf +++ b/roles/people/templates/people.conf @@ -277,3 +277,7 @@ AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml text/css a FileETag MTime Size + +ServerLimit 2500 +MaxRequestWorkers 2500 +MaxRequestsPerChild 10000 diff --git a/roles/rabbit/queue/handlers/main.yml b/roles/rabbit/queue/handlers/main.yml deleted file mode 100644 index 978e3ed49c..0000000000 --- a/roles/rabbit/queue/handlers/main.yml +++ /dev/null @@ -1,10 +0,0 @@ ---- -- name: Restart nagios on noc - ansible.builtin.shell: nagios -v /etc/nagios/nagios.cfg && systemctl restart nagios - delegate_to: "{{ queue_nagios_server }}" - -- name: Restart nrpe on rabbitmq - service: - name: nrpe - state: restarted - delegate_to: "{{ queue_rabbitmq_server }}" diff --git a/roles/rabbit/queue/tasks/main.yml b/roles/rabbit/queue/tasks/main.yml index 282e708dcd..77e27b25e3 100644 --- a/roles/rabbit/queue/tasks/main.yml +++ b/roles/rabbit/queue/tasks/main.yml @@ -17,9 +17,6 @@ # # - user_sent_topics (str): A regular expression that must match the topic when a # message is published by this user. -# - queue_thresholds (dict): A dictionary with two keys: "warning" and "critical". -# The values are numbers. Generate an alert in Nagios if -# the number of messages go above these values. - name: Validate queue name {{ queue_name }} assert: @@ -77,28 +74,3 @@ tags: - fedora-messaging - rabbitmq_cluster - -- name: Monitor the {{ queue_name }} queue in Nagios (NRPE) - when: queue_thresholds and env == "production" - delegate_to: "{{ queue_rabbitmq_server }}" - ansible.builtin.template: - src: nrpe.cfg.j2 - dest: /etc/nrpe.d/check_rabbitmq_queue_{{ queue_name }}.cfg - owner: root - group: root - mode: "0644" - notify: Restart nrpe on rabbitmq - tags: - - fedora-messaging - - rabbitmq_cluster - -- name: Monitor the {{ queue_name }} queue in Nagios - when: queue_thresholds and env == "production" - delegate_to: "{{ queue_nagios_server }}" - ansible.builtin.template: - src: nagios.cfg.j2 - dest: /etc/nagios/services/rabbitmq-queue-{{ queue_name }}.cfg -# notify: Restart nagios on noc - tags: - - fedora-messaging - - rabbitmq_cluster diff --git a/roles/rabbit/queue/templates/nagios.cfg.j2 b/roles/rabbit/queue/templates/nagios.cfg.j2 deleted file mode 100644 index bfb64cc6b1..0000000000 --- a/roles/rabbit/queue/templates/nagios.cfg.j2 +++ /dev/null @@ -1,6 +0,0 @@ -define service { - host_name {{ queue_rabbitmq_server }} - service_description Check queue {{ queue_name }} - check_command check_by_nrpe!check_rabbitmq_queue_{{ queue_vhost.replace("/", "") }}_{{ queue_name }} - use defaulttemplate -} diff --git a/roles/rabbit/queue/templates/nrpe.cfg.j2 b/roles/rabbit/queue/templates/nrpe.cfg.j2 deleted file mode 100644 index 3bd91c03c7..0000000000 --- a/roles/rabbit/queue/templates/nrpe.cfg.j2 +++ /dev/null @@ -1 +0,0 @@ -command[check_rabbitmq_queue_{{ queue_vhost.replace("/", "") }}_{{ queue_name }}]=/usr/lib64/nagios/plugins-rabbitmq/check_rabbitmq_queue --extra-opts=common@/etc/nrpe.d/rabbitmq_args.ini --vhost {{ queue_vhost }} --queue {{ queue_name }} {% if queue_thresholds.warning %}-w {{ queue_thresholds.warning }}{% endif %} {% if queue_thresholds.critical %}-c {{ queue_thresholds.critical }}{% endif %} diff --git a/roles/rabbitmq_cluster/files/nrpe_rabbitmq.te b/roles/rabbitmq_cluster/files/nrpe_rabbitmq.te deleted file mode 100644 index 7da96acd2b..0000000000 --- a/roles/rabbitmq_cluster/files/nrpe_rabbitmq.te +++ /dev/null @@ -1,11 +0,0 @@ -module nrpe_rabbitmq 1.0; - -require { - type amqp_port_t; - type nrpe_t; - class tcp_socket name_connect; -} - -#============= nrpe_t ============== -# let nagios plugin connect to rabbitmq management interface -allow nrpe_t amqp_port_t:tcp_socket name_connect; \ No newline at end of file diff --git a/roles/rabbitmq_cluster/files/selinux-load.sh b/roles/rabbitmq_cluster/files/selinux-load.sh deleted file mode 100644 index e126f235a4..0000000000 --- a/roles/rabbitmq_cluster/files/selinux-load.sh +++ /dev/null @@ -1,9 +0,0 @@ -#!/bin/sh - -set -e -set -x - -checkmodule -M -m -o /etc/nagios/nrpe_rabbitmq.mod /etc/nagios/nrpe_rabbitmq.te -semodule_package -o /etc/nagios/nrpe_rabbitmq.pp -m /etc/nagios/nrpe_rabbitmq.mod -semodule -i /etc/nagios/nrpe_rabbitmq.pp -rm /etc/nagios/nrpe_rabbitmq.mod /etc/nagios/nrpe_rabbitmq.pp \ No newline at end of file diff --git a/roles/rabbitmq_cluster/files/zabbix/template-rabbitmq.yml b/roles/rabbitmq_cluster/files/zabbix/template-rabbitmq.yml index d73637d58f..ca4c93208f 100644 --- a/roles/rabbitmq_cluster/files/zabbix/template-rabbitmq.yml +++ b/roles/rabbitmq_cluster/files/zabbix/template-rabbitmq.yml @@ -1196,6 +1196,15 @@ zabbix_export: - tag: vhost value: '{#VHOST}' trigger_prototypes: + - uuid: bc63d110be214843b6bcd8116ef01a09 + expression: 'min(/RabbitMQ Monitoring/rabbitmq.queue.messages["{#VHOST}/{#QUEUE}"],5m)>{$RABBITMQ.MESSAGES.MAX.HIGH:"{#QUEUE}"}' + name: 'RabbitMQ node: Too many messages in queue [{#VHOST}][{#QUEUE}]' + event_name: 'RabbitMQ node: Too many messages in queue [{#VHOST}][{#QUEUE}] (over {$RABBITMQ.MESSAGES.MAX.HIGH:"{#QUEUE}"} for 5m)' + opdata: 'Current value: {ITEM.LASTVALUE1}' + priority: HIGH + tags: + - tag: scope + value: performance - uuid: b6f52153a22144338e6d002287446ac1 expression: 'min(/RabbitMQ Monitoring/rabbitmq.queue.messages["{#VHOST}/{#QUEUE}"],5m)>{$RABBITMQ.MESSAGES.MAX.WARN:"{#QUEUE}"}' name: 'RabbitMQ node: Too many messages in queue [{#VHOST}][{#QUEUE}]' @@ -1420,6 +1429,9 @@ zabbix_export: - macro: '{$RABBITMQ.LLD.FILTER.QUEUE.NOT_MATCHES}' value: CHANGE_IF_NEEDED description: 'This macro is used in the discovery of queues. It can be overridden at host level or its linked template level.' + - macro: '{$RABBITMQ.MESSAGES.MAX.HIGH}' + value: '2000' + description: 'The maximum number of messages in the queue for a Matrix alert.' - macro: '{$RABBITMQ.MESSAGES.MAX.WARN}' value: '1000' description: 'The maximum number of messages in the queue for a trigger expression.' diff --git a/roles/rabbitmq_cluster/tasks/apps.yml b/roles/rabbitmq_cluster/tasks/apps.yml index 968f3ebba1..bf0a0b8fc1 100644 --- a/roles/rabbitmq_cluster/tasks/apps.yml +++ b/roles/rabbitmq_cluster/tasks/apps.yml @@ -179,3 +179,11 @@ # - "#.buildsys.tag" # # ELN END + +- name: LogDetective - Packit + run_once: true + include_role: + name: rabbit/user + vars: + user_name: logdetective-packit{{ env_suffix }} + user_sent_topics: ^org\.fedoraproject\.{{ env_short }}\.logdetective\..* diff --git a/roles/rabbitmq_cluster/tasks/main.yml b/roles/rabbitmq_cluster/tasks/main.yml index 4af2c94ee3..0db46cb39b 100644 --- a/roles/rabbitmq_cluster/tasks/main.yml +++ b/roles/rabbitmq_cluster/tasks/main.yml @@ -23,7 +23,6 @@ state: present name: - rabbitmq-server - - nagios-plugins-rabbitmq tags: - rabbitmq_cluster - packages @@ -269,6 +268,7 @@ # Users with the "monitoring" tag have read-only access vhosts, connections, # channels, node-level resource usage, and cluster stats. # (this needs to be done after all vhost that the user has access to have been created) +# TODO: The Zabbix monitoring currently reuses this user, we should rename it - name: Create the Nagios monitoring user in staging rabbitmq_user: user: nagios-monitoring @@ -297,6 +297,7 @@ - rabbitmq_cluster - config +# TODO: The Zabbix monitoring currently reuses this user, we should rename it - name: Create the Nagios monitoring user in production rabbitmq_user: user: nagios-monitoring @@ -432,23 +433,6 @@ federation-upstream: "pubsub-to-public_pubsub" vhost: /public_pubsub -# SELinux: allow the Nagios NRPE plugin to access the management interface -- name: Install the selinux module compilation script - ansible.builtin.copy: - src: selinux-load.sh - dest: /etc/nagios/selinux-load.sh - mode: "0755" - -- name: Copy over our custom selinux module - ansible.builtin.copy: - src: nrpe_rabbitmq.te - dest: /etc/nagios/nrpe_rabbitmq.te - register: selinux_module - -- name: Compile and install our custom selinux module - ansible.builtin.command: /etc/nagios/selinux-load.sh - when: selinux_module is changed - # RabbitMQ monitoring for Fedora Zabbix - name: Configure RabbitMQ monitoring in Zabbix ansible.builtin.include_tasks: monitoring.yml diff --git a/roles/security.txt/files/security.txt b/roles/security.txt/files/security.txt index f1d6f9edde..51d3d1b07c 100644 --- a/roles/security.txt/files/security.txt +++ b/roles/security.txt/files/security.txt @@ -1,2 +1,5 @@ +Canonical: https://admin.fedoraproject.org/.well-known/security.txt Contact: mailto:infra-security@fedoraproject.org -Expires: 2025-12-31T21:01:00.000Z +Expires: 2027-02-10T21:01:00.000Z +Bug-Bounty: False +Policy: https://fedoraproject.org/wiki/Infrastructure-security-reports diff --git a/roles/torrent/files/torrent-hashes.py b/roles/torrent/files/torrent-hashes.py index 69ad1674f1..ec70a3b6ad 100755 --- a/roles/torrent/files/torrent-hashes.py +++ b/roles/torrent/files/torrent-hashes.py @@ -1,50 +1,54 @@ -#!/usr/bin/python -# by Matt Domsch -# License: BitTorrent -# -# This simply prints the bittorrent hashes for each file -# to stdout or an output file. -# To be used as the whitelist with opentracker +#!/usr/bin/env python3 import os import sys import hashlib -from optparse import OptionParser -from BitTorrent.bencode import bencode, bdecode -from BitTorrent.btformats import check_message +import fastbencode def torrent_hash(fname): - f = open(fname, 'rb') - d = bdecode(f.read()) - f.close() - check_message(d) - hash = hashlib.sha1(bencode(d['info'])).hexdigest().upper() - fn = os.path.basename(fname) - return '%s - %s' % (hash,fn) + try: + with open(fname, 'rb') as f: + torrent_data = f.read() + decoded_torrent = fastbencode.bdecode(torrent_data) + info_hash = hashlib.sha1(fastbencode.bencode(decoded_torrent[b'info'])).hexdigest().upper() + fn = os.path.basename(fname) + return f"{info_hash} - {fn}" + except FileNotFoundError: + print(f"Error: File '{fname}' not found.", file=sys.stderr) + except IsADirectoryError: + print(f"Error: '{fname}' is a directory.", file=sys.stderr) + except Exception as e: + print(f"Error reading or decoding hash from {fname}: {e}", file=sys.stderr) + return None def main(): - parser = OptionParser(usage=sys.argv[0] + " [options] [torrentfiles] ...") - parser.add_option("-o", "--output", type="string", metavar="FILE", - dest="output", default='-', - help="write hashes to FILE, default=stdout") - (options, args) = parser.parse_args() + if len(sys.argv) < 2: + print("Usage: ./torrent-hashes.py [-o output] [torrentfiles] ...", file=sys.stderr) + sys.exit(1) - outfd = sys.stdout - if options.output != '-': + redir = False + output_file = sys.stdout + if len(sys.argv) >= 3 and sys.argv[1] == "-o": try: - outfd = open(options.output, 'w') - except: - sys.stderr.write("Error: unable to open output file %s\n" % options.output) - return 1 - - for a in args: - try: - hash = torrent_hash(a) - outfd.write(hash + '\n') - except: - sys.stderr.write("Error reading hash from %s\n" % a) + output_file = open(sys.argv[2], 'w') + redir = True + except Exception as e: + print(f"Error: unable to open output file {sys.argv[2]}: {e}", file=sys.stderr) + sys.exit(1) + del sys.argv[1:3] - return 0 + for torrent_file in sys.argv[1:]: + try: + if redir: + print(f"Processing torrent file: {torrent_file}") + hash_str = torrent_hash(torrent_file) + if hash_str: + output_file.write(hash_str + '\n') + except Exception as e: + print(f"Error reading hash from {torrent_file}: {e}", file=sys.stderr) + + if output_file is not sys.stdout: + output_file.close() if __name__ == "__main__": - sys.exit(main()) + main() diff --git a/roles/varnish/templates/kojipkgs.vcl.j2 b/roles/varnish/templates/kojipkgs.vcl.j2 index 68f8603a25..43aa57ab08 100644 --- a/roles/varnish/templates/kojipkgs.vcl.j2 +++ b/roles/varnish/templates/kojipkgs.vcl.j2 @@ -83,7 +83,7 @@ sub vcl_recv { } # Rewrite toplink URLs to improve cache hit ratio - # See: https://forge.fedoraproject.org/infra/tickets/7383 + # See: https://forge.fedoraproject.org/infra/tickets/issues/7383 set req.url = regsub(req.url, "^/repos/[^/]+/[^/]+/[^/]+/toplink/", "/"); return (hash); diff --git a/roles/varnish/templates/proxies.vcl.j2 b/roles/varnish/templates/proxies.vcl.j2 index c4717d8f5c..20cc9ace60 100644 --- a/roles/varnish/templates/proxies.vcl.j2 +++ b/roles/varnish/templates/proxies.vcl.j2 @@ -36,8 +36,8 @@ backend registry { } backend src { - .host = "localhost"; - .port = "10057"; + .host = "pkgs01"; + .port = "http"; } acl purge { @@ -116,7 +116,7 @@ sub vcl_recv { if (req.http.X-Forwarded-Server ~ "^src.fedoraproject.org" || req.http.X-Forwarded-Server ~ "^src.stg.fedoraproject.org" ) { set req.backend_hint = src; # Do not cache .crate files as it messes up checksums - # see https://forge.fedoraproject.org/infra/tickets/12812 + # see https://forge.fedoraproject.org/infra/tickets/issues/12812 if (req.url ~ ".*crate") { return (pipe); } diff --git a/roles/varnish/templates/s390kojipkgs.vcl.j2 b/roles/varnish/templates/s390kojipkgs.vcl.j2 index cc3708313e..34b3343717 100644 --- a/roles/varnish/templates/s390kojipkgs.vcl.j2 +++ b/roles/varnish/templates/s390kojipkgs.vcl.j2 @@ -98,7 +98,7 @@ sub vcl_recv { } # Rewrite toplink URLs to improve cache hit ratio - # See: https://forge.fedoraproject.org/infra/tickets/7383 + # See: https://forge.fedoraproject.org/infra/tickets/issues/7383 set req.url = regsub(req.url, "^/repos/[^/]+/[^/]+/[^/]+/toplink/", "/"); return (hash); diff --git a/roles/web-data-analysis/files/combineHttpLogs.sh b/roles/web-data-analysis/files/combineHttpLogs.sh index 386a43f305..34d74fb667 100644 --- a/roles/web-data-analysis/files/combineHttpLogs.sh +++ b/roles/web-data-analysis/files/combineHttpLogs.sh @@ -26,8 +26,7 @@ export MSGTOPIC_PREFIX=logging.stats export MSGBODY_PRESET="loghost=$(hostname) run_id=$(uuidgen -r)" # simple_message_to_bus is in /usr/local/bin which isn't in the default path -# Put it at the back, so it doesn't override anything. -export PATH="$PATH:/usr/local/bin" +export PATH="/usr/local/bin:$PATH" simple_message_to_bus combinehttplogs.start diff --git a/roles/web-data-analysis/files/condense-mirrorlogs.sh b/roles/web-data-analysis/files/condense-mirrorlogs.sh index f06847777f..921ca765b1 100644 --- a/roles/web-data-analysis/files/condense-mirrorlogs.sh +++ b/roles/web-data-analysis/files/condense-mirrorlogs.sh @@ -26,6 +26,8 @@ # We have dropped this down to 3 days on 2019-10-01 +export PATH="/usr/local/bin:$PATH" + export MSGTOPIC_PREFIX=logging.stats export MSGBODY_PRESET="loghost=$(hostname) run_id=$(uuidgen -r)" simple_message_to_bus condense-mirrorlogs.start diff --git a/roles/web-data-analysis/files/countme-centos-update.sh b/roles/web-data-analysis/files/countme-centos-update.sh index b895418f94..e734e92fc2 100644 --- a/roles/web-data-analysis/files/countme-centos-update.sh +++ b/roles/web-data-analysis/files/countme-centos-update.sh @@ -1,5 +1,7 @@ #!/bin/bash +export PATH="/usr/local/bin:$PATH" + # What are we called (used for message bus so don't just use cmd $0) CMD_NAME=countme-centos-update @@ -102,11 +104,6 @@ if [ -d "$COUNTME_CHECKOUT" ]; then PATH="$COUNTME_CHECKOUT:$COUNTME_CHECKOUT/scripts:$PATH" fi -# Hardcoding /usr/local/bin here is hacky; should be pulled from pip, but -# parsing pip output is nontrivial, and my father always told me: -# "Son, life's too damn short write a RFC2822 parser in bash." -PATH="$PATH:/usr/local/bin" - # Check for required commands command -v $UPDATE_RAWDB >/dev/null || die "can't find '$UPDATE_RAWDB'" command -v $UPDATE_TOTALS >/dev/null || die "can't find '$UPDATE_TOTALS'" diff --git a/roles/web-data-analysis/files/countme-update.sh b/roles/web-data-analysis/files/countme-update.sh index 761c5ffa16..92ee6fd064 100644 --- a/roles/web-data-analysis/files/countme-update.sh +++ b/roles/web-data-analysis/files/countme-update.sh @@ -1,5 +1,7 @@ #!/bin/bash +export PATH="/usr/local/bin:$PATH" + # What are we called (used for message bus so don't just use cmd $0) CMD_NAME=countme-update @@ -101,11 +103,6 @@ if [ -d "$COUNTME_CHECKOUT" ]; then PATH="$COUNTME_CHECKOUT:$COUNTME_CHECKOUT/scripts:$PATH" fi -# Hardcoding /usr/local/bin here is hacky; should be pulled from pip, but -# parsing pip output is nontrivial, and my father always told me: -# "Son, life's too damn short write a RFC2822 parser in bash." -PATH="$PATH:/usr/local/bin" - # Check for required commands command -v $UPDATE_RAWDB >/dev/null || die "can't find '$UPDATE_RAWDB'" command -v $UPDATE_TOTALS >/dev/null || die "can't find '$UPDATE_TOTALS'" diff --git a/roles/web-data-analysis/files/mirrors-data.gp b/roles/web-data-analysis/files/mirrors-data.gp index c59e63bcb3..4b1204cea0 100644 --- a/roles/web-data-analysis/files/mirrors-data.gp +++ b/roles/web-data-analysis/files/mirrors-data.gp @@ -10,7 +10,7 @@ set key outside right top Right title 'Legend' box 3 ## set output "/var/www/html/csv-reports/images/mirrors-all-points.png" set title "Fedora+Epel Yum Unique IPs" -plot ["2007-05-17":"2024-12-31"] \ +plot ["2007-05-17":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:2 title 'epel4' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:3 title 'epel5' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:4 title 'epel6' with lines lw 3,\ @@ -101,13 +101,13 @@ unset output # set output "/var/www/html/csv-reports/images/fedora-daily.png" # set title "Fedora Daily Totals Unique IPs" -# plot ["2007-05-17":"2024-12-31"] \ +# plot ["2007-05-17":] \ # '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:36 title 'Fedora' with lines lw 3 # unset output set output "/var/www/html/csv-reports/images/fedora-os-all.png" set title "Fedora OS Yum Unique IPs" -plot ["2007-05-17":"2024-12-31"] \ +plot ["2007-05-17":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:6 title 'fed03' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:7 title 'fed04' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:8 title 'fed05' with lines lw 3,\ @@ -166,7 +166,7 @@ unset output set output "/var/www/html/csv-reports/images/fedora-os-modular.png" set title "Fedora Modular Users Unique IPs" -plot ["2018-01-01":"2024-12-31"] \ +plot ["2018-01-01":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:59 title 'modular' with lines lw 3, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:60 title 'modular_rawhide' with lines lw 3, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:61 title 'modular_f27' with lines lw 3, \ @@ -186,7 +186,7 @@ unset output set output "/var/www/html/csv-reports/images/fedora-os-latest.png" set title "Fedora Selected Versions Unique IPs" -plot ["2022-01-01":"2024-12-31"] \ +plot ["2022-01-01":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:36 title 'Fedora' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:76 title 'fed36' with lines lw 3, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:77 title 'fed37' with lines lw 3, \ @@ -198,7 +198,7 @@ unset output set output "/var/www/html/csv-reports/images/fedora-os-latest-stacked.png" set title "Fedora Selected Versions Unique IPs" -plot ["2022-01-01":"2024-12-31"] \ +plot ["2022-01-01":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:36 title 'Fedora' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($76+$77+$78+$79+$33) title 'fed36' with filledcurves x1,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($77+$78+$79+$33) title 'fed37' with filledcurves x1,\ @@ -209,7 +209,7 @@ unset output set output "/var/www/html/csv-reports/images/fedora-hardware-full.png" set title "Fedora Hardware via Unique IPs" -plot ["2007-05-17":"2024-12-31"] \ +plot ["2007-05-17":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:39 title 'ARM64' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:58 title 'ppc64le' with lines lw 3, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:43 title 's390' with lines lw 3,\ @@ -219,7 +219,7 @@ unset output set output "/var/www/html/csv-reports/images/fedora-hardware-2nd.png" set title "Fedora Secondary via Unique IPs" -plot ["2007-05-17":"2024-12-31"] \ +plot ["2007-05-17":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:37 title 'alpha' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:38 title 'ARM' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:40 title 'ia64' with lines lw 3,\ @@ -235,7 +235,7 @@ unset output set output "/var/www/html/csv-reports/images/fedora-epel-stacked.png" set title "Fedora Yum Unique IPs" -plot ["2007-05-17":"2024-12-31"] \ +plot ["2007-05-17":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($2+$3+$4+$5+$72+$73+$6+$7+$8+$9+$10+$11+$12+$13+$14+$15+$16+$17+$18+$19+$20+$21+$22+$23+$24+$25+$26+$27+$28+$29+$30+$31+$32+$65+$66+$67+$68+$74+$75+$76+$77+$78+$79+$33) title 'rawhide' w filledcurves x1, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($2+$3+$4+$5+$72+$73+$6+$7+$8+$9+$10+$11+$12+$13+$14+$15+$16+$17+$18+$19+$20+$21+$22+$23+$24+$25+$26+$27+$28+$29+$30+$31+$32+$65+$66+$67+$68+$74+$75+$76+$77+$78+$79) title 'f39' w filledcurves x1, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($2+$3+$4+$5+$72+$73+$6+$7+$8+$9+$10+$11+$12+$13+$14+$15+$16+$17+$18+$19+$20+$21+$22+$23+$24+$25+$26+$27+$28+$29+$30+$31+$32+$65+$66+$67+$68+$74+$75+$76+$77+$78) title 'f38' w filledcurves x1, \ @@ -285,7 +285,7 @@ unset output set output "/var/www/html/csv-reports/images/fedora-stacked.png" set title "Fedora Yum Unique IPs" -plot ["2007-05-17":"2024-12-31"] \ +plot ["2007-05-17":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($6+$7+$8+$9+$10+$11+$12+$13+$14+$15+$16+$17+$18+$19+$20+$21+$22+$23+$24+$25+$26+$27+$28+$29+$30+$31+$32+$65+$66+$67+$68+$74+$75+$76+$77+$78+$79+$33) title 'rawhide' w filledcurves x1, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($6+$7+$8+$9+$10+$11+$12+$13+$14+$15+$16+$17+$18+$19+$20+$21+$22+$23+$24+$25+$26+$27+$28+$29+$30+$31+$32+$65+$66+$67+$68+$74+$75+$76+$77+$78+$79) title 'f39' w filledcurves x1, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($6+$7+$8+$9+$10+$11+$12+$13+$14+$15+$16+$17+$18+$19+$20+$21+$22+$23+$24+$25+$26+$27+$28+$29+$30+$31+$32+$65+$66+$67+$68+$74+$75+$76+$77+$78) title 'f38' w filledcurves x1, \ @@ -328,7 +328,7 @@ unset output set output "/var/www/html/csv-reports/images/fedora-rev-all-stacked.png" set title "Fedora Yum Reverse Unique IPs" -plot ["2007-05-17":"2024-12-31"] \ +plot ["2007-05-17":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($33+$68+$67+$66+$65+$32+$31+$30+$29+$28+$27+$26+$25+$24+$23+$22+$21+$20+$19+$18+$17+$16+$15+$14+$13+$12+$11+$10+$9+$8+$7+$6) title 'fed03' w filledcurves x1 lc rgb "#F0F0F0", \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($33+$68+$67+$66+$65+$32+$31+$30+$29+$28+$27+$26+$25+$24+$23+$22+$21+$20+$19+$18+$17+$16+$15+$14+$13+$12+$11+$10+$9+$8+$7) title 'fed04' w filledcurves x1 lc rgb "#F0F0F0", \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($33+$68+$67+$66+$65+$32+$31+$30+$29+$28+$27+$26+$25+$24+$23+$22+$21+$20+$19+$18+$17+$16+$15+$14+$13+$12+$11+$10+$9+$8) title 'fed05' w filledcurves x1 lc rgb "#F0F0F0", \ @@ -365,7 +365,7 @@ unset output set output "/var/www/html/csv-reports/images/fedora-select-stacked.png" set title "Fedora Yum Unique IPs" -plot ["2007-05-17":"2024-12-31"] \ +plot ["2007-05-17":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($6+$7+$8+$9+$10+$11+$12+$13+$14+$15+$16+$17+$18+$19+$20+$21+$22+$23+$24+$25+$26+$27+$28+$29+$30+$31+$32+$65+$66+$67+$68+$74+$75+$76+$77+$78+$79+$33) title 'fedora-future' w filledcurves x1, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($6+$7+$8+$9+$10+$11+$12+$13+$14+$15+$16+$17+$18+$19+$20+$21+$22+$23+$24+$25+$26+$27+$28+$29+$30+$31+$32+$65+$66+$67+$68) title 'fedora 30-33' w filledcurves x1, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($6+$7+$8+$9+$10+$11+$12+$13+$14+$15+$16+$17+$18+$19+$20+$21+$22+$23+$24+$25+$26+$27+$28+$29+$30+$31+$32+$65) title 'fedora 26-30' w filledcurves x1, \ @@ -383,7 +383,7 @@ unset output set output "/var/www/html/csv-reports/images/epel-all.png" set title "Epel Yum Unique IPs" -plot ["2007-05-17":"2024-12-31"] \ +plot ["2007-05-17":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:2 title 'epel4' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:3 title 'epel5' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:4 title 'epel6' with lines lw 3,\ @@ -395,7 +395,7 @@ unset output set output "/var/www/html/csv-reports/images/epel-all-short.png" set title "Epel Yum Unique IPs" -plot ["2019-12-31":"2024-12-31"] \ +plot ["2019-12-31":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:2 title 'epel4' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:3 title 'epel5' with lines lw 3,\ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:4 title 'epel6' with lines lw 3,\ @@ -407,7 +407,7 @@ unset output set output "/var/www/html/csv-reports/images/epel-stacked.png" set title "Epel Releases Unique IPs" -plot ["2007-05-17":"2024-12-31"] \ +plot ["2007-05-17":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($2+$3+$4+$5+$72+$73) title 'epel9' w filledcurves x1, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($2+$3+$4+$5+$72) title 'epel8' w filledcurves x1, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($2+$3+$4+$5) title 'epel7' w filledcurves x1, \ @@ -418,7 +418,7 @@ unset output set output "/var/www/html/csv-reports/images/epel-stacked-short.png" set title "Epel Releases Unique IPs" -plot ["2019-12-31":"2024-12-31"] \ +plot ["2019-12-31":] \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($2+$3+$4+$5+$72+$73) title 'epel9' w filledcurves x1, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($2+$3+$4+$5+$72) title 'epel8' w filledcurves x1, \ '/var/www/html/csv-reports/mirrors/mirrorsdata-all.csv' using 1:($2+$3+$4+$5) title 'epel7' w filledcurves x1, \ diff --git a/roles/zabbix/httpchecks/files/proxy-template.yml b/roles/zabbix/httpchecks/files/proxy-template.yml index 084376ae70..911e84a10d 100644 --- a/roles/zabbix/httpchecks/files/proxy-template.yml +++ b/roles/zabbix/httpchecks/files/proxy-template.yml @@ -94,6 +94,27 @@ zabbix_export: value: availability - tag: scope value: connectivity + - uuid: dc1cf4b6e1a643e5802428a6d37ab36c + name: http-release-monitoring.org + type: ZABBIX_ACTIVE + key: 'web.page.regexp[https://release-monitoring.org/,,,Watch for releases of your favorite projects]' + delay: 5m + value_type: CHAR + trends: '0' + timeout: 6s + tags: + - tag: component + value: infra + triggers: + - uuid: 1861890b34894f27b1be9380f58c82b9 + expression: 'last(/Proxy HTTP checks/web.page.regexp[https://release-monitoring.org/,,,Watch for releases of your favorite projects])<>"Watch for releases of your favorite projects"' + name: 'http-release-monitoring.org connection failed' + priority: AVERAGE + tags: + - tag: scope + value: availability + - tag: scope + value: connectivity tags: - tag: application value: proxies diff --git a/roles/zabbix/sslchecks/tasks/sslcheck.yml b/roles/zabbix/sslchecks/tasks/sslcheck.yml index 049f536a45..1a2954981d 100644 --- a/roles/zabbix/sslchecks/tasks/sslcheck.yml +++ b/roles/zabbix/sslchecks/tasks/sslcheck.yml @@ -12,7 +12,6 @@ tags: - zabbix_agent - zabbix_api - - pagure block: - name: Create {{ item.name }} cert age item community.zabbix.zabbix_item: @@ -25,9 +24,12 @@ units: 'days' timeout: '10s' interval: '12h' + preprocessing: + - type: check_unsupported + error_handler: set_custom_error_message + error_handler_params: 'failed to execute zext_ssl_cert.sh' + params: '-1' # corresponds to 'any value' in the UI params tags: - - tag: application - value: pagure - tag: component value: ssl diff --git a/roles/zabbix/zabbix_agent/defaults/main.yml b/roles/zabbix/zabbix_agent/defaults/main.yml index ad8ee86baf..4f1e870872 100644 --- a/roles/zabbix/zabbix_agent/defaults/main.yml +++ b/roles/zabbix/zabbix_agent/defaults/main.yml @@ -1,6 +1,7 @@ --- # Defaults variables for role zabbix-agent zabbix_host: "zabbix.server" +zabbix_agentd: /etc/zabbix/zabbix_agentd.conf zabbix_agent_hostname: "{{ ansible_hostname }}" zabbix_base_templates: - Linux by Zabbix agent active diff --git a/roles/zabbix/zabbix_agent/vars/CentOS-7.yml b/roles/zabbix/zabbix_agent/vars/CentOS-7.yml index 35073d0ccf..18a2ceb347 100644 --- a/roles/zabbix/zabbix_agent/vars/CentOS-7.yml +++ b/roles/zabbix/zabbix_agent/vars/CentOS-7.yml @@ -7,6 +7,4 @@ zabbix_pkgs_list: - zabbix7.0 - bc -zabbix_agentd: /etc/zabbix_agentd.conf - zabbix_agentd_dir: /etc/zabbix/zabbix_agentd.d diff --git a/roles/zabbix/zabbix_agent/vars/CentOS-8.yml b/roles/zabbix/zabbix_agent/vars/CentOS-8.yml index fc4a30065d..a6a668f0df 100644 --- a/roles/zabbix/zabbix_agent/vars/CentOS-8.yml +++ b/roles/zabbix/zabbix_agent/vars/CentOS-8.yml @@ -7,6 +7,4 @@ zabbix_pkgs_list: - zabbix7.0 - bc -zabbix_agentd: /etc/zabbix_agentd.conf - zabbix_agentd_dir: /etc/zabbix/zabbix_agentd.d diff --git a/roles/zabbix/zabbix_agent/vars/CentOS-9.yml b/roles/zabbix/zabbix_agent/vars/CentOS-9.yml index fc4a30065d..a6a668f0df 100644 --- a/roles/zabbix/zabbix_agent/vars/CentOS-9.yml +++ b/roles/zabbix/zabbix_agent/vars/CentOS-9.yml @@ -7,6 +7,4 @@ zabbix_pkgs_list: - zabbix7.0 - bc -zabbix_agentd: /etc/zabbix_agentd.conf - zabbix_agentd_dir: /etc/zabbix/zabbix_agentd.d diff --git a/roles/zabbix/zabbix_agent/vars/Fedora.yml b/roles/zabbix/zabbix_agent/vars/Fedora.yml index 5b780a6d44..66d3d59717 100644 --- a/roles/zabbix/zabbix_agent/vars/Fedora.yml +++ b/roles/zabbix/zabbix_agent/vars/Fedora.yml @@ -9,6 +9,4 @@ zabbix_pkgs_list: - bc - ncurses-compat-libs # Needed for some megacli tools and raid monitoring checks through zabbix_sender -zabbix_agentd: /etc/zabbix_agentd.conf - zabbix_agentd_dir: /etc/zabbix/zabbix_agentd.d diff --git a/roles/zabbix/zabbix_agent/vars/common.yml b/roles/zabbix/zabbix_agent/vars/common.yml index a8ad2cef5c..a72ac72971 100644 --- a/roles/zabbix/zabbix_agent/vars/common.yml +++ b/roles/zabbix/zabbix_agent/vars/common.yml @@ -7,5 +7,3 @@ zabbix_pkgs_list: - zabbix-agent - zabbix-sender - bc - -zabbix_agentd: /etc/zabbix/zabbix_agentd.conf diff --git a/roles/zabbix/zabbix_server/files/externalscripts/zext_ssl_cert.sh b/roles/zabbix/zabbix_server/files/externalscripts/zext_ssl_cert.sh index 0f9e70925c..788ff84310 100755 --- a/roles/zabbix/zabbix_server/files/externalscripts/zext_ssl_cert.sh +++ b/roles/zabbix/zabbix_server/files/externalscripts/zext_ssl_cert.sh @@ -14,7 +14,7 @@ EXPIRY=$(echo | timeout 5 openssl s_client -servername $HOST -connect $HOST:$POR openssl x509 -noout -enddate 2>/dev/null | cut -d= -f2) if [ -z "$EXPIRY" ]; then - echo 0 + echo "timeout connecting to $HOST" exit 1 fi diff --git a/tasks/cloud_setup_basic.yml b/tasks/cloud_setup_basic.yml index d5b2b3844c..db166e8f6f 100644 --- a/tasks/cloud_setup_basic.yml +++ b/tasks/cloud_setup_basic.yml @@ -44,7 +44,7 @@ tags: - config - sshkeys - # https://forge.fedoraproject.org/infra/tickets/10509 + # https://forge.fedoraproject.org/infra/tickets/issues/10509 ignore_errors: true - name: Enable ssh_sysadm_login sebool diff --git a/vars/all/FedoraBranchedBodhi.yaml b/vars/all/FedoraBranchedBodhi.yaml index 8c20a8b084..f1d6556d69 100644 --- a/vars/all/FedoraBranchedBodhi.yaml +++ b/vars/all/FedoraBranchedBodhi.yaml @@ -3,4 +3,4 @@ # prebeta: After bodhi enablement/beta freeze and before beta release # postbeta: After beta release and before final release --- -FedoraBranchedBodhi: prebeta +FedoraBranchedBodhi: postbeta diff --git a/vars/all/Frozen.yaml b/vars/all/Frozen.yaml index 8a4b4163ce..d9bc53b221 100644 --- a/vars/all/Frozen.yaml +++ b/vars/all/Frozen.yaml @@ -1,7 +1,7 @@ --- # is the infrastructure freeze currently in place? -InfraFrozen: True +InfraFrozen: False # is the pending release (Branched) currently frozen? -NextReleaseFrozen: True +NextReleaseFrozen: False # for 'backwards compatibility' Frozen: "{{ InfraFrozen }}"