feat: add reproducible deploy script and rootless Podman docs #13

Closed
gtfrans2re wants to merge 1 commit from feat/reproducible-deploy into main
2 changed files with 234 additions and 0 deletions

100
docs/rootless-podman.md Normal file
View file

@ -0,0 +1,100 @@
# Running the pipeline rootless with Podman
This describes how to run the RAG pipeline as an unprivileged user with rootless
Podman, so the workload is standardized and does not depend on running as root.
Rootless operation is the safer default: the containers ramalama launches run
without root on the host, which limits the blast radius if anything goes wrong.
This complements docs/gpu-deployment.md. The GPU passthrough there uses CDI,
which is designed to work rootless, so the same GPU setup applies here.
## Why rootless
The pipeline has so far been run as root on the deployment host. Running it
rootless instead means:
- The workload is reproducible for any user, not just root.
- Containers run under the user's own namespace, without host root.
- It matches how ramalama and Podman are intended to be used, and how a
standardized, shareable environment should behave.
## One-time host setup
These steps are done once, and some require root to prepare the unprivileged
user. Creating the user and granting subordinate ID ranges is inherently an
administrative action; everything after that runs unprivileged.
### 1. Create an unprivileged user
```bash
useradd -m -s /bin/bash raguser
```
### 2. Ensure subordinate UID/GID ranges exist
Rootless Podman maps container users into a range of subordinate IDs on the
host. Most distributions add these automatically; confirm they are present.
```bash
grep raguser /etc/subuid /etc/subgid || \
usermod --add-subuids 100000-165535 --add-subgids 100000-165535 raguser
```
### 3. Enable lingering so the user's services persist
This lets the user's Podman run without an active login session, which is
useful for long builds and background serving.
```bash
loginctl enable-linger raguser
```
### 4. Confirm GPU access is available rootless
CDI passthrough works rootless. As raguser, verify the GPU is visible from a
container (see docs/gpu-deployment.md for the CDI setup):
```bash
su - raguser
podman run --rm --device nvidia.com/gpu=all \
docker.io/nvidia/cuda:12.4.1-base-ubi9 nvidia-smi
```
If this prints the GPU table, rootless GPU access works.
## Running the pipeline rootless
As raguser, the workflow is identical to the root workflow; only the user
changes. Clone the repo and run the deploy script:
```bash
su - raguser
git clone https://forge.fedoraproject.org/ai-ml/editorial-guide-ramalama.git
cd editorial-guide-ramalama
./scripts/deploy.sh --publication commblog
```
ramalama and Podman run entirely in the user's own namespace. Images the user
builds are stored in the user's local container storage under their home
directory, not in a system-wide location.
## Notes and caveats
- Rootless containers cannot bind to ports below 1024 by default. The Streamlit
app defaults to port 8501, which is fine. If a privileged port is ever
needed, prefer a higher port or a reverse proxy rather than granting extra
privileges.
- Rootless Podman uses fuse-overlayfs for storage. On most current systems this
is already in place; if image builds are slow or fail on overlay, installing
fuse-overlayfs resolves it.
- Pushing built images to Quay still requires the user to run `podman login`
with their own credentials.
- The deploy script (scripts/deploy.sh) is user-agnostic: it does not assume
root and runs the same way for raguser as for any other account.
## Status
This documents the intended rootless setup. It has not yet been executed on the
deployment host; running it and recording any host-specific adjustments is the
next step, to be aligned with the mentors as part of standardizing the
environment.

134
scripts/deploy.sh Executable file
View file

@ -0,0 +1,134 @@
#!/usr/bin/env bash
#
# deploy.sh
#
# Reproducible, end-to-end deployment of the editorial-guide-ramalama RAG
# pipeline on a fresh host. The goal is to treat the server as stateless: this
# script rebuilds everything from source, so a host can be torn down and
# recreated without losing anything that is not in git or on Quay.
#
# It performs: dependency setup, corpus fetch and staging, model pull, vector
# store build, and an optional smoke query. GPU acceleration is used when
# available (see docs/gpu-deployment.md for the CDI setup); otherwise it falls
# back to CPU with a warning.
#
# Usage:
# ./scripts/deploy.sh [--publication commblog|magazine|editorial]
# [--chunk-size N] [--no-build] [--smoke]
#
# Examples:
# ./scripts/deploy.sh # full deploy, commblog, chunk 256
# ./scripts/deploy.sh --publication magazine # deploy the magazine corpus
# ./scripts/deploy.sh --smoke # deploy then run one test query
#
# Environment overrides:
# EGR_MODEL model to pull/run (default: hf://bartowski/google_gemma-4-E4B-it-GGUF)
# EGR_IMAGE_REPO Quay repo (default: quay.io/fedora/editorial-guide-ramalama)
# EGR_VENV virtualenv path (default: .venv)
set -euo pipefail
# -- Defaults -----------------------------------------------------------------
PUBLICATION="commblog"
CHUNK_SIZE="256"
DO_BUILD=1
DO_SMOKE=0
MODEL="${EGR_MODEL:-hf://bartowski/google_gemma-4-E4B-it-GGUF}"
IMAGE_REPO="${EGR_IMAGE_REPO:-quay.io/fedora/editorial-guide-ramalama}"
VENV="${EGR_VENV:-.venv}"
# -- Parse arguments ----------------------------------------------------------
while [ "$#" -gt 0 ]; do
case "$1" in
--publication) PUBLICATION="$2"; shift 2 ;;
--chunk-size) CHUNK_SIZE="$2"; shift 2 ;;
--no-build) DO_BUILD=0; shift ;;
--smoke) DO_SMOKE=1; shift ;;
-h|--help) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) echo "Unknown argument: $1" >&2; exit 2 ;;
esac
done
IMAGE="${IMAGE_REPO}:${PUBLICATION}"
log() { printf '\n=== %s ===\n' "$1"; }
# -- Preflight ----------------------------------------------------------------
log "Preflight: checking required tools"
for tool in python3 git podman ramalama; do
if ! command -v "$tool" >/dev/null 2>&1; then
echo "Missing required tool: $tool" >&2
echo "Install it and re-run. See docs/GUIDE.md." >&2
exit 1
fi
printf ' found %s\n' "$tool"
done
# GPU check: warn (do not fail) if the GPU is not visible to containers.
log "Preflight: checking GPU passthrough"
if podman run --rm --device nvidia.com/gpu=all \
docker.io/nvidia/cuda:12.4.1-base-ubi9 nvidia-smi >/dev/null 2>&1; then
echo " GPU passthrough is working; the build will use the GPU."
else
echo " WARNING: GPU not visible to containers (CDI not configured?)."
echo " The pipeline will run on CPU, which is much slower."
echo " See docs/gpu-deployment.md to enable GPU passthrough."
fi
# -- Python environment -------------------------------------------------------
log "Setting up the Python environment"
if [ ! -d "$VENV" ]; then
python3 -m venv "$VENV"
fi
# shellcheck disable=SC1091
source "$VENV/bin/activate"
pip install --quiet --upgrade pip
pip install --quiet -r requirements.txt
echo " dependencies installed into $VENV"
# -- Fetch and stage the corpus ----------------------------------------------
log "Fetching articles"
python scripts/fetch_articles.py
log "Staging the corpus"
bash scripts/stage_corpus.sh
# -- Pull the model -----------------------------------------------------------
log "Pulling the model: $MODEL"
ramalama pull "$MODEL"
# -- Build the vector store ---------------------------------------------------
if [ "$DO_BUILD" -eq 1 ]; then
log "Building the vector store: $IMAGE (chunk-size $CHUNK_SIZE)"
CORPUS_DIR="data/source/${PUBLICATION}"
if [ ! -d "$CORPUS_DIR" ]; then
echo "Corpus directory not found: $CORPUS_DIR" >&2
echo "Check the --publication value." >&2
exit 1
fi
time ramalama rag --chunk-size "$CHUNK_SIZE" "$CORPUS_DIR" "$IMAGE"
echo " built $IMAGE"
else
echo "Skipping vector store build (--no-build)."
fi
# -- Optional smoke query -----------------------------------------------------
if [ "$DO_SMOKE" -eq 1 ]; then
log "Smoke query against $IMAGE"
printf 'What are the key editorial guidelines for a good article?\n' \
| ramalama run --rag "$IMAGE" "$MODEL" || true
fi
log "Deploy complete"
echo "Image: $IMAGE"
echo "Run interactively with:"
echo " ramalama run --rag $IMAGE $MODEL"