feat: add reproducible deploy script and rootless Podman docs #13
2 changed files with 234 additions and 0 deletions
feat: add reproducible deploy script and rootless Podman docs
Add scripts/deploy.sh, an end-to-end deploy that rebuilds the pipeline from source on a fresh host (deps, fetch, stage, model pull, vector store build), so the server can be treated as stateless. It uses the GPU when available and warns and falls back to CPU otherwise. Add docs/rootless-podman.md describing how to run the workload unprivileged with rootless Podman, to standardize the environment. Not yet executed on the host; that is the next step. Assisted-by: Claude Opus 4.8 Signed-off-by: gtfrans2re <francoisgonothitoure@gmail.com>
commit
d2944bddd2
100
docs/rootless-podman.md
Normal file
100
docs/rootless-podman.md
Normal file
|
|
@ -0,0 +1,100 @@
|
|||
# Running the pipeline rootless with Podman
|
||||
|
||||
This describes how to run the RAG pipeline as an unprivileged user with rootless
|
||||
Podman, so the workload is standardized and does not depend on running as root.
|
||||
Rootless operation is the safer default: the containers ramalama launches run
|
||||
without root on the host, which limits the blast radius if anything goes wrong.
|
||||
|
||||
This complements docs/gpu-deployment.md. The GPU passthrough there uses CDI,
|
||||
which is designed to work rootless, so the same GPU setup applies here.
|
||||
|
||||
## Why rootless
|
||||
|
||||
The pipeline has so far been run as root on the deployment host. Running it
|
||||
rootless instead means:
|
||||
|
||||
- The workload is reproducible for any user, not just root.
|
||||
- Containers run under the user's own namespace, without host root.
|
||||
- It matches how ramalama and Podman are intended to be used, and how a
|
||||
standardized, shareable environment should behave.
|
||||
|
||||
## One-time host setup
|
||||
|
||||
These steps are done once, and some require root to prepare the unprivileged
|
||||
user. Creating the user and granting subordinate ID ranges is inherently an
|
||||
administrative action; everything after that runs unprivileged.
|
||||
|
||||
### 1. Create an unprivileged user
|
||||
|
||||
```bash
|
||||
useradd -m -s /bin/bash raguser
|
||||
```
|
||||
|
||||
### 2. Ensure subordinate UID/GID ranges exist
|
||||
|
||||
Rootless Podman maps container users into a range of subordinate IDs on the
|
||||
host. Most distributions add these automatically; confirm they are present.
|
||||
|
||||
```bash
|
||||
grep raguser /etc/subuid /etc/subgid || \
|
||||
usermod --add-subuids 100000-165535 --add-subgids 100000-165535 raguser
|
||||
```
|
||||
|
||||
### 3. Enable lingering so the user's services persist
|
||||
|
||||
This lets the user's Podman run without an active login session, which is
|
||||
useful for long builds and background serving.
|
||||
|
||||
```bash
|
||||
loginctl enable-linger raguser
|
||||
```
|
||||
|
||||
### 4. Confirm GPU access is available rootless
|
||||
|
||||
CDI passthrough works rootless. As raguser, verify the GPU is visible from a
|
||||
container (see docs/gpu-deployment.md for the CDI setup):
|
||||
|
||||
```bash
|
||||
su - raguser
|
||||
podman run --rm --device nvidia.com/gpu=all \
|
||||
docker.io/nvidia/cuda:12.4.1-base-ubi9 nvidia-smi
|
||||
```
|
||||
|
||||
If this prints the GPU table, rootless GPU access works.
|
||||
|
||||
## Running the pipeline rootless
|
||||
|
||||
As raguser, the workflow is identical to the root workflow; only the user
|
||||
changes. Clone the repo and run the deploy script:
|
||||
|
||||
```bash
|
||||
su - raguser
|
||||
git clone https://forge.fedoraproject.org/ai-ml/editorial-guide-ramalama.git
|
||||
cd editorial-guide-ramalama
|
||||
./scripts/deploy.sh --publication commblog
|
||||
```
|
||||
|
||||
ramalama and Podman run entirely in the user's own namespace. Images the user
|
||||
builds are stored in the user's local container storage under their home
|
||||
directory, not in a system-wide location.
|
||||
|
||||
## Notes and caveats
|
||||
|
||||
- Rootless containers cannot bind to ports below 1024 by default. The Streamlit
|
||||
app defaults to port 8501, which is fine. If a privileged port is ever
|
||||
needed, prefer a higher port or a reverse proxy rather than granting extra
|
||||
privileges.
|
||||
- Rootless Podman uses fuse-overlayfs for storage. On most current systems this
|
||||
is already in place; if image builds are slow or fail on overlay, installing
|
||||
fuse-overlayfs resolves it.
|
||||
- Pushing built images to Quay still requires the user to run `podman login`
|
||||
with their own credentials.
|
||||
- The deploy script (scripts/deploy.sh) is user-agnostic: it does not assume
|
||||
root and runs the same way for raguser as for any other account.
|
||||
|
||||
## Status
|
||||
|
||||
This documents the intended rootless setup. It has not yet been executed on the
|
||||
deployment host; running it and recording any host-specific adjustments is the
|
||||
next step, to be aligned with the mentors as part of standardizing the
|
||||
environment.
|
||||
134
scripts/deploy.sh
Executable file
134
scripts/deploy.sh
Executable file
|
|
@ -0,0 +1,134 @@
|
|||
#!/usr/bin/env bash
|
||||
#
|
||||
# deploy.sh
|
||||
#
|
||||
# Reproducible, end-to-end deployment of the editorial-guide-ramalama RAG
|
||||
# pipeline on a fresh host. The goal is to treat the server as stateless: this
|
||||
# script rebuilds everything from source, so a host can be torn down and
|
||||
# recreated without losing anything that is not in git or on Quay.
|
||||
#
|
||||
# It performs: dependency setup, corpus fetch and staging, model pull, vector
|
||||
# store build, and an optional smoke query. GPU acceleration is used when
|
||||
# available (see docs/gpu-deployment.md for the CDI setup); otherwise it falls
|
||||
# back to CPU with a warning.
|
||||
#
|
||||
# Usage:
|
||||
# ./scripts/deploy.sh [--publication commblog|magazine|editorial]
|
||||
# [--chunk-size N] [--no-build] [--smoke]
|
||||
#
|
||||
# Examples:
|
||||
# ./scripts/deploy.sh # full deploy, commblog, chunk 256
|
||||
# ./scripts/deploy.sh --publication magazine # deploy the magazine corpus
|
||||
# ./scripts/deploy.sh --smoke # deploy then run one test query
|
||||
#
|
||||
# Environment overrides:
|
||||
# EGR_MODEL model to pull/run (default: hf://bartowski/google_gemma-4-E4B-it-GGUF)
|
||||
# EGR_IMAGE_REPO Quay repo (default: quay.io/fedora/editorial-guide-ramalama)
|
||||
# EGR_VENV virtualenv path (default: .venv)
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# -- Defaults -----------------------------------------------------------------
|
||||
|
||||
PUBLICATION="commblog"
|
||||
CHUNK_SIZE="256"
|
||||
DO_BUILD=1
|
||||
DO_SMOKE=0
|
||||
|
||||
MODEL="${EGR_MODEL:-hf://bartowski/google_gemma-4-E4B-it-GGUF}"
|
||||
IMAGE_REPO="${EGR_IMAGE_REPO:-quay.io/fedora/editorial-guide-ramalama}"
|
||||
VENV="${EGR_VENV:-.venv}"
|
||||
|
||||
# -- Parse arguments ----------------------------------------------------------
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--publication) PUBLICATION="$2"; shift 2 ;;
|
||||
--chunk-size) CHUNK_SIZE="$2"; shift 2 ;;
|
||||
--no-build) DO_BUILD=0; shift ;;
|
||||
--smoke) DO_SMOKE=1; shift ;;
|
||||
-h|--help) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
|
||||
*) echo "Unknown argument: $1" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
IMAGE="${IMAGE_REPO}:${PUBLICATION}"
|
||||
|
||||
log() { printf '\n=== %s ===\n' "$1"; }
|
||||
|
||||
# -- Preflight ----------------------------------------------------------------
|
||||
|
||||
log "Preflight: checking required tools"
|
||||
for tool in python3 git podman ramalama; do
|
||||
if ! command -v "$tool" >/dev/null 2>&1; then
|
||||
echo "Missing required tool: $tool" >&2
|
||||
echo "Install it and re-run. See docs/GUIDE.md." >&2
|
||||
exit 1
|
||||
fi
|
||||
printf ' found %s\n' "$tool"
|
||||
done
|
||||
|
||||
# GPU check: warn (do not fail) if the GPU is not visible to containers.
|
||||
log "Preflight: checking GPU passthrough"
|
||||
if podman run --rm --device nvidia.com/gpu=all \
|
||||
docker.io/nvidia/cuda:12.4.1-base-ubi9 nvidia-smi >/dev/null 2>&1; then
|
||||
echo " GPU passthrough is working; the build will use the GPU."
|
||||
else
|
||||
echo " WARNING: GPU not visible to containers (CDI not configured?)."
|
||||
echo " The pipeline will run on CPU, which is much slower."
|
||||
echo " See docs/gpu-deployment.md to enable GPU passthrough."
|
||||
fi
|
||||
|
||||
# -- Python environment -------------------------------------------------------
|
||||
|
||||
log "Setting up the Python environment"
|
||||
if [ ! -d "$VENV" ]; then
|
||||
python3 -m venv "$VENV"
|
||||
fi
|
||||
# shellcheck disable=SC1091
|
||||
source "$VENV/bin/activate"
|
||||
pip install --quiet --upgrade pip
|
||||
pip install --quiet -r requirements.txt
|
||||
echo " dependencies installed into $VENV"
|
||||
|
||||
# -- Fetch and stage the corpus ----------------------------------------------
|
||||
|
||||
log "Fetching articles"
|
||||
python scripts/fetch_articles.py
|
||||
|
||||
log "Staging the corpus"
|
||||
bash scripts/stage_corpus.sh
|
||||
|
||||
# -- Pull the model -----------------------------------------------------------
|
||||
|
||||
log "Pulling the model: $MODEL"
|
||||
ramalama pull "$MODEL"
|
||||
|
||||
# -- Build the vector store ---------------------------------------------------
|
||||
|
||||
if [ "$DO_BUILD" -eq 1 ]; then
|
||||
log "Building the vector store: $IMAGE (chunk-size $CHUNK_SIZE)"
|
||||
CORPUS_DIR="data/source/${PUBLICATION}"
|
||||
if [ ! -d "$CORPUS_DIR" ]; then
|
||||
echo "Corpus directory not found: $CORPUS_DIR" >&2
|
||||
echo "Check the --publication value." >&2
|
||||
exit 1
|
||||
fi
|
||||
time ramalama rag --chunk-size "$CHUNK_SIZE" "$CORPUS_DIR" "$IMAGE"
|
||||
echo " built $IMAGE"
|
||||
else
|
||||
echo "Skipping vector store build (--no-build)."
|
||||
fi
|
||||
|
||||
# -- Optional smoke query -----------------------------------------------------
|
||||
|
||||
if [ "$DO_SMOKE" -eq 1 ]; then
|
||||
log "Smoke query against $IMAGE"
|
||||
printf 'What are the key editorial guidelines for a good article?\n' \
|
||||
| ramalama run --rag "$IMAGE" "$MODEL" || true
|
||||
fi
|
||||
|
||||
log "Deploy complete"
|
||||
echo "Image: $IMAGE"
|
||||
echo "Run interactively with:"
|
||||
echo " ramalama run --rag $IMAGE $MODEL"
|
||||
Loading…
Add table
Add a link
Reference in a new issue