feat: add reproducible deploy script and rootless Podman docs
Some checks failed
CI / checks (push) Has been cancelled
Some checks failed
CI / checks (push) Has been cancelled
Add scripts/deploy.sh, an end-to-end deploy that rebuilds the pipeline from source on a fresh host, so the server can be treated as stateless. Add docs/rootless-podman.md for running the workload unprivileged with rootless Podman. Assisted-by: Claude Opus 4.8 Signed-off-by: gtfrans2re <francoisgonothitoure@gmail.com>
This commit is contained in:
parent
bc1d7f296b
commit
10cb1945f4
2 changed files with 234 additions and 0 deletions
100
docs/rootless-podman.md
Normal file
100
docs/rootless-podman.md
Normal file
|
|
@ -0,0 +1,100 @@
|
|||
# Running the pipeline rootless with Podman
|
||||
|
||||
This describes how to run the RAG pipeline as an unprivileged user with rootless
|
||||
Podman, so the workload is standardized and does not depend on running as root.
|
||||
Rootless operation is the safer default: the containers ramalama launches run
|
||||
without root on the host, which limits the blast radius if anything goes wrong.
|
||||
|
||||
This complements docs/gpu-deployment.md. The GPU passthrough there uses CDI,
|
||||
which is designed to work rootless, so the same GPU setup applies here.
|
||||
|
||||
## Why rootless
|
||||
|
||||
The pipeline has so far been run as root on the deployment host. Running it
|
||||
rootless instead means:
|
||||
|
||||
- The workload is reproducible for any user, not just root.
|
||||
- Containers run under the user's own namespace, without host root.
|
||||
- It matches how ramalama and Podman are intended to be used, and how a
|
||||
standardized, shareable environment should behave.
|
||||
|
||||
## One-time host setup
|
||||
|
||||
These steps are done once, and some require root to prepare the unprivileged
|
||||
user. Creating the user and granting subordinate ID ranges is inherently an
|
||||
administrative action; everything after that runs unprivileged.
|
||||
|
||||
### 1. Create an unprivileged user
|
||||
|
||||
```bash
|
||||
useradd -m -s /bin/bash raguser
|
||||
```
|
||||
|
||||
### 2. Ensure subordinate UID/GID ranges exist
|
||||
|
||||
Rootless Podman maps container users into a range of subordinate IDs on the
|
||||
host. Most distributions add these automatically; confirm they are present.
|
||||
|
||||
```bash
|
||||
grep raguser /etc/subuid /etc/subgid || \
|
||||
usermod --add-subuids 100000-165535 --add-subgids 100000-165535 raguser
|
||||
```
|
||||
|
||||
### 3. Enable lingering so the user's services persist
|
||||
|
||||
This lets the user's Podman run without an active login session, which is
|
||||
useful for long builds and background serving.
|
||||
|
||||
```bash
|
||||
loginctl enable-linger raguser
|
||||
```
|
||||
|
||||
### 4. Confirm GPU access is available rootless
|
||||
|
||||
CDI passthrough works rootless. As raguser, verify the GPU is visible from a
|
||||
container (see docs/gpu-deployment.md for the CDI setup):
|
||||
|
||||
```bash
|
||||
su - raguser
|
||||
podman run --rm --device nvidia.com/gpu=all \
|
||||
docker.io/nvidia/cuda:12.4.1-base-ubi9 nvidia-smi
|
||||
```
|
||||
|
||||
If this prints the GPU table, rootless GPU access works.
|
||||
|
||||
## Running the pipeline rootless
|
||||
|
||||
As raguser, the workflow is identical to the root workflow; only the user
|
||||
changes. Clone the repo and run the deploy script:
|
||||
|
||||
```bash
|
||||
su - raguser
|
||||
git clone https://forge.fedoraproject.org/ai-ml/editorial-guide-ramalama.git
|
||||
cd editorial-guide-ramalama
|
||||
./scripts/deploy.sh --publication commblog
|
||||
```
|
||||
|
||||
ramalama and Podman run entirely in the user's own namespace. Images the user
|
||||
builds are stored in the user's local container storage under their home
|
||||
directory, not in a system-wide location.
|
||||
|
||||
## Notes and caveats
|
||||
|
||||
- Rootless containers cannot bind to ports below 1024 by default. The Streamlit
|
||||
app defaults to port 8501, which is fine. If a privileged port is ever
|
||||
needed, prefer a higher port or a reverse proxy rather than granting extra
|
||||
privileges.
|
||||
- Rootless Podman uses fuse-overlayfs for storage. On most current systems this
|
||||
is already in place; if image builds are slow or fail on overlay, installing
|
||||
fuse-overlayfs resolves it.
|
||||
- Pushing built images to Quay still requires the user to run `podman login`
|
||||
with their own credentials.
|
||||
- The deploy script (scripts/deploy.sh) is user-agnostic: it does not assume
|
||||
root and runs the same way for raguser as for any other account.
|
||||
|
||||
## Status
|
||||
|
||||
This documents the intended rootless setup. It has not yet been executed on the
|
||||
deployment host; running it and recording any host-specific adjustments is the
|
||||
next step, to be aligned with the mentors as part of standardizing the
|
||||
environment.
|
||||
134
scripts/deploy.sh
Executable file
134
scripts/deploy.sh
Executable file
|
|
@ -0,0 +1,134 @@
|
|||
#!/usr/bin/env bash
|
||||
#
|
||||
# deploy.sh
|
||||
#
|
||||
# Reproducible, end-to-end deployment of the editorial-guide-ramalama RAG
|
||||
# pipeline on a fresh host. The goal is to treat the server as stateless: this
|
||||
# script rebuilds everything from source, so a host can be torn down and
|
||||
# recreated without losing anything that is not in git or on Quay.
|
||||
#
|
||||
# It performs: dependency setup, corpus fetch and staging, model pull, vector
|
||||
# store build, and an optional smoke query. GPU acceleration is used when
|
||||
# available (see docs/gpu-deployment.md for the CDI setup); otherwise it falls
|
||||
# back to CPU with a warning.
|
||||
#
|
||||
# Usage:
|
||||
# ./scripts/deploy.sh [--publication commblog|magazine|editorial]
|
||||
# [--chunk-size N] [--no-build] [--smoke]
|
||||
#
|
||||
# Examples:
|
||||
# ./scripts/deploy.sh # full deploy, commblog, chunk 256
|
||||
# ./scripts/deploy.sh --publication magazine # deploy the magazine corpus
|
||||
# ./scripts/deploy.sh --smoke # deploy then run one test query
|
||||
#
|
||||
# Environment overrides:
|
||||
# EGR_MODEL model to pull/run (default: hf://bartowski/google_gemma-4-E4B-it-GGUF)
|
||||
# EGR_IMAGE_REPO Quay repo (default: quay.io/fedora/editorial-guide-ramalama)
|
||||
# EGR_VENV virtualenv path (default: .venv)
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# -- Defaults -----------------------------------------------------------------
|
||||
|
||||
PUBLICATION="commblog"
|
||||
CHUNK_SIZE="256"
|
||||
DO_BUILD=1
|
||||
DO_SMOKE=0
|
||||
|
||||
MODEL="${EGR_MODEL:-hf://bartowski/google_gemma-4-E4B-it-GGUF}"
|
||||
IMAGE_REPO="${EGR_IMAGE_REPO:-quay.io/fedora/editorial-guide-ramalama}"
|
||||
VENV="${EGR_VENV:-.venv}"
|
||||
|
||||
# -- Parse arguments ----------------------------------------------------------
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--publication) PUBLICATION="$2"; shift 2 ;;
|
||||
--chunk-size) CHUNK_SIZE="$2"; shift 2 ;;
|
||||
--no-build) DO_BUILD=0; shift ;;
|
||||
--smoke) DO_SMOKE=1; shift ;;
|
||||
-h|--help) grep '^#' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
|
||||
*) echo "Unknown argument: $1" >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
IMAGE="${IMAGE_REPO}:${PUBLICATION}"
|
||||
|
||||
log() { printf '\n=== %s ===\n' "$1"; }
|
||||
|
||||
# -- Preflight ----------------------------------------------------------------
|
||||
|
||||
log "Preflight: checking required tools"
|
||||
for tool in python3 git podman ramalama; do
|
||||
if ! command -v "$tool" >/dev/null 2>&1; then
|
||||
echo "Missing required tool: $tool" >&2
|
||||
echo "Install it and re-run. See docs/GUIDE.md." >&2
|
||||
exit 1
|
||||
fi
|
||||
printf ' found %s\n' "$tool"
|
||||
done
|
||||
|
||||
# GPU check: warn (do not fail) if the GPU is not visible to containers.
|
||||
log "Preflight: checking GPU passthrough"
|
||||
if podman run --rm --device nvidia.com/gpu=all \
|
||||
docker.io/nvidia/cuda:12.4.1-base-ubi9 nvidia-smi >/dev/null 2>&1; then
|
||||
echo " GPU passthrough is working; the build will use the GPU."
|
||||
else
|
||||
echo " WARNING: GPU not visible to containers (CDI not configured?)."
|
||||
echo " The pipeline will run on CPU, which is much slower."
|
||||
echo " See docs/gpu-deployment.md to enable GPU passthrough."
|
||||
fi
|
||||
|
||||
# -- Python environment -------------------------------------------------------
|
||||
|
||||
log "Setting up the Python environment"
|
||||
if [ ! -d "$VENV" ]; then
|
||||
python3 -m venv "$VENV"
|
||||
fi
|
||||
# shellcheck disable=SC1091
|
||||
source "$VENV/bin/activate"
|
||||
pip install --quiet --upgrade pip
|
||||
pip install --quiet -r requirements.txt
|
||||
echo " dependencies installed into $VENV"
|
||||
|
||||
# -- Fetch and stage the corpus ----------------------------------------------
|
||||
|
||||
log "Fetching articles"
|
||||
python scripts/fetch_articles.py
|
||||
|
||||
log "Staging the corpus"
|
||||
bash scripts/stage_corpus.sh
|
||||
|
||||
# -- Pull the model -----------------------------------------------------------
|
||||
|
||||
log "Pulling the model: $MODEL"
|
||||
ramalama pull "$MODEL"
|
||||
|
||||
# -- Build the vector store ---------------------------------------------------
|
||||
|
||||
if [ "$DO_BUILD" -eq 1 ]; then
|
||||
log "Building the vector store: $IMAGE (chunk-size $CHUNK_SIZE)"
|
||||
CORPUS_DIR="data/source/${PUBLICATION}"
|
||||
if [ ! -d "$CORPUS_DIR" ]; then
|
||||
echo "Corpus directory not found: $CORPUS_DIR" >&2
|
||||
echo "Check the --publication value." >&2
|
||||
exit 1
|
||||
fi
|
||||
time ramalama rag --chunk-size "$CHUNK_SIZE" "$CORPUS_DIR" "$IMAGE"
|
||||
echo " built $IMAGE"
|
||||
else
|
||||
echo "Skipping vector store build (--no-build)."
|
||||
fi
|
||||
|
||||
# -- Optional smoke query -----------------------------------------------------
|
||||
|
||||
if [ "$DO_SMOKE" -eq 1 ]; then
|
||||
log "Smoke query against $IMAGE"
|
||||
printf 'What are the key editorial guidelines for a good article?\n' \
|
||||
| ramalama run --rag "$IMAGE" "$MODEL" || true
|
||||
fi
|
||||
|
||||
log "Deploy complete"
|
||||
echo "Image: $IMAGE"
|
||||
echo "Run interactively with:"
|
||||
echo " ramalama run --rag $IMAGE $MODEL"
|
||||
Loading…
Add table
Add a link
Reference in a new issue