quality/testdays-web
6
0
Fork 0
Code Issues 12 Pull requests 1 Projects Releases Packages Wiki Activity Actions

Implement MCP server
All checks were successful
Run tests / test (pull_request) Successful in 31s
Details

Browse source 
- add simple read-only MCP server for reading testdays and test results
- add MCP server docs
- add example testdays-mcp skill
- extend tests to cover new features

Fixes #108

Assisted-by: Claude Code
This commit is contained in:
Jaroslav Groman 2026-05-21 13:47:50 +02:00 • committed by Jaroslav Groman
commit 80f4c961df
7 changed files with 786 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

18
asgi.py
View file

@ -39,16 +39,16 @@ class _McpAccessLogMiddleware(BaseHTTPMiddleware):
# Build the top-level ASGI app directly from the MCP instance.
# streamable_http_app() registers the MCP handler at "/mcp" (set explicitly
# in mcp_server.py) and wires its own lifespan to start/stop the session
# manager — no manual lifespan wiring needed here.
#
# We then append a catch-all Mount("/") for the Flask app. Mount uses
# prefix matching, so it handles all paths not already claimed by "/mcp".
# Route order matters: Starlette matches routes in declaration order, and
# the Flask mount is appended last so it never shadows the MCP endpoint.
# streamable_http_app() creates a Starlette application with the MCP
# handler at "/mcp" (the default but also set explicitly in
# mcp_server.py) We then mount the WSGI-converted Flask app at /.
# Since this happens after the "/mcp" route is set up, it will route
# everything except /mcp to the Flask app.
# The MCP instance creates its own Flask app instance (cached per-process)
# separate from the WSGI app below. They share the same config and database.
#
# Pattern adapted from: github.com/modelcontextprotocol/python-sdk/issues/1367
flask_app = create_app()
application = mcp.streamable_http_app()
application.router.routes.append(Mount("/", app=WsgiToAsgi(create_app())))
application.router.routes.append(Mount("/", app=WsgiToAsgi(flask_app)))
application.add_middleware(_McpAccessLogMiddleware)

163
docs/MCP_SERVER.md Normal file
View file

@ -0,0 +1,163 @@
# MCP Server
The testdays-web project includes a read-only MCP (Model Context Protocol) server that allows
AI assistants to query testday data. It exposes two tools:
- **list_testdays** -- List all published testdays
- **get_testday** -- Get details of a specific testday with result summaries
The MCP server is defined in `mcp_server.py` and runs embedded inside the web
application via `asgi.py`. It shares the Flask app context and database connection
with the main application -- it is not run as a standalone process.
## Running
The MCP server starts automatically as part of the ASGI application:
The MCP endpoint is available at `/mcp` (Streamable HTTP transport).
## MCP Client Configuration
MCP clients that support Streamable HTTP can connect directly to the running
web application.
### OpenCode
Add to `opencode.json` in the project root (or `~/.config/opencode/config.json`
for global availability):
```json
{
"mcp": {
"testdays": {
"type": "remote",
"url": "http://localhost:5050/mcp"
}
}
}
```
### Claude Code
Run the CLI command:
```shell
claude mcp add --transport http testdays http://localhost:5050/mcp
```
Or add to `.mcp.json` in the project root:
```json
{
"mcpServers": {
"testdays": {
"type": "http",
"url": "http://localhost:5050/mcp"
}
}
}
```
### Cursor
Add to `.cursor/mcp.json` in the project root:
```json
{
"mcpServers": {
"testdays": {
"url": "http://localhost:5050/mcp"
}
}
}
```
### VS Code (GitHub Copilot)
Add to `.vscode/mcp.json` in the project root:
```json
{
"servers": {
"testdays": {
"type": "http",
"url": "http://localhost:5050/mcp"
}
}
}
```
### Claude Desktop
Claude Desktop does not support remote MCP servers via its config file.
Connect through the Claude.ai web UI instead: Settings > Connectors > Add
custom connector, then enter the server URL.
### Other Clients
Any MCP client that supports Streamable HTTP transport can connect to the
`/mcp` endpoint. Replace `localhost:5050` with your deployment hostname as
needed.
## Adding Tools
Tools are defined in `mcp_server.py` using the [FastMCP](https://gofastmcp.com/)
library. Each tool is a plain Python function decorated with `@mcp.tool()`.
The docstring becomes the tool description sent to MCP clients, and type
annotations define the parameter schema.
Example -- adding a new read-only tool:
```python
@mcp.tool()
def get_testday_summary(testday_id: int) -> dict:
"""Get a brief summary of a testday's overall results.
Args:
testday_id: The ID of the testday to summarize.
Returns total pass/fail/info counts and the number of distinct testers.
"""
td = db.session.get(Testday, testday_id)
if td is None:
return {"error": f"Testday with id {testday_id} not found"}
# ... query and aggregate ...
return {"total_pass": ..., "total_fail": ..., "testers": ...}
```
Key conventions:
- Return `{"error": "..."}` for invalid requests rather than raising exceptions.
- Tool functions run synchronously inside a Flask app context provided by the
`flask_lifespan` context manager.
- Use the shared `db.session` for database access (same session as the Flask app).
## AI Agent Skill
An AI agent skill for working with the MCP tools is available at
`docs/skills/testdays-mcp/SKILL.md`. It provides domain context, usage patterns,
and result interpretation guidance to help AI assistants use the testdays MCP
tools effectively.
To install the skill, copy or symlink the `docs/skills/testdays-mcp/` directory
into your agent's skills location:
- **OpenCode:** Copy to `.opencode/skills/testdays-mcp/` in the project root, or
to `~/.config/opencode/skills/testdays-mcp/` for global availability.
- **Claude Code:** Copy to `.claude/skills/testdays-mcp/`.
Once installed, the agent will automatically load the skill when querying testday
data via MCP tools.
## Testing
MCP server unit tests are in `tests/unit/test_mcp_server.py`. Run them with:
```shell
make test -- tests/unit/test_mcp_server.py -v
```
Or run the full test suite:
```shell
make test
```

69
docs/skills/testdays-mcp/SKILL.md Normal file
View file

@ -0,0 +1,69 @@
---
name: testdays-mcp
description: Use when querying Fedora Testdays data via MCP tools, interpreting test day results, or reasoning about testday sections, testcases, and pass/fail/info result counts.
---
# Testdays MCP Server
## Overview
Read-only MCP access to Fedora Testdays data. Two tools: `list_testdays` (find testdays) and `get_testday` (get full details with result counts).
## Domain Context
A **Test Day** is a coordinated Fedora QA community event where volunteers test specific areas of the upcoming Fedora release and record their results. Common test areas include Changes (new features), kernel updates, desktop environments (GNOME, KDE, Xfce), graphical drivers, upgrades, internationalization, and more.
**Purpose:** Verify that the upcoming release has sufficient quality, or identify important issues before full release. Testing before or after the official date is also encouraged -- results continue to arrive for roughly two weeks after the event.
**Participants:** Both Red Hat employees (developers, QA engineers) and community volunteers participate. Component maintainers are typically available during the event to help debug issues in real time. The `displayname` field does not distinguish between Red Hat staff and community members.
**Communication:** Real-time discussion during events happens on Matrix in `#test-day:fedoraproject.org`. Some test days use a different channel, specified on the wiki page.
**Data hierarchy:**
```
Testday
-> Sections (groups of related tests, e.g. "Suspend/Resume")
-> Testcases (individual tests, e.g. "pm-suspend")
-> Results (one per user per hardware profile)
```
- `testday_url` links to the wiki page -- the authoritative source for test instructions, prerequisites, setup steps, available developers, and bug reporting guidance. Always reference this URL when directing users to testing instructions.
- `profile_text` overrides the default "Profile" column header for a section (e.g. "Hardware", "VM Type")
- Each testcase `url` links to a wiki page with precise, reproducible test steps. Some test days include an "Exploratory Testing" testcase for free-form testing beyond scripted cases.
**Result values:**
- **PASS** -- test succeeded on the participant's hardware/configuration
- **FAIL** -- test failed; the `bugs` field typically contains URLs to bug reports filed in Red Hat Bugzilla or upstream trackers (free-text string; may contain multiple URLs separated by whitespace, or be empty)
- **INFO** -- informational, neither pass nor fail (supplementary observations)
**Draft vs published:** Testdays start as drafts (`is_draft=True`). Only published testdays are visible through MCP. Requesting a draft returns an error.
**Removed results:** Results can be soft-deleted (`removed=True`). These are excluded from all counts and do not appear in the results array.
**Filtered testcase entries:** The raw testday data can contain two special entry types that are automatically excluded from the MCP response and never appear in `testcases`:
- **Comment lines** -- editorial notes in the test matrix, not real testcases
- **Disabled testcases** -- testcases marked as inactive for the event
You will only ever see active, real testcases in `get_testday` output.
## Usage Patterns
### List-then-get workflow
Always start with `list_testdays` to discover available testdays and their IDs, then use `get_testday` with a specific ID for details. Don't guess testday IDs.
### Interpreting results
Each testcase includes a `results_summary` with pass/fail/info counts and a `results` array with individual per-user records.
- **High fail count** relative to pass count signals problems with that test area
- **All zeros** means no one has tested that testcase yet
- **Info-only results** are supplementary data, not pass/fail judgments
- **FAIL results with empty `bugs`** may indicate the tester didn't file a bug report -- the comment may still describe the issue
- **Multiple FAILs on the same testcase** across different profiles suggest a widespread problem rather than a hardware-specific issue
- When summarizing a test day, count distinct testers (unique `displayname` values) to gauge participation level
### Error handling
`get_testday` returns an `"error"` key (not an exception) for missing or draft testdays. Always check for the `"error"` key before processing sections.

182
mcp_server.py
View file

@ -1,12 +1,68 @@
"""
Stub MCP server for ASGI integration development.
Read-only MCP server for Testdays.
This file provides a minimal FastMCP instance so that asgi.py can import
and mount it. The real MCP server will be implemented later.
Exposes two tools:
- list_testdays: List all published testdays
- get_testday: Get details of a specific testday with result summaries
This module is not run standalone. It is imported by asgi.py (production)
and by tests. In production, the ``flask_lifespan`` context manager
provides the Flask app context. Tests provide their own app context
via fixtures.
"""
from contextlib import asynccontextmanager
from functools import cache
from flask import has_app_context
from sqlalchemy import func
from mcp.server.fastmcp import FastMCP
from mcp.server.transport_security import TransportSecuritySettings
from testdays import create_app
from testdays.extensions import db
from testdays.models.testday import Testday, Result
from testdays.models.user import User
@cache
def _create_flask_app():
"""Create and cache a Flask app for MCP database access.
Uses ``functools.cache`` so the app is created once per process.
In unit tests, the test fixtures provide their own app context, so
this function is never called. In ASGI integration tests (where
``TestClient`` runs in a separate thread), this function is called
to create a Flask app that connects to the test database.
"""
return create_app()
@asynccontextmanager
async def flask_lifespan(server: FastMCP):
"""Push a Flask app context for the duration of each MCP request.
In stateless HTTP mode, ``Server.run()`` is called per-request and
enters this context manager each time. The Flask ``app_context()``
uses ``ContextVar``, so the context propagates through the
``anyio`` task tree to the synchronous tool handler functions.
When a Flask app context is already active (e.g. in unit tests
where the ``app`` fixture pushes one), the lifespan is a no-op to
avoid interfering with the test transactional session. In ASGI
integration tests (where ``TestClient`` runs in a separate thread),
``has_app_context()`` returns ``False`` and the lifespan creates
its own app via ``_create_flask_app()``.
"""
if has_app_context():
yield {}
else:
with _create_flask_app().app_context():
yield {}
# --- MCP server ---
mcp = FastMCP(
"testdays",
instructions="Read-only access to Fedora Testdays data.",
@ -16,8 +72,124 @@ mcp = FastMCP(
# but setting it here makes the routing intent visible and ensures
# asgi.py can mount Flask at "/" without any path conflicts.
streamable_http_path="/mcp",
lifespan=flask_lifespan,
# Override the default DNS rebinding protection which only allows
# localhost connections. We keep it enabled but add the production
# and staging hostnames so the MCP endpoint is reachable remotely.
transport_security=TransportSecuritySettings(
enable_dns_rebinding_protection=True,
allowed_hosts=[
"127.0.0.1:*",
"localhost:*",
"[::1]:*",
"testdays.stg.fedoraproject.org",
"testdays.fedoraproject.org",
],
),
)
if __name__ == "__main__":
mcp.run(transport="stdio")
@mcp.tool()
def list_testdays() -> list[dict]:
"""List all published (non-draft) testdays.
Returns a list of testday objects with id, name, testday_url, start, and end fields,
ordered by start date descending (most recent first).
"""
testdays = (
Testday.query
.filter(Testday.is_draft.is_(False))
.order_by(Testday.start.desc())
.all()
)
return [
{
"id": td.id,
"name": td.name,
"testday_url": td.testday_url,
"start": td.start.isoformat() if td.start else None,
"end": td.end.isoformat() if td.end else None,
}
for td in testdays
]
@mcp.tool()
def get_testday(testday_id: int) -> dict:
"""Get details of a specific testday including sections, testcases, and result summaries.
Args:
testday_id: The ID of the testday to retrieve.
Returns a testday object with metadata, sections containing testcases,
aggregated result counts (pass/fail/info) per testcase, and individual
result records with displayname, profile, result, comment, and bugs.
Only published (non-draft) testdays are accessible.
"""
td = db.session.get(Testday, testday_id)
if td is None:
return {"error": f"Testday with id {testday_id} not found"}
if td.is_draft:
return {"error": f"Testday with id {testday_id} is not published"}
# Fetch all non-removed results with user info in one query
results_query = (
db.session.query(Result, User)
.join(User, Result.user_id == User.id)
.filter(
Result.testday_id == testday_id,
Result.removed.is_(False),
)
.all()
)
# Build per-testcase aggregated counts and individual result records
result_counts: dict[str, dict[str, int]] = {}
result_records: dict[str, list[dict]] = {}
for result_obj, user_obj in results_query:
ulid = result_obj.testcase_ulid
if ulid not in result_counts:
result_counts[ulid] = {"pass": 0, "fail": 0, "info": 0}
key = result_obj.result.lower() if result_obj.result else ""
if key in result_counts[ulid]:
result_counts[ulid][key] += 1
result_records.setdefault(ulid, []).append({
"displayname": user_obj.displayname or "",
"profile": result_obj.profile or "",
"result": result_obj.result or "",
"comment": result_obj.comment or "",
"bugs": result_obj.bugs or "",
})
# Build sections from structure
structure = td.structure
sections = []
for section in structure.get("Sections", []):
if "Comment" in section:
continue
testcases = []
for tc in section.get("Testcases", []):
if "Comment" in tc or tc.get("Disabled"):
continue
ulid = tc.get("Ulid", "")
testcases.append({
"ulid": ulid,
"name": tc.get("Name", ""),
"url": tc.get("Url", ""),
"results_summary": result_counts.get(ulid, {"pass": 0, "fail": 0, "info": 0}),
"results": result_records.get(ulid, []),
})
sections.append({
"name": section.get("Name", ""),
"profile_text": section.get("ProfileText", ""),
"testcases": testcases,
})
return {
"id": td.id,
"name": td.name,
"testday_url": td.testday_url,
"start": td.start.isoformat() if td.start else None,
"end": td.end.isoformat() if td.end else None,
"sections": sections,
}

1
requirements.txt
View file

@ -26,3 +26,4 @@ uvicorn>=0.31.1
asgiref~=3.8.1
psycopg2-binary~=2.9.11
psycopg2-pool==1.2
mcp~=1.27.1

50
tests/functional/test_asgi.py
View file

@ -95,3 +95,53 @@ class TestMcpEndpointViaAsgi:
"""
response = asgi_client.get("/mcp")
assert response.status_code == 406
def test_mcp_tool_call_has_flask_context(self, asgi_client: TestClient) -> None:
"""MCP tool call succeeds with Flask app context (via lifespan).
Performs a full MCP lifecycle: initialize, then call the
list_testdays tool. This exercises the flask_lifespan context
manager that provides the app context for DB access.
"""
# Initialize MCP session
init_resp = asgi_client.post(
"/mcp",
json={
"jsonrpc": "2.0",
"id": 1,
"method": "initialize",
"params": {
"protocolVersion": "2025-03-26",
"capabilities": {},
"clientInfo": {"name": "test", "version": "0.1.0"},
},
},
headers={
"Content-Type": "application/json",
"Accept": "application/json",
},
)
assert init_resp.status_code == 200
# Call the list_testdays tool
tool_resp = asgi_client.post(
"/mcp",
json={
"jsonrpc": "2.0",
"id": 2,
"method": "tools/call",
"params": {
"name": "list_testdays",
"arguments": {},
},
},
headers={
"Content-Type": "application/json",
"Accept": "application/json",
},
)
assert tool_resp.status_code == 200
body = tool_resp.json()
# The tool should return a result, not an app context error
assert "result" in body
assert "error" not in body

317
tests/unit/test_mcp_server.py Normal file
View file

@ -0,0 +1,317 @@
"""
MCP server tool unit tests.
Tests the list_testdays and get_testday tool handler functions
using the existing test database fixtures.
"""
import datetime
import pytest
from mcp_server import list_testdays, get_testday
from testdays.models.testday import Testday, Testcase, Result
from testdays.models.user import User
@pytest.fixture(name="sample_testday")
def _sample_testday(db_session):
"""Create a published testday with sections, testcases, and results."""
structure = {
"Sections": [
{
"Name": "Suspend Tests",
"ProfileText": "",
"Testcases": [
{
"Ulid": "TC-001",
"Name": "pm-suspend",
"Url": "https://example.com/tc1",
},
{
"Ulid": "TC-002",
"Name": "pm-hibernate",
"Url": "https://example.com/tc2",
},
{"Comment": "this is a comment line"},
{
"Ulid": "TC-003",
"Name": "disabled-test",
"Url": "https://example.com/tc3",
"Disabled": True,
},
],
},
],
}
td = Testday("Test Day Alpha", structure)
td.is_draft = False
td.start = datetime.datetime(2026, 5, 1, tzinfo=datetime.timezone.utc)
td.end = datetime.datetime(2026, 5, 2, tzinfo=datetime.timezone.utc)
db_session.add(td)
db_session.flush()
# Create testcases in the testcase table
tc1 = Testcase("https://example.com/tc1", td, ulid="TC-001")
tc2 = Testcase("https://example.com/tc2", td, ulid="TC-002")
db_session.add_all([tc1, tc2])
db_session.flush()
# Create users for results
user = User(sub="anon:tester", username="tester")
user.displayname = "Test User"
user2 = User(sub="fedora:alice", username="alice")
user2.displayname = "Alice A."
db_session.add_all([user, user2])
db_session.flush()
# Create results (with comments and bugs on some)
db_session.add(Result(td.id, "TC-001", user, "Laptop X", "PASS",
comment="Works fine"))
db_session.add(Result(td.id, "TC-001", user, "Desktop Y", "PASS"))
db_session.add(Result(td.id, "TC-001", user2, "VM Z", "FAIL",
bugs="https://bugzilla.redhat.com/12345",
comment="Crashes on resume"))
db_session.add(Result(td.id, "TC-002", user, "Laptop X", "INFO"))
# Add a removed result that should not be counted
removed = Result(td.id, "TC-001", user, "Old HW", "FAIL")
removed.removed = True
db_session.add(removed)
db_session.flush()
return td
@pytest.fixture(name="draft_testday")
def _draft_testday(db_session):
"""Create a draft (unpublished) testday."""
td = Testday("Draft Day", {"Sections": []})
td.is_draft = True
td.start = datetime.datetime(2026, 6, 1, tzinfo=datetime.timezone.utc)
td.end = datetime.datetime(2026, 6, 2, tzinfo=datetime.timezone.utc)
db_session.add(td)
db_session.flush()
return td
class TestListTestdays:
"""Tests for the list_testdays MCP tool."""
@pytest.mark.usefixtures("sample_testday", "draft_testday")
def test_returns_published_testdays(self):
"""Verify only published testdays appear in the listing."""
result = list_testdays()
assert isinstance(result, list)
assert len(result) == 1
names = [td["name"] for td in result]
assert "Test Day Alpha" in names
@pytest.mark.usefixtures("sample_testday")
def test_returns_expected_fields(self):
"""Verify each testday dict contains all required fields."""
result = list_testdays()
td = next(t for t in result if t["name"] == "Test Day Alpha")
assert "id" in td
assert "name" in td
assert "testday_url" in td
assert "start" in td
assert "end" in td
@pytest.mark.usefixtures("draft_testday")
def test_empty_when_no_published_testdays(self):
"""Verify an empty list is returned when only drafts exist."""
result = list_testdays()
assert result == []
@pytest.mark.usefixtures("sample_testday")
def test_ordered_by_start_descending(self, db_session):
"""Verify testdays are returned most-recent-first."""
# Create an older published testday
older = Testday("Older Day", {"Sections": []})
older.is_draft = False
older.start = datetime.datetime(2025, 1, 1, tzinfo=datetime.timezone.utc)
older.end = datetime.datetime(2025, 1, 2, tzinfo=datetime.timezone.utc)
db_session.add(older)
db_session.flush()
result = list_testdays()
names = [td["name"] for td in result]
assert names == ["Test Day Alpha", "Older Day"]
class TestGetTestday:
"""Tests for the get_testday MCP tool."""
def test_returns_testday_details(self, sample_testday):
"""Verify basic testday metadata is returned."""
result = get_testday(sample_testday.id)
assert result["name"] == "Test Day Alpha"
assert result["id"] == sample_testday.id
assert "sections" in result
def test_sections_contain_testcases(self, sample_testday):
"""Verify sections include their active testcases."""
result = get_testday(sample_testday.id)
section = result["sections"][0]
assert section["name"] == "Suspend Tests"
testcase_names = [tc["name"] for tc in section["testcases"]]
assert "pm-suspend" in testcase_names
assert "pm-hibernate" in testcase_names
def test_excludes_disabled_testcases(self, sample_testday):
"""Verify disabled testcases are not included."""
result = get_testday(sample_testday.id)
section = result["sections"][0]
testcase_names = [tc["name"] for tc in section["testcases"]]
assert "disabled-test" not in testcase_names
def test_excludes_comment_lines(self, sample_testday):
"""Verify comment lines in the structure are filtered out."""
result = get_testday(sample_testday.id)
section = result["sections"][0]
# Should have 2 testcases (pm-suspend, pm-hibernate), not 4
assert len(section["testcases"]) == 2
def test_result_summaries(self, sample_testday):
"""Verify aggregated pass/fail/info counts per testcase."""
result = get_testday(sample_testday.id)
section = result["sections"][0]
tc1 = next(tc for tc in section["testcases"]
if tc["name"] == "pm-suspend")
# 2 PASS + 1 FAIL (removed FAIL excluded)
assert tc1["results_summary"]["pass"] == 2
assert tc1["results_summary"]["fail"] == 1
assert tc1["results_summary"]["info"] == 0
def test_excluded_removed_results(self, sample_testday):
"""Verify removed results are not counted in summaries."""
result = get_testday(sample_testday.id)
section = result["sections"][0]
tc1 = next(tc for tc in section["testcases"]
if tc["name"] == "pm-suspend")
# Total non-removed: 2 PASS + 1 FAIL = 3
total = sum(tc1["results_summary"].values())
assert total == 3
@pytest.mark.usefixtures("db_session")
def test_not_found(self):
"""Verify error response for a non-existent testday ID."""
result = get_testday(99999)
assert "error" in result
assert "not found" in result["error"]
def test_draft_not_accessible(self, draft_testday):
"""Verify error response when requesting a draft testday."""
result = get_testday(draft_testday.id)
assert "error" in result
assert "not published" in result["error"]
def test_testday_with_no_results(self, db_session):
"""Verify a testday with testcases but no results returns zero counts."""
structure = {
"Sections": [
{
"Name": "Empty Section",
"ProfileText": "",
"Testcases": [
{
"Ulid": "TC-EMPTY",
"Name": "no-results-tc",
"Url": "https://example.com/empty",
},
],
},
],
}
td = Testday("No Results Day", structure)
td.is_draft = False
td.start = datetime.datetime(2026, 7, 1, tzinfo=datetime.timezone.utc)
td.end = datetime.datetime(2026, 7, 2, tzinfo=datetime.timezone.utc)
db_session.add(td)
db_session.flush()
result = get_testday(td.id)
tc = result["sections"][0]["testcases"][0]
assert tc["results_summary"] == {"pass": 0, "fail": 0, "info": 0}
def test_results_contain_individual_records(self, sample_testday):
"""Verify each testcase includes a results list with per-user records."""
result = get_testday(sample_testday.id)
section = result["sections"][0]
tc1 = next(tc for tc in section["testcases"]
if tc["name"] == "pm-suspend")
# pm-suspend has 3 non-removed results
assert "results" in tc1
assert len(tc1["results"]) == 3
def test_results_have_expected_fields(self, sample_testday):
"""Verify each result record has displayname, profile, result, comment, bugs."""
result = get_testday(sample_testday.id)
section = result["sections"][0]
tc1 = next(tc for tc in section["testcases"]
if tc["name"] == "pm-suspend")
for r in tc1["results"]:
assert "displayname" in r
assert "profile" in r
assert "result" in r
assert "comment" in r
assert "bugs" in r
def test_results_include_displayname(self, sample_testday):
"""Verify displayname is the human-readable name, not ulid or username."""
result = get_testday(sample_testday.id)
section = result["sections"][0]
tc1 = next(tc for tc in section["testcases"]
if tc["name"] == "pm-suspend")
displaynames = {r["displayname"] for r in tc1["results"]}
assert "Test User" in displaynames
assert "Alice A." in displaynames
def test_results_include_comments_and_bugs(self, sample_testday):
"""Verify comment and bugs fields are populated from result data."""
result = get_testday(sample_testday.id)
section = result["sections"][0]
tc1 = next(tc for tc in section["testcases"]
if tc["name"] == "pm-suspend")
fail_result = next(r for r in tc1["results"]
if r["result"] == "FAIL")
assert fail_result["comment"] == "Crashes on resume"
assert fail_result["bugs"] == "https://bugzilla.redhat.com/12345"
def test_removed_results_excluded_from_results_list(self, sample_testday):
"""Verify soft-deleted results don't appear in the results list."""
result = get_testday(sample_testday.id)
section = result["sections"][0]
tc1 = next(tc for tc in section["testcases"]
if tc["name"] == "pm-suspend")
# The removed result had profile "Old HW"
profiles = [r["profile"] for r in tc1["results"]]
assert "Old HW" not in profiles
def test_results_empty_for_untested_testcase(self, db_session):
"""Verify testcases with no results return an empty results list."""
structure = {
"Sections": [
{
"Name": "Empty Section",
"ProfileText": "",
"Testcases": [
{
"Ulid": "TC-EMPTY",
"Name": "no-results-tc",
"Url": "https://example.com/empty",
},
],
},
],
}
td = Testday("No Results Day", structure)
td.is_draft = False
td.start = datetime.datetime(2026, 7, 1, tzinfo=datetime.timezone.utc)
td.end = datetime.datetime(2026, 7, 2, tzinfo=datetime.timezone.utc)
db_session.add(td)
db_session.flush()
result = get_testday(td.id)
tc = result["sections"][0]["testcases"][0]
assert tc["results"] == []