Decision / Feedback: Adopt bruce in CI? #69

Open
opened 2026-10-25 14:13:33 +00:00 by mwinters · 2 comments
Member

What is bruce?

bruce is a tool I've created (still WIP) to crawl sites like docs.fp.o and produce reports on broken links. What's unique is that bruce can report links to a broken anchor, e.g. https://example.com/foo#this_anchor_doesnt_exist. This surfaces issues where we linked to a specific section on a page, and then that section went away. The URL is not 404, but the content is gone.

There are >700 such links currently across our docs -- almost as many of these as actual 404 links (>800).

You can see some example reports at https://mwinters.fedorapeople.org/bruce/ . The tool is still WIP, so newer reports look better than old ones, but I think they are useful enough now to be actionable. The underlying data is also available as json and duckdb for further tinkering.

My plan / request

Phase 1

  1. Get feedback on the html reports to see if people find that sort of information useful / any tweaks requested.
  2. Deploy this (once) to our docs team CI as a nightly job, so that there is a central place people can point to and see where these reports come from instead of my own homelab infra. Presumably we'd also find a better / more official place to publish the latest report than my fedorapeople.org account.
  3. Update our own docs to point newcomers who are looking to help over to the broken link report.

Phase 2

Update the tool so that it can automatically file "broken link" issues in the Forge repos. The tool must also be capable of detecting when a link is no longer broken and automatically closing these automatically-created issues, so that they are helpful instead of annoying.

If our AI folks wanted to, this would provide an easy index for some other bot to discover such issues and propose PRs.

Phase "maybe someday"

Since the tool builds a central database, it would be possible to configure each docs repo with a lint step in CI which checks the central database, to see whether the current commit contains broken links and either warn or prevent the commit.

The difficult part is not building this step, it's the fact that if you applied it today it would warn/block almost every single repo: 83% of them have broken links today. So each repo would need to fix all of their links first to get to green.


How to read the report (FAQ)

I'll continue to update this issue as questions come up, and I'll incorporate these docs directly into the report.

What does "broken through" and/or "doubled slash" mean?

We have a situation where page B is fine if you go there the right way, but page A links to it in a way that adds an extra slash, e.g. docs.fp.o/foo//page_B/ or docs.fp.o/foo/page_B//.

If you navigate from A -> B, page B will work but all of its relative links (../../blah) are broken because of the extra slash in the path. I didn't want to count these numbers against page B, because there's nothing wrong there -- the problem is in page A. This is labeled as "broken through it" if the slash is in the middle of the URL, and "doubled slash" if the slash is at the end.

What is a "pedantic" error?

Humans generally write URLs like example.com/foo but technically the correct form is example.com/foo/ -- notice the trailing slash. These are actually two separate web pages as far as most web servers are concerned, and if you visit the first one the web server will helpfully tell your browser, "Hey dummy, there's no page here it's over at example.com/foo/". So your browser has to make a second request.

Should we count the "wrong" URL as broken, considering that the user eventually ends up in the right place? It's debatable, but certainly lower priority than actually-broken links (e.g. 404). These are not currently marked as broken, landing instead under Housekeeping as "pedantic".

## What is bruce? [`bruce`](https://github.com/mwinters0/bruce) is a tool I've created (still WIP) to crawl sites like docs.fp.o and produce reports on broken links. What's unique is that bruce can report links to a broken anchor, e.g. `https://example.com/foo#this_anchor_doesnt_exist`. This surfaces issues where we linked to a specific section on a page, and then that section went away. **The URL is not 404, but the content is gone.** There are >700 such links currently across our docs -- almost as many of these as actual 404 links (>800). You can see some example reports at https://mwinters.fedorapeople.org/bruce/ . The tool is still WIP, so newer reports look better than old ones, but I think they are useful enough now to be actionable. The underlying data is also available as json and duckdb for further tinkering. ## My plan / request ### Phase 1 1. Get feedback on the html reports to see if people find that sort of information useful / any tweaks requested. - ✅ DONE: Approved! See: https://forge.fedoraproject.org/docs/tickets/issues/69#issuecomment-1371168 2. Deploy this (once) to our docs team CI as a nightly job, so that there is a central place people can point to and see where these reports come from instead of my own homelab infra. Presumably we'd also find a better / more official place to publish the latest report than my fedorapeople.org account. - [ ] WIP: See: https://forge.fedoraproject.org/docs/tickets/issues/69#issuecomment-1374696 3. Update our own docs to point newcomers who are looking to help over to the broken link report. ### Phase 2 Update the tool so that it can automatically file "broken link" issues in the Forge repos. The tool must also be capable of detecting when a link is no longer broken and automatically closing these automatically-created issues, so that they are helpful instead of annoying. If our AI folks wanted to, this would provide an easy index for some other bot to discover such issues and propose PRs. ### Phase "maybe someday" Since the tool builds a central database, it would be possible to configure each docs repo with a lint step in CI which checks the central database, to see whether the current commit contains broken links and either warn or prevent the commit. The difficult part is not building this step, it's the fact that if you applied it today it would warn/block almost every single repo: 83% of them have broken links today. So each repo would need to fix all of their links first to get to green. --- ## How to read the report (FAQ) _I'll continue to update this issue as questions come up, and I'll incorporate these docs directly into the report._ ### What does "broken through" and/or "doubled slash" mean? We have a situation where page B is fine if you go there the right way, but page A links to it in a way that adds an extra slash, e.g. `docs.fp.o/foo//page_B/` or `docs.fp.o/foo/page_B//`. If you navigate from A -> B, page B will work but all of its relative links (`../../blah`) are broken because of the extra slash in the path. I didn't want to count these numbers against page B, because there's nothing wrong there -- the problem is in page A. This is labeled as "broken through it" if the slash is in the middle of the URL, and "doubled slash" if the slash is at the end. ### What is a "pedantic" error? Humans generally write URLs like `example.com/foo` but _technically_ the correct form is `example.com/foo/` -- notice the trailing slash. These are actually two separate web pages as far as most web servers are concerned, and if you visit the first one the web server will helpfully tell your browser, "Hey dummy, there's no page here it's over at example.com/foo/". So your browser has to make a second request. Should we count the "wrong" URL as broken, considering that the user eventually ends up in the right place? It's debatable, but certainly lower priority than actually-broken links (e.g. 404). These are not currently marked as broken, landing instead under Housekeeping as "pedantic".
Owner

Discussed at the 2026-11-03 team meeting: https://discussion.fedoraproject.org/t/fedora-docs-team-meeting-2026-11-03/206179/2

  • Michael Winters presented a script named “Bruce” that checks for broken links across all documentation.
  • The team supports integrating this tool into the CI/CD pipeline.
  • Discussion covered how to manage the output, with suggestions including a static status page, creating “Good First Issues” for newcomers, and using AI to suggest fixes.
  • The rollout will happen in phases:
    • Phase 1: Get the script running in CI and publishing a static report.
    • Phase 2: Determine how to act on the findings, such as by automatically filing issues.
Discussed at the 2026-11-03 team meeting: https://discussion.fedoraproject.org/t/fedora-docs-team-meeting-2026-11-03/206179/2 * Michael Winters presented a script named “Bruce” that checks for broken links across all documentation. * The team supports integrating this tool into the CI/CD pipeline. * Discussion covered how to manage the output, with suggestions including a static status page, creating “Good First Issues” for newcomers, and using AI to suggest fixes. * The rollout will happen in phases: * Phase 1: Get the script running in CI and publishing a static report. * Phase 2: Determine how to act on the findings, such as by automatically filing issues.
Author
Member

My near-term roadmap then:

  1. Finish the Github dev deploy to prove that this is working as intended to do a nightly crawl + report in CI.
  2. Deploy the same to docs Forge somewhere.
  3. Work out how to publish the results to /static/
My near-term roadmap then: 1. Finish the Github dev deploy to prove that this is working as intended to do a nightly crawl + report in CI. - ✅ DONE: [Digest Index](https://mwinters0.github.io/bruce-fedora-docs-ci/) (via Github Pages), [Repo](https://github.com/mwinters0/bruce-fedora-docs-ci/) 2. Deploy the same to docs Forge somewhere. 3. Work out how to publish the results to `/static/`
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
docs/tickets#69
No description provided.