Fix all yamllint errors and warnings #3143
No reviewers
Labels
No labels
ai-review-please
freeze-break-request
post-freeze
Backlog Status
Needs Review
Backlog Status
Ready
chore
documentation
points
01
points
02
points
03
points
05
points
08
points
13
Priority
High
Priority
Low
Priority
Medium
Sprint Status
Blocked
Sprint Status
Done
Sprint Status
In Progress
Sprint Status
Review
Sprint Status
To Do
Technical Debt
Work Item
Bug
Work Item
Epic
Work Item
Spike
Work Item
Task
Work Item
User Story
No milestone
No project
No assignees
6 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
infra/ansible!3143
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "ryanlerch/ansible:yamllint-fixes"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
fixes all yamllint errors and warnings across the Ansible repository.
Note too, that i changed the CI for yamllint to now run on all files, as there should be no linting errors in the codebase.
Note too the ansible-lint check is likely to fail since this touches so many files, and many still fail if you run ansible-lint on all the files in the ansible repo
1010260b5e15b84fbd4eWow. Thanks for this.
I need to look it over in detail... and it should wait for freeze to be over, but thanks!
I have to be honest—I’m not a fan of mass linter fixes. Code should generally only be changed when necessary, and our priority should be to make sure no new warnings are added. Maintaining 'git blame' stability provides much higher value than simply having happy linters. While some might argue that you can't CI-lint a project without 'cleaning house' first, implementing differential scanning is actually quite easy these days (ie, scanning both the base and the proposed code with the same linter version, and only failing if new errors are introduced).
I'm not a super fan of them either, but I think it''s probibly worth doing at this point so we got to a point where people stop ignoring ci checks because they are in things they didn't touch. ;(
But, now this has conflicts... @ryanlerch can you rebase/fix them up? Then we could run the handy ai review on it and see what it says?
The problem is that we cannot meet this goal consistently, full-scan linters would anyway bother people in the future. Linter results often change even when the code remains untouched—typically when a new version of yamllint is released or a new policy becomes the default. When this happens, you are forced into another round of mass fixes.
With Adam we previously discussed why differential scanning is the superior approach. Now that we have action runners (moved from Pagure), it seems like low-hanging fruit to implement. I’m happy to contribute the code if the community is interested.
@praiskup This is actually how it's being done. It's only running ansible-lint and yamllint on files that were changed. Unfortunately this is not testing only changes, but whole files. In any case it doesn't run yamllint against the whole repository.
While this may seem like an advantage, it has a serious consequence: you lose the necessary context for those linters. The only correct way to handle linting is to:
Analyzing files in isolation provides the worst of both worlds: you lose the necessary context (e.g., code calling "ansible handler" that is nowhere defined), yet you are still burdened with pre-existing issues in that file that you didn't introduce. It is a more awkward compromise than simply performing a scan of a "diff file".
(You may refer to the previous discussion with Adam)
Yeah, I agree with @praiskup 's proposal. For many languages there are tools for doing this - I use diff-cover for Python, for e.g, which also includes a
diff-qualityimplementation which does more or less this with linters (only tells you about new linter errors introduced by the PR). But if something like that doesn't exist for ansible-lint and yamllint, implementing it shouldn't be too hard, if they have useful machine-readable output formats at least.Yeah, that all makes sense... and I am open to changing to that.
However, we still need to fix things so we have a known good/no errors/warnings state right?
No, that's the benefit of the diff approach if you can swing it. It works even if the base state has 50,000 errors; if the checks on the PR show the same 50,000 errors, then it passes. If they show 50,001 errors, it fails. For each PR, the 'base state' is "whatever the linter shows on the branch the PR is against", the 'comparison state' is "whatever the linter shows on the PR branch", and the test passes if there are no new errors. If there are new errors, the test fails and the output is any new errors found.
There are some tricky bits to implementation, though. Say the PR sticks 500 new (perfectly linted) lines into a file which has pre-existing lint errors after those new lines. Now the linter output will be different because the line numbers are different, but it's still the same errors. The implementation needs to be able, somehow, to know that the two outputs are equivalent and call this a pass. It's probably not trivial to solve this (and other related situations). It'd probably be useful to look at how diff-cover does it (and maybe steal it; maybe we can even just extend diff-cover to work with yamllint and ansible-lint?). Maybe the trick is to examine the diff and adjust the line numbers appropriately?
edit: note you can't just look for errors which are on lines in the diff, because it's possible for a change to cause an error on an existing line elsewhere.
Sure, I get that, but then if you never touch the stuff that fails already it never gets cleaned up?
I'm fine with the flow you are all describing, but I still think it might be valuable to clean up at least easy ones so there's not 50,000 errors in things we don't touch much. Not all the time, but since our linting was utterly broken for a long time (it was returning fine 100% of the time/not actually running the checks)
Not through this process, no. The idea is that you do that as and when you want to, and the differential checks prevent backsliding.
I know this is for the yamllint portion, but AIUI, ansible-lint is a bit harder to effectively run on just changed files or changed parts of files.
If a clean blame is what you are trying to save, would it be better to do all linting changes in a single commit, so you can use --
ignore-revswhen trying to git blame and running into the linting commit?@adamwill
Yes, some of those tricky line movements are trickier than others; but in general, csdiff solves this problem a decent way for years (we've been using it for pylint in vcs-diff-lint).
@kevin
Yeah, but, ... typically we only care about very specific warnings (only those that have real potential to strike, e.g. security related families of errors). The rest is just better to ignore.
This is where I disagree, but it's just an individual's perspective (fixing linter warnings only ever caused hard times to me, never anything good).
@ryanlerch
Well, I think we can create a simple hook for ansible-lint into the linter? Off hand, I don't see any reason this should be challenging?
Certainly! One commit is much better than a dozen.
To clarify, I’m referring to the practice of 'fixing lint just for the sake of having warning-free code.' The reality is that, over the long term, the codebase cleans itself: differential scanning prevents new warnings from being introduced, while legacy warnings naturally disappear as those sections of code are refactored or removed.
Just a quick experiment with vcs-diff-lint (csdiff command-line wrapper) and yamllint here:
https://github.com/fedora-copr/vcs-diff-lint/pull/39
If vcs-diff-lint is installed (from the Copr build triggered by Packit), one can run the command against this PR:
vcs-diff-lint --compare-against 5134a6c604e05852e68a07f752d2477d039b7a3e --linter-tag yamllint --print-fixed-errors(no new yaml errors added, many are fixed)
Take a look at the approach here, feedback is welcome: #3304
So, I am fine with the differential scan, thats fine.
I still like the idea of this as a one off thing to fix all the outstanding junk so we start from a good / better baseline.
I'll be happily shouted down if all of you disagree.
related discussion in https://discussion.fedoraproject.org/t/should-we-restore-pre-commit-in-infra-ansible/195507/2
15b84fbd4ecc1db32428cc1db3242804dbd9cbfe@kevin have rebased and fixed conflicts.
also fixed a bunch of new warnings that were introduced since this was last rebased.
all but one were in existing categories, so i put the changes into each commit for them.
this was the new change, which should be checked -- !3143 (commit
04dbd9cbfe)i assume we just want to squash and merge here in forge when we do merge this.
Ok. Yeah, that change was my bad... I meant to delete the 1 there, but apparently I didn't. ;(
@adamwill @praiskup I'm still inclined to merge this so we start from some kind of 'mostly passing' baseline...
but feel free to talk me out of it?
you have successfully poked this ticket enough times that I no longer care and just want the emails to stop
Sorry. Lets see what the clanker has to say?
AI Code Review
Detailed Code Review
This is a massive and well-executed cleanup effort. By bringing the entire repository to a zero-warning state, you provide a clear path forward for maintaining linting hygiene.
I have thoroughly reviewed the automated and manual fixes applied throughout the repository. Specifically, quoting octal representations for file permissions (e.g.,
mode: "0755") is exactly the right approach for Ansible and prevents implicit octal conversion issues in strict YAML parsers. Furthermore, standardizing on lowercase booleans (falseinstead ofFalse) guarantees compatibility with both YAML 1.1 and 1.2 specifications.The removal of the duplicate
persistentvolumeclaimskey correctly retained the parameterized version, which resolves the lint error while preserving the intended dynamic behavior. No regressions, logic errors, or security concerns were identified during this review.✅ Summary
🤖 AI Code Review | Generated with ai-code-review | Model:
gemini-3.1-pro-preview⚠️ AI-generated suggestions may be incorrect. Verify before applying. Not a replacement for human review.
it loves it! What could possibly go wrong! :)