forge/forge
10
7
Fork 5
Code Issues 85 Pull requests 1 Projects 3 Wiki Activity Actions

Explore the option of mount of an OS secret with runner configuration into the VM (explored and deployed)) #436

New issue
Closed
opened 2026-03-04 14:40:32 +00:00 by lenkaseg · 10 comments
lenkaseg commented 2026-03-04 14:40:32 +00:00
Member
Copy link

Documentation: https://kubevirt.io/user-guide/storage/disks_and_volumes/#secret

  • create a VM (use a template from the current runnerhost VM) with mounted service account that would allow to watch the secret with the runners configuration
  • create service account
  • create role
  • create role binding
  • create secret with the runners
  • make a POC that proves the VM can get the secrets
Documentation: https://kubevirt.io/user-guide/storage/disks_and_volumes/#secret - create a VM (use a template from the current runnerhost VM) with mounted service account that would allow to watch the secret with the runners configuration - create service account - create role - create role binding - create secret with the runners - make a POC that proves the VM can get the secrets
lenkaseg added this to the Backlog project 2026-03-04 14:40:33 +00:00
lenkaseg commented 2026-03-11 14:37:03 +00:00
Author
Member
Copy link

First POC:

  • create a secret in forgejo OC namespace

  • create a pod that can fetch the secret and read it

First POC: - [x] create a secret in forgejo OC namespace - [x] create a pod that can fetch the secret and read it
lenkaseg commented 2026-03-12 14:08:17 +00:00
Author
Member
Copy link
  • exploring virtiofs mount, seems that we would need to enable virtiofs on a cluster, it is not enabled now
  • exploring mount as a disc, but that would require reboot every time we want to fetch a fresh secret, so not an option
  • exploring VM with a service account, role, rolebinding - this option would enable us to watch the secret, so it could run the playbook every time the secret changes, probably no need for cron then
- exploring virtiofs mount, seems that we would need to enable virtiofs on a cluster, it is not enabled now - exploring mount as a disc, but that would require reboot every time we want to fetch a fresh secret, so not an option - exploring VM with a service account, role, rolebinding - this option would enable us to watch the secret, so it could run the playbook every time the secret changes, probably no need for cron then
lenkaseg commented 2026-03-13 11:15:10 +00:00
Author
Member
Copy link
  • created a service account
  • created role to be able to get a secret called forgejo-runner-config (the test secret I created earlier)
  • created rolebinding to enable the service account to act as the defined role
  • created a test runner vm on openshift staging with the service account mounted as a disc
  • logged inside the VM and fetched the secret successfully
- [x] created a service account - [x] created role to be able to get a secret called forgejo-runner-config (the test secret I created earlier) - [x] created rolebinding to enable the service account to act as the defined role - [x] created a test runner vm on openshift staging with the service account mounted as a disc - [x] logged inside the VM and fetched the secret successfully
lenkaseg commented 2026-03-13 15:36:55 +00:00
Author
Member
Copy link
  • Let claude create a watcher - piece of code that keep watching the secret and at an event of change, it fetches the secret and stores it on the VM

Code of the watcher: https://gist.github.com/LenkaSeg/fd5fc87370e98b063ef33fdecd0f43b0

  • Started the watcher on a testing VM
  • Changed the secret
  • Troubleshot(/shooted?) the watcher until it started working, fetched the token well
  • Watched the watcher watch and fetch the changed secret and store it in the VM.
  • Copy the entire playbook ansible-role-forgejo-runner inside the VM and try to make it fetch the secret with runner definitions and run them as a runner services.
  • Add/remove some more runners in the secret and watch if they get synced and the runner services get added/removed.
- [x] Let claude create a watcher - piece of code that keep watching the secret and at an event of change, it fetches the secret and stores it on the VM Code of the watcher: https://gist.github.com/LenkaSeg/fd5fc87370e98b063ef33fdecd0f43b0 - [x] Started the watcher on a testing VM - [x] Changed the secret - [x] Troubleshot(/shooted?) the watcher until it started working, fetched the token well - [x] Watched the watcher watch and fetch the changed secret and store it in the VM. - [x] Copy the entire playbook ansible-role-forgejo-runner inside the VM and try to make it fetch the secret with runner definitions and run them as a runner services. - [x] Add/remove some more runners in the secret and watch if they get synced and the runner services get added/removed.
lenkaseg modified the project from Backlog to Sprint 17 2026-03-23 10:27:09 +00:00
dkirwan commented 2026-03-23 13:00:47 +00:00
Member
Copy link
  • Create the systemd unit to install and run this watcher
  • watcher needs to deploy the config to /home/fedora/forgejo-runners-vars.yaml
- Create the systemd unit to install and run this watcher - watcher needs to deploy the config to /home/fedora/forgejo-runners-vars.yaml
lenkaseg commented 2026-03-26 10:10:35 +00:00
Author
Member
Copy link

  • Create the systemd unit to install and run this watcher

  • watcher needs to deploy the config to /home/fedora/forgejo-runners-vars.yaml

- [x] Create the systemd unit to install and run this watcher - [x] watcher needs to deploy the config to /home/fedora/forgejo-runners-vars.yaml
lenkaseg commented 2026-03-30 13:16:00 +00:00
Author
Member
Copy link

The watcher now watches the secret and synchronizes the runner configs in a file inside the runner vm.
Running the playbook from the VM, the runner service start and runners get deployed to staging.

Required to solve a problem with networking, all good and working now.

The watcher now watches the secret and synchronizes the runner configs in a file inside the runner vm. Running the playbook from the VM, the runner service start and runners get deployed to staging. Required to solve a problem with networking, all good and working now.
lenkaseg commented 2026-03-30 13:17:50 +00:00
Author
Member
Copy link

Next steps:

  • automate the playbook run at the event of secret sync
  • open a PR on ansible-role-forgejo-runner repo with the VM side changes
  • open a PR on infra/ansible with the infra side changes (service account, role, rolebinding, updated vm definition)
Next steps: - [x] automate the playbook run at the event of secret sync - [x] open a PR on ansible-role-forgejo-runner repo with the VM side changes - [x] open a PR on infra/ansible with the infra side changes (service account, role, rolebinding, updated vm definition)
lenkaseg commented 2026-03-30 16:14:05 +00:00
Author
Member
Copy link

VM side changes: https://codeberg.org/fedora/ansible-role-forgejo-runner/pulls/6

VM side changes: https://codeberg.org/fedora/ansible-role-forgejo-runner/pulls/6
lenkaseg commented 2026-03-31 14:02:36 +00:00
Author
Member
Copy link

Ansible infra side changes: infra/ansible#3253

Ansible infra side changes: https://forge.fedoraproject.org/infra/ansible/pulls/3253
lenkaseg 2026-03-31 14:02:52 +00:00
lenkaseg changed title from Explore the option of mount of an OS secret with runner configuration into the VM to Explore the option of mount of an OS secret with runner configuration into the VM (explored and deployed)) 2026-03-31 15:33:23 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
test
docs
No results found.
Labels
Clear labels   
Backlog Status
Needs Review
Work items that have been created or updated but need review, refinement, or approval before they can be considered ready for sprint planning.

  
Backlog Status
Ready
Work items that have been reviewed, refined, and are ready to be committed to a sprint. These items have clear requirements and could be worked on immediately when capacity is available.

  
Chore

Tasks that are necessary but don't directly deliver new value to the end-user. These are internal tasks like updating a library, managing user accounts, or setting up a new service.

  
points
01
A task that is simple, small, and has very little uncertainty. It should be easy enough for any team member to pick up and complete quickly, without a lot of discussion or research.

  
points
02
This is a task that is slightly more complex or larger than a "1." It might involve two or three simple steps, but the outcome is still very clear and there's little uncertainty.

  
points
03
medium-sized task with some noticeable complexity. It might require a bit more thought or collaboration. The steps are well-defined, but it's not a trivial change.

  
points
05
a significant piece of work. It has more complexity and a higher degree of uncertainty than smaller tasks. It might require some research, a design discussion, or collaboration across multiple teams.

  
points
08
complex task with a lot of uncertainty. It's often the largest size that a team will consider for a single sprint. It likely requires a detailed breakdown and multiple team members to complete.

  
points
13
Any task that feels like it's a "13" or more is considered an Epic. These are too large and uncertain to be worked on in a single sprint.

  
Priority
High
This is for work that is critical and needs to be addressed in the next sprint. It's often for urgent bugs, important features, or a major piece of technical debt that is blocking other work.

  
Priority
Low
This is for work that is good to do but not urgent. It's a candidate for being worked on when there's extra time or when it rises in importance. It can stay in the backlog for a longer period.

  
Priority
Medium
This is for important work that should be done soon, but isn't as urgent as a high-priority item. It will likely be considered for the next 1-2 sprints.

  
Sprint Status
Blocked
Indicates the work cannot proceed. There is a dependency or an issue that is preventing the team from completing the task. This is a flag for the Scrum Master to help unblock the team.

  
Sprint Status
Done
Work items that have been completed, reviewed, and integrated. No further work is needed within the current sprint.

  
Sprint Status
In Progress
This label is for work that a developer is actively working on. This is usually what you'll see on the "In Progress" column of your project board.

  
Sprint Status
Review
This indicates that the work is complete, but it is now in a review state. This could mean a peer code review, a QA check, or a review by the Product Owner.

  
Sprint Status
To Do
Work items that are committed to the sprint but haven't been started yet. These items are ready to be picked up by team members.

  
Technical Debt

work that improves the codebase without adding new features. It's work that pays down the "debt" you've accumulated, such as refactoring a messy piece of code, improving test coverage.

  
Work Item
Bug
Issues where the system is not working as intended or designed. Bugs represent deviations from expected behavior.

  
Work Item
Epic

  
Work Item
Spike

  
Work Item
Task

  
Work Item
User Story

  
Backlog Status
Needs Review
Work items that have been created or updated but need review, refinement, or approval before they can be considered ready for sprint planning.

  
Backlog Status
Ready
Work items that have been reviewed, refined, and are ready to be committed to a sprint. These items have clear requirements and can be worked on immediately when capacity is available.

  
chore

Work that needs to be done but doesn't add new functionality or fix bugs. Maintenance tasks that keep the system running smoothly and the codebase clean.

  
documentation

Work focused on creating, updating, or improving documentation for users, developers, or system administrators.

  
points
01
A task that is simple, small, and has very little uncertainty. It should be easy enough for any team member to pick up and complete quickly, without a lot of discussion or research.

  
points
02
This is a task that is slightly more complex or larger than a "1." It might involve two or three simple steps, but the outcome is still very clear and there's little uncertainty.

  
points
03
Medium-sized task with some noticeable complexity. It might require a bit more thought or collaboration. The steps are well-defined, but it's not a trivial change.

  
points
05
A significant piece of work. It has more complexity and a higher degree of uncertainty than smaller tasks. It might require some research, a design discussion, or collaboration across multiple teams.

  
points
08
Complex task with a lot of uncertainty. It's often the largest size that a team will consider for a single sprint. It likely requires a detailed breakdown and multiple team members to complete.

  
points
13
Any task that feels like it's a "13" or more is considered an Epic. These are too large and uncertain to be worked on in a single sprint and should be broken down.

  
Priority
High
This is for work that is critical and needs to be addressed in the next sprint. It's often for urgent bugs, important features, or a major piece of technical debt that is blocking other work.

  
Priority
Low
This is for work that is good to do but not urgent. It's a candidate for being worked on when there's extra time or when it rises in importance. It can stay in the backlog for a longer period.

  
Priority
Medium
This is for important work that should be done soon, but isn't as urgent as a high-priority item. It will likely be considered for the next 1-2 sprints.

  
Sprint Status
Blocked
Work items that cannot progress due to external dependencies, blockers, or issues that need resolution.

  
Sprint Status
Done
Work items that have been completed, reviewed, and integrated. No further work is needed within the current sprint.

  
Sprint Status
In Progress
Work items currently being worked on by a team member. Active development, research, or implementation is happening.

  
Sprint Status
Review
Work items that are complete but need review, testing, or approval before being marked as done.

  
Sprint Status
To Do
Work items that are committed to the sprint but haven't been started yet. These items are ready to be picked up by team members.

  
Technical Debt

Work that addresses shortcuts, workarounds, or suboptimal implementations that were done previously to meet deadlines. These items improve code quality, maintainability, or performance without adding new features.

  
Work Item
Bug
Issues where the system is not working as intended or designed. Bugs represent deviations from expected behavior.

  
Work Item
Epic
Large initiatives that are broken down into multiple smaller work items (user stories, tasks, bugs). Epics represent significant features or projects that span multiple sprints.

  
Work Item
Spike
Time-boxed research, investigation, or proof-of-concept work to reduce uncertainty or explore solutions. Spikes help inform future development decisions.

  
Work Item
Task
Used to break down larger user stories into smaller, manageable implementation pieces. They focus on the specific technical steps needed to deliver the user story's value.

  
Work Item
User Story
User-facing features or functionality that can be framed from a user perspective, describing what someone wants to accomplish and why.

No labels
Backlog Status
Needs Review
Backlog Status
Ready
Chore
points
01
points
02
points
03
points
05
points
08
points
13
Priority
High
Priority
Low
Priority
Medium
Sprint Status
Blocked
Sprint Status
Done
Sprint Status
In Progress
Sprint Status
Review
Sprint Status
To Do
Technical Debt
Work Item
Bug
Work Item
Epic
Work Item
Spike
Work Item
Task
Work Item
User Story
Backlog Status
Needs Review
Backlog Status
Ready
chore
documentation
points
01
points
02
points
03
points
05
points
08
points
13
Priority
High
Priority
Low
Priority
Medium
Sprint Status
Blocked
Sprint Status
Done
Sprint Status
In Progress
Sprint Status
Review
Sprint Status
To Do
Technical Debt
Work Item
Bug
Work Item
Epic
Work Item
Spike
Work Item
Task
Work Item
User Story
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Sprint 17
Assignees
Clear assignees
No assignees
lenkaseg
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
forge/forge#436
No description provided.