infra/tickets

Fork 1

disk auto-decryption problem on vmhost-x86-copr04 #13273

New issue

Closed

opened 2026-04-16 21:17:33 +00:00 by praiskup · 6 comments

praiskup commented

2026-04-16 21:17:33 +00:00

Member

Description of request

I am able to get the console output, but I fail to understand the error.
It seems some disks are successfully decrypted, but some are not.

### Description of request I am able to get the console output, but I fail to understand the error. It seems some disks are successfully decrypted, but some are not.

praiskup commented

2026-04-16 21:22:26 +00:00

Author

Member

I just evacuated the host; it's OK to reboot it anytime you need (in case debugging is needed).

Right now we face some weird FD-leak issue in virtqemud, causing headaches in Copr (and causing other FD leaks on the copr side). After some time, the SSH on the affected host dies :-/ and I need to reboot over idrac/ipmi (so it would be nice if the machine went online automatically).

I just evacuated the host; it's OK to reboot it anytime you need (in case debugging is needed). Right now we face some weird FD-leak issue in virtqemud, causing headaches in Copr (and causing other FD leaks on the copr side). After some time, the SSH on the affected host dies :-/ and I need to reboot over idrac/ipmi (so it would be nice if the machine went online automatically).

kevin commented

2026-04-16 21:31:52 +00:00

Owner

Oh. I thought something was wrong with the hardware and it was rebooting. :(

I kept entering the luks passphrase and it booted then it rebooted, so I assume that was you?

I do need to fix the luks not unlocking on boot for sure.

Oh. I thought something was wrong with the hardware and it was rebooting. :( I kept entering the luks passphrase and it booted then it rebooted, so I assume that was you? I do need to fix the luks not unlocking on boot for sure.

kevin commented

2026-04-16 22:31:51 +00:00

Owner

So, do the other ones all unlock properly?

I can't see whats different about 04 off hand...

So, do the other ones all unlock properly? I can't see whats different about 04 off hand...

praiskup commented

2026-04-17 07:03:18 +00:00

Author

Member

Oh. I thought something was wrong with the hardware and it was rebooting. :(

We were on it at the same time probably, sorry for the inconvenience. :(

I kept entering the luks passphrase and it booted then it rebooted, so I assume that was you?

I bet. Sorry, I was a bit sleepy already. 🤦

I do need to fix the luks not unlocking on boot for sure.
So, do the other ones all unlock properly?
I can't see whats different about 04 off hand...

If you can fix that, it would be awesome.

No there aren't any differences, FWICT. There are some significant problems with libvirt on all of those machines, but I haven't tried to reboot all of them -- I just "started experimenting with 04", and it failed to reboot (after sshd stopped working).

> Oh. I thought something was wrong with the hardware and it was rebooting. :( We were on it at the same time probably, sorry for the inconvenience. :( > I kept entering the luks passphrase and it booted then it rebooted, so I assume that was you? I bet. Sorry, I was a bit sleepy already. 🤦 > I do need to fix the luks not unlocking on boot for sure. > So, do the other ones all unlock properly? > I can't see whats different about 04 off hand... If you can fix that, it would be awesome. No there aren't any differences, FWICT. There are some significant problems with libvirt *on all of those machines*, but I haven't tried to reboot all of them -- I just "started experimenting with 04", and it failed to reboot (after sshd stopped working).

kevin added the

labels

2026-04-17 17:57:22 +00:00

kevin added this to the Sprint ( 2026-04-06 to 2026-04-20 ) project

2026-04-17 17:57:27 +00:00

kevin self-assigned this

2026-04-17 17:57:29 +00:00

kevin commented

2026-04-17 18:14:44 +00:00

Owner

The problem was that there were 2 more encrypted volumes... /dev/md3 and /dev/md4.

We needed to bind them in ansible so it would use tang to unlock them.

I have done that and I think this should be all fixed now.

Let us know if you still see any problems.

The problem was that there were 2 more encrypted volumes... /dev/md3 and /dev/md4. We needed to bind them in ansible so it would use tang to unlock them. I have done that and I think this should be all fixed now. Let us know if you still see any problems.

kevin closed this issue

2026-04-17 18:14:44 +00:00