riscv/planning
11
1
Fork 0
Code Issues 28 Pull requests Projects Releases Packages Wiki Activity Actions

XFS + LVM for omni/server images result in root filesystem mis-mounting errors #33

New issue
Open
opened 2026-09-23 20:17:56 +00:00 by kashyapc · 5 comments
kashyapc commented 2026-09-23 20:17:56 +00:00
Owner
Copy link

Description

We tried testing by reverting to to XFS + LVM[1] for images in ki, but the boot fails with:

[...]

[  549.764298] dracut-initqueue[400]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks:
[  549.833989] dracut-initqueue[400]: Warning: /var/lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-uuid\x2fa43ddfbf-bbc6-480c-a863-9a5846a3fd57.sh: "if ! grep -qs After=remote-fs-pre.target /run/systemd/generator/systemd-cryptsetup@*.service; then
[  549.840494] dracut-initqueue[400]:     [ -e "/dev/disk/by-uuid/a43ddfbf-bbc6-480c-a863-9a5846a3fd57" ]
[  549.845953] dracut-initqueue[400]: fi"
[     *] (1 of 2) Job dev-disk-by\x2duuid-a43ddfbf\x2dbbc6\x2d480c\x2da863\x2d9a5846a3fd57.device/start running (8min 37s / no limit)
[  551.210839] dracut-initqueue[400]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks:
[  551.277299] dracut-initqueue[400]: Warning: /var/lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-uuid\x2fa43ddfbf-bbc6-480c-a863-9a5846a3fd57.sh: "if ! grep -qs After=remote-fs-pre.target /run/systemd/generator/systemd-cryptsetup@*.service; then
[  551.284383] dracut-initqueue[400]:     [ -e "/dev/disk/by-uuid/a43ddfbf-bbc6-480c-a863-9a5846a3fd57" ]
[  551.289562] dracut-initqueue[400]: fi"
[  *** ] (2 of 2) Job dracut-initqueue.service/start running (8min 38s / no limit)
[  552.626758] dracut-initqueue[400]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks:
[  552.692899] dracut-initqueue[400]: Warning: /var/lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-uuid\x2fa43ddfbf-bbc6-480c-a863-9a5846a3fd57.sh: "if ! grep -qs After=remote-fs-pre.target /run/systemd/generator/systemd-cryptsetup@*.service; then
[  552.698467] dracut-initqueue[400]:     [ -e "/dev/disk/by-uuid/a43ddfbf-bbc6-480c-a863-9a5846a3fd57" ]
[  552.702597] dracut-initqueue[400]: fi"
[  552.755761] dracut-initqueue[400]: Warning: dracut-initqueue: starting timeout scripts
         Starting dracut-emergency.service - Dracut Emergency Shell...
Warning: /dev/disk/by-uuid/a43ddfbf-bbc6-480c-a863-9a5846a3fd57 does not exist

Generating "/run/initramfs/rdsosreport.txt"


Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
after mounting them and attach it to a bug report.


Press Enter for system maintenance
(or press Control-D to continue):
dracut:/root#

[1] http://fedora.riscv.rocks:3000/davidlt/fedora-kiwi-descriptions/commits/branch/f45-riscv64

Reproducer

  1. Download the Fedora-Server-Host-Omni-45-20260923.0.riscv64.raw image from: https://riscv-koji.fedoraproject.org/koji/buildinfo?buildID=133351

  2. Import it into libvirt:

$ virt-install \
    --import \
    --name f45-omni \
    --osinfo fedora-rawhide \
    --arch riscv64 \
    --cpu mode=maximum \
    --vcpus 4 \
    --ram 4096 \
    --boot uefi \
    --disk path=/var/lib/libvirt/images/Fedora-Server-Host-Omni-45-20260923.0.riscv64.raw \
    --network default \
    --tpm none \
    --graphics none \
    --controller scsi,model=virtio-scsi \

Other info

The output of blkid and /proc/cmdline:

dracut:/root# blkid
/dev/sr0: BLOCK_SIZE="2048" UUID="2026-09-23-15-19-47-00" LABEL="cidata" TYPE="iso9660"
/dev/mapper/systemVG: LABEL="fedora" UUID="a43ddfbf-bbc6-480c-a863-9a5846a3fd57" BLOCK_SIZE="512" TYPE="xfs"
/dev/vda2: LABEL="BOOT" UUID="ba5c566d-cd86-4238-9ca0-6a220a823d53" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="p.lxboot" PARTUUID="918c4426-77e5-4807-b548-9a5c3f08c5bc"
/dev/vda3: UUID="OEqtHA-ypmA-kzYy-bccS-gEtW-AxOO-SpGj06" TYPE="LVM2_member" PARTLABEL="p.lxlvm" PARTUUID="e9a4aac6-d31e-40a2-8baf-baa4779ab066"
/dev/vda1: SEC_TYPE="msdos" LABEL_FATBOOT="EFI" LABEL="EFI" UUID="CC24-87A3" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="p.UEFI" PARTUUID="cd670361-3c52-49bb-a1a7-32a94eacd382"
dracut:/root#
dracut:/root# cat /proc/cmdline
BOOT_IMAGE=(hd0,gpt2)/vmlinuz-7.2.7-300.0.riscv64.omni.fc45.riscv64 earlycon root=UUID=a43ddfbf-bbc6-480c-a863-9a5846a3fd57 clk_ignore_unused
dracut:/root#
## Description We tried testing by reverting to to XFS + LVM[1] for images in ki, but the boot fails with: ``` [...] [ 549.764298] dracut-initqueue[400]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks: [ 549.833989] dracut-initqueue[400]: Warning: /var/lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-uuid\x2fa43ddfbf-bbc6-480c-a863-9a5846a3fd57.sh: "if ! grep -qs After=remote-fs-pre.target /run/systemd/generator/systemd-cryptsetup@*.service; then [ 549.840494] dracut-initqueue[400]: [ -e "/dev/disk/by-uuid/a43ddfbf-bbc6-480c-a863-9a5846a3fd57" ] [ 549.845953] dracut-initqueue[400]: fi" [ *] (1 of 2) Job dev-disk-by\x2duuid-a43ddfbf\x2dbbc6\x2d480c\x2da863\x2d9a5846a3fd57.device/start running (8min 37s / no limit) [ 551.210839] dracut-initqueue[400]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks: [ 551.277299] dracut-initqueue[400]: Warning: /var/lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-uuid\x2fa43ddfbf-bbc6-480c-a863-9a5846a3fd57.sh: "if ! grep -qs After=remote-fs-pre.target /run/systemd/generator/systemd-cryptsetup@*.service; then [ 551.284383] dracut-initqueue[400]: [ -e "/dev/disk/by-uuid/a43ddfbf-bbc6-480c-a863-9a5846a3fd57" ] [ 551.289562] dracut-initqueue[400]: fi" [ *** ] (2 of 2) Job dracut-initqueue.service/start running (8min 38s / no limit) [ 552.626758] dracut-initqueue[400]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks: [ 552.692899] dracut-initqueue[400]: Warning: /var/lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fdisk\x2fby-uuid\x2fa43ddfbf-bbc6-480c-a863-9a5846a3fd57.sh: "if ! grep -qs After=remote-fs-pre.target /run/systemd/generator/systemd-cryptsetup@*.service; then [ 552.698467] dracut-initqueue[400]: [ -e "/dev/disk/by-uuid/a43ddfbf-bbc6-480c-a863-9a5846a3fd57" ] [ 552.702597] dracut-initqueue[400]: fi" [ 552.755761] dracut-initqueue[400]: Warning: dracut-initqueue: starting timeout scripts Starting dracut-emergency.service - Dracut Emergency Shell... Warning: /dev/disk/by-uuid/a43ddfbf-bbc6-480c-a863-9a5846a3fd57 does not exist Generating "/run/initramfs/rdsosreport.txt" Entering emergency mode. Exit the shell to continue. Type "journalctl" to view system logs. You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot after mounting them and attach it to a bug report. Press Enter for system maintenance (or press Control-D to continue): dracut:/root# ``` [1] http://fedora.riscv.rocks:3000/davidlt/fedora-kiwi-descriptions/commits/branch/f45-riscv64 ## Reproducer 1. Download the `Fedora-Server-Host-Omni-45-20260923.0.riscv64.raw` image from: https://riscv-koji.fedoraproject.org/koji/buildinfo?buildID=133351 2. Import it into libvirt: ``` $ virt-install \ --import \ --name f45-omni \ --osinfo fedora-rawhide \ --arch riscv64 \ --cpu mode=maximum \ --vcpus 4 \ --ram 4096 \ --boot uefi \ --disk path=/var/lib/libvirt/images/Fedora-Server-Host-Omni-45-20260923.0.riscv64.raw \ --network default \ --tpm none \ --graphics none \ --controller scsi,model=virtio-scsi \ ``` ## Other info The output of `blkid` and `/proc/cmdline`: ``` dracut:/root# blkid /dev/sr0: BLOCK_SIZE="2048" UUID="2026-09-23-15-19-47-00" LABEL="cidata" TYPE="iso9660" /dev/mapper/systemVG: LABEL="fedora" UUID="a43ddfbf-bbc6-480c-a863-9a5846a3fd57" BLOCK_SIZE="512" TYPE="xfs" /dev/vda2: LABEL="BOOT" UUID="ba5c566d-cd86-4238-9ca0-6a220a823d53" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="p.lxboot" PARTUUID="918c4426-77e5-4807-b548-9a5c3f08c5bc" /dev/vda3: UUID="OEqtHA-ypmA-kzYy-bccS-gEtW-AxOO-SpGj06" TYPE="LVM2_member" PARTLABEL="p.lxlvm" PARTUUID="e9a4aac6-d31e-40a2-8baf-baa4779ab066" /dev/vda1: SEC_TYPE="msdos" LABEL_FATBOOT="EFI" LABEL="EFI" UUID="CC24-87A3" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="p.UEFI" PARTUUID="cd670361-3c52-49bb-a1a7-32a94eacd382" dracut:/root# dracut:/root# cat /proc/cmdline BOOT_IMAGE=(hd0,gpt2)/vmlinuz-7.2.7-300.0.riscv64.omni.fc45.riscv64 earlycon root=UUID=a43ddfbf-bbc6-480c-a863-9a5846a3fd57 clk_ignore_unused dracut:/root# ```
kashyapc commented 2026-09-23 20:22:02 +00:00
Author
Owner
Copy link

@jmontleon did some digging and points out the UUID is getting wrongly truncated:

dracut:/root# cat /sys/block/dm-0/dm/name /sys/block/dm-0/dm/uuid
systemVG                                                         
LVM-h3Ul

After further digging, he confirms disabling the Zbb CPU feature lets it boot:

  <cpu mode='custom' match='exact' check='none'>
    <model fallback='forbid'>rv64</model>
    <feature policy='disable' name='zbb'/>
  </cpu>

Jason, please add/amend as you see fit.

@jmontleon did some digging and points out the UUID is getting wrongly truncated: ``` dracut:/root# cat /sys/block/dm-0/dm/name /sys/block/dm-0/dm/uuid systemVG LVM-h3Ul ``` After further digging, he confirms disabling the `Zbb` CPU feature lets it boot: ``` <cpu mode='custom' match='exact' check='none'> <model fallback='forbid'>rv64</model> <feature policy='disable' name='zbb'/> </cpu> ``` Jason, please add/amend as you see fit.
kashyapc commented 2026-09-23 20:39:25 +00:00
Author
Owner
Copy link

I can confirm the above workaround works.

There's a related upstream in-progress kernel patch that Jason found (and reported this issue there):

https://lkml.org/lkml/2026/9/23/2879 -- riscv: lib: Fix address overflow due to large count values in strnlen ZBB path

I can confirm the above workaround works. There's a related upstream in-progress kernel patch that Jason found (and reported this issue there): https://lkml.org/lkml/2026/9/23/2879 -- riscv: lib: Fix address overflow due to large count values in strnlen ZBB path
jmontleon commented 2026-09-23 21:17:56 +00:00
Owner
Copy link

Yes, this looks correct. launching it with --cpu model=rv64,-zbb also seems to work. I found two mailing list threads about this as well.

https://lkml.org/lkml/2026/8/19/546
https://lists.infradead.org/pipermail/linux-riscv/2026-September/098614.html

I responded to the first today.
https://lkml.org/lkml/2026/9/23/2879

We can always test a patch if an update is proposed, but for now it might be wise to revert to btrfs and maybe xfs this again once the ZBB issue is resolved.

Yes, this looks correct. launching it with `--cpu model=rv64,-zbb` also seems to work. I found two mailing list threads about this as well. https://lkml.org/lkml/2026/8/19/546 https://lists.infradead.org/pipermail/linux-riscv/2026-September/098614.html I responded to the first today. https://lkml.org/lkml/2026/9/23/2879 We can always test a patch if an update is proposed, but for now it might be wise to revert to btrfs and maybe xfs this again once the ZBB issue is resolved.
kashyapc commented 2026-09-24 08:53:07 +00:00
Author
Owner
Copy link

@jmontleon wrote in #33 (comment):

Yes, this looks correct. launching it with --cpu model=rv64,-zbb also seems to work. I found two mailing list threads about this as well.

https://lkml.org/lkml/2026/8/19/546 https://lists.infradead.org/pipermail/linux-riscv/2026-September/098614.html

I responded to the first today. https://lkml.org/lkml/2026/9/23/2879

Thank you!

We can always test a patch if an update is proposed, but for now it might be wise to revert to btrfs and maybe xfs this again once the ZBB issue is resolved.

Yes, already reverted and built a Server-Host-Omni image: https://riscv-koji.fedoraproject.org/koji/buildinfo?buildID=133357

@jmontleon wrote in https://forge.fedoraproject.org/riscv/planning/issues/33#issuecomment-1359650: > Yes, this looks correct. launching it with `--cpu model=rv64,-zbb` also seems to work. I found two mailing list threads about this as well. > > https://lkml.org/lkml/2026/8/19/546 https://lists.infradead.org/pipermail/linux-riscv/2026-September/098614.html > > I responded to the first today. https://lkml.org/lkml/2026/9/23/2879 Thank you! > We can always test a patch if an update is proposed, but for now it might be wise to revert to btrfs and maybe xfs this again once the ZBB issue is resolved. Yes, already reverted and built a Server-Host-Omni image: https://riscv-koji.fedoraproject.org/koji/buildinfo?buildID=133357
jmontleon commented 2026-09-25 13:16:43 +00:00
Owner
Copy link

I tested the updated v2 of the first patch as well as the second and they both allowed the image to boot normally with ZBB enabled. I provided feedback to that effect on the list. I can't speak to the merits of either beyond that.
https://gitlab.com/jmontleon/kernel-ark/-/tree/fedora-7.2.7-riscv-omni.0.zbb-fix-1
https://gitlab.com/jmontleon/kernel-ark/-/tree/fedora-7.2.7-riscv-omni.0.zbb-fix-2

I tested the updated v2 of the first patch as well as the second and they both allowed the image to boot normally with ZBB enabled. I provided feedback to that effect on the list. I can't speak to the merits of either beyond that. https://gitlab.com/jmontleon/kernel-ark/-/tree/fedora-7.2.7-riscv-omni.0.zbb-fix-1 https://gitlab.com/jmontleon/kernel-ark/-/tree/fedora-7.2.7-riscv-omni.0.zbb-fix-2
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
riscv/planning#33
No description provided.