Create advisory installation validation test cases for VirtualBox

adamwill commented

2012-02-27 21:59:28 +00:00

Owner

Although kernel team (and the rest of the development team) specifically don't want to commit to supporting VirtualBox, the practical fact is that a lot of people like to run Fedora in VirtualBox, and we don't at present have formal testing in place to find out if it's actually working during release validation.

In practice, we usually find out about any bugs via people trying it and then posting their results to test@ or the forums, but this is pretty rough and ready and it's easy to lose the information. If we add formal VBox testing to the release validation process we'll probably find out about VBox fails sooner and do a better job of tracking them for possible fixes or at least documentation at release time.

So, ideally, we should write test cases for deploying Fedora as a VBox guest (and possibly using it as a VBox host), and add these to the installation validation matrix as 'advisory' tests (tests not associated with Alpha, Beta or Final release phases, and without a corresponding release criterion).

Dan Mashal, who I hope I've added to CC, is interested in this topic and may choose to be an awesome rock star and contribute the test cases :)

Although kernel team (and the rest of the development team) specifically don't want to commit to supporting VirtualBox, the practical fact is that a lot of people like to run Fedora in VirtualBox, and we don't at present have formal testing in place to find out if it's actually working during release validation. In practice, we usually find out about any bugs via people trying it and then posting their results to test@ or the forums, but this is pretty rough and ready and it's easy to lose the information. If we add formal VBox testing to the release validation process we'll probably find out about VBox fails sooner and do a better job of tracking them for possible fixes or at least documentation at release time. So, ideally, we should write test cases for deploying Fedora as a VBox guest (and possibly using it as a VBox host), and add these to the installation validation matrix as 'advisory' tests (tests not associated with Alpha, Beta or Final release phases, and without a corresponding release criterion). Dan Mashal, who I hope I've added to CC, is interested in this topic and may choose to be an awesome rock star and contribute the test cases :)

technologov commented

2012-03-04 17:44:48 +00:00

Replying to [comment:1 robatino]:

I see one major problem with your wish: no one to fullfill it.

VirtualBox team will say they don't support unreleased OS.
Fedora team will say, that vbox is not part of upstream kernel and not part of Fedora. End of story.

The only person, that can attempt to fullfill it is you.

-Technologov

VirtualBox community member.

Replying to [comment:1 robatino]: I see one major problem with your wish: no one to fullfill it. VirtualBox team will say they don't support unreleased OS. Fedora team will say, that vbox is not part of upstream kernel and not part of Fedora. End of story. The only person, that can attempt to fullfill it is *you*. -Technologov VirtualBox community member.

vicodan commented

2012-03-04 17:46:10 +00:00

So first simple test case is to try to get VirtualBox guest additions to install on a Fedora guest (host OS should not matter):BR
BR
BR
Steps to test:BR
BR

Install VirtualBox on any host OSBR
Install Fedora (preferably using the DVD ISO) choosing all desktop managers and dev libsBR
In Vbox console window for Fedora VM click devices -> "Install guest additions"BR
Open terminal, become rootBR
cd /media/vbox directory or mount -t iso9660 /dev/cdrom /mediaBR
run ./VBoxLinuxAdditions.runBR
Output should look like the following:BR
BR
http://fpaste.org/J7Tn/ BR
BR
reboot machine, test copy/paste from guest to host and vice versaBR
Test "3d effects/eye candy"BR
Resize console window and check that guest desktop resolution automatically resizes.BR
Test shared folders between host and guest.BR

So first simple test case is to try to get VirtualBox guest additions to install on a Fedora guest (host OS should not matter):[[BR]] [[BR]] [[BR]] Steps to test:[[BR]] [[BR]] 1) Install VirtualBox on any host OS[[BR]] 2) Install Fedora (preferably using the DVD ISO) choosing all desktop managers and dev libs[[BR]] 3) In Vbox console window for Fedora VM click devices -> "Install guest additions"[[BR]] 4) Open terminal, become root[[BR]] 5) cd /media/vbox directory or mount -t iso9660 /dev/cdrom /media[[BR]] 6) run ./VBoxLinuxAdditions.run[[BR]] 7) Output should look like the following:[[BR]] [[BR]] http://fpaste.org/J7Tn/ [[BR]] [[BR]] 8) reboot machine, test copy/paste from guest to host and vice versa[[BR]] 9) Test "3d effects/eye candy"[[BR]] 10) Resize console window and check that guest desktop resolution automatically resizes.[[BR]] 11) Test shared folders between host and guest.[[BR]]

vicodan commented

2012-03-04 18:54:33 +00:00

vbox additions installation log
vboxadd-install.log

vbox additions installation log [vboxadd-install.log](/fedora-qa/issue/raw/files/9a522b8c2630f8b5851ab7dee58807e4c87251174a6022d154543306337a4c8a-vboxadd-install.log)

vicodan commented

2012-03-04 18:54:43 +00:00

Example of VirtualBox guest additions failing on Fedora 17 alpha:BR
BR
BR
[root@Fedora17 media]# ./VBoxLinuxAdditions.run BR
Verifying archive integrity... All good.BR
Uncompressing VirtualBox 4.1.8 Guest Additions for Linux.........BR
VirtualBox Guest Additions installerBR
Removing installed version 4.1.8 of VirtualBox Guest Additions...BR
Removing existing VirtualBox DKMS kernel modules [ OK ]BR
Removing existing VirtualBox non-DKMS kernel modules [ OK ]BR
Building the VirtualBox Guest Additions kernel modulesBR
Building the main Guest Additions module [ OK ]BR
Building the shared folder support module [ OK ]BR
Building the OpenGL support module [FAILED]BR
(Look at /var/log/vboxadd-install.log to find out what went wrong)BR
Doing non-kernel setup of the Guest Additions [ OK ]BR
Installing the Window System driversBR
Warning: unsupported pre-release version of X.Org Server installed. Not
installing the X.Org drivers.BR
Installing modules [ OK ]BR
Installing graphics libraries and desktop services componen[ OK ]BR
[root@Fedora17 media]# BR

Example of VirtualBox guest additions failing on Fedora 17 alpha:[[BR]] [[BR]] [[BR]] [root@Fedora17 media]# ./VBoxLinuxAdditions.run [[BR]] Verifying archive integrity... All good.[[BR]] Uncompressing VirtualBox 4.1.8 Guest Additions for Linux.........[[BR]] VirtualBox Guest Additions installer[[BR]] Removing installed version 4.1.8 of VirtualBox Guest Additions...[[BR]] Removing existing VirtualBox DKMS kernel modules [ OK ][[BR]] Removing existing VirtualBox non-DKMS kernel modules [ OK ][[BR]] Building the VirtualBox Guest Additions kernel modules[[BR]] Building the main Guest Additions module [ OK ][[BR]] Building the shared folder support module [ OK ][[BR]] Building the OpenGL support module [FAILED][[BR]] (Look at /var/log/vboxadd-install.log to find out what went wrong)[[BR]] Doing non-kernel setup of the Guest Additions [ OK ][[BR]] Installing the Window System drivers[[BR]] Warning: unsupported pre-release version of X.Org Server installed. Not installing the X.Org drivers.[[BR]] Installing modules [ OK ][[BR]] Installing graphics libraries and desktop services componen[ OK ][[BR]] [root@Fedora17 media]# [[BR]]

technologov commented

2012-03-04 19:13:03 +00:00

Current Red Hat policy disallows to support out-of-tree kernel modules.

One possible (long-term) solution is: port VirtualBox to KVM, resulting to VirtualBox/KVM (just like Qemu/KVM).

Needless to say, that I expect both Oracle and Red Hat to refuse this idea (for political reasons), but here it is:

https://forums.virtualbox.org/viewtopic.php?f=9&t=47825

-Technologov

Current Red Hat policy disallows to support out-of-tree kernel modules. One possible (long-term) solution is: port VirtualBox to KVM, resulting to VirtualBox/KVM (just like Qemu/KVM). Needless to say, that I expect both Oracle and Red Hat to refuse this idea (for political reasons), but here it is: https://forums.virtualbox.org/viewtopic.php?f=9&t=47825 -Technologov

adamwill commented

2012-03-05 17:50:16 +00:00

Author

Owner

technologv: we're well aware of the limitations to how much 'support' Fedora and VBox can possibly have for each other given the policies you mention, but that's really okay. What we want from a QA perspective is to know with certainty whether VBox is working when we do a Fedora (pre-)release, to know what issues are preventing it working if it's not working, to document those issues together with any potential workarounds, and to file them with the appropriate project if they are issues that are actually amenable to being fixed.

The existence of test cases does not imply that we require or even expect VBox to be working. That's not the case. Rather, we just want to be systematic and consistent about knowing whether or not it's working, and if not, why not.

technologv: we're well aware of the limitations to how much 'support' Fedora and VBox can possibly have for each other given the policies you mention, but that's really okay. What we want from a QA perspective is to know with certainty whether VBox is working when we do a Fedora (pre-)release, to know what issues are preventing it working if it's not working, to document those issues together with any potential workarounds, and to file them with the appropriate project if they are issues that are actually amenable to being fixed. The existence of test cases does not imply that we require or even expect VBox to be working. That's not the case. Rather, we just want to be systematic and consistent about knowing whether or not it's working, and if not, why not.

adamwill commented

2012-03-05 17:52:57 +00:00

Author

Owner

vicodan: comment #3 looks like a good start for a test case. However, can you draft up your test cases as wiki pages, using the test case wiki template, as recommended in https://fedoraproject.org/wiki/QA:SOP_test_case_creation ? You can look at, say, https://fedoraproject.org/w/index.php?title=QA:Testcase_Install_to_Current_KVM&action=edit as a general example of the format used for a test case in the Wiki.

robatino commented

2012-03-05 18:37:45 +00:00

Not sure how important this is, but there's manual installation of the guest additions, and also automatic installation as documented at http://www.virtualbox.org/manual/ch04.html#idp11277648 . The latter appears to only work under F15 and F16, but not F17 or Rawhide. Manual installation seems to work for all of them (except that for F17 and Rawhide, getting the OpenGL support module to build requires the workaround at http://freeoracletrainings.com/index.php/blogs/item/15-install-guest-additions-building-the-opengl-support-module-failed . (There is a ticket for this issue at https://www.virtualbox.org/ticket/10293 .)

vicodan commented

2012-03-05 20:12:18 +00:00

robatino: Thank you for your reply but this actually does not fix the problem and breaks X windows.

If you have any other possible solutions they would be appreciated. The main problem I'm seeing is when compiling from source there are sections in their makefiles that check for the kernel version and build appropriately. Some modules were updated for kernel 3.2 but not all and on Fedora 17 we are using 3.3 and a prerelease version of X.org.

robatino: Thank you for your reply but this actually does not fix the problem and breaks X windows. If you have any other possible solutions they would be appreciated. The main problem I'm seeing is when compiling from source there are sections in their makefiles that check for the kernel version and build appropriately. Some modules were updated for kernel 3.2 but not all and on Fedora 17 we are using 3.3 and a prerelease version of X.org.

robatino commented

2012-03-05 20:55:13 +00:00

Replying to [comment:10 vicodan]:

robatino: Thank you for your reply but this actually does not fix the problem and breaks X windows.

Assuming you're talking about the OpenGL error, can you add this information to https://www.virtualbox.org/ticket/10293 ? (Superficially at least it seems to work for me - after setting $MAKE, the guest additions appear to build without error and X seems to work normally - or at least as normally as it did before, given the "Oh no" screens that frequently appear either way.)

Replying to [comment:10 vicodan]: > robatino: Thank you for your reply but this actually does not fix the problem and breaks X windows. Assuming you're talking about the OpenGL error, can you add this information to https://www.virtualbox.org/ticket/10293 ? (Superficially at least it seems to work for me - after setting $MAKE, the guest additions appear to build without error and X seems to work normally - or at least as normally as it did before, given the "Oh no" screens that frequently appear either way.)

adamwill commented

2018-02-06 22:59:05 +00:00

Author

Owner

Metadata Update from @adamwill:

Issue close_status updated to: None
Issue priority set to: wishlist (was: normal)
Issue set to the milestone: Undetermined Future

**Metadata Update from @adamwill**: - Issue close_status updated to: None - Issue priority set to: wishlist (was: normal) - Issue set to the milestone: Undetermined Future

adamwill commented

2024-02-22 01:17:29 +00:00

Author

Owner

this...would still be useful, honestly.

adamwill commented

2024-02-22 01:17:30 +00:00

Author

Owner

Metadata Update from @adamwill:

Issue set to the milestone: None (was: Undetermined Future)

**Metadata Update from @adamwill**: - Issue set to the milestone: None (was: Undetermined Future)

adamwill commented

2024-02-22 01:17:36 +00:00

Author

Owner

Metadata Update from @adamwill:

Issue set to the milestone: Undefined Future

**Metadata Update from @adamwill**: - Issue set to the milestone: Undefined Future

adamwill commented

2024-02-22 01:49:40 +00:00

Author

Owner

OK, there's my good deed for the day. Added to the matrix, too.

OK, [there's my good deed for the day](https://fedoraproject.org/wiki/QA:Testcase_Install_to_VirtualBox). Added to [the matrix, too](https://fedoraproject.org/w/index.php?title=Template:Installation_test_matrix&diff=701680&oldid=701657).

adamwill commented

2024-02-22 01:49:41 +00:00

Author

Owner

Metadata Update from @adamwill: