Extend 'navigation' test to cover QA:Testcase_window_manager #301

Open
opened 2023-08-10 06:06:53 +00:00 by adamwill · 13 comments
Owner

The initial GNOME 45 updates had a pretty bad bug - alt-tab didn't work correctly. The openQA tests didn't catch this, so we let it through.

This seems like something we should catch, really. I'm thinking of a test which opens several windows and tests that alt-tab and alt-(keyabovetab) work correctly. Also probably testing window switch via overview and a basic test that virtual desktops work correctly would be good.

The initial GNOME 45 updates had a pretty bad bug - [alt-tab didn't work correctly](https://gitlab.gnome.org/GNOME/mutter/-/issues/2950). The openQA tests didn't catch this, so we let it through. This seems like something we should catch, really. I'm thinking of a test which opens several windows and tests that alt-tab and alt-(keyabovetab) work correctly. Also probably testing window switch via overview and a basic test that virtual desktops work correctly would be good.
Author
Owner

we should probably write a wiki test case for this too and add it to the matrices.

we should probably write a wiki test case for this too and add it to the matrices.
Owner

I am taking it.

I am taking it.
Owner

Metadata Update from @lruzicka:

  • Issue assigned to lruzicka
**Metadata Update from @lruzicka**: - Issue assigned to lruzicka
Owner

So, there are some points we might test in this test:

  1. Use Alt-Tab to switch between two running applications.
  2. Use Alt-Tab to cycle through more than two applications.
  3. Test the reverse navigation using Alt-Shift-Tab.
  4. Test Alt-Tab when some applications are minimized.
  5. Test Alt-Tab functionality when an application is in full-screen mode.
So, there are some points we might test in this test: 1. Use Alt-Tab to switch between two running applications. 2. Use Alt-Tab to cycle through more than two applications. 3. Test the reverse navigation using Alt-Shift-Tab. 4. Test Alt-Tab when some applications are minimized. 5. Test Alt-Tab functionality when an application is in full-screen mode.
Owner
https://pagure.io/fedora-qa/os-autoinst-distri-fedora/pull-request/304
Author
Owner

@kparal created a wiki test case for this last year: https://fedoraproject.org/wiki/QA:Testcase_window_manager . The current PR implements only part of this. I'm probably going to merge the PR anyway, but I don't think it fully closes this ticket, I think we need to extend the test further to cover more of the stuff in the test case first.

@kparal created a wiki test case for this last year: https://fedoraproject.org/wiki/QA:Testcase_window_manager . The current PR implements only part of this. I'm probably going to merge the PR anyway, but I don't think it fully closes this ticket, I think we need to extend the test further to cover more of the stuff in the test case first.
Owner

@kparal created a wiki test case for this last year: https://fedoraproject.org/wiki/QA:Testcase_window_manager . The current PR implements only part of this. I'm probably going to merge the PR anyway, but I don't think it fully closes this ticket, I think we need to extend the test further to cover more of the stuff in the test case first.

I can take a look at this once more and see if I can make it more compatible with what the test case says.

> @kparal created a wiki test case for this last year: https://fedoraproject.org/wiki/QA:Testcase_window_manager . The current PR implements only part of this. I'm probably going to merge the PR anyway, but I don't think it fully closes this ticket, I think we need to extend the test further to cover more of the stuff in the test case first. I can take a look at this once more and see if I can make it more compatible with what the test case says.
Owner

Metadata Update from @lruzicka:

  • Custom field story_points adjusted to 15
  • Issue priority set to: Normal (was: Low)
**Metadata Update from @lruzicka**: - Custom field story_points adjusted to 15 - Issue priority set to: Normal (was: Low)
Owner

Metadata Update from @lruzicka:

  • Custom field story_points adjusted to 6 (was: 15)
**Metadata Update from @lruzicka**: - Custom field story_points adjusted to 6 (was: 15)
Owner

Metadata Update from @lruzicka:

  • Custom field story_points adjusted to 8 (was: 6)
**Metadata Update from @lruzicka**: - Custom field story_points adjusted to 8 (was: 6)
Author
Owner

@lruzicka , can you update this? does the current test sufficiently cover the wiki test case?

@lruzicka , can you update this? does the current test sufficiently cover the wiki test case?
Author
Owner

@lruzicka ping again on this?

@lruzicka ping again on this?
Owner

The current suite covers most of the required test case reasonably well for openQA.

It covers common window operations through deterministic window-manager actions: maximize, restore, minimize/hide, close, and tiling. Freeform mouse dragging should not be required, because it would be unreliable and hard to assert safely. Moving/resizing should instead be covered with stable keyboard shortcuts or title-bar/window-menu actions.

Fullscreen handling is covered well, including switching between normal and fullscreen modes and distinguishing fullscreen from maximized state. Application switching is also covered well across multiple applications, including keyboard and mouse-based switching. Workspace handling is covered, including moving windows between workspaces and switching between them. Game playback is covered, including switching between windowed and fullscreen mode. Local video playback is also covered.

What still needs to be added:

  1. Add common window operations for a few more applications, not only the current one.
  2. Add a deterministic resize test, using keyboard shortcuts or title-bar/window-menu actions if available.
  3. Add testing with multiple windows of the same application, for example multiple Terminal, Files, or Text Editor windows.
  4. Add browser video playback, including switching between windowed and fullscreen video.

So the suite is mostly aligned with the required test case, but it still needs these missing pieces before I would call it fully covered.

The current suite covers most of the required test case reasonably well for openQA. It covers common window operations through deterministic window-manager actions: maximize, restore, minimize/hide, close, and tiling. Freeform mouse dragging should not be required, because it would be unreliable and hard to assert safely. Moving/resizing should instead be covered with stable keyboard shortcuts or title-bar/window-menu actions. Fullscreen handling is covered well, including switching between normal and fullscreen modes and distinguishing fullscreen from maximized state. Application switching is also covered well across multiple applications, including keyboard and mouse-based switching. Workspace handling is covered, including moving windows between workspaces and switching between them. Game playback is covered, including switching between windowed and fullscreen mode. Local video playback is also covered. What still needs to be added: 1. Add common window operations for a few more applications, not only the current one. 2. Add a deterministic resize test, using keyboard shortcuts or title-bar/window-menu actions if available. 3. Add testing with multiple windows of the same application, for example multiple Terminal, Files, or Text Editor windows. 4. Add browser video playback, including switching between windowed and fullscreen video. So the suite is mostly aligned with the required test case, but it still needs these missing pieces before I would call it fully covered.
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
quality/os-autoinst-distri-fedora#301
No description provided.