gnome-software update notification needs a new testing approach #899
Labels
No labels
agile
anacondawebui
arm
blockerfe
Closed As
Duplicate
Closed As
Fixed
Closed As
Invalid
Closed As
Wontfix
Closed As
Worksforme
coreos
criteria
defect
easyfix
enhancement
iot
meeting
meta
onboarding call
proventesters
retrospective
silverblue
sponsor
test cases
test days
wiki
ai-review-please
Backlog Status
Needs Review
Backlog Status
Ready
chore
documentation
points
01
points
02
points
03
points
05
points
08
points
13
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Sprint Status
Blocked
Sprint Status
Done
Sprint Status
In Progress
Sprint Status
Review
Sprint Status
To Do
Technical Debt
Work Item
Bug
Work Item
Epic
Work Item
Spike
Work Item
Task
Work Item
User Story
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
quality/tickets#899
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Testcase https://fedoraproject.org/wiki/QA:Testcase_Gnome_software_update_preparation shows up in Bodhi for gnome-software, we reference it from https://fedoraproject.org/wiki/QA:Testcase_desktop_update_notification and it's also used in OpenQA. But, with the plumbing change from PackageKit to DNF5, the described steps no longer work.
The suggested approach was:
The
update-notification-timestampshows a notification only once in 14 days. But in order to show it, the updates need to be downloaded first. Downloading is now handled by dnf5daemon-server, and only happens after 14 days from theinstall-timestamp, or if any update is critical. The new problem is that overwritinginstall-timestampis no longer sufficient - on each gnome-software start (which initializes dnf5daemon-server), it updates the timestamp with whatever DNF determines to be the latest time it installed any package. Because it has its own database (dnf history list), it's not enough to reset the gsettings value, it will always overwrite it back.Which means the current approach works for a fresh new installed OS (which is why it probably works in OpenQA), but it doesn't work for any system where you installed any RPM in the last <14 days.
We need to figure out a new approach for testing gnome-software update notifications. Milan Crha is available for gnome-software related discussions, but we need to talk to DNF folks as well.
Or we will give up on persuading the tools to do what we want, and simply adjust system time to test this. That's not a great approach to recommend to general audience for release validation (and Bodhi updates validation), though, because resetting the time back to the present can potentially cause issues for certain apps/services.
One potential idea is that we could create a repository with a fake critical update, which could convince the tools to show it due to its importance. Needs to be tested, if that would even work. (But maybe OpenQA does this already?)
A quick note: Kamil summarized it well. It worked earlier, in "the gnome-software with PackageKit", because PacakgeKit has its own "offline updates" file, from which it reads when it was done the last time. The dnf5 plugin uses the dnf5daemon-server, which also sees changes from the command line, which is sometimes nice to have, as you can see what had been changed not only from the gnome-software itself (or through
pkconfor the PackageKit), but, unfortunately, it stays in the way of this functionality.Thinking of it, I'd say the test uncovered a difference in the behavior, aka the test did its job well. The test showed the things can be broken for the people relying on the gnome-software update (notifications) when they use dnf command line from time to time.
I'm afraid the only chance is to add dnf5-plugin specific timestamp into the GSettings for the last offline update ran by the gnome-software itself, which will be also used as a clamp on the discovered past updates.
I'll think about it a bit more. Feel free to comment and/or suggest things.
This is what OpenQA does:
https://openqa.fedoraproject.org/tests/4690934#step/desktop_notifications/28
Sorry, a better source is here. That's most of the relevant test source; you can see all the messing about we do to try and get notifications to appear reliably. Note some of the things are probably obsolete/unneeded; we don't touch this unless/until it stops working.
One thing that isn't there is the thing we do to make sure an update is actually available. That's
prepare_test_packageswhich is called on line 55. You can see the implementation here. Notably it does run dnf commands.This is interesting and it just means that we still don't fully understand the current behavior.
Anyway, our current assumption is that if the user performs some RPM operations (through DNF or gnome-software) every <14 days, they will never get a "new updates available" notification (unless there's a critical update in the set, which might trigger a different code path). It needs to be tested, but would pose a bigger (and general) problem than this ticket. If that's the case, we should file it separately (might even propose it as an F45 blocker), and Milan could think about a different implementation (that would only track system updates, possibly done just through gnome-software, for the purpose of future updates notifications).
Right, right, I remember the acpica-tools package, I used it for some testing in the past too :)
I still cannot think of any better way than a new timestamp key (there are never enough timestamp keys, right?), I'll look on it tomorrow or so. I'll let you know with a test package once I have any.
That's what I meant as the test worked, it uncovered a (hidden?) problem. I would not open a new ticket for it, it might be just duplicate of this one, as when i fix this, it'll fix that as well.
Could you give a try to this test build, please?
https://koji.fedoraproject.org/koji/taskinfo?taskID=145218699
It contains a patch which seems to work better here, but I often miss some detail and it works slightly differently on other machines than on mine, thus better to test it before that. If you'd want to be super-sure, then also reset
packagekit-historical-updates-timestampkey, which the dnf5 plugin already uses, I extended its use on other places and use it as an upper date/time clamp of a package install, thus any later package changes are not interfering.Other plugins can still interfere, like the fwupd, but that would be a question on the gnome-software upstream and the gs-update-monitor, how to deal with it, because it's independent of the dnf5 plugin. Of course, such change would void a change on the dnf5 plugin side, but I guess its fine for now.
@mcrha wrote in #899 (comment):
This seems to be working great for me. With this approach:
I have no problem making gnome-software pop up "Updates Ready" notification any time I tried.
Will you make this change go into the main branch?
Sure thing, I've been only waiting for a (second) confirmation, aka also have it tested from someone else than me.
Great. I've updated our test case with new instructions.
@adamwill @lruzicka Can we also update OpenQA and make the additional timestamp key reset as well?
Once there's a new official GS build, this should be hopefully done.
Expected versions with the fix are
gnome-software-50.1-5.fc45andgnome-software-50.1-5.fc44(currently building). I won't spam here once the builds are done, it'll be reflected in bodhi.//Update: F44 Bodhi update
@kparal wrote in #899 (comment):
sure, adding one more to the pile can't hurt I guess. You can send a PR if you like? Just add it to all the others. :D
@adamwill wrote in #899 (comment):
Not really...? It took me 20 minutes just to clone the repo! Oh well, here it is: quality/os-autoinst-distri-fedora#548
Closing this ticket as done.
Hum. I think this is causing the updates to actually get installed? See https://openqa.fedoraproject.org/tests/4722563#step/desktop_notifications/124 . We don't really want that...
The
packagekit-historical-updates-timestampis used to determine how old package/dnf changes are considered as "recent updates". Anything beyond that date is considered a recent change and is announced by the referenced notification. Did it notify about the acpi-tool downgrade/install?I don't that believe really installs updates, because it need a reboot cycle to install updates (and that didn't happen in that test case). But I have a suspicion what happens there. I also saw some unexpected "Software updates installed" notifications in certain cases during my testing. I believe that's a notification about the previous RPM transaction (might not even be initiated by gnome-software), which in this case might be the
acpi-tooldowngrade, as Milan suggests. Resetting the additional timestamp perhaps caused a difference in behavior for these "previous transaction complete" notifications.Correct, it can be the other
dnfcommands interfering. When you rungnome-software --mode=updatedyou can check what changes it found.To add to the confusion, the dnf5daemon does not return the install/change time, the only time information is the build time of the (installed) packages, thus it's currently used as the install time. It mostly works, but can misbehave. I opened a bug against the dnf5daemon to provide the transaction time in the response.
The
acpi-toolpackage should be fine, it has an ancient build time, due to that I suppose some other operation triggered the notification.Well that's annoying. Is there some way to prevent this happening?
It depends. Can you find out what it sees as the recent updates, please? It'll help to narrow the problem.
The
packagekit-historical-updates-timestampis supposed to help. Maybe set it only a minute in the past, not two weeks, aka just before the update? Err, but even with it, as the transaction time is not known, the updated package will be missed... That's... not ideal... Trusting the dnf5daemon server returning only transaction since that time, I can ignore the buildtime of the package and use thepackagekit-historical-updates-timestampas the packages install time, until/if the dnf5 folks will add the real transaction time to the response. I know they return transactions since that time, thus it should be fine. Opinions?By the way, this happens when the tools are accurate ;)
Actually, my PR that resets
packagekit-historical-updates-timestampis not yet deployed to OpenQA production, as far as I can say. So this change we're seeing in OpenQA seems to be related to GS code changes only, not the adjustments of our tests. I'm trying to dig more into it.//Edit: Ok, I officially don't understand it. The executed script doesn't contain this line:
but the video from the test shows it as being executed. I have no idea what's happening here.
I've done a lot of local testing with Milan's help. This DNF bug complicates the testing a lot. However, these claims currently seem to be true:
packagekit-historical-updates-timestamp, andinstall-timestamp<packagekit-historical-updates-timestamp.packagekit-historical-updates-timestampis 0, it suppresses any "Updates installed" notification, because it's considered as a clean OS installation state.I'm not sure why we're seeing "Updates installed" notification in OpenQA, because we reset the timestamps to the exactly same value. That timestamp can also be adjusted during GS startup, but we can't look more into it without adding debug info to the OpenQA test. I tried but I can't reproduce it locally.
However, the second point could help us avoid that problem. Simply set
packagekit-historical-updates-timestamp = 0, and it should prevent these notifications (but still allow "Updates available" notifications, at least in my local testing).@adamwill Can you test if quality/os-autoinst-distri-fedora#551 fixes the problem?
@kparal wrote in #899 (comment):
I reverted your commit directly on openQA prod (which I meant as a temporary test but then forgot about). I did that at
Date: Wed May 13 06:46:15 2026 +0000. A test run after your PR merged but before that revert would have run your step. A test run after that point would not.Note openQA - at least openQA the way we run it, we're still using the very old-fashioned way - always shows the current state of the test code when you click on those links in the web UI. It does not display the code as it existed at the time the test was executed.
There's a newer-style way to do things in openQA where I think it knows what state of the test repo was used to execute each test and can display the right source, but...we're not using that.
quality/os-autoinst-distri-fedora#551 seems to have fixed our tests. Closing again.
For what it's worth, the dnf5 folks have some good changes, which I'll incorporate into the gnome-software sooner or later. I guess you'll notice soon if I break anything with it ;)