Zabbix monitoring for bugzilla2fedmsg #13376
Labels
No labels
announcement
anubis
authentication
aws
backlog
blocked
bodhi
ci
cloud
communishift
copr
database
day-to-day
dc-move
deprecated
dev
discourse
dns
downloads
easyfix
epel
firmitas
forgejo_migration
Gain
High
Gain
Low
Gain
Medium
gitlab
greenwave
hardware
help wanted
high-trouble
koji
koschei
lists
low-trouble
medium-trouble
mirrorlists
monitoring
Needs investigation
odcs
OpenShift
ops
outage
packager_workflow_blocker
pagure
permissions
Priority
Needs Review
Priority
Next Meeting
Priority
🔥 URGENT 🔥
Priority
Waiting on Assignee
Priority
Waiting on External
Priority
Waiting on Reporter
rabbitmq
release-monitoring
releng
request-for-resources
s390x
security
SMTP
sprint-0
sprint-1
src.fp.o
staging
unfreeze
waiverdb
websites-general
wiki
Backlog Status
Needs Review
Backlog Status
Ready
chore
documentation
points
01
points
02
points
03
points
05
points
08
points
13
Priority
High
Priority
Low
Priority
Medium
Sprint Status
Blocked
Sprint Status
Done
Sprint Status
In Progress
Sprint Status
Review
Sprint Status
To Do
Technical Debt
Work Item
Bug
Work Item
Epic
Work Item
Spike
Work Item
Task
Work Item
User Story
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
infra/tickets#13376
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Description of request
We discussed with @gwmngilfen that it would be nice to have Zabbix monitoring for the
bugzilla2fedmgservice because it still gets often stuck (for example, it is stuck now).The issue is detailed in https://github.com/FrostyX/fedora-review-service/issues/42
I am not sure what is the best way to monitor it, though.
Zabbix monitor for bugzilla2fedmsgto Zabbix monitoring for bugzilla2fedmsgWe are actually getting notifications from zabbix when
bugzilla2fedmsgis stuck. Do I miss something or what kind of monitoring we are looking for here?Hey thanks for the ping @frostyx. So we do have a check for this, we get an alert in Zabbix when it goes stale. It still requires someone to go kick it though. The check is here in Zabbix
So I guess my questions are:
When I was creating this ticket, there was ~8 hours outage in progress. Looking at Zabbix, it probably noticed, I think it was this one: https://zabbix.fedoraproject.org/tr_events.php?triggerid=52427&eventid=3291635
Personally, I don't need that notification because I cannot do anything about it, only report the problem. I usually ping @kevin or @abompard
Yeah, if we set this check too low we will get alerts when there's just not much bugzilla activity.
If we set it too high we will not be notified and people will notice it not processing.
Additionally, they are moving to a new broker at some point here and fedora may be moving away from bugzilla later this year anyhow, so... I'd say perhaps we just close this and limp along for now in hopes of better days ahead?
But open to ideas...
I think what we have is sufficient given the difficulty of improving it and the fact it'll go away entirely at some point. I suggest we close this, if that is ok for you @frostyx ?
Do we have a ticket for the kafka migration? I want to link it from my Fedora Review Service ticket.
Edit: Also, if there is any ticket about the migration out of Bugzilla ...
We had #13103 but... when we asked for access they said they were not ready for anyone external to try and connect and asked us to revisit later. I can try and find out some current status, but I suspect it will not be anytime soon.
Thank you @kevin, feel free to close this ticket then. And please tag me on future tickets that may be relevant for the Fedora Review Service.