Retired packages should close rawhide bugzilla as WONTFIX or EOL

churchyard commented

2019-06-24 22:04:27 +00:00

Member

Describe the issue

When a package gets retired, all the bugzilla filed for rawhide should be closed as WONTFIX or EOL.

When do you need this?

Any time.

When is this no longer needed or useful?

Once we don't retire packages or use bugzilla.

If we cannot complete your request, what is the impact?

A lot of open bugzillas for retired components. I think this used to happen automatically before, but maybe it was done by a human.

* Describe the issue When a package gets retired, all the bugzilla filed for rawhide should be closed as WONTFIX or EOL. * When do you need this? Any time. * When is this no longer needed or useful? Once we don't retire packages or use bugzilla. * If we cannot complete your request, what is the impact? A lot of open bugzillas for retired components. I think this used to happen automatically before, but maybe it was done by a human.

mohanboddu commented

2019-06-25 21:23:47 +00:00

Contributor

Metadata Update from @mohanboddu:

Issue tagged with: automation

**Metadata Update from @mohanboddu**: - Issue tagged with: automation

churchyard commented

2019-09-22 10:16:39 +00:00

Author

Member

I have a prototype script that closes all bugzillas for retired components.

However, when trying to get list of all rawhide bugzillas, bugzilla API crashes with Proxy error.

For 31 it seems to work (apparently, less 31 than rawhide Fedora bugzillas). Will do more testing and post something soon.

I have a prototype script that closes all bugzillas for retired components. However, when trying to get list of all rawhide bugzillas, bugzilla API crashes with Proxy error. For 31 it seems to work (apparently, less 31 than rawhide Fedora bugzillas). Will do more testing and post something soon.

churchyard commented

2019-09-22 13:57:58 +00:00

Author

Member

import configparser
import bugzilla
import pathlib
import sys
import logging
import subprocess
from click import progressbar


assert sys.version_info[0] > 2, 'Needs Python 3'


# XXX Update this after branching!
RAWHIDE = '32'


LOGGER = logging.getLogger()
LOGGER.setLevel(logging.DEBUG)
fh = logging.FileHandler('bugzilla.log')
fh.setLevel(logging.DEBUG)
LOGGER.addHandler(fh)


# Get credentials from config
config_path = pathlib.Path('./close_retired_bugzillas.cfg')
if not config_path.exists():
    config_path = pathlib.Path('/etc/close_retired_bugzillas.cfg')
if not config_path.exists():
    raise RuntimeError('Create a config file as ./close_retired_bugzillas.cfg '
                       'or /etc/close_retired_bugzillas.cfg')

config = configparser.ConfigParser()
config.read(config_path)

URL = config['bugzilla'].get('url', 'https://bugzilla.redhat.com')
USERNAME = config['bugzilla']['username']
PASSWORD = config['bugzilla'].get('password', '')
FEDORA = (config['bugzilla'].get('fedora', '').upper().replace('F', '')
          or RAWHIDE)


TEMPLATE = f"""Automation has figured out the package is retired in Fedora {FEDORA}.

If you like it to be unretired, please open a ticket at https://pagure.io/releng/new_issue?template=package_unretirement
"""  # noqa


if USERNAME == '-interactive-':
    bzapi = bugzilla.Bugzilla(URL)
    if not bzapi.logged_in:
        print(f'Log in to Bugzilla ta {URL}')
        bzapi.interactive_login()
else:
    bzapi = bugzilla.Bugzilla(URL, user=USERNAME, password=PASSWORD)


def open_bugz(version=FEDORA):
    if version == RAWHIDE:
        version = 'rawhide'
    query = bzapi.build_query(product='Fedora', version=version)
    query['status'] = [
        'NEW',
        'ASSIGNED',
        'POST',
        'MODIFIED',
        'ON_QA',
        'VERIFIED',
        'RELEASE_PENDING',
    ]
    return bzapi.query(query)


def is_retired(component, *, version=FEDORA):
    if component == 'distribution' or ' ' in component:
        return False
    if version == 'rawhide':
        version = RAWHIDE
    tag = f'f{version}'
    cmd = ('koji', 'list-pkgs', '--show-blocked',
           '--tag', tag, '--package', component)
    p = subprocess.run(cmd, stdout=subprocess.PIPE, text=True)
    return '[BLOCKED]' in p.stdout


def close_bugzillas(bug_ids, comment=TEMPLATE):
    update = bzapi.build_update(comment=comment,
                                status='CLOSED', resolution='WONTFIX')
    try:
        bzapi.update_bugs(bug_ids, update)
    except Exception:
        LOGGER.exception()


print('Gathering bugz, this can take several minutes...')
bugz = open_bugz()
components = set(bug.component for bug in bugz)
print(f'There are {len(bugz)} unclosed bugz for {len(components)} components')

print('Checking which components are retired...')
retired = set()
with progressbar(components, item_show_func=lambda c: c or 'Finished!') as cb:
    for component in cb:
        if is_retired(component):
            retired.add(component)


bug_ids = [b.id for b in bugz if b.component in retired]
print(f'There are {len(retired)} retired components, '
      f'closing their {len(bug_ids)} bugzillas.')

close_bugzillas(bug_ids)

It could use some concurrency of is_retired calls (now it takes ~1 hour), but otherwise appears to work.

I cannot longer close Bugzillas as EOL, so I went with WONTFIX.

This is unfortunately already causing confusion: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/4554GEM6XE3ZBAE53SU26GV2HLTU7S4O/

```python import configparser import bugzilla import pathlib import sys import logging import subprocess from click import progressbar assert sys.version_info[0] > 2, 'Needs Python 3' # XXX Update this after branching! RAWHIDE = '32' LOGGER = logging.getLogger() LOGGER.setLevel(logging.DEBUG) fh = logging.FileHandler('bugzilla.log') fh.setLevel(logging.DEBUG) LOGGER.addHandler(fh) # Get credentials from config config_path = pathlib.Path('./close_retired_bugzillas.cfg') if not config_path.exists(): config_path = pathlib.Path('/etc/close_retired_bugzillas.cfg') if not config_path.exists(): raise RuntimeError('Create a config file as ./close_retired_bugzillas.cfg ' 'or /etc/close_retired_bugzillas.cfg') config = configparser.ConfigParser() config.read(config_path) URL = config['bugzilla'].get('url', 'https://bugzilla.redhat.com') USERNAME = config['bugzilla']['username'] PASSWORD = config['bugzilla'].get('password', '') FEDORA = (config['bugzilla'].get('fedora', '').upper().replace('F', '') or RAWHIDE) TEMPLATE = f"""Automation has figured out the package is retired in Fedora {FEDORA}. If you like it to be unretired, please open a ticket at https://pagure.io/releng/new_issue?template=package_unretirement """ # noqa if USERNAME == '-interactive-': bzapi = bugzilla.Bugzilla(URL) if not bzapi.logged_in: print(f'Log in to Bugzilla ta {URL}') bzapi.interactive_login() else: bzapi = bugzilla.Bugzilla(URL, user=USERNAME, password=PASSWORD) def open_bugz(version=FEDORA): if version == RAWHIDE: version = 'rawhide' query = bzapi.build_query(product='Fedora', version=version) query['status'] = [ 'NEW', 'ASSIGNED', 'POST', 'MODIFIED', 'ON_QA', 'VERIFIED', 'RELEASE_PENDING', ] return bzapi.query(query) def is_retired(component, *, version=FEDORA): if component == 'distribution' or ' ' in component: return False if version == 'rawhide': version = RAWHIDE tag = f'f{version}' cmd = ('koji', 'list-pkgs', '--show-blocked', '--tag', tag, '--package', component) p = subprocess.run(cmd, stdout=subprocess.PIPE, text=True) return '[BLOCKED]' in p.stdout def close_bugzillas(bug_ids, comment=TEMPLATE): update = bzapi.build_update(comment=comment, status='CLOSED', resolution='WONTFIX') try: bzapi.update_bugs(bug_ids, update) except Exception: LOGGER.exception() print('Gathering bugz, this can take several minutes...') bugz = open_bugz() components = set(bug.component for bug in bugz) print(f'There are {len(bugz)} unclosed bugz for {len(components)} components') print('Checking which components are retired...') retired = set() with progressbar(components, item_show_func=lambda c: c or 'Finished!') as cb: for component in cb: if is_retired(component): retired.add(component) bug_ids = [b.id for b in bugz if b.component in retired] print(f'There are {len(retired)} retired components, ' f'closing their {len(bug_ids)} bugzillas.') close_bugzillas(bug_ids) ``` It could use some concurrency of `is_retired` calls (now it takes ~1 hour), but otherwise appears to work. I cannot longer close Bugzillas as EOL, so I went with WONTFIX. This is unfortunately already causing confusion: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/4554GEM6XE3ZBAE53SU26GV2HLTU7S4O/

kevin commented

2020-09-29 15:35:01 +00:00

Owner

So... why do we need this? The bugs should get EOLed as fedora EOL's releases.

If someone unretires a package, it won't be easy for them to see what things were outstanding when it was retired.

Is there some advantage to doing this?

So... why do we need this? The bugs should get EOLed as fedora EOL's releases. If someone unretires a package, it won't be easy for them to see what things were outstanding when it was retired. Is there some advantage to doing this?

churchyard commented

2020-09-29 15:39:57 +00:00

Author

Member

So... why do we need this? The bugs should get EOLed as fedora EOL's releases.

Some bugzillas are forever rawhide and never EOLed.

> So... why do we need this? The bugs should get EOLed as fedora EOL's releases. Some bugzillas are forever rawhide and never EOLed.

kevin commented

2020-09-29 15:58:08 +00:00

Owner

Wouldn't they get moved to branched at branching time? unless they have FutureFeature or whatever... in which case, wouldn't they still be of use to the package in case it's ever unretired?

mohanboddu commented

2020-09-29 16:07:20 +00:00

Contributor

From our releng meeting today, we decided to implement this as part of toddlers but when someone unretires we will send them a list of bugs that got closed due to retirement and let them handle it however they want to.

But we need help as this is something that we can live with, if someone from community would like to pick it up for us, that would be great.

From our releng meeting today, we decided to implement this as part of toddlers but when someone unretires we will send them a list of bugs that got closed due to retirement and let them handle it however they want to. But we need help as this is something that we can live with, if someone from community would like to pick it up for us, that would be great.

mohanboddu commented

2023-05-02 17:01:18 +00:00

Contributor

From our releng meeting today, we decided to implement this as part of toddlers but when someone unretires we will send them a list of bugs that got closed due to retirement and let them handle it however they want to.

But we need help as this is something that we can live with, if someone from community would like to pick it up for us, that would be great.

This still seems to be the plan, it needs someone working on writing the code in toddlers

> From our releng meeting today, we decided to implement this as part of toddlers but when someone unretires we will send them a list of bugs that got closed due to retirement and let them handle it however they want to. > > But we need help as this is something that we can live with, if someone from community would like to pick it up for us, that would be great. This still seems to be the plan, it needs someone working on writing the code in toddlers

lbalhar commented

2025-12-02 08:25:04 +00:00

I'd like to help with this, but I have no idea what toddlers means. I recently discovered that we have more than 400 open bugs from release monitoring for retired packages.

kevin commented

2025-12-04 22:55:14 +00:00

Owner

toddlers is a 'app' , where the source is: https://pagure.io/fedora-infra/toddlers

We run that application in the fedora-infrastructure openshift cluster.

It listens for messages on our message bus and then has various workers that do things based on them.

So, for this case we would I think to adjust an existing one that handles when we get retirement events and have it go out and close bugs.

toddlers is a 'app' , where the source is: https://pagure.io/fedora-infra/toddlers We run that application in the fedora-infrastructure openshift cluster. It listens for messages on our message bus and then has various workers that do things based on them. So, for this case we would I think to adjust an existing one that handles when we get retirement events and have it go out and close bugs.

lbalhar commented

2025-12-15 12:14:03 +00:00

I've found this toddler that runs every time a branch is retired and blocks the package in Koji. This shares a lot of logic, and it might make sense to extend this one to also close bugzillas reported by "Upstream Release Monitoring" for the retired branch/version and the package.

What do you think?

I've found [this toddler](https://pagure.io/fedora-infra/toddlers/blob/main/f/toddlers/plugins/koji_block_retired.py) that runs every time a branch is retired and blocks the package in Koji. This shares a lot of logic, and it might make sense to extend this one to also close bugzillas reported by "Upstream Release Monitoring" for the retired branch/version and the package. What do you think?

humaton added the

labels

2026-01-22 12:11:22 +00:00

lbalhar commented

2026-03-06 09:26:33 +00:00

@humaton, do you plan to work on this? Do I understand the labels added last month correctly?

amedvede commented

2026-03-09 12:16:34 +00:00

Owner

@lbalhar Hey, those labels were added as part of grooming old tickets. I'll bring this ticket up during the releng weekly meeting, and we will see how we would like to proceed with it. You are welcome to join the meeting happening at 4 pm CEST time in the #meeting-3:fedoraproject.org matrix channel

Rows
Columns

Retired packages should close rawhide bugzilla as WONTFIX or EOL #8478