infra/tickets

Fork 2

Collection of this group's data for a LLM-powered weekly report #13552

New issue

Closed

opened 2026-09-09 10:06:12 +00:00 by abompard · 7 comments

abompard commented

2026-09-09 10:06:12 +00:00

Member

Description of request

Hi folks,

I'm generating a weekly report of activity in the Fedora project centered on teams and SIGs, it's called This Week in Fedora. This generation uses an LLM. The goal is to make it easier to catch up with everything that's going on in the project, and to give visibility to each group's work and requests for help. I think that from a newcomer's point of view, having an insight on what a group is working on at a point in time is helpful to decide whether it's interesting and worth contributing to.
I'm collecting activity from:

mailing-lists
discourse posts with the infrastructure-team, infrastructure-projects, and fedora-forge tags
meetings on Matrix meeting channels
this tracker

In the latest version, I've added the possiblity to gather messages exchanged in Matrix channels. The channels are public and their history is available to anyone, but a quite a few people in mutiple Matrix channels were unhappy about the LLM collecting this info, for various reasons.
I would like to know whether this group is OK with my program collecting and submitting info to an LLM from the following sources:

mailing-lists, discourse posts and meetings
this tracker
the following matrix channels: #admin, and #forge

I would like to make the case for source 3 (the Matrix channels) because I think it's less obvious: knowing the "tone" of a conversation and the currently discussed subjects before joining is very important to help contribute to it. It's like when you're added to an empty group chat and you don't know how to start. That's why we show history to new channel members, and this would be the same idea at a higher level. Plus there's actual work going on in there for a lot of groups.
That said, I recognize that the form of the tool influences its use, chat messages are usually less thought-through than emails or forum posts, and members could be indimidated that something they write there and regret end up in a visible summary (the program respects message edits, but anyway).
Since my goal is to foster participation, the one thing I don't want to do is to hinder it in another way.

Please decide which collected sources, if any, you are OK with (and if you work in other places that you would like to have collected, of course (I can do RSS feeds!)). If the group is OK with Matrix channel collection but some individuals aren't, I can also exclude certain Matrix usernames from collection throughout the Matrix channels that are not meetings.

I'm happy to answer questions if you have any. Thanks!

### Description of request Hi folks, I'm generating a weekly report of activity in the Fedora project centered on teams and SIGs, it's called [This Week in Fedora](https://abompard.fedorapeople.org/twif/). This generation uses an LLM. The goal is to make it easier to catch up with everything that's going on in the project, and to give visibility to each group's work and requests for help. I think that from a newcomer's point of view, having an insight on what a group is working on at a point in time is helpful to decide whether it's interesting and worth contributing to. I'm collecting activity from: - mailing-lists - discourse posts with the `infrastructure-team`, `infrastructure-projects`, and `fedora-forge` tags - meetings on Matrix meeting channels - this tracker In the latest version, I've added the possiblity to gather messages exchanged in Matrix channels. The channels are public and their history is available to anyone, but a quite a few people in mutiple Matrix channels were unhappy about the LLM collecting this info, for various reasons. I would like to know whether this group is OK with my program collecting and submitting info to an LLM from the following sources: 1. mailing-lists, discourse posts and meetings 2. this tracker 3. the following matrix channels: `#admin`, and `#forge` I would like to make the case for source 3 (the Matrix channels) because I think it's less obvious: knowing the "tone" of a conversation and the currently discussed subjects before joining is very important to help contribute to it. It's like when you're added to an empty group chat and you don't know how to start. That's why we show history to new channel members, and this would be the same idea at a higher level. Plus there's actual work going on in there for a lot of groups. That said, I recognize that the form of the tool influences its use, chat messages are usually less thought-through than emails or forum posts, and members could be indimidated that something they write there and regret end up in a visible summary (the program respects message edits, but anyway). Since my goal is to foster participation, the one thing I don't want to do is to hinder it in another way. Please decide which collected sources, if any, you are OK with (and if you work in other places that you would like to have collected, of course (I can do RSS feeds!)). If the group is OK with Matrix channel collection but some individuals aren't, I can also exclude certain Matrix usernames from collection throughout the Matrix channels that are not meetings. I'm happy to answer questions if you have any. Thanks!

zlopez commented

2026-09-09 10:18:10 +00:00

Owner

I'm OK with this as all the work is done in public anyway. Having the work summarized is helpful not only for newcomers.

kevin commented

2026-09-09 19:03:17 +00:00

Owner

It's ok with me... at least for now. If it causes a problem we may revisit it.

james added the

points

Priority

Low

labels

2026-09-09 19:04:09 +00:00

gwmngilfen commented

2026-09-10 09:01:54 +00:00

Member

+1 as we're working in the open anyway.

abompard commented

2026-09-14 07:59:22 +00:00

Author

Member

Alright, I'll collect the channels as indicated. It's of course entirely revocable at any point in time. What's harder to do is to expunge Matrix sources from existing summaries, so I'd rather not do that if possible. But I can turn Matrix collection off for the next report easily.

kevin commented

2026-09-14 17:46:30 +00:00

Owner

Thinking a bit more about this, I suspect (but can only speak for myself) that a lot of the concern around this is that the bot will ingest/summarize things that are not related to fedora/the group.

Many people consider the folks working in some area of fedora to be their friends, and there's often off topic discussion in channels for unrelated things that friends talk about, ex tv shows or books you are watching or reading, medical or personal issues you are working through or the like. I don't think very many people would appreciate these sorts of things to be ingested and summarized.

That said, lets try it and see...

Thinking a bit more about this, I suspect (but can only speak for myself) that a lot of the concern around this is that the bot will ingest/summarize things that are not related to fedora/the group. Many people consider the folks working in some area of fedora to be their friends, and there's often off topic discussion in channels for unrelated things that friends talk about, ex tv shows or books you are watching or reading, medical or personal issues you are working through or the like. I don't think very many people would appreciate these sorts of things to be ingested and summarized. That said, lets try it and see...

kevin closed this issue

2026-09-14 17:46:30 +00:00

abompard commented

2026-09-15 15:16:42 +00:00

Author

Member

I understand the sentiment, in theory the LLM should discard these topics as it is instructed to only consider what's relevant to the project. That said, you know LLMs... The human review (currently: me) should get rid of the off topic discussions that get through the LLM filter.
And if it ends up being unreliable, we can totally reconsider this source. I'm not overly attached to collecting the Matrix channels, I want to capture work where work happens, and if it's not in Matrix then fine :-)

I understand the sentiment, in theory the LLM should discard these topics as it is instructed to only consider what's relevant to the project. That said, you know LLMs... The human review (currently: me) should get rid of the off topic discussions that get through the LLM filter. And if it ends up being unreliable, we can totally reconsider this source. I'm not overly attached to collecting the Matrix channels, I want to capture work where work happens, and if it's not in Matrix then fine :-)

kevin commented

2026-09-15 20:52:00 +00:00

Owner

Yeah, just thought I would mention a concern I think people have. No worries, we can adjust...