releng/tickets

Fork 2

scripts: Introduce organized directory structure with conventions #12936

Merged

jnsamyak merged 1 commit from releng_improv_pt2 into main

2025-09-16 04:02:17 +00:00

jnsamyak commented

2025-09-04 05:41:50 +00:00

Owner

Introduce Organized Scripts Directory Structure

Overview

This PR introduces a new, organized directory structure for the scripts/ called scripts_new/ directory to address long-standing maintainability issues and prepare for systematic script migration.

Current Problems Addressed

1. Poor Organization

65+ scripts in flat directory structure
Inconsistent naming (mix of hyphens, underscores, missing extensions)
Related scripts scattered across different locations
Difficult for new contributors to navigate

2. Obsolete Content Removal

Removed ODCS scripts (retired post-F41)
Previously removed module scripts (retired post-F39)
Previously removed PDC scripts (retired post-F40)

New Structure Introduced

As per the issue: https://pagure.io/releng/issue/12925, the scripts_new directory is introduced, so when the migration happens this will become the scripts directory again.

Standards Established

CONVENTIONS.md (have been introduced in the new directory structure for uniformity)

File naming: snake_case with mandatory extensions (.py, .sh)
Directory naming: lowercase-with-hyphens
Migration guidelines: Backward compatibility with symlinks
Validation tools: Scripts to check convention adherence

Documentation

README files in each directory explaining the purpose (If I missed something, we will add it in the upcoming pull-requests)
Examples of scripts that will be migrated to each location
Clear migration status indicators

Migration Strategy

Phase 1 (This PR) - @jnsamyak

Create directory structure
Establish conventions and documentation
Remove obsolete ODCS scripts

Phase 2 (Upcoming)

Research and categorize existing scripts (tickets being filed)
Ticket 1: Scripts Directory Restructuring Research - Build & Release Processes (Part 1/2) #12934
Ticket 2: Scripts Directory Restructuring Research - Build & Release Processes (Part 1/2) #12935

Phase 3 (Future)

Once this becomes robust, we will move to the new forge.
And, finally, we will change the documentation at https://docs.fedoraproject.org/en-US/infra/.

Things to review:

Directory structure validated for logical organization
CONVENTIONS.md reviewed for completeness and clarity
README files checked for accuracy and helpfulness
No functional changes to existing scripts

This foundation enables systematic improvement of our release engineering tooling while maintaining full backward compatibility. The new structure will make the releng repository more maintainable and accessible for current and future contributors.

Addresses script organization concerns raised in team discussions
Supports onboarding improvements for new contributors
Enables better maintenance practices for release engineering tools

/cc @fedora-releng-team

# Introduce Organized Scripts Directory Structure ## Overview This PR introduces a new, organized directory structure for the `scripts/` called `scripts_new/` directory to address long-standing maintainability issues and prepare for systematic script migration. ## Current Problems Addressed ### 1. **Poor Organization** - 65+ scripts in flat directory structure - Inconsistent naming (mix of hyphens, underscores, missing extensions) - Related scripts scattered across different locations - Difficult for new contributors to navigate ### 2. **Obsolete Content Removal** - Removed ODCS scripts (retired post-F41) - Previously removed module scripts (retired post-F39) - Previously removed PDC scripts (retired post-F40) ## New Structure Introduced As per the issue: https://pagure.io/releng/issue/12925, the scripts_new directory is introduced, so when the migration happens this will become the scripts directory again. ## Standards Established ### **CONVENTIONS.md** (have been introduced in the new directory structure for uniformity) - **File naming**: `snake_case` with mandatory extensions (`.py`, `.sh`) - **Directory naming**: `lowercase-with-hyphens` - **Migration guidelines**: Backward compatibility with symlinks - **Validation tools**: Scripts to check convention adherence ### **Documentation** - README files in each directory explaining the purpose (If I missed something, we will add it in the upcoming pull-requests) - Examples of scripts that will be migrated to each location - Clear migration status indicators ## Migration Strategy ### **Phase 1** (This PR) - @jnsamyak - Create directory structure - Establish conventions and documentation - Remove obsolete ODCS scripts ### **Phase 2** (Upcoming) - Research and categorize existing scripts (tickets being filed) Ticket 1: Scripts Directory Restructuring Research - Build & Release Processes (Part 1/2) #12934 Ticket 2: Scripts Directory Restructuring Research - Build & Release Processes (Part 1/2) #12935 ### **Phase 3** (Future) - Once this becomes robust, we will move to the new forge. - And, finally, we will change the documentation at https://docs.fedoraproject.org/en-US/infra/. Things to review: - Directory structure validated for logical organization - CONVENTIONS.md reviewed for completeness and clarity - README files checked for accuracy and helpfulness - No functional changes to existing scripts --- This foundation enables systematic improvement of our release engineering tooling while maintaining full backward compatibility. The new structure will make the releng repository more maintainable and accessible for current and future contributors. ## Related Issues - Addresses script organization concerns raised in team discussions - Supports onboarding improvements for new contributors - Enables better maintenance practices for release engineering tools /cc @fedora-releng-team

jnsamyak commented

2025-09-04 05:44:25 +00:00

Author

Owner

rebased onto f9357b1e20

rebased onto f9357b1e20ed2884cf392d556bc6cea02bb0c4f4

kevin commented

2025-09-04 19:43:35 +00:00

Owner

Seems ok, but a few minor comments:

Are you sure you want to make a new tree and then keep symlinks for 6 months? IMHO it just means that when we drop those we have to fix anything that breaks instead of just doing it right now when it's in mind. (Although, not in freeze please...).
I'm a little unclear on if the compatiblity stuff is a link or a wrapper? it describes both, but it can't be both?
Do we need all those .gitignore files in subdirs? perhaps just one at the top level?

Seems ok, but a few minor comments: * Are you sure you want to make a new tree and then keep symlinks for 6 months? IMHO it just means that when we drop those we have to fix anything that breaks instead of just doing it right now when it's in mind. (Although, not in freeze please...). * I'm a little unclear on if the compatiblity stuff is a link or a wrapper? it describes both, but it can't be both? * Do we need all those .gitignore files in subdirs? perhaps just one at the top level?

gotmax23 commented

2025-09-04 21:09:41 +00:00

Member

Thanks for working on this. Some feedback:

I think the FTBFS, FTI, and orphaned packages process scripts should be in the same parent directory. They're all FESCo policies that involve cleaning up the distribution by orphaning and/or retiring packages.
The CONVENTIONS doc seems excessively verbose and LLM-y, IMHO. The long examples and emojis and many subheadings make it hard to read. Engineers working in this repository should know what "Filenames should be snake_case" or "Always include file extensions in scripts" means or at least not require such a lengthy explanation.

Thanks for working on this. Some feedback: - I think the FTBFS, FTI, and orphaned packages process scripts should be in the same parent directory. They're all FESCo policies that involve cleaning up the distribution by orphaning and/or retiring packages. - The CONVENTIONS doc seems excessively verbose and LLM-y, IMHO. The long examples and emojis and many subheadings make it hard to read. Engineers working in this repository should know what "Filenames should be snake_case" or "Always include file extensions in scripts" means or at least not require such a lengthy explanation.

jnsamyak commented

2025-09-05 06:23:42 +00:00

Author

Owner

Are you sure you want to make a new tree and then keep symlinks for 6 months? IMHO it just means that when we drop those we have to fix anything that breaks instead of just doing it right now when it's in mind. (Although, not in freeze please...).

NO, waiting for six months, I'm just creating this structure to take the place of scripts after the freeze is done. And, yes, I'm also in favour of fixing it now, then in the future. I have just added the directory structure in order to copy the contents now, and just move the script directory to the new one at that time.

I'm a little unclear on if the compatiblity stuff is a link or a wrapper? it describes both, but it can't be both?

I gave it a thought, and we should just drop that, it will not require if we will not do the 6 month of grace waiting. I tried generating it using cursor module by giving it reference of other fedora repos, and I think it caught that, and my changes on the top of it got lost. I'll correct those up

The CONVENTIONS doc seems excessively verbose and LLM-y, IMHO. The long examples and emojis and many subheadings make it hard to read. Engineers working in this repository should know what "Filenames should be snake_case" or "Always include file extensions in scripts" means or at least not require such a lengthy explanation.

Yeah! Make sense, I have addressed it above. Anyways point of keeping it verbose (not this much) was I was asked to write as much as nitty gritty details for other releng associates.

I think the FTBFS, FTI, and orphaned packages process scripts should be in the same parent directory. They're all FESCo policies that involve cleaning up the distribution by orphaning and/or retiring packages.

It is a good suggestion, can you also add it at https://pagure.io/releng/issue/12925,

> Are you sure you want to make a new tree and then keep symlinks for 6 months? IMHO it just means that when we drop those we have to fix anything that breaks instead of just doing it right now when it's in mind. (Although, not in freeze please...). NO, waiting for six months, I'm just creating this structure to take the place of scripts after the freeze is done. And, yes, I'm also in favour of fixing it now, then in the future. I have just added the directory structure in order to copy the contents now, and just move the script directory to the new one at that time. > I'm a little unclear on if the compatiblity stuff is a link or a wrapper? it describes both, but it can't be both? I gave it a thought, and we should just drop that, it will not require if we will not do the 6 month of grace waiting. I tried generating it using cursor module by giving it reference of other fedora repos, and I think it caught that, and my changes on the top of it got lost. I'll correct those up > The CONVENTIONS doc seems excessively verbose and LLM-y, IMHO. The long examples and emojis and many subheadings make it hard to read. Engineers working in this repository should know what "Filenames should be snake_case" or "Always include file extensions in scripts" means or at least not require such a lengthy explanation. Yeah! Make sense, I have addressed it above. Anyways point of keeping it verbose (not this much) was I was asked to write as much as nitty gritty details for other releng associates. > I think the FTBFS, FTI, and orphaned packages process scripts should be in the same parent directory. They're all FESCo policies that involve cleaning up the distribution by orphaning and/or retiring packages. It is a good suggestion, can you also add it at https://pagure.io/releng/issue/12925,

jnsamyak commented

2025-09-05 06:32:51 +00:00

Author

Owner

rebased onto f9357b1e20

rebased onto f9357b1e20ed2884cf392d556bc6cea02bb0c4f4

jnsamyak commented

2025-09-05 06:43:29 +00:00

Author

Owner

rebased onto f9357b1e20

rebased onto f9357b1e20ed2884cf392d556bc6cea02bb0c4f4