Migrate Dist Git from Puppet to Ansible

bochecha commented

2014-07-17 22:50:07 +00:00

SSIA.

bochecha commented

2014-07-17 22:52:39 +00:00

Author

I've been working on this for some time now, and I think I have something ready to be tested.

I'll be attaching individual patches to this ticket, but for a more graphical overview, see:

https://github.com/bochecha/fedora-infra-ansible/compare/master...pkgs

I've been working on this for some time now, and I think I have something ready to be tested. I'll be attaching individual patches to this ticket, but for a more graphical overview, see: https://github.com/bochecha/fedora-infra-ansible/compare/master...pkgs

bochecha commented

2014-07-17 22:54:42 +00:00

Author

[PATCH 02/13] Add a new git/server role
0002-Add-a-new-git-server-role.patch

[PATCH 02/13] Add a new git/server role [0002-Add-a-new-git-server-role.patch](/fedora-infrastructure/issue/raw/files/ec60f2d5b45519c161ffe201580c7014206f6bd5a728a8ee4d52b6fa7324b2fa-0002-Add-a-new-git-server-role.patch)

bochecha commented

2014-07-17 22:54:57 +00:00

Author

[PATCH 03/13] Add a new gitolite/base role
0003-Add-a-new-gitolite-base-role.patch

[PATCH 03/13] Add a new gitolite/base role [0003-Add-a-new-gitolite-base-role.patch](/fedora-infrastructure/issue/raw/files/2fdb7ceef8087b20f375a947795a8631a03c7b29ed4301698beb46b9df228b64-0003-Add-a-new-gitolite-base-role.patch)

bochecha commented

2014-07-17 22:55:28 +00:00

Author

[PATCH 05/13] Add a new cgit/clean_lock_cron role
0005-Add-a-new-cgit-clean_lock_cron-role.patch

[PATCH 05/13] Add a new cgit/clean_lock_cron role [0005-Add-a-new-cgit-clean_lock_cron-role.patch](/fedora-infrastructure/issue/raw/files/fbac9e79255ad9748a14bc1e23b914a3654113932541d51e7b0261f008e70473-0005-Add-a-new-cgit-clean_lock_cron-role.patch)

bochecha commented

2014-07-17 22:55:45 +00:00

Author

[PATCH 06/13] Add a new clamav role
0006-Add-a-new-clamav-role.patch

[PATCH 06/13] Add a new clamav role [0006-Add-a-new-clamav-role.patch](/fedora-infrastructure/issue/raw/files/618f27527c27b42e6cdad80dac3e5f947421cfb1916ef898bf20b38866ceae60-0006-Add-a-new-clamav-role.patch)

bochecha commented

2014-07-17 22:56:02 +00:00

Author

[PATCH 07/13] Add a new gitolite/check_fedmsg_hooks role
0007-Add-a-new-gitolite-check_fedmsg_hooks-role.patch

[PATCH 07/13] Add a new gitolite/check_fedmsg_hooks role [0007-Add-a-new-gitolite-check_fedmsg_hooks-role.patch](/fedora-infrastructure/issue/raw/files/03bcbacdd991e06f61511c9c5a826ecb72374140a16c545fc916dac6f0cdf8ff-0007-Add-a-new-gitolite-check_fedmsg_hooks-role.patch)

bochecha commented

2014-07-17 22:56:31 +00:00

Author

[PATCH 09/13] Add a new git/make_checkout_seed role
0009-Add-a-new-git-make_checkout_seed-role.patch

[PATCH 09/13] Add a new git/make_checkout_seed role [0009-Add-a-new-git-make_checkout_seed-role.patch](/fedora-infrastructure/issue/raw/files/d8d2db1bafe113c44a2b27b9d5a9b9f4b55571322cdd3b5bf0819908519d23b7-0009-Add-a-new-git-make_checkout_seed-role.patch)

bochecha commented

2014-07-17 22:56:47 +00:00

Author

[PATCH 10/13] Add the confine_ssh task
0010-Add-the-confine_ssh-task.patch

[PATCH 10/13] Add the confine_ssh task [0010-Add-the-confine_ssh-task.patch](/fedora-infrastructure/issue/raw/files/2c9e2d453530d142d9b1822db7a31d5b7a609109423b917a08773cbcfc379b62-0010-Add-the-confine_ssh-task.patch)

bochecha commented

2014-07-17 22:57:05 +00:00

Author

[PATCH 11/13] Add a drbackupkey task
0011-Add-a-drbackupkey-task.patch

[PATCH 11/13] Add a drbackupkey task [0011-Add-a-drbackupkey-task.patch](/fedora-infrastructure/issue/raw/files/291d215077cb29c36cbf7a3f7cd695925f03a41acad0a97ce22f947c4fbf178a-0011-Add-a-drbackupkey-task.patch)

bochecha commented

2014-07-17 22:57:33 +00:00

Author

[PATCH 13/13] Setup the production and staging Dist Git
0013-Setup-the-production-and-staging-Dist-Git.patch

[PATCH 13/13] Setup the production and staging Dist Git [0013-Setup-the-production-and-staging-Dist-Git.patch](/fedora-infrastructure/issue/raw/files/a3c15c991a3e225b77e94a7dd195ab539c9dfd2301bd2ae958afe8a2f259675f-0013-Setup-the-production-and-staging-Dist-Git.patch)

bochecha commented

2014-07-17 22:59:11 +00:00

Author

Just a note: this is the very first time I ever am in contact with Puppet and Ansible, I never used either of them before, never wrote any module for either of them, etc...

I totally expect that I missed some things, made stupid mistakes, and generally got stuff wrong. :)

Just a note: this is the very first time I ever am in contact with Puppet and Ansible, I never used either of them before, never wrote any module for either of them, etc... I totally expect that I missed some things, made stupid mistakes, and generally got stuff wrong. :)

misc commented

2014-07-25 00:41:12 +00:00

Member

So, commenting on every patch is gonna take time, so i will just look at them in order :)

On the first patch, a comment say that this part requires fedmsg/base. I think such dependencies should be expressed using a role dependencies :

http://docs.ansible.com/playbooks_roles.html#role-dependencies

And on roles/git/hooks/files/gnome-post-receive-email , there is a module called kitchen. I do not see it being required or installed, so I suspect this would be missing on a new installation.

So, commenting on every patch is gonna take time, so i will just look at them in order :) On the first patch, a comment say that this part requires fedmsg/base. I think such dependencies should be expressed using a role dependencies : http://docs.ansible.com/playbooks_roles.html#role-dependencies And on roles/git/hooks/files/gnome-post-receive-email , there is a module called kitchen. I do not see it being required or installed, so I suspect this would be missing on a new installation.

misc commented

2014-07-25 00:46:21 +00:00

Member

Patch 2

I think we need to make sure that xinetd is started and enabled ( see the service module :

http://docs.ansible.com/service_module.html )

And in fact, I would put a specific role for xinetd, so you could just have the git-server role requires the xinetd role, who would start xinetd, install it and have a proper handler ( since currently, the notify is not going anywhere IMHO ). See
http://docs.ansible.com/playbooks_intro.html for how handlers work.

Patch 2 I think we need to make sure that xinetd is started and enabled ( see the service module : http://docs.ansible.com/service_module.html ) And in fact, I would put a specific role for xinetd, so you could just have the git-server role requires the xinetd role, who would start xinetd, install it and have a proper handler ( since currently, the notify is not going anywhere IMHO ). See http://docs.ansible.com/playbooks_intro.html for how handlers work.

misc commented

2014-07-25 00:48:40 +00:00

Member

patch 3, I am not sure why there is a specific owner change on gitolite directory, shouldn't it be fixed in the package directly ?

misc commented

2014-07-25 00:56:06 +00:00

Member

On patch 4, I think the yaml around line 10 to be incorrect. IE, you need to have 1 name for each module :
{{{

name: foo
file: src=foo dest=/etc/foo
name: foo 2
file: src=bar dest=/etc/bar
}}}

rather than placing the 2 file and copy under the same item ( ie, each time you do a action, you need to have a new yaml element, ie start by - )

On patch 4, I think the yaml around line 10 to be incorrect. IE, you need to have 1 name for each module : {{{ - name: foo file: src=foo dest=/etc/foo - name: foo 2 file: src=bar dest=/etc/bar }}} rather than placing the 2 file and copy under the same item ( ie, each time you do a action, you need to have a new yaml element, ie start by - )

bochecha commented

2014-07-28 19:17:39 +00:00

Author

[PATCH 01/13] Add a new git/hooks role
0001-Add-a-new-git-hooks-role.patch

[PATCH 01/13] Add a new git/hooks role [0001-Add-a-new-git-hooks-role.patch](/fedora-infrastructure/issue/raw/files/f6da6c330745c369eb029b61940a736630d6e87c52fd705d726e343e4f5d8900-0001-Add-a-new-git-hooks-role.patch)

bochecha commented

2014-07-28 19:25:11 +00:00

Author

Replying to [comment:5 misc]:

patch 3, I am not sure why there is a specific owner change on gitolite directory, shouldn't it be fixed in the package directly ?

Because:

{{{
$ repoquery -l gitolite | grep etc
$
}}}

AIUI, {{{/etc/gitolite}}} is something that is used in the Fedora Dist Git deployment, but it's not where gitolite configuration is necessarily hosted. (for example, the gitolire rc file usuqlly lives in the gitolite user's home directory)

So it's not just an owner change, it's actually creating the directory.

Replying to [comment:5 misc]: > patch 3, I am not sure why there is a specific owner change on gitolite directory, shouldn't it be fixed in the package directly ? Because: {{{ $ repoquery -l gitolite | grep etc $ }}} AIUI, {{{/etc/gitolite}}} is something that is used in the Fedora Dist Git deployment, but it's not where gitolite configuration is necessarily hosted. (for example, the gitolire rc file usuqlly lives in the gitolite user's home directory) So it's not just an owner change, it's actually creating the directory.

bochecha commented

2014-07-28 19:29:39 +00:00

Author

Replying to [comment:3 misc]:

On the first patch, a comment say that this part requires fedmsg/base. I think such dependencies should be expressed using a role dependencies :

http://docs.ansible.com/playbooks_roles.html#role-dependencies

Fixed, didn't know about that, thanks.

Note that I've done that quite often, so I'll go ahead and fix my other patches for this. Do let me know if you find other instances, though.

And on roles/git/hooks/files/gnome-post-receive-email , there is a module called kitchen. I do not see it being required or installed, so I suspect this would be missing on a new installation.

Fixed, good catch.

Replying to [comment:3 misc]: > On the first patch, a comment say that this part requires fedmsg/base. I think such dependencies should be expressed using a role dependencies : > > http://docs.ansible.com/playbooks_roles.html#role-dependencies Fixed, didn't know about that, thanks. Note that I've done that quite often, so I'll go ahead and fix my other patches for this. Do let me know if you find other instances, though. > And on roles/git/hooks/files/gnome-post-receive-email , there is a module called kitchen. I do not see it being required or installed, so I suspect this would be missing on a new installation. Fixed, good catch.

bochecha commented

2014-08-01 15:27:42 +00:00

Author

Replying to [comment:6 misc]:

rather than placing the 2 file and copy under the same item ( ie, each time you do a action, you need to have a new yaml element, ie start by - )

Fixed.

Replying to [comment:6 misc]: > rather than placing the 2 file and copy under the same item ( ie, each time you do a action, you need to have a new yaml element, ie start by - ) Fixed.

bochecha commented

2014-08-01 15:28:36 +00:00

Author

[PATCH 04/13] Add a new cgit/base role
0004-Add-a-new-cgit-base-role.patch

[PATCH 04/13] Add a new cgit/base role [0004-Add-a-new-cgit-base-role.patch](/fedora-infrastructure/issue/raw/files/e2d9e71051ca1fb3b70dcd441cf2f70c6f8da2cfd874a68ea5411e33b123670c-0004-Add-a-new-cgit-base-role.patch)

bochecha commented

2014-08-01 15:35:21 +00:00

Author

Replying to [comment:4 misc]:

And in fact, I would put a specific role for xinetd, so you could just have the git-server role requires the xinetd role, who would start xinetd, install it and have a proper handler ( since currently, the notify is not going anywhere IMHO ). See
http://docs.ansible.com/playbooks_intro.html for how handlers work.

This one I don't really understand what you mean. I basically just redid the same thing as what's done in other roles/playbooks with regards to services. :-/

There is a handler for xinetd in {{{handlers/restart_services.yml}}}, and it is included in the playbook, like other playbooks (I took example on the Bodhi one).

Should I instead include the handler in the role?

Replying to [comment:4 misc]: > And in fact, I would put a specific role for xinetd, so you could just have the git-server role requires the xinetd role, who would start xinetd, install it and have a proper handler ( since currently, the notify is not going anywhere IMHO ). See > http://docs.ansible.com/playbooks_intro.html for how handlers work. This one I don't really understand what you mean. I basically just redid the same thing as what's done in other roles/playbooks with regards to services. :-/ There is a handler for xinetd in {{{handlers/restart_services.yml}}}, and it is included in the playbook, like other playbooks (I took example on the Bodhi one). Should I instead include the handler in the role?

bochecha commented

2014-08-01 15:36:17 +00:00

Author

[PATCH 08/13] Add a new cgit/make_pkgs_list role
0008-Add-a-new-cgit-make_pkgs_list-role.patch

[PATCH 08/13] Add a new cgit/make_pkgs_list role [0008-Add-a-new-cgit-make_pkgs_list-role.patch](/fedora-infrastructure/issue/raw/files/eb74cb8c71501e7d326acbc7f1baf02283e3b763402e6b3717b92cd1a51903e3-0008-Add-a-new-cgit-make_pkgs_list-role.patch)

bochecha commented

2014-08-01 15:36:50 +00:00

Author

Just fixed patch 8, it was adding lots of empty lines at the end of a file, for no reason (not sure what I did there)

bochecha commented

2014-08-01 15:51:06 +00:00

Author

[PATCH 12/13] Add a new distgit role
0012-Add-a-new-distgit-role.patch

[PATCH 12/13] Add a new distgit role [0012-Add-a-new-distgit-role.patch](/fedora-infrastructure/issue/raw/files/fc2aa522d9b34fd62a659288038e1026f083044d064d1606a20cdec68476ed6c-0012-Add-a-new-distgit-role.patch)

bochecha commented

2014-08-01 15:51:59 +00:00

Author

Just fixed patch 12, which contained some of the same mistakes you had found:

not using role dependencies
too many tasks in one

Just fixed patch 12, which contained some of the same mistakes you had found: * not using role dependencies * too many tasks in one

kevin commented

2014-08-01 23:46:19 +00:00

Owner

I didn't get to this this week, sorry... ;(

We can look at it at flock if we have time, or just after. ;)

I didn't get to this this week, sorry... ;( We can look at it at flock if we have time, or just after. ;)

bochecha commented

2014-08-02 14:58:00 +00:00

Author

Replying to [comment:13 kevin]:

I didn't get to this this week, sorry... ;(

No worries, I had enough to do fixing the issues Michael found. :)

We can look at it at flock if we have time, or just after. ;)

Unfortunately I'm not going to Flock. :(

Replying to [comment:13 kevin]: > I didn't get to this this week, sorry... ;( No worries, I had enough to do fixing the issues Michael found. :) > We can look at it at flock if we have time, or just after. ;) Unfortunately I'm not going to Flock. :(

kevin commented

2014-08-27 01:33:13 +00:00

Owner

ok. I have commited this to ansible git, done some cleanups and tweaks and am trying to build a new pkgs01.stg. ;)

Will let you know if I run into any issues. Once it's up we will want to test it out quite a lot...

ok. I have commited this to ansible git, done some cleanups and tweaks and am trying to build a new pkgs01.stg. ;) Will let you know if I run into any issues. Once it's up we will want to test it out quite a lot...

bochecha commented

2014-09-18 15:43:17 +00:00

Author

Reopening, as I found some stuff passed through the cracks (e.g stuff I corrected in new patch versions, but the old ones were applied, or things that got done in puppet after the patches were made/applied).

I'll attach the new patch series. :)

Reopening, as I found some stuff passed through the cracks (e.g stuff I corrected in new patch versions, but the old ones were applied, or things that got done in puppet after the patches were made/applied). I'll attach the new patch series. :)

bochecha commented

2014-09-18 15:45:17 +00:00

Author

git/hooks: Add missing package dependency
0001-git-hooks-Add-missing-package-dependency.patch

git/hooks: Add missing package dependency [0001-git-hooks-Add-missing-package-dependency.patch](/fedora-infrastructure/issue/raw/files/9db6c20f1c39334ea154f250c6e605e024eee82beb27455babfc8affe15ea39a-0001-git-hooks-Add-missing-package-dependency.patch)

bochecha commented

2014-09-18 15:45:34 +00:00

Author

git/hooks: Express the role dependency properly
0002-git-hooks-Express-the-role-dependency-properly.patch

git/hooks: Express the role dependency properly [0002-git-hooks-Express-the-role-dependency-properly.patch](/fedora-infrastructure/issue/raw/files/aa504eb13a6378ac3e400d3092c60508b0c26b1fb2a96d863e2666e7197a1c51-0002-git-hooks-Express-the-role-dependency-properly.patch)

bochecha commented

2014-09-18 15:45:50 +00:00

Author

cgit/make_pkgs_list: Remove useless new lines
0003-cgit-make_pkgs_list-Remove-useless-new-lines.patch

cgit/make_pkgs_list: Remove useless new lines [0003-cgit-make_pkgs_list-Remove-useless-new-lines.patch](/fedora-infrastructure/issue/raw/files/b3efd95817b290f62ec07f8754e53b333d7f9e70782121b3b7c17d5dbc1c8fca-0003-cgit-make_pkgs_list-Remove-useless-new-lines.patch)

bochecha commented

2014-09-18 15:46:15 +00:00

Author

Add support for packaging groups
0004-Add-support-for-packaging-groups.patch

Add support for packaging groups [0004-Add-support-for-packaging-groups.patch](/fedora-infrastructure/issue/raw/files/4d4dbfd14d989155ea26962fe387df88d8d04039b01b52aeb4a45b5d97b35435-0004-Add-support-for-packaging-groups.patch)

bochecha commented

2014-09-18 15:46:31 +00:00

Author

Setup for F21 branching
0005-Setup-for-F21-branching.patch

Setup for F21 branching [0005-Setup-for-F21-branching.patch](/fedora-infrastructure/issue/raw/files/a9a97d943228ddd630ae292d4e143200fb633dc35e8b29a69df3195c3fe9132c-0005-Setup-for-F21-branching.patch)

bochecha commented

2014-09-18 15:46:48 +00:00

Author

distgit: Express the role dependency properly
0006-distgit-Express-the-role-dependency-properly.patch

distgit: Express the role dependency properly [0006-distgit-Express-the-role-dependency-properly.patch](/fedora-infrastructure/issue/raw/files/0be32bcef5b7710bbb44bbbf3638692c8d5c64a846f420a17d4a1dc5938f0f4b-0006-distgit-Express-the-role-dependency-properly.patch)

bochecha commented

2014-09-18 15:47:06 +00:00

Author

distgit: One action per task
0007-distgit-One-action-per-task.patch

distgit: One action per task [0007-distgit-One-action-per-task.patch](/fedora-infrastructure/issue/raw/files/ac766a8860571cc8cc18281d4f26b96b46e0d0310ae1d36f401a18a96e706ab5-0007-distgit-One-action-per-task.patch)

bochecha commented

2014-09-18 15:51:35 +00:00

Author

To make things a bit easier, the new patches are:

[https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0001-git-hooks-Add-missing-package-dependency.patch 0001-git-hooks-Add-missing-package-dependency.patch]
[https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0002-git-hooks-Express-the-role-dependency-properly.patch 0002-git-hooks-Express-the-role-dependency-properly.patch]
[https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0003-cgit-make_pkgs_list-Remove-useless-new-lines.patch 0003-cgit-make_pkgs_list-Remove-useless-new-lines.patch]
[https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0004-Add-support-for-packaging-groups.patch 0004-Add-support-for-packaging-groups.patch]
[https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0005-Setup-for-F21-branching.patch 0005-Setup-for-F21-branching.patch]
[https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0006-distgit-Express-the-role-dependency-properly.patch 0006-distgit-Express-the-role-dependency-properly.patch]
[https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0007-distgit-One-action-per-task.patch 0007-distgit-One-action-per-task.patch]

To make things a bit easier, the new patches are: * [https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0001-git-hooks-Add-missing-package-dependency.patch 0001-git-hooks-Add-missing-package-dependency.patch] * [https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0002-git-hooks-Express-the-role-dependency-properly.patch 0002-git-hooks-Express-the-role-dependency-properly.patch] * [https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0003-cgit-make_pkgs_list-Remove-useless-new-lines.patch 0003-cgit-make_pkgs_list-Remove-useless-new-lines.patch] * [https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0004-Add-support-for-packaging-groups.patch 0004-Add-support-for-packaging-groups.patch] * [https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0005-Setup-for-F21-branching.patch 0005-Setup-for-F21-branching.patch] * [https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0006-distgit-Express-the-role-dependency-properly.patch 0006-distgit-Express-the-role-dependency-properly.patch] * [https://fedorahosted.org/fedora-infrastructure/attachment/ticket/4452/0007-distgit-One-action-per-task.patch 0007-distgit-One-action-per-task.patch]

pingou commented

2014-09-24 16:26:19 +00:00

Member

Applied and pushed

Rows
Columns

Migrate Dist Git from Puppet to Ansible #4452