Malte Janduda
2012-Dec-18 17:59 UTC
[Puppet Users] How to manage puppet modules? git submodules? hg subrepositories? own solution?
Hi folks, we are looking for a way to manage our puppet modules in a convenient way. At the moment we have about 30 modules which are versionized with git or mercurial. Most of them are self-made, some are from github. We thought it would be a good idea to use mercurial subrepositories to manage the versions of those modules. Well, it is not. Everytime you push your commits to the server all subrepositories are checked for updates. It takes minutes to push your changes! And if you have to merge something you do not really know what happens. You just try every command which appears in your mind - multiple times and in different order :D It happened three times that every developer had to delete the whole module tree and check it out again because something was broken. git submodules do not seem to be fun either. I have looked for an alternative to submodules/subrepositories and found librarian-puppet (https://github.com/rodjek/librarian-puppet). Unfortunately it does not really work good for me. A lot of useless error messages if something is configured wrong. This is pretty important for us. All developers should be able to use the versioning tool. It has to be easy to use. Further the librarian-puppet does not seem to be backed by a healthy community. Just every few months a commit and a lot of open issues and pull-requests. I have hacked some Python code to realize something similar: dyps - https://github.com/MalteJ/dyps You have a yaml file which defines your modules: name, source, version and DVCS system (git/hg): - dep: mysql source: https://github.com/puppetlabs/puppetlabs-mysql.git type: git version: af4b8bff0ad59f21211be2a98d31588066c41e0c All modules are downloaded to the folder "dependencies" when executing "dyps run". You can decide if you want to get a specific changeset at the next run or if you want to get the latest changes - just discard the version property or execute "dyps release your-module-name". If you want to keep the current version execute "dyps keep your-module". You can update one or all versionized modules via "dyps update your-module" or "dyps update-all". The modules without a defined versioned are updated at every dyps run. I have added an import function for mercurial subrepository .hgsub files. Feel free to extend it with an importGitSubmodules function ;-) What do you think about this approach? How are you managing your modules/dependencies? Best, Malte -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/wXsxrloQTVYJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Benjamin Priestman
2012-Dec-20 13:36 UTC
[Puppet Users] Re: How to manage puppet modules? git submodules? hg subrepositories? own solution?
I''ve tried using https://github.com/azimux/externals (referenced in Version Control with Git <http://shop.oreilly.com/product/9780596520137.do>) which kind of works, but is a bit buggy and I''ve yet to come up with an easily understood workflow. On Tuesday, 18 December 2012 17:59:48 UTC, Malte Janduda wrote:> > Hi folks, > > we are looking for a way to manage our puppet modules in a convenient way. > At the moment we have about 30 modules which are versionized with git or > mercurial. Most of them are self-made, some are from github. > We thought it would be a good idea to use mercurial subrepositories to > manage the versions of those modules. Well, it is not. Everytime you push > your commits to the server all subrepositories are checked for updates. It > takes minutes to push your changes! And if you have to merge something you > do not really know what happens. You just try every command which appears > in your mind - multiple times and in different order :D > It happened three times that every developer had to delete the whole > module tree and check it out again because something was broken. > git submodules do not seem to be fun either. > > I have looked for an alternative to submodules/subrepositories and found > librarian-puppet (https://github.com/rodjek/librarian-puppet). > Unfortunately it does not really work good for me. A lot of useless error > messages if something is configured wrong. This is pretty important for us. > All developers should be able to use the versioning tool. It has to be easy > to use. Further the librarian-puppet does not seem to be backed by a > healthy community. Just every few months a commit and a lot of open issues > and pull-requests. > > I have hacked some Python code to realize something similar: dyps - > https://github.com/MalteJ/dyps > You have a yaml file which defines your modules: name, source, version and > DVCS system (git/hg): > > - dep: mysql > source: https://github.com/puppetlabs/puppetlabs-mysql.git > type: git > version: af4b8bff0ad59f21211be2a98d31588066c41e0c > > All modules are downloaded to the folder "dependencies" when executing > "dyps run". > You can decide if you want to get a specific changeset at the next run or > if you want to get the latest changes - just discard the version property > or execute "dyps release your-module-name". If you want to keep the current > version execute "dyps keep your-module". You can update one or all > versionized modules via "dyps update your-module" or "dyps update-all". The > modules without a defined versioned are updated at every dyps run. > > I have added an import function for mercurial subrepository .hgsub files. > Feel free to extend it with an importGitSubmodules function ;-) > > > What do you think about this approach? > How are you managing your modules/dependencies? > > Best, > Malte >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/VZjQExuTnvAJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Mason Turner
2012-Dec-20 18:28 UTC
Re: [Puppet Users] Re: How to manage puppet modules? git submodules? hg subrepositories? own solution?
We are still using svn, but looking on adapt this to git next year. This is a simplified view of our svn repo: - nodes -- dev --- trunk --- tags ---- 20121220-121212 -- prod --- trunk --- tags - modules -- apache --- trunk --- tags ---- 20121220-121212 -- something_else --- trunk --- tags ---- 20121220-121212 We then have a utility (link_puppet) that: * takes a module, tag and environment as command line options * Updates the svn sandbox in /opt/puppet/svn/modules/$module * links /opt/puppet/svn/modules/$module/tags/$tag to /etc/puppet/environments/$env/modules/$module It also validates the module before linking, logs who did what when. Pretty handy. — Mason Turner On Dec 20, 2012, at 8:36 AM, Benjamin Priestman <benjamin@miniverse.me.uk> wrote:> I''ve tried using https://github.com/azimux/externals (referenced in Version Control with Git) which kind of works, but is a bit buggy and I''ve yet to come up with an easily understood workflow. > > On Tuesday, 18 December 2012 17:59:48 UTC, Malte Janduda wrote: >> >> Hi folks, >> >> we are looking for a way to manage our puppet modules in a convenient way. At the moment we have about 30 modules which are versionized with git or mercurial. Most of them are self-made, some are from github. >> We thought it would be a good idea to use mercurial subrepositories to manage the versions of those modules. Well, it is not. Everytime you push your commits to the server all subrepositories are checked for updates. It takes minutes to push your changes! And if you have to merge something you do not really know what happens. You just try every command which appears in your mind - multiple times and in different order :D >> It happened three times that every developer had to delete the whole module tree and check it out again because something was broken. >> git submodules do not seem to be fun either. >> >> I have looked for an alternative to submodules/subrepositories and found librarian-puppet (https://github.com/rodjek/librarian-puppet). Unfortunately it does not really work good for me. A lot of useless error messages if something is configured wrong. This is pretty important for us. All developers should be able to use the versioning tool. It has to be easy to use. Further the librarian-puppet does not seem to be backed by a healthy community. Just every few months a commit and a lot of open issues and pull-requests. >> >> I have hacked some Python code to realize something similar: dyps - https://github.com/MalteJ/dyps >> You have a yaml file which defines your modules: name, source, version and DVCS system (git/hg): >> >> - dep: mysql >> source: https://github.com/puppetlabs/puppetlabs-mysql.git >> type: git >> version: af4b8bff0ad59f21211be2a98d31588066c41e0c >> >> All modules are downloaded to the folder "dependencies" when executing "dyps run". >> You can decide if you want to get a specific changeset at the next run or if you want to get the latest changes - just discard the version property or execute "dyps release your-module-name". If you want to keep the current version execute "dyps keep your-module". You can update one or all versionized modules via "dyps update your-module" or "dyps update-all". The modules without a defined versioned are updated at every dyps run. >> >> I have added an import function for mercurial subrepository .hgsub files. Feel free to extend it with an importGitSubmodules function ;-) >> >> >> What do you think about this approach? >> How are you managing your modules/dependencies? >> >> Best, >> Malte > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/VZjQExuTnvAJ. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Nikola Petrov
2012-Dec-24 12:16 UTC
Re: [Puppet Users] Re: How to manage puppet modules? git submodules? hg subrepositories? own solution?
On Thu, Dec 20, 2012 at 01:28:04PM -0500, Mason Turner wrote:> We are still using svn, but looking on adapt this to git next year. > > This is a simplified view of our svn repo: > > - nodes > -- dev > --- trunk > --- tags > ---- 20121220-121212 > -- prod > --- trunk > --- tags > - modules > -- apache > --- trunk > --- tags > ---- 20121220-121212 > -- something_else > --- trunk > --- tags > ---- 20121220-121212 > > We then have a utility (link_puppet) that:Can you please post the link_puppet utility. I will be happy if we can burrow some code and not start from scratch on almost the same setup(we are doing it manually at the moment) Best, Nikola -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.