Puppet users - Jul 2013 - Puppet, Yum, Cassandra, openjdk and --nodeps

Hello, puppet n00b here.

Trying to install cassandra via puppet. Works great, only caveat is 
cassandra (dsc12 package) lists openjdk as a dependency. Ironically enough, 
the datastax guys themselves recommend using Oracle JRE instead of openjdk, 
and there is even a bug
<https://bugzilla.redhat.com/show_bug.cgi?id=907485>that prevents
cassandra from starting if it''s using openjdk. Anyway, I
fixed it by adding an exec to my puppet-java module that sets the Oracle 
JRE runtime as the defaults via alternatives, and it works fine. However, I 
still end up with two different java runtimes installed which I find to be 
a bit unclean.

The root of the problem to me, though, is that by having puppet install 
dsc12, I lose visibility into what all those dependencies were that got 
installed along with it (I didn''t realize openjdk was even installing
until
I started investigating why cassandra wasn''t starting). So what
I''d prefer
to do is add each individual package dependency into my cassandra module 
itself, thereby explicitly installing only what I intend to install, and 
nothing else.

The only way this works, though, is if I can somehow pass the
"--nodeps"
option into yum during puppet apply time. Otherwise, regardless of whether 
I already installed Oracle JRE, using yum to install dsc12 will 
automatically install openjdk.



How have you guys handled scenarios like this? I tried searching through 
the topics here for "yum nodeps" but it seems people found different
ways
of solving their individual problems rather than sending flags to the yum 
provider itself. I also noticed this puppet feature
request<https://projects.puppetlabs.com/issues/4976>which unfortunately
has remained open(?) for 3 years. I''ve also seen people
suggest that nodeps should never be used with yum since the purpose of yum 
is to handle dependencies... but we also like some of the other features of 
yum, like being able to pull packages down by name automatically from our 
yum repo (which we manage in-house).

Anyway, any insights would be great! Thanks,
Paul

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

This is rather unfortunate. The only way that I can think of is to
change the deb/rpm package and serve that on your own. 

We are using cassandra on some deployment too but force JAVA_HOME
explicitly in configuration files to the oracle jdk


-- 
Nikola

On Wed, Jul 24, 2013 at 02:29:44PM -0700, Paul Pham
wrote:
> Hello, puppet n00b here.
> 
> Trying to install cassandra via puppet. Works great, only caveat is 
> cassandra (dsc12 package) lists openjdk as a dependency. Ironically enough,
> the datastax guys themselves recommend using Oracle JRE instead of openjdk,
> and there is even a bug
<https://bugzilla.redhat.com/show_bug.cgi?id=907485>that prevents
cassandra from starting if it''s using openjdk. Anyway, I
> fixed it by adding an exec to my puppet-java module that sets the Oracle 
> JRE runtime as the defaults via alternatives, and it works fine. However, I
> still end up with two different java runtimes installed which I find to be 
> a bit unclean.
> 
> The root of the problem to me, though, is that by having puppet install 
> dsc12, I lose visibility into what all those dependencies were that got 
> installed along with it (I didn''t realize openjdk was even
installing until
> I started investigating why cassandra wasn''t starting). So what
I''d prefer
> to do is add each individual package dependency into my cassandra module 
> itself, thereby explicitly installing only what I intend to install, and 
> nothing else.
> 
> The only way this works, though, is if I can somehow pass the
"--nodeps"
> option into yum during puppet apply time. Otherwise, regardless of whether 
> I already installed Oracle JRE, using yum to install dsc12 will 
> automatically install openjdk.
> 
> 
> 
> How have you guys handled scenarios like this? I tried searching through 
> the topics here for "yum nodeps" but it seems people found
different ways
> of solving their individual problems rather than sending flags to the yum 
> provider itself. I also noticed this puppet feature
request<https://projects.puppetlabs.com/issues/4976>which unfortunately
has remained open(?) for 3 years. I''ve also seen people
> suggest that nodeps should never be used with yum since the purpose of yum 
> is to handle dependencies... but we also like some of the other features of
> yum, like being able to pull packages down by name automatically from our 
> yum repo (which we manage in-house).
> 
> Anyway, any insights would be great! Thanks,
> Paul
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
email to puppet-users+unsubscribe@googlegroups.com.
> To post to this group, send email to puppet-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users.
> For more options, visit https://groups.google.com/groups/opt_out.
> 
> 
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On Wednesday, July 24, 2013 4:29:44 PM UTC-5, Paul Pham
wrote:
>
> Hello, puppet n00b here.
>
> Trying to install cassandra via puppet. Works great, only caveat is 
> cassandra (dsc12 package) lists openjdk as a dependency. Ironically enough,
> the datastax guys themselves recommend using Oracle JRE instead of openjdk,
> and there is even a
bug<https://bugzilla.redhat.com/show_bug.cgi?id=907485>that prevents
cassandra from starting if it''s using openjdk. Anyway, I
> fixed it by adding an exec to my puppet-java module that sets the Oracle 
> JRE runtime as the defaults via alternatives, and it works fine. However, I
> still end up with two different java runtimes installed which I find to be 
> a bit unclean.
>
> The root of the problem to me, though, is that by having puppet install 
> dsc12, I lose visibility into what all those dependencies were that got 
> installed along with it (I didn''t realize openjdk was even
installing until
> I started investigating why cassandra wasn''t starting). So what
I''d prefer
> to do is add each individual package dependency into my cassandra module 
> itself, thereby explicitly installing only what I intend to install, and 
> nothing else.
>
> The only way this works, though, is if I can somehow pass the
"--nodeps"
> option into yum during puppet apply time. Otherwise, regardless of whether 
> I already installed Oracle JRE, using yum to install dsc12 will 
> automatically install openjdk.
>
>
>
> How have you guys handled scenarios like this? I tried searching through 
> the topics here for "yum nodeps" but it seems people found
different ways
> of solving their individual problems rather than sending flags to the yum 
> provider itself. I also noticed this puppet feature
request<https://projects.puppetlabs.com/issues/4976>which unfortunately
has remained open(?) for 3 years. I''ve also seen people
> suggest that nodeps should never be used with yum since the purpose of yum 
> is to handle dependencies... but we also like some of the other features of
> yum, like being able to pull packages down by name automatically from our 
> yum repo (which we manage in-house).
>
> Anyway, any insights would be great! Thanks,
>

The best solution to the immediate problem would be to rebuild the RPM to 
correct the dependency information.  There are even tools available by 
which you could edit your existing RPM without rebuilding it from scratch.

As far as --nodeps goes, I am not aware that yum implements it, or anything 
similar.  I find explicit documentation to the contrary, in fact.  If you 
want to go that direction (very unwise) then you need to use rpm directly.

If you use rpm directly, however, (including by explicitly telling Puppet 
to use the ''rpm'' Package provider) then you probably still
don''t want
--nodeps.  Instead, you want rpm to fail when a package''s dependencies
are
not satisfied, so that you can fix the problem.  This is the route to take 
if you want to explicitly specify every package and avoid any others being 
installed.  You do not then get automatic package downloads, but rpm 
understands http: URLs, so you can still pull packages from your local 
repository (or external ones) by specifying their complete URLs.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Appreciate the input guys. You''re right about the --nodeps flag being
an
RPM thing, I was hoping there might be something similar I could leverage 
from the yum provider, but it looks like the yum provider does not 
currently support "install_options" anyway, so it''s kind of
moot.

It seems like there''s a bug/feature request in here somewhere, but I
still
can''t put my finger on it. I''m talking about Java, but in
general if you
have some type of runtime that has multiple vendors/providers, and
you''re
using Alternatives to configure your default provider for that type of 
runtime, how does/should puppet handle it? Or should I not be relying on 
alternatives to set defaults? Is the main problem that the cassandra RPM 
requires openjdk, or is that standard practice (and I should learn to deal 
with it?)

Do most folks who use puppet not use yum? Or are most people okay with 
letting puppet install dependencies automagically?

Paul

On Monday, July 29, 2013 10:58:11 AM UTC-7, jcbollinger
wrote:
>
>
>
> On Wednesday, July 24, 2013 4:29:44 PM UTC-5, Paul Pham wrote:
>>
>> Hello, puppet n00b here.
>>
>> Trying to install cassandra via puppet. Works great, only caveat is 
>> cassandra (dsc12 package) lists openjdk as a dependency. Ironically
enough,
>> the datastax guys themselves recommend using Oracle JRE instead of
openjdk,
>> and there is even a
bug<https://bugzilla.redhat.com/show_bug.cgi?id=907485>that prevents
cassandra from starting if it''s using openjdk. Anyway, I
>> fixed it by adding an exec to my puppet-java module that sets the
Oracle
>> JRE runtime as the defaults via alternatives, and it works fine.
However, I
>> still end up with two different java runtimes installed which I find to
be
>> a bit unclean.
>>
>> The root of the problem to me, though, is that by having puppet install
>> dsc12, I lose visibility into what all those dependencies were that got
>> installed along with it (I didn''t realize openjdk was even
installing until
>> I started investigating why cassandra wasn''t starting). So
what I''d prefer
>> to do is add each individual package dependency into my cassandra
module
>> itself, thereby explicitly installing only what I intend to install,
and
>> nothing else.
>>
>> The only way this works, though, is if I can somehow pass the
"--nodeps"
>> option into yum during puppet apply time. Otherwise, regardless of
whether
>> I already installed Oracle JRE, using yum to install dsc12 will 
>> automatically install openjdk.
>>
>>
>>
>> How have you guys handled scenarios like this? I tried searching
through
>> the topics here for "yum nodeps" but it seems people found
different ways
>> of solving their individual problems rather than sending flags to the
yum
>> provider itself. I also noticed this puppet feature
request<https://projects.puppetlabs.com/issues/4976>which unfortunately
has remained open(?) for 3 years. I''ve also seen people
>> suggest that nodeps should never be used with yum since the purpose of
yum
>> is to handle dependencies... but we also like some of the other
features of
>> yum, like being able to pull packages down by name automatically from
our
>> yum repo (which we manage in-house).
>>
>> Anyway, any insights would be great! Thanks,
>>
>
>
> The best solution to the immediate problem would be to rebuild the RPM to 
> correct the dependency information.  There are even tools available by 
> which you could edit your existing RPM without rebuilding it from scratch.
>
> As far as --nodeps goes, I am not aware that yum implements it, or 
> anything similar.  I find explicit documentation to the contrary, in fact.
> If you want to go that direction (very unwise) then you need to use rpm 
> directly.
>
> If you use rpm directly, however, (including by explicitly telling Puppet 
> to use the ''rpm'' Package provider) then you probably
still don''t want
> --nodeps.  Instead, you want rpm to fail when a package''s
dependencies are
> not satisfied, so that you can fix the problem.  This is the route to take 
> if you want to explicitly specify every package and avoid any others being 
> installed.  You do not then get automatic package downloads, but rpm 
> understands http: URLs, so you can still pull packages from your local 
> repository (or external ones) by specifying their complete URLs.
>
>
> John
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Thanks for this suggestion btw Nikola, this might be the most practical and 
easiest way to solve this specific problem. I''m just also curious about
how
folks are solving the general case, as well.

On Friday, July 26, 2013 12:50:03 AM UTC-7, nikolavp
wrote:
>
> This is rather unfortunate. The only way that I can think of is to 
> change the deb/rpm package and serve that on your own. 
>
> We are using cassandra on some deployment too but force JAVA_HOME 
> explicitly in configuration files to the oracle jdk 
>
>
> -- 
> Nikola 
>
> On Wed, Jul 24, 2013 at 02:29:44PM -0700, Paul Pham wrote: 
> > Hello, puppet n00b here. 
> > 
> > Trying to install cassandra via puppet. Works great, only caveat is 
> > cassandra (dsc12 package) lists openjdk as a dependency. Ironically 
> enough, 
> > the datastax guys themselves recommend using Oracle JRE instead of 
> openjdk, 
> > and there is even a bug <
> https://bugzilla.redhat.com/show_bug.cgi?id=907485>that prevents 
> cassandra from starting if it''s using openjdk. Anyway, I 
> > fixed it by adding an exec to my puppet-java module that sets the
Oracle
> > JRE runtime as the defaults via alternatives, and it works fine. 
> However, I 
> > still end up with two different java runtimes installed which I find
to
> be 
> > a bit unclean. 
> > 
> > The root of the problem to me, though, is that by having puppet
install
> > dsc12, I lose visibility into what all those dependencies were that
got
> > installed along with it (I didn''t realize openjdk was even
installing
> until 
> > I started investigating why cassandra wasn''t starting). So
what I''d
> prefer 
> > to do is add each individual package dependency into my cassandra
module
> > itself, thereby explicitly installing only what I intend to install,
and
> > nothing else. 
> > 
> > The only way this works, though, is if I can somehow pass the
"--nodeps"
> > option into yum during puppet apply time. Otherwise, regardless of 
> whether 
> > I already installed Oracle JRE, using yum to install dsc12 will 
> > automatically install openjdk. 
> > 
> > 
> > 
> > How have you guys handled scenarios like this? I tried searching
through
> > the topics here for "yum nodeps" but it seems people found
different
> ways 
> > of solving their individual problems rather than sending flags to the 
> yum 
> > provider itself. I also noticed this puppet feature request<
> https://projects.puppetlabs.com/issues/4976>which unfortunately has 
> remained open(?) for 3 years. I''ve also seen people 
> > suggest that nodeps should never be used with yum since the purpose of
> yum 
> > is to handle dependencies... but we also like some of the other
features
> of 
> > yum, like being able to pull packages down by name automatically from 
> our 
> > yum repo (which we manage in-house). 
> > 
> > Anyway, any insights would be great! Thanks, 
> > Paul 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "Puppet Users" group. 
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com <javascript:>. 
> > To post to this group, send email to
puppet...@googlegroups.com<javascript:>.
>
> > Visit this group at http://groups.google.com/group/puppet-users. 
> > For more options, visit https://groups.google.com/groups/opt_out. 
> > 
> > 
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On 30 Jul 2013 18:57, "Paul Pham" <me@paulpham.net>
wrote:
> Do most folks who use puppet not use yum? Or are most people okay with
letting puppet install dependencies automagically?

Puppet has different package providers and I use the appropriate one
depending on what I want it to do.

If I want Puppet to just install a package and fail if it''s
dependencies
are not met I use the rpm provider and specify the location of the rpm file.

If I want Puppet to install a package with all its dependencies then I use
the yum provider (the default) usually with a local repository so I can
make local changes - try fpm, pulp and similar tools.

Of course there are similar providers for non-RHEL systems.
> Paul
  - Keith
>
> On Monday, July 29, 2013 10:58:11 AM UTC-7, jcbollinger wrote:
>>
>>
>>
>> On Wednesday, July 24, 2013 4:29:44 PM UTC-5, Paul Pham wrote:
>>>
>>> Hello, puppet n00b here.
>>>
>>> Trying to install cassandra via puppet. Works great, only caveat is
cassandra (dsc12 package) lists openjdk as a dependency. Ironically enough,
the datastax guys themselves recommend using Oracle JRE instead of openjdk,
and there is even a bug that prevents cassandra from starting if it''s
using
openjdk. Anyway, I fixed it by adding an exec to my puppet-java module that
sets the Oracle JRE runtime as the defaults via alternatives, and it works
fine. However, I still end up with two different java runtimes installed
which I find to be a bit unclean.
>>>
>>> The root of the problem to me, though, is that by having puppet
install
dsc12, I lose visibility into what all those dependencies were that got
installed along with it (I didn''t realize openjdk was even installing
until
I started investigating why cassandra wasn''t starting). So what
I''d prefer
to do is add each individual package dependency into my cassandra module
itself, thereby explicitly installing only what I intend to install, and
nothing else.
>>>
>>> The only way this works, though, is if I can somehow pass the
"--nodeps" option into yum during puppet apply time. Otherwise,
regardless
of whether I already installed Oracle JRE, using yum to install dsc12 will
automatically install openjdk.
>>>
>>>
>>>
>>> How have you guys handled scenarios like this? I tried searching
through the topics here for "yum nodeps" but it seems people found
different ways of solving their individual problems rather than sending
flags to the yum provider itself. I also noticed this puppet feature
request which unfortunately has remained open(?) for 3 years. I''ve also
seen people suggest that nodeps should never be used with yum since the
purpose of yum is to handle dependencies... but we also like some of the
other features of yum, like being able to pull packages down by name
automatically from our yum repo (which we manage
in-house).
>>>
>>> Anyway, any insights would be great! Thanks,
>>
>>
>>
>> The best solution to the immediate problem would be to rebuild the RPM
to correct the dependency information.  There are even tools available by
which you could edit your existing RPM without rebuilding it from
scratch.
>>
>> As far as --nodeps goes, I am not aware that yum implements it, or
anything similar.  I find explicit documentation to the contrary, in fact.
If you want to go that direction (very unwise) then you need to use rpm
directly.
>>
>> If you use rpm directly, however, (including by explicitly telling
Puppet to use the ''rpm'' Package provider) then you probably
still don''t
want --nodeps.  Instead, you want rpm to fail when a package''s
dependencies
are not satisfied, so that you can fix the problem.  This is the route to
take if you want to explicitly specify every package and avoid any others
being installed.  You do not then get automatic package downloads, but rpm
understands http: URLs, so you can still pull packages from your local
repository (or external ones) by specifying their complete
URLs.
>>
>>
>> John
>>
> --
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
email to puppet-users+unsubscribe@googlegroups.com.
> To post to this group, send email to puppet-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On Tuesday, July 30, 2013 12:56:56 PM UTC-5, Paul Pham
wrote:
>
> Appreciate the input guys. You''re right about the --nodeps flag
being an
> RPM thing, I was hoping there might be something similar I could leverage 
> from the yum provider, but it looks like the yum provider does not 
> currently support "install_options" anyway, so it''s kind
of moot.
>
> It seems like there''s a bug/feature request in here somewhere, but
I still
> can''t put my finger on it. I''m talking about Java, but in
general if you
> have some type of runtime that has multiple vendors/providers, and
you''re
> using Alternatives to configure your default provider for that type of 
> runtime, how does/should puppet handle it? Or should I not be relying on 
> alternatives to set defaults? Is the main problem that the cassandra RPM 
> requires openjdk, or is that standard practice (and I should learn to deal 
> with it?)
>
> Do most folks who use puppet not use yum? Or are most people okay with 
> letting puppet install dependencies automagically?
>
>
I suspect that most people who use Puppet with yum-based clients let Puppet 
install dependencies automagically via yum, at least in most cases.  I 
certainly do.  If you need finer control then that''s what the
''rpm'' command
and Puppet''s direct rpm provider are for.

I exercise a measure of control by configuring yum to use local mirrors of 
most of the repositories we rely on.  In principle, that affords me the 
opportunity to audit all packages available for installation, but in 
practice I do very little of that.  It also allows me to control when 
package updates become available for installation, which I make much less 
frequent than the upstream providers do.  Additionally, it allows me to 
pick and choose what versions of what packages from upstream are available 
at all, but I exercise that capability only very rarely.

The careful among us may go so far as to maintain test environments that 
encompass repository updates in addition to Puppet manifest and data 
updates.

I think you should give a bit of thought to *why* you are uncomfortable 
with using yum via Puppet (and that''s really what you''re
saying, since yum
doesn''t offer any form of nodeps operation, Puppet notwithstanding). 
There
may be a way to mitigate your concerns without greatly complicating your 
Puppet manifests by explicitly declaring all dependencies, recursively, of 
every package you want Puppet to manage.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.