Gavin Williams
2013-Feb-15 12:08 UTC
[Puppet Users] Puppet & Automatic Resource State changing...
Morning all, Firstly, apologies for the length of this post, however I thought it probably most useful to fully outline the challenge and the desired result... Ok, so we''re in the process of Puppetizing our Oracle/NetApp platform for Live/DR running. The current manual process, upon setting up a new database, a set of volumes are created to contain the various Oracle DB elements, and these are then SnapMirror''d to the DR site. This SnapMirror process requires a period of time to copy the base data over... This time period is directly relational to the amount of data required... I.e. a copy of 20Gb may take an hour, 200Gb may take 10 hours... During this period, the SnapMirror resource is in an ''initializing'' state. Once the data copy is complete, then the resource will change to an ''initialized'' state. The next step in the process is then to break the relationship so that the DR end can be used in a R/W mode... Now, in order to Puppetize this, I need to be able to replicate the above behaviour... I''ve got Puppet to create and initialize the relationship, and that works as expected. However Puppet doesn''t currently care about the relationship state. Now that''s easy enough to add in as a new property against the type/provider. However what I''m struggling to understand is how, or if it''s even possible, to automate the switch from ''Initialized'' state to a ''Broken'' state upon completion of the initialization stage??? Now these databases definitions are currently driven from a YAML backend which maintains information such as database name, volume information, primary netapp controller, replication netapp controller, etc... Currently, this YAML file is just a file on the puppet master... However there are ambitions to move this into a more dynamic backend, such as CouchDB or similar... So that opens the possibility to automatically update the YAML resource state.. However Puppet still needs to be able to support updating that backend based on the information it gets from the actual resource... So to flow it out: 1. Create a new database in backend -> 2. Puppet creates volumes on primary -> 3. Data is added to volumes -> 4. Backend updated to indicate replication is required -> 5. Puppet creates volumes on Secondary and adds Snapmirror relationship -> 6. Snapmirror initializes in background -> 7. Puppet periodically runs against network device and checks resource state -> 8. Backend resource state is updated following each run? -> 9. Snapmirror initialization completes -> 10. Puppet runs, detects new resource state and then triggers break? 11. Backend resource state updated to ''broken''? Now 1 to 7 above are fine, but 8 to 11 are where I get a bit unsure... So, that''s the challenge... Am I barking up the wrong tree, or is this something that Puppet could manage? Cheers in advance for any responses. Regards Gavin -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Gavin Williams
2013-Feb-18 08:13 UTC
[Puppet Users] Re: Puppet & Automatic Resource State changing...
No-one doing anything similar? Cheers Gav On Friday, 15 February 2013 12:08:37 UTC, Gavin Williams wrote:> > Morning all, > > Firstly, apologies for the length of this post, however I thought it > probably most useful to fully outline the challenge and the desired > result... > > Ok, so we''re in the process of Puppetizing our Oracle/NetApp platform for > Live/DR running. > > The current manual process, upon setting up a new database, a set of > volumes are created to contain the various Oracle DB elements, and these > are then SnapMirror''d to the DR site. > This SnapMirror process requires a period of time to copy the base data > over... This time period is directly relational to the amount of data > required... I.e. a copy of 20Gb may take an hour, 200Gb may take 10 > hours... > During this period, the SnapMirror resource is in an ''initializing'' state. > Once the data copy is complete, then the resource will change to an > ''initialized'' state. > The next step in the process is then to break the relationship so that the > DR end can be used in a R/W mode... > > Now, in order to Puppetize this, I need to be able to replicate the above > behaviour... > I''ve got Puppet to create and initialize the relationship, and that works > as expected. However Puppet doesn''t currently care about the relationship > state. Now that''s easy enough to add in as a new property against the > type/provider. > However what I''m struggling to understand is how, or if it''s even > possible, to automate the switch from ''Initialized'' state to a ''Broken'' > state upon completion of the initialization stage??? > > Now these databases definitions are currently driven from a YAML backend > which maintains information such as database name, volume information, > primary netapp controller, replication netapp controller, etc... Currently, > this YAML file is just a file on the puppet master... However there are > ambitions to move this into a more dynamic backend, such as CouchDB or > similar... So that opens the possibility to automatically update the YAML > resource state.. However Puppet still needs to be able to support updating > that backend based on the information it gets from the actual resource... > > So to flow it out: > > 1. Create a new database in backend -> > 2. Puppet creates volumes on primary -> > 3. Data is added to volumes -> > 4. Backend updated to indicate replication is required -> > 5. Puppet creates volumes on Secondary and adds Snapmirror > relationship -> > 6. Snapmirror initializes in background -> > 7. Puppet periodically runs against network device and checks resource > state -> > 8. Backend resource state is updated following each run? -> > 9. Snapmirror initialization completes -> > 10. Puppet runs, detects new resource state and then triggers break? > 11. Backend resource state updated to ''broken''? > > Now 1 to 7 above are fine, but 8 to 11 are where I get a bit unsure... > > So, that''s the challenge... Am I barking up the wrong tree, or is this > something that Puppet could manage? > > Cheers in advance for any responses. > > Regards > Gavin >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
jcbollinger
2013-Feb-18 15:08 UTC
[Puppet Users] Re: Puppet & Automatic Resource State changing...
On Monday, February 18, 2013 2:13:26 AM UTC-6, Gavin Williams wrote:> > No-one doing anything similar? > >Puppet is all about managing nodes to a known-in-advance state. You can encapsulate stateful logic inside ''providers'' for custom or built-in resource types. You also might be able to model steps or groups of steps as a separate classes, provided that no node state set by one such class subsequently needs to be modified by another. Alternatively, you might need or want to use multiple Puppet runs to step through this process. Indeed, that may be necessary if you need the same resource (as Puppet sees it) to be managed to two or more different states during the course of the process. I don''t have any knowledge of the specific software stack you are configuring, so I''m afraid I can''t offer detailed suggestions. One way or another, Puppet can probably be made to perform this process for you, but that doesn''t necessarily mean it''s the best tool for the job. One alternative might be to script multiple puppet runs in series, each selecting different classes to apply via tags. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
fatmcgav
2013-Feb-18 15:15 UTC
Re: [Puppet Users] Re: Puppet & Automatic Resource State changing...
John Cheers again for the response... Commented in-line below... Cheers Gav On 18 February 2013 15:08, jcbollinger <John.Bollinger@stjude.org> wrote:> > > On Monday, February 18, 2013 2:13:26 AM UTC-6, Gavin Williams wrote: >> >> No-one doing anything similar? >> >> > Puppet is all about managing nodes to a known-in-advance state. You can > encapsulate stateful logic inside ''providers'' for custom or built-in > resource types. You also might be able to model steps or groups of steps > as a separate classes, provided that no node state set by one such class > subsequently needs to be modified by another. > > Alternatively, you might need or want to use multiple Puppet runs to step > through this process. Indeed, that may be necessary if you need the same > resource (as Puppet sees it) to be managed to two or more different states > during the course of the process. > >I was expecting the full state transition to take several Puppet runs, as the initial data replication to get from start to an ''Initialized'' state could take anything from a couple of minutes to a couple of days... So changing the required state of the resource in-between runs was an expected step - ideally it would be automated, but could be manual I guess... I don''t have any knowledge of the specific software stack you are> configuring, so I''m afraid I can''t offer detailed suggestions. One way or > another, Puppet can probably be made to perform this process for you, but > that doesn''t necessarily mean it''s the best tool for the job. One > alternative might be to script multiple puppet runs in series, each > selecting different classes to apply via tags. > > > John > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-users+unsubscribe@googlegroups.com. > To post to this group, send email to puppet-users@googlegroups.com. > Visit this group at http://groups.google.com/group/puppet-users?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
jcbollinger
2013-Feb-19 14:36 UTC
Re: [Puppet Users] Re: Puppet & Automatic Resource State changing...
On Monday, February 18, 2013 9:15:05 AM UTC-6, Gavin Williams wrote:> > > I was expecting the full state transition to take several Puppet runs, as > the initial data replication to get from start to an ''Initialized'' state > could take anything from a couple of minutes to a couple of days... > So changing the required state of the resource in-between runs was an > expected step - ideally it would be automated, but could be manual I > guess... > >That was not my point. You can have multiple classes that manage the same state detail in different ways. You just can''t include more than one of them in the same catalog. Since you were planning multiple runs anyway, that is probably not a major issue. At this point, however, I''m not sure what the question is. You asked "is this something Puppet could manage?" and the answer is "probably", especially considering that you''re open to doing it across several runs. If you''re looking for specific advice on how to describe steps 8-11 to Puppet then you''ll need to provide more information. How can any person or agent determine to what stage the process has progressed? What are the system properties to be managed, and by what tool and/or operation are they managed? John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.