Hello, I have another quick one about puppet dashboard. We are using the Debian one for Squeeze (1.0.4) and as the database has significantly grown, if/when we click on a list of unresponsive nodes to check them, the web interface sits there doing nothing then the puppetmaster process starts timing out for all nodes compiling their catalogs. Has anyone got a workaround for this? Does it involve cleaning the database from time to time? The box is a VM with 3G of allocated RAM, our ibdata1 file is pretty big, reached 8G recently. So what do others do? Cheers, Steph -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Mon, 4 Apr 2011 11:33:54 +0100 FRLinux FRLinux wrote:> Hello,Hi,> I have another quick one about puppet dashboard. We are using the > Debian one for Squeeze (1.0.4) and as the database has significantly > grown, if/when we click on a list of unresponsive nodes to check them, > the web interface sits there doing nothing then the puppetmaster > process starts timing out for all nodes compiling their catalogs. > > Has anyone got a workaround for this? Does it involve cleaning the > database from time to time?We added a cron which does: rake RAILS_ENV=production reports:prune upto=1 unit=wk rake RAILS_ENV=production db:raw:optimize from /opt/puppet-dashboard once per week. since then, the server has increased its performance significally and we see no more timeout errors.> The box is a VM with 3G of allocated RAM, our ibdata1 file is pretty > big, reached 8G recently. > > So what do others do? > > Cheers, > StephHTH, Arnau -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Apr 4, 2011, at 3:33 AM, FRLinux wrote:> Hello, > > I have another quick one about puppet dashboard. We are using the > Debian one for Squeeze (1.0.4) and as the database has significantly > grown, if/when we click on a list of unresponsive nodes to check them, > the web interface sits there doing nothing then the puppetmaster > process starts timing out for all nodes compiling their catalogs. > > Has anyone got a workaround for this? Does it involve cleaning the > database from time to time? > > The box is a VM with 3G of allocated RAM, our ibdata1 file is pretty > big, reached 8G recently. > > So what do others do?I run: cd /usr/share/puppet-dashboard/ nice -n +1 rake RAILS_ENV=production reports:prune upto=1 unit=mon && nice -n +1 rake RAILS_ENV=production db:raw:optimize This removes all entires older than a month, which keeps it small enough for me. My machine is a P4 with 1GB of RAM, and serves 80 nodes. Note that with MySQL, the size of the ibdata1 file won''t actually shrink. The explanation is complicated, and can be found using google. If you want to get the space back, an easy way is to wipe the whole database. Put this into your config: [mysqld] innodb_file_per_table=true Then the db:raw:optimize command will actually be able to shrink the DB. Note, that attribute will cause all tables to be stored in separate files. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Mon, Apr 4, 2011 at 11:53 AM, Patrick <kc7zzv@gmail.com> wrote:> Then the db:raw:optimize command will actually be able to shrink the DB. > Note, that attribute will cause all tables to be stored in separate files.Nice one, thanks for all the replies, I have a responsive dashboard once again :) Steph -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Apr 4, 2011, at 4:03 AM, FRLinux wrote:> On Mon, Apr 4, 2011 at 11:53 AM, Patrick <kc7zzv@gmail.com> wrote: >> Then the db:raw:optimize command will actually be able to shrink the DB. >> Note, that attribute will cause all tables to be stored in separate files. > > Nice one, thanks for all the replies, I have a responsive dashboard > once again :)Steph & Patrick ~ We''re aware of some performance problems in Dashboard and are keen to fix them. How much data do you have? How many nodes? How many unresponsive? What''s your update interval? At what point does it start to fall over? Thank you! r -- Randall Hansen • Director of User Experience • randall@puppetlabs.com -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Apr 4, 2011, at 8:26 AM, Randall Hansen wrote:> On Apr 4, 2011, at 4:03 AM, FRLinux wrote: > >> On Mon, Apr 4, 2011 at 11:53 AM, Patrick <kc7zzv@gmail.com> wrote: >>> Then the db:raw:optimize command will actually be able to shrink the DB. >>> Note, that attribute will cause all tables to be stored in separate files. >> >> Nice one, thanks for all the replies, I have a responsive dashboard >> once again :) > > Steph & Patrick ~ > > We''re aware of some performance problems in Dashboard and are keen to fix them. How much data do you have? How many nodes? How many unresponsive? What''s your update interval? At what point does it start to fall over?I''m using Passenger for the puppetmaster+storedconfigs and for dashbaord. processor0 => Intel(R) Celeron(R) CPU 2.53GHz I''ve currently got 77147 reports I''ve got about 80 hosts. runinterval = 1800 I found that upgrading to dashboard 1.0.4 gave me huge performance gains. I found that turning on a 24MB query cache in mysqld gave me some moderate performance gains too. I assume this is because the results for the sidebar are cached, but this is mostly baseless speculation. The load time on the first page is usually around 2-2.5 seconds. Any page after that takes around 1.5 seconds to load the first time, and about 0.7 seconds if called again "soon". I don''t know what "soon" is except that it can be longer than 5 seconds, and doesn''t apply a day later. The query cache might mean some data is out of date. I don''t know how mysql works, but I don''t need realtime updates. Just reports on the last week. I always found that using Webrick was often painfully slow. I''m guessing that this is because I either got "suck behind" a puppetmaster uploading its report, or that I wasn''t able to load all the little files (images, etc.) very fast. I am using RubyEE. My reports.ibd is 908MB. Anything else that would help? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Mon, Apr 4, 2011 at 4:26 PM, Randall Hansen <randall@puppetlabs.com> wrote:> We''re aware of some performance problems in Dashboard and are keen to fix them. How much data do you have? How many nodes? How many unresponsive? What''s your update interval? At what point does it start to fall over?Hello! As Patrick, upgraded to 1.0.4 a few months back and loved it, much faster and better. I had some performance issue lately where i could get to the front page but no longer look at any report/non reporting node. The server would then be stuck and would start timing out client nodes. I can remove the cron and take a look at it week after week until the point where it fails again. I''ll see if i can restore the mysql server from backups (should be a few days old) then will mimic the behavior again if that helps. Have 80 nodes reporting which is not much. Wiped out the whole DB when upgraded to 1.0.4, Think it was back in the beginning of November so it is same to assume that we''ve been logging since. Only 10 nodes were added in the last month (March). Any other info I can provide, i can take a look tomorrow at work so let me know :) Cheers, Steph -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Apr 4, 2011, at 3:58 PM, FRLinux wrote:> Have 80 nodes reporting which is not much. Wiped out the whole DB when > upgraded to 1.0.4, Think it was back in the beginning of November so > it is same to assume that we''ve been logging since. Only 10 nodes were > added in the last month (March).So about 70 nodes over about 5 months slowed Dashboard to the point of uselessness. That sucks; I''m sorry. What''s your update interval? We have a tool (coming in the next release) which will generate reports and populate Dashboard with them. I''ll guess a 30-minute update interval and crank out a bunch of fake reports to see where it starts to fall over. r -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.