Hello, The attached graphs are from a server running FreeBSD 7.1-i386 (now) with the typical Apache2+MySQL with forums, Joomla... I just cannot explain this. Disk I/O bandwidth was suffering a lot, and after the update the disks are almost idle. Any ideas? I cannot imagine a change between versions to justify this. The latest update to 7-STABLE had been done in August. Maybe it was just degradation due to not being rebooted since August? But it's strange anyway, and I had restarted MySQL and Apache some times during this period. -------------- next part -------------- -------------- next part --------------
On Jan 30, 2009, at 10:12 AM, Borja Marcos wrote:> > Hello, > > The attached graphs are from a server running FreeBSD 7.1-i386 (now) > with the typical Apache2+MySQL with forums, Joomla...I see that the attachments didn't make it. Disk I/O bandwidth was an average 40 - 60 % before the update and reboot. After the update and reboot it's now less than 10 %. Most of the I/O activity was on the /var filesystem. It houses the MySQL database and system logs, mostly. So, either: 1- there's an important improvement on file caching from 7.0-STABLE- August to 7.1-RELEASE? 2- Something was rotten since the last reboot in August 3- Something was malfunctioning in the "amr" array controller? (It's a Dell server, and it has two hard disks in mirror) Apache and MySQL had been restarted some times last week, trying to tune some parameters and it made no difference. Any ideas? It's quite weird. Same kernel configuration, same hardware... Borja.
On Fri, 30 Jan 2009, Borja Marcos wrote:> The attached graphs are from a server running FreeBSD 7.1-i386 (now) with > the typical Apache2+MySQL with forums, Joomla... > > I just cannot explain this. Disk I/O bandwidth was suffering a lot, and > after the update the disks are almost idle. > > Any ideas? I cannot imagine a change between versions to justify this. The > latest update to 7-STABLE had been done in August. > > Maybe it was just degradation due to not being rebooted since August? But > it's strange anyway, and I had restarted MySQL and Apache some times during > this period.There are basically three ways to go about exploring this, none particularly good: (1) Do a more formal before and after analysis of performance on the box, perhaps using tools like kernel profiling, hwpmc, dtrace, etc. (2) Do a binary search to narrow down the date of the change that improved things until it becomes clear which mattered. (3) Hope someone annecdotally remembers something that might or might not be it and assume they're right. Of these, I'd guess (2) is actually the most effective way to go about it, but is potentially time-consuming. As you point out, the most interesting question is whether, when you go back to 7.0, things suddenly get slower again, or not. Typically long uptimes don't lead to performance problems on FreeBSD (in my experience) so I think that's unlikely to be the source. There are a lot of improvements in 7.1 relating to performance, but none particularly stands out for me as having the effect you describe. If you're really curious, I would try to pin it down with a binary search. Robert N M Watson Computer Laboratory University of Cambridge
On Jan 31, 2009, at 7:27 PM, Robert Watson wrote:> There are basically three ways to go about exploring this, none > particularly good: > > (1) Do a more formal before and after analysis of performance on the > box, > perhaps using tools like kernel profiling, hwpmc, dtrace, etc.Machine in production, I cannot do it :(> (2) Do a binary search to narrow down the date of the change that > improved > things until it becomes clear which mattered.> (3) Hope someone annecdotally remembers something that might or > might not be > it and assume they're right. > > Of these, I'd guess (2) is actually the most effective way to go > about it, but is potentially time-consuming. As you point out, the > most interesting question is whether, when you go back to 7.0, > things suddenly get slower again, or not. Typically long uptimes > don't lead to performance problems on FreeBSD (in my experience) so > I think that's unlikely to be the source. There are a lot of > improvements in 7.1 relating to performance, but none particularly > stands out for me as having the effect you describe. If you're > really curious, I would try to pin it down with a binary search.I will have to learn how to use dtrace, I think. This is quite weird. And in a lot of years I haven't seen a FreeBSD system degraded because of a long uptime. Something in userland must be the culprit... As I see (I don't administer the machine but co-administer it) there's a Qmail system with some AV crap... and now I see that active memory had gone up, and is much lower after the update. I'll keep investigating.. the kind of answer I was looking for was a "oh, yes, there was a problem that degraded, blah, blah, blah". The graphs can be accessed here: http://194.30.110.21/orca/ It's behind an ADSL, so expect slow performance. Borja.