Stefan Kaes
2005-Jul-15 11:32 UTC
[Rails-core] corrected performance numbers for 0.13, trunk and perf branch
I rushed it a bit to produce the new performance data for 13.0 and so didn''t notice 2 mistakes that crept into the tests. One of them was caused by the fact that the new ActiveRecord session class requires one to call save! instead of save and also needs one access to it''s data to make it store correctly in the database. As a result, most tested page only produced "Invalid access", which is naturally pretty fast. I have remeasured 0.13 and the current trunk. Results: 0.12.1 compared to 0.13 : configuration 1: r121.06-28 requests=1000, optionsconfiguration 2: r130.07-15.gc100 requests=1000, options=-r130 -gc100 page c1 real c2 real c1 r/s c2 r/s c1/c2 /empty/index 6.99982 4.71259 142.9 212.2 1.49 /welcome/index 7.15241 4.57944 139.8 218.4 1.56 /rezept/index 5.16318 4.06296 193.7 246.1 1.27 /rezept/myknzlpzl 5.16549 4.07408 193.6 245.5 1.27 /rezept/show/713 19.36915 10.32029 51.6 96.9 1.88 /rezept/cat/Hauptspeise 23.04381 12.04498 43.4 83.0 1.91 /rezept/cat/Hauptspeise?page=5 23.42356 12.74783 42.7 78.4 1.84 /rezept/letter/G 21.94957 11.82389 45.6 84.6 1.86 So the improvement is not as big as posted in my previous mail. 0.13 compared to trunk: configuration 1: r130.07-15.gc100 requests=1000, options=-r130 -gc100 configuration 2: trunk.07-15.gc100 requests=1000, options=-trunk -gc100 page c1 real c2 real c1 r/s c2 r/s c1/c2 /empty/index 4.71259 4.84225 212.2 206.5 0.97 /welcome/index 4.57944 4.69578 218.4 213.0 0.98 /rezept/index 4.06296 4.14789 246.1 241.1 0.98 /rezept/myknzlpzl 4.07408 4.14633 245.5 241.2 0.98 /rezept/show/713 10.32029 10.45727 96.9 95.6 0.99 /rezept/cat/Hauptspeise 12.04498 10.90006 83.0 91.7 1.11 /rezept/cat/Hauptspeise?page=5 12.74783 11.18895 78.4 89.4 1.14 /rezept/letter/G 11.82389 11.22321 84.6 89.1 1.05 Which shows that the pagination intensive pages have improved quite a bit over 0.13. trunk compared to perf branch: configuration 1: trunk.07-15.gc100 requests=1000, options=-trunk -gc100 configuration 2: svn.07-15.gc100 requests=1000, options=-svn -gc100 page c1 real c2 real c1 r/s c2 r/s c1/c2 /empty/index 4.84225 1.61744 206.5 618.3 2.99 /welcome/index 4.69578 1.78653 213.0 559.7 2.63 /rezept/index 4.14789 1.82140 241.1 549.0 2.28 /rezept/myknzlpzl 4.14633 1.81707 241.2 550.3 2.28 /rezept/show/713 10.45727 4.36503 95.6 229.1 2.40 /rezept/cat/Hauptspeise 10.90006 4.76995 91.7 209.6 2.29 /rezept/cat/Hauptspeise?page=5 11.18895 4.82623 89.4 207.2 2.32 /rezept/letter/G 11.22321 4.81651 89.1 207.6 2.33 Thsi comparison is actually a bit unfair, as trunk uses ActiveRecord for session storage. Since ActiveRecordStore is not a easily adaptable as I had hoped, I used my own session storage class. configuration 1: trunk.07-15.gc100 requests=1000, options=-trunk -gc100 configuration 2: trunk.07-15.mysql_session.gc100 requests=1000, options=-trunk -mysql_session -gc100 page c1 real c2 real c1 r/s c2 r/s c1/c2 /empty/index 4.84225 2.49370 206.5 401.0 1.94 /welcome/index 4.69578 2.38781 213.0 418.8 1.97 /rezept/index 4.14789 2.32662 241.1 429.8 1.78 /rezept/myknzlpzl 4.14633 2.32979 241.2 429.2 1.78 /rezept/show/713 10.45727 8.68052 95.6 115.2 1.20 /rezept/cat/Hauptspeise 10.90006 9.13140 91.7 109.5 1.19 /rezept/cat/Hauptspeise?page=5 11.18895 9.27107 89.4 107.9 1.21 /rezept/letter/G 11.22321 9.22866 89.1 108.4 1.22 This shows how much can be gained by a hand crafted session storage class. Now we can compare the perf branch and trunk: configuration 1: trunk.07-15.mysql_session.gc100 requests=1000, options=-trunk -mysql_session -gc100 configuration 2: svn.07-15.gc100 requests=1000, options=-svn -gc100 page c1 real c2 real c1 r/s c2 r/s c1/c2 /empty/index 2.49370 1.61744 401.0 618.3 1.54 /welcome/index 2.38781 1.78653 418.8 559.7 1.34 /rezept/index 2.32662 1.82140 429.8 549.0 1.28 /rezept/myknzlpzl 2.32979 1.81707 429.2 550.3 1.28 /rezept/show/713 8.68052 4.36503 115.2 229.1 1.99 /rezept/cat/Hauptspeise 9.13140 4.76995 109.5 209.6 1.91 /rezept/cat/Hauptspeise?page=5 9.27107 4.82623 107.9 207.2 1.92 /rezept/letter/G 9.22866 4.81651 108.4 207.6 1.92 Using the patched GC with 500K heap slots gives the folowing results: configuration 1: trunk.07-15.mysql_session.serv requests=1000, options=-trunk -mysql_session configuration 2: svn.07-15.serv requests=1000, options=-svn page c1 real c2 real c1 r/s c2 r/s c1/c2 /empty/index 2.23567 1.34449 447.3 743.8 1.66 /welcome/index 2.08676 1.53648 479.2 650.8 1.36 /rezept/index 2.11274 1.50949 473.3 662.5 1.40 /rezept/myknzlpzl 2.11316 1.50729 473.2 663.4 1.40 /rezept/show/713 8.73529 4.12088 114.5 242.7 2.12 /rezept/cat/Hauptspeise 9.39169 4.66484 106.5 214.4 2.01 /rezept/cat/Hauptspeise?page=5 9.57409 4.71551 104.4 212.1 2.03 /rezept/letter/G 9.53702 4.69499 104.9 213.0 2.03 Which shows that there is still ample room for improvement. -- stefan
David Heinemeier Hansson
2005-Jul-15 18:52 UTC
[Rails-core] corrected performance numbers for 0.13, trunk and perf branch
> Thsi comparison is actually a bit unfair, as trunk uses ActiveRecord for > session storage. Since ActiveRecordStore is not a easily adaptable as I > had hoped, I used my own session storage class.Could we not include a default custom class for each of the major databases? Create one for MySQL, PG, and so on that goes straight to the metal. It definitely seems like there''s significant gain to be had. And with custom classes that were auto-configured depending on the adapter used, it should just be "free performance" for all.> Which shows that there is still ample room for improvement.Great numbers. It would be awesome if each of the new patches could have a profile like this attached to it. I''m really looking forward to seeing the new patch of patches and the opportunity of shipping custom session classes by default. -- David Heinemeier Hansson http://www.loudthinking.com -- Broadcasting Brain http://www.basecamphq.com -- Online project management http://www.backpackit.com -- Personal information manager http://www.rubyonrails.com -- Web-application framework
Stefan Kaes
2005-Jul-19 03:50 UTC
[Rails-core] corrected performance numbers for 0.13, trunk and perf branch
David Heinemeier Hansson wrote:>>Thsi comparison is actually a bit unfair, as trunk uses ActiveRecord for >>session storage. Since ActiveRecordStore is not a easily adaptable as I >>had hoped, I used my own session storage class. >> >> > >Could we not include a default custom class for each of the major >databases? Create one for MySQL, PG, and so on that goes straight to >the metal. It definitely seems like there''s significant gain to be >had. And with custom classes that were auto-configured depending on >the adapter used, it should just be "free performance" for all. > > > >>Which shows that there is still ample room for improvement. >> >> > >Great numbers. It would be awesome if each of the new patches could >have a profile like this attached to it. I''m really looking forward to >seeing the new patch of patches and the opportunity of shipping custom >session classes by default. > >I have had some time now to experiment with the SqlBypass class in ActiveRecordStore. It turned out that is wasn''t as difficult to modify as I thought initially. I tried 2 mods that both support created_on and updated_on, which is what I use in my app. One was coded in the original SqlBypass style (supporting all Dbs), the other coded by dropping fingerprinting and using MySql syntax (see attached code). configuration 1: 07-19.sqlbypass requests=1000, options=-svn configuration 2: 07-19.sqlbypass.mysql requests=1000, options=-svn page c1 real c2 real c1 r/s c2 r/s c1/c2 /empty/index 1.42047 1.35615 704.0 737.4 1.05 /welcome/index 1.60013 1.54894 624.9 645.6 1.03 /rezept/index 1.63891 1.52768 610.2 654.6 1.07 /rezept/myknzlpzl 1.62170 1.51340 616.6 660.8 1.07 /rezept/show/713 4.30114 4.17546 232.5 239.5 1.03 /rezept/cat/Hauptspeise 4.81616 4.60559 207.6 217.1 1.05 /rezept/cat/Hauptspeise?page=5 4.78235 4.74832 209.1 210.6 1.01 /rezept/letter/G 4.83253 4.71175 206.9 212.2 1.03 The second version is faster, but maybe not so much faster that it would warrant coding up special versions for all adapters. Instead, I would suggest to extract the sql code for update, insert into separate functions. That way modification of SqlBypass becomes much easier, avoiding the need to rewrite the save function. Regards, Stefan -------------- next part -------------- A non-text attachment was scrubbed... Name: sql_bypass.rb Type: application/x-ruby Size: 1705 bytes Desc: not available Url : http://wrath.rubyonrails.org/pipermail/rails-core/attachments/20050719/b8dd9cd7/sql_bypass.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: sql_bypass.mysql.rb Type: application/x-ruby Size: 1287 bytes Desc: not available Url : http://wrath.rubyonrails.org/pipermail/rails-core/attachments/20050719/b8dd9cd7/sql_bypass.mysql.bin