Here''s a ticket for a simple patch to use Marshal instead of YAML for attribute serialization. Marshaling is significantly faster (see in link), and fixes some YAML load issues (including an outstanding ticket). http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/1191-marshal-serialized-attributes Simple enough? Stephen --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On 8 Oct 2008, at 17:07, Stephen Celis wrote:> > Here''s a ticket for a simple patch to use Marshal instead of YAML for > attribute serialization. Marshaling is significantly faster (see in > link), and fixes some YAML load issues (including an outstanding > ticket). > > http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/1191-marshal-serialized-attributes >While the option is fair enough, I don''t thing all existing apps wouldn''t want this turned on "silently": - if other people use your database yaml is ok as there are parsers for it in many languages whereas Marshal would be a PITA - if your existing column is not a blob column (which it wouldn''t have to be previously since yaml generates plain text), the database will throw a hissy fit (or just truncate the data) when you try to insert a character that is not legal in the charset used. - should you be calling string_to_binary if the column supports it ? Fred> Simple enough? > > Stephen > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
To add additional point for -1: Marshal uses a binary format and does not guarantee any compatibility with anything except the ruby on the computer which created it. Marshal uses internal format versioning numbers which do not correspond to ruby versions. This means your database backups are potentially non-portable to other OS/ruby/ computer version. This in itself makes this option a non-starter, imho. You could add the option, but it should never become a default. And even an option should come with a big warning. izidor On 8.10.2008, at 18:23, Frederick Cheung wrote:> > > On 8 Oct 2008, at 17:07, Stephen Celis wrote: > >> >> Here''s a ticket for a simple patch to use Marshal instead of YAML for >> attribute serialization. Marshaling is significantly faster (see in >> link), and fixes some YAML load issues (including an outstanding >> ticket). >> >> http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/1191-marshal-serialized-attributes >> > While the option is fair enough, I don''t thing all existing apps > wouldn''t want this turned on "silently": > - if other people use your database yaml is ok as there are parsers > for it in many languages whereas Marshal would be a PITA > - if your existing column is not a blob column (which it wouldn''t have > to be previously since yaml generates plain text), the database will > throw a hissy fit (or just truncate the data) when you try to insert a > character that is not legal in the charset used. > - should you be calling string_to_binary if the column supports it ? > > Fred >> Simple enough? >> >> Stephen >>> > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
Yes; after Fred''s post I did a bit more research and realize it''s a big no for portable apps. Not an impossible hurdle to deal with, but not a desirable default. I also realized that AR in its current state has no simple interface for such an option: quoting''s serialization is abstracted away from table and model information. While MySQL handled Marshal swimmingly in string columns, I don''t think other adapters would agree. I''ve tried to stay away from serialize where possible, but am dealing with it on a current project and noticed complaints from others regarding the speed at which large groups of objects with serialized attributes are instantiated from the database. YAML has been the culprit. Stephen On Oct 9, 2:45 am, Izidor Jerebic <ij.rubyl...@gmail.com> wrote:> To add additional point for -1: Marshal uses a binary format and does > not guarantee any compatibility with anything except the ruby on the > computer which created it. Marshal uses internal format versioning > numbers which do not correspond to ruby versions. This means your > database backups are potentially non-portable to other OS/ruby/ > computer version. This in itself makes this option a non-starter, imho. > > You could add the option, but it should never become a default. And > even an option should come with a big warning. > > izidor > > On 8.10.2008, at 18:23, Frederick Cheung wrote: > > > > > > > On 8 Oct 2008, at 17:07, Stephen Celis wrote: > > >> Here''s a ticket for a simple patch to use Marshal instead of YAML for > >> attribute serialization. Marshaling is significantly faster (see in > >> link), and fixes some YAML load issues (including an outstanding > >> ticket). > > >>http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/11... > > > While the option is fair enough, I don''t thing all existing apps > > wouldn''t want this turned on "silently": > > - if other people use your database yaml is ok as there are parsers > > for it in many languages whereas Marshal would be a PITA > > - if your existing column is not a blob column (which it wouldn''t have > > to be previously since yaml generates plain text), the database will > > throw a hissy fit (or just truncate the data) when you try to insert a > > character that is not legal in the charset used. > > - should you be calling string_to_binary if the column supports it ? > > > Fred > >> Simple enough? > > >> Stephen--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
There are also faster YAML dumpers out there. A recent Portland code sprint produced ZAML, a work-in-progress that offers (if I recall correctly) something like a 14x speed boost over vanilla YAML.dump http://github.com/hallettj/zaml/tree/master/zaml.rb YAML''s slowness has been a pain point, but we don''t have to sacrifice portability. On Thu, Oct 9, 2008 at 5:17 AM, Stephen Celis <stephen.celis@gmail.com>wrote:> > Yes; after Fred''s post I did a bit more research and realize it''s a > big no for portable apps. Not an impossible hurdle to deal with, but > not a desirable default. I also realized that AR in its current state > has no simple interface for such an option: quoting''s serialization is > abstracted away from table and model information. While MySQL handled > Marshal swimmingly in string columns, I don''t think other adapters > would agree. > > I''ve tried to stay away from serialize where possible, but am dealing > with it on a current project and noticed complaints from others > regarding the speed at which large groups of objects with serialized > attributes are instantiated from the database. YAML has been the > culprit. > > Stephen > > > On Oct 9, 2:45 am, Izidor Jerebic <ij.rubyl...@gmail.com> wrote: > > To add additional point for -1: Marshal uses a binary format and does > > not guarantee any compatibility with anything except the ruby on the > > computer which created it. Marshal uses internal format versioning > > numbers which do not correspond to ruby versions. This means your > > database backups are potentially non-portable to other OS/ruby/ > > computer version. This in itself makes this option a non-starter, imho. > > > > You could add the option, but it should never become a default. And > > even an option should come with a big warning. > > > > izidor > > > > On 8.10.2008, at 18:23, Frederick Cheung wrote: > > > > > > > > > > > > > On 8 Oct 2008, at 17:07, Stephen Celis wrote: > > > > >> Here''s a ticket for a simple patch to use Marshal instead of YAML for > > >> attribute serialization. Marshaling is significantly faster (see in > > >> link), and fixes some YAML load issues (including an outstanding > > >> ticket). > > > > >>http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/11. > .. > > > > > While the option is fair enough, I don''t thing all existing apps > > > wouldn''t want this turned on "silently": > > > - if other people use your database yaml is ok as there are parsers > > > for it in many languages whereas Marshal would be a PITA > > > - if your existing column is not a blob column (which it wouldn''t have > > > to be previously since yaml generates plain text), the database will > > > throw a hissy fit (or just truncate the data) when you try to insert a > > > character that is not legal in the charset used. > > > - should you be calling string_to_binary if the column supports it ? > > > > > Fred > > >> Simple enough? > > > > >> Stephen > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
I did a quick benchmark of ZAML and found it to be slower: http://pastie.org/288592 On Oct 9, 11:52 am, "Duncan Beevers" <duncanbeev...@gmail.com> wrote:> There are also faster YAML dumpers out there. A recent Portland code sprint > produced ZAML, a work-in-progress that offers (if I recall correctly) > something like a 14x speed boost over vanilla YAML.dumphttp://github.com/hallettj/zaml/tree/master/zaml.rb > > YAML''s slowness has been a pain point, but we don''t have to sacrifice > portability. > > On Thu, Oct 9, 2008 at 5:17 AM, Stephen Celis <stephen.ce...@gmail.com>wrote: > > > > > > > Yes; after Fred''s post I did a bit more research and realize it''s a > > big no for portable apps. Not an impossible hurdle to deal with, but > > not a desirable default. I also realized that AR in its current state > > has no simple interface for such an option: quoting''s serialization is > > abstracted away from table and model information. While MySQL handled > > Marshal swimmingly in string columns, I don''t think other adapters > > would agree. > > > I''ve tried to stay away from serialize where possible, but am dealing > > with it on a current project and noticed complaints from others > > regarding the speed at which large groups of objects with serialized > > attributes are instantiated from the database. YAML has been the > > culprit. > > > Stephen > > > On Oct 9, 2:45 am, Izidor Jerebic <ij.rubyl...@gmail.com> wrote: > > > To add additional point for -1: Marshal uses a binary format and does > > > not guarantee any compatibility with anything except the ruby on the > > > computer which created it. Marshal uses internal format versioning > > > numbers which do not correspond to ruby versions. This means your > > > database backups are potentially non-portable to other OS/ruby/ > > > computer version. This in itself makes this option a non-starter, imho. > > > > You could add the option, but it should never become a default. And > > > even an option should come with a big warning. > > > > izidor > > > > On 8.10.2008, at 18:23, Frederick Cheung wrote: > > > > > On 8 Oct 2008, at 17:07, Stephen Celis wrote: > > > > >> Here''s a ticket for a simple patch to use Marshal instead of YAML for > > > >> attribute serialization. Marshaling is significantly faster (see in > > > >> link), and fixes some YAML load issues (including an outstanding > > > >> ticket). > > > > >>http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/11. > > .. > > > > > While the option is fair enough, I don''t thing all existing apps > > > > wouldn''t want this turned on "silently": > > > > - if other people use your database yaml is ok as there are parsers > > > > for it in many languages whereas Marshal would be a PITA > > > > - if your existing column is not a blob column (which it wouldn''t have > > > > to be previously since yaml generates plain text), the database will > > > > throw a hissy fit (or just truncate the data) when you try to insert a > > > > character that is not legal in the charset used. > > > > - should you be calling string_to_binary if the column supports it ? > > > > > Fred > > > >> Simple enough? > > > > >> Stephen--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---
On 9 Oct 2008, at 19:44, Stephen Celis wrote:> > I did a quick benchmark of ZAML and found it to be slower: > > http://pastie.org/288592 >I swapped the IO for a StringIO and ZAML was then twice as fast: user system total real yaml 0.360000 0.010000 0.370000 ( 0.375689) zaml 0.150000 0.000000 0.150000 ( 0.154976)>making the dumped object a little less trivial (seems to me like the initial test only really tests the overhead in getting things set up) further increases the difference to_dump = [''foo'', ''bar'', ''baz'', Time.now, {''key'' => ''value'', ''bar'' => {1 => ''hello'', 2 => ''world''}}]*5 yaml 9.260000 0.060000 9.320000 ( 9.413077) zaml 2.790000 0.010000 2.800000 ( 2.827708)> On Oct 9, 11:52 am, "Duncan Beevers" <duncanbeev...@gmail.com> wrote: >> There are also faster YAML dumpers out there. A recent Portland >> code sprint >> produced ZAML, a work-in-progress that offers (if I recall correctly) >> something like a 14x speed boost over vanilla YAML.dumphttp://github.com/hallettj/zaml/tree/master/zaml.rb >> >> YAML''s slowness has been a pain point, but we don''t have to sacrifice >> portability. >> >> On Thu, Oct 9, 2008 at 5:17 AM, Stephen Celis >> <stephen.ce...@gmail.com>wrote: >> >> >> >> >> >>> Yes; after Fred''s post I did a bit more research and realize it''s a >>> big no for portable apps. Not an impossible hurdle to deal with, but >>> not a desirable default. I also realized that AR in its current >>> state >>> has no simple interface for such an option: quoting''s >>> serialization is >>> abstracted away from table and model information. While MySQL >>> handled >>> Marshal swimmingly in string columns, I don''t think other adapters >>> would agree. >> >>> I''ve tried to stay away from serialize where possible, but am >>> dealing >>> with it on a current project and noticed complaints from others >>> regarding the speed at which large groups of objects with serialized >>> attributes are instantiated from the database. YAML has been the >>> culprit. >> >>> Stephen >> >>> On Oct 9, 2:45 am, Izidor Jerebic <ij.rubyl...@gmail.com> wrote: >>>> To add additional point for -1: Marshal uses a binary format and >>>> does >>>> not guarantee any compatibility with anything except the ruby on >>>> the >>>> computer which created it. Marshal uses internal format versioning >>>> numbers which do not correspond to ruby versions. This means your >>>> database backups are potentially non-portable to other OS/ruby/ >>>> computer version. This in itself makes this option a non-starter, >>>> imho. >> >>>> You could add the option, but it should never become a default. And >>>> even an option should come with a big warning. >> >>>> izidor >> >>>> On 8.10.2008, at 18:23, Frederick Cheung wrote: >> >>>>> On 8 Oct 2008, at 17:07, Stephen Celis wrote: >> >>>>>> Here''s a ticket for a simple patch to use Marshal instead of >>>>>> YAML for >>>>>> attribute serialization. Marshaling is significantly faster >>>>>> (see in >>>>>> link), and fixes some YAML load issues (including an outstanding >>>>>> ticket). >> >>>>>> http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/11 >>>>>> . >>> .. >> >>>>> While the option is fair enough, I don''t thing all existing apps >>>>> wouldn''t want this turned on "silently": >>>>> - if other people use your database yaml is ok as there are >>>>> parsers >>>>> for it in many languages whereas Marshal would be a PITA >>>>> - if your existing column is not a blob column (which it >>>>> wouldn''t have >>>>> to be previously since yaml generates plain text), the database >>>>> will >>>>> throw a hissy fit (or just truncate the data) when you try to >>>>> insert a >>>>> character that is not legal in the charset used. >>>>> - should you be calling string_to_binary if the column supports >>>>> it ? >> >>>>> Fred >>>>>> Simple enough? >> >>>>>> Stephen > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group. To post to this group, send email to rubyonrails-core@googlegroups.com To unsubscribe from this group, send email to rubyonrails-core+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en -~----------~----~----~----~------~----~------~--~---