Rails core - Oct 2009 - Make model.number = "1 234" store 1234 instead of 1?

Since our users have been complaining about putting "1 234" in a form
and having 1 stored instead of 1234 (because of how to_i/to_f works),
I made a simple monkeypatch to get their expected behavior:

https://gist.github.com/3d99d172175ee3bc64a1

I''d be happy to submit a patch to Rails, but I wanted to check here
first. Is there good reason not to make this behavior default?

I think if you are validating numercality it should be an error.
Removing spaces seems like unexpected behavior to me.

On Mon, Oct 26, 2009 at 4:17 AM, Henrik N <henrik@nyh.se>
wrote:
>
> Since our users have been complaining about putting "1 234" in a
form
> and having 1 stored instead of 1234 (because of how to_i/to_f works),
> I made a simple monkeypatch to get their expected behavior:
>
> https://gist.github.com/3d99d172175ee3bc64a1
>
> I''d be happy to submit a patch to Rails, but I wanted to check
here
> first. Is there good reason not to make this behavior default?
>
> >
>


-- 
Amos King
http://dirtyInformation.com
http://github.com/Adkron
--
Looking for something to do? Visit http://ImThere.com

On Mon, Oct 26, 2009 at 7:17 AM, Henrik N <henrik@nyh.se> wrote:
>
> Since our users have been complaining about putting "1 234" in a
form
> and having 1 stored instead of 1234 (because of how to_i/to_f works),
> I made a simple monkeypatch to get their expected behavior:
>
>
This is not something that should go in core, IMO.

If on your specific application it makes sense to do that, then use a
before_validation callback to strip the spaces off the value on the fields
for which this makes sense.

Cheers

> https://gist.github.com/3d99d172175ee3bc64a1
>
> I''d be happy to submit a patch to Rails, but I wanted to check
here
> first. Is there good reason not to make this behavior default?
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en
-~----------~----~----~----~------~----~------~--~---

On Mon, Oct 26, 2009 at 10:17 AM, Henrik N <henrik@nyh.se> wrote:
> Since our users have been complaining about putting "1 234" in a
form
> and having 1 stored instead of 1234 (because of how to_i/to_f works),
> I made a simple monkeypatch to get their expected behavior:
Don''t see this in core.

Looks like a custom numerification that makes sense in your
application. Both that one and Rails'' make arbitrary assumptions about
what they tolerate, but Rails is coherent with Ruby and I think the
current behaviour is better because of that.

Custom rules, yours or others, can be programmed as needed.

On Oct 26, 2009, at 5:39 PM, Xavier Noria wrote:
>
> On Mon, Oct 26, 2009 at 10:17 AM, Henrik N <henrik@nyh.se> wrote:
>
>> Since our users have been complaining about putting "1 234"
in a form
>> and having 1 stored instead of 1234 (because of how to_i/to_f works),
>> I made a simple monkeypatch to get their expected behavior:
>
> Don''t see this in core.
>
> Looks like a custom numerification that makes sense in your
> application. Both that one and Rails'' make arbitrary assumptions
about
> what they tolerate, but Rails is coherent with Ruby and I think the
> current behaviour is better because of that.
Couldn''t agree more. This isn''t by any means a
"feature" for Rails-
Core. Maybe
you should look at the attribute_normalizer plugin.

Cheers,

Amol Hatwar (rubygem)

Thanks for the feedback, all.

Can anyone think of a specific example of when the behavior I proposed
would be undesirable? I can''t think of anything.

For most users, I imagine the suggested behavior follows the principle
of least surprise and the current behavior does not. End users likely
care less about consistency with to_i/to_f and more about the web app
doing what they mean.

On Oct 26, 12:53 pm, Amos King <amos.l.k...@gmail.com>
wrote:
> I think if you are validating numercality it should be an error.
Why would you want to show the user a validation error instead of just
doing what they would expect, though?
> Removing spaces seems like unexpected behavior to me.
Definitely if it was a string field, but for an integer or decimal
column? Per above: when would the user *expect* the number to be cut
off at the first space?

On Oct 26, 1:09 pm, Xavier Noria <f...@hashref.com>
wrote:
> Don''t see this in core.
>
> Looks like a custom numerification that makes sense in your
> application. Both that one and Rails'' make arbitrary assumptions
about
> what they tolerate, but Rails is coherent with Ruby and I think the
> current behaviour is better because of that.
I wouldn''t call it arbitrary. My proposed behavior is based on how end
users realistically may input numbers, and it improves their
experience without any added ambiguity that I can think of. Support
for commas/periods as thousand separators *would* add ambiguity since
that is locale-dependent, but to my knowledge spaces in numbers are
unambiguous across locales.

On Mon, Oct 26, 2009 at 1:48 PM, Henrik N <henrik@nyh.se> wrote:
> On Oct 26, 1:09 pm, Xavier Noria <f...@hashref.com> wrote:
>> Don''t see this in core.
>>
>> Looks like a custom numerification that makes sense in your
>> application. Both that one and Rails'' make arbitrary
assumptions about
>> what they tolerate, but Rails is coherent with Ruby and I think the
>> current behaviour is better because of that.
>
> I wouldn''t call it arbitrary. My proposed behavior is based on how
end
> users realistically may input numbers, and it improves their
> experience without any added ambiguity that I can think of. Support
> for commas/periods as thousand separators *would* add ambiguity since
> that is locale-dependent, but to my knowledge spaces in numbers are
> unambiguous across locales.
I understand it is not arbitrary in the sense that it is based on the
feedback of some users of *your* application. I didn''t mean it is a
blind customization, but that is as arbitrary a priori as removing any
spurious character.

I have applications where numeric input filters remove any \D that is
not a comma or a period becaue that make sense for them, so I
understand your motivation. Removing \D is as arbitrary as removing \s
in my view, and thus I prefer the programmer to do that, instead of
core to do that by default.

On Oct 26, 2:43 pm, Xavier Noria <f...@hashref.com>
wrote:
> On Mon, Oct 26, 2009 at 1:48 PM, Henrik N <hen...@nyh.se> wrote:
> > On Oct 26, 1:09 pm, Xavier Noria <f...@hashref.com> wrote:
> >> Don''t see this in core.
>
> >> Looks like a custom numerification that makes sense in your
> >> application. Both that one and Rails'' make arbitrary
assumptions about
> >> what they tolerate, but Rails is coherent with Ruby and I think
the
> >> current behaviour is better because of that.
>
> > I wouldn''t call it arbitrary. My proposed behavior is based
on how end
> > users realistically may input numbers, and it improves their
> > experience without any added ambiguity that I can think of. Support
> > for commas/periods as thousand separators *would* add ambiguity since
> > that is locale-dependent, but to my knowledge spaces in numbers are
> > unambiguous across locales.
>
> I understand it is not arbitrary in the sense that it is based on the
> feedback of some users of *your* application. I didn''t mean it is
a
> blind customization, but that is as arbitrary a priori as removing any
> spurious character.
Is it, though?

I can think of no case when a space can''t be just dropped from numeric
input without losing anything.

Letters and special characters on the other hand should probably be
caught by validations in most apps, since we can''t know if they hit
"q" but meant to hit 1, or if their comma was intended as a decimal
comma and not a thousand separator, etc.

These are reasons not to remove those characters. I can think of no
reasons not to remove spaces. So that''s not very arbitrary :)

I would like to see specific examples of why it might be a bad idea to
remove the spaces. I''m hearing nays but I don''t get what
issues you
see with this. I do see the benefits, though: more intuitive and human-
friendly input handling.

On Mon, Oct 26, 2009 at 4:38 PM, Henrik N <henrik@nyh.se> wrote:
> I would like to see specific examples of why it might be a bad idea to
> remove the spaces. I''m hearing nays but I don''t get what
issues you
> see with this. I do see the benefits, though: more intuitive and human-
> friendly input handling.
Hey, this is not a back and white issue. You propose a patch in -core,
and you get feedback. My feedback is that I do not see it in core.

In my view being tolerant to spaces can make sense for some, but that means that

   12                          678

is goona be 12678 by default in Rails. I don''t buy that''s
good.

I don''t think that interpretation is any better than 12 a priori. So
albeit I understand your motivation, as I said, I think the current
default is just as valid as yours, and in addition coherent with what
a Ruby programmer would expect from a string to number conversion.

On Oct 26, 2009, at 8:38 AM, Henrik N wrote:
> I can think of no case when a space can''t be just dropped from
numeric
> input without losing anything.
What we''d lose is consistency in our framework.

It would be an error to try to provide you with use-cases for why this  
is a bad idea.  To do so would miss a very important part of what  
makes Rails (and ruby for that matter) great.  The main reason it''s a  
bad idea is that it introduces an inconsistency into an otherwise  
consistent framework and language.

One of the huge wins that Ruby has over other languages, and something  
which Rails remains (mostly) true to, is that it''s logically  
consistent.  One can learn the principles of ruby and proceed to make  
accurate deductions on language structure and behavior based on those  
principles.  Rails is (mostly) the same (and is getting more  
consistent as it matures).

I go further than Xavier who said that "12   678".to_i => 12 is
"just
as valid as" => 12678.  A space is not a number, no one can argue that  
it is.  Also, one could never deduce the behavior you''re suggesting  
Henrik.  One would have to accept that "removing spaces from numbers  
is just the way it is".  Why a space and not a tab?  Why a space and  
not a comma?  "It just is."

If you want a tool that covers this type of special use-case in the  
core, then I''d suggest you try PHP - it''s one of the things it
does
really well.  It encodes common use cases without any regard to  
consistency.

Jason

On Oct 26, 5:06 pm, Xavier Noria <f...@hashref.com>
wrote:
> On Mon, Oct 26, 2009 at 4:38 PM, Henrik N <hen...@nyh.se> wrote:
> > I would like to see specific examples of why it might be a bad idea to
> > remove the spaces. I''m hearing nays but I don''t get
what issues you
> > see with this. I do see the benefits, though: more intuitive and
human-
> > friendly input handling.
>
> Hey, this is not a back and white issue. You propose a patch in -core,
> and you get feedback. My feedback is that I do not see it in core.
Very glad for the feedback. Was just trying to get to specifics beyond
just "I do not see it".

So I''m getting that you (and others that commented) prefer consistency
(with to_i/to_f) for the Ruby programmer. Fair enough. I disagree, but
it''s an easy override per-app, so no big deal.

On Oct 26, 5:35 pm, Jason King <smathy.w...@gmail.com>
wrote:
> I go further than Xavier who said that "12   678".to_i => 12
is "just
> as valid as" => 12678.  A space is not a number, no one can argue
that
> it is.
That is how computers think, but not non-programmer end users. But I
see your point about consistency. Perhaps it would be better to use
some sort of form/submission object that handles spaces (and periods
and commas if you want to get into that mess) as the user would
expect, and leave the attribute writers as is.

Also, one could never deduce the behavior you''re
suggesting
> Henrik.  One would have to accept that "removing spaces from numbers
> is just the way it is".  Why a space and not a tab?  Why a space and
> not a comma?  "It just is."
The point about consistency is valid, but this isn''t quite. Whitespace
can be unambiguously removed; commas (and periods) in numbers are
ambiguous. That''s why.
> If you want a tool that covers this type of special use-case in the
> core, then I''d suggest you try PHP - it''s one of the
things it does
> really well.  It encodes common use cases without any regard to
> consistency.
No need to get nasty ;) I proposed something that would improve the
experience for end users without any practical negative effects that I
can tell. It seems we simply disagree about whether user experience
trumps consistency.

In general I have found the core teams advice to be very very helpful  
and listening to their (positive and negative) feedback has helped me  
write patches and contribute. Their technical and gut reactions come  
from a wealth of experience. It reminds of a book I was reading by the  
pragmatic programmers on agile learning and skill development. For  
such a small thing, some on the core team might not be able to fully  
express their "intuition" and expertise on their feedback. Despite  
their articulation, it is good advice.

My opinion on the matter is aligned with those that have already  
spoken. Sure I might put something like your code into my own app, I  
can totally see it. But it''s too far and assuming to much for core  
code. For instance, in my big day job app, I use this simple attribute  
stripper for string based columns.

http://gist.github.com/219579

A good idea... hell yea. My users are probably the worst low tech  
users that type in all uppercase, using IE5 that you can find. Does  
this belong in core... my gut says no... although there are probably  
tons of smarter reasons that I can not express.


  - Ken Collins


On Oct 27, 2009, at 6:46 AM, Henrik N wrote:
>
> On Oct 26, 5:06 pm, Xavier Noria <f...@hashref.com> wrote:
>> On Mon, Oct 26, 2009 at 4:38 PM, Henrik N <hen...@nyh.se> wrote:
>>> I would like to see specific examples of why it might be a bad  
>>> idea to
>>> remove the spaces. I''m hearing nays but I don''t
get what issues you
>>> see with this. I do see the benefits, though: more intuitive and  
>>> human-
>>> friendly input handling.
>>
>> Hey, this is not a back and white issue. You propose a patch in - 
>> core,
>> and you get feedback. My feedback is that I do not see it in core.
>
> Very glad for the feedback. Was just trying to get to specifics beyond
> just "I do not see it".
>
> So I''m getting that you (and others that commented) prefer
consistency
> (with to_i/to_f) for the Ruby programmer. Fair enough. I disagree, but
> it''s an easy override per-app, so no big deal.
>
> On Oct 26, 5:35 pm, Jason King <smathy.w...@gmail.com> wrote:
>> I go further than Xavier who said that "12   678".to_i =>
12 is "just
>> as valid as" => 12678.  A space is not a number, no one can
argue
>> that
>> it is.
>
> That is how computers think, but not non-programmer end users. But I
> see your point about consistency. Perhaps it would be better to use
> some sort of form/submission object that handles spaces (and periods
> and commas if you want to get into that mess) as the user would
> expect, and leave the attribute writers as is.
>
> Also, one could never deduce the behavior you''re suggesting
>> Henrik.  One would have to accept that "removing spaces from
numbers
>> is just the way it is".  Why a space and not a tab?  Why a space
and
>> not a comma?  "It just is."
>
> The point about consistency is valid, but this isn''t quite.
Whitespace
> can be unambiguously removed; commas (and periods) in numbers are
> ambiguous. That''s why.
>
>> If you want a tool that covers this type of special use-case in the
>> core, then I''d suggest you try PHP - it''s one of the
things it does
>> really well.  It encodes common use cases without any regard to
>> consistency.
>
> No need to get nasty ;) I proposed something that would improve the
> experience for end users without any practical negative effects that I
> can tell. It seems we simply disagree about whether user experience
> trumps consistency.
> >

On Tue, Oct 27, 2009 at 11:46 AM, Henrik N <henrik@nyh.se> wrote:
> It seems we simply disagree about whether user experience
> trumps consistency.
I think we rather disagree on whether that gives a significant better
user experience.

For you it is clear it does, for me (and I guess a few others) the
patch just relaxes number input in one of a miriad ways. In particular
I don''t see why given

    12                             678

you one should pick 12678. I don''t see why you don''t pick 1000
out of

    $1000

Wouldn''t that be useful? What would a user could possibly mean other
than 1000? One could ask.

Given that I think the weakness in the patch is as arbitrary as any
other numerification that accepts strings that do not contain
well-formed numbers. That is, everything being equal (again, in my
opinion), I chose the behaviour that is consistent with Ruby. Because
everything being equal we get *in addition* expected behaviour from
for the programmer. And in any case it is configurable if an
application has custom rules.

I understand you disagree, that''s OK :).

Em 27-10-2009 17:59, Xavier Noria escreveu:
> On Tue, Oct 27, 2009 at 11:46 AM, Henrik N<henrik@nyh.se>  wrote:
>
>    
>> It seems we simply disagree about whether user experience
>> trumps consistency.
>>      
>
> I think we rather disagree on whether that gives a significant better
> user experience.
>
> For you it is clear it does, for me (and I guess a few others) the
> patch just relaxes number input in one of a miriad ways. In particular
> I don''t see why given
>
>      12                             678
>
> you one should pick 12678. I don''t see why you don''t pick
1000 out of
>
>      $1000
>
> Wouldn''t that be useful? What would a user could possibly mean
other
> than 1000? One could ask.
>
> Given that I think the weakness in the patch is as arbitrary as any
> other numerification that accepts strings that do not contain
> well-formed numbers. That is, everything being equal (again, in my
> opinion), I chose the behaviour that is consistent with Ruby. Because
> everything being equal we get *in addition* expected behaviour from
> for the programmer. And in any case it is configurable if an
> application has custom rules.
>    
What I really think is that the current behaviour would also be 
unacceptable... I would prefer an exception to be generated when 
converting "12 34" to integer, instead of a result of 12 that would be
inconsistent with the user input...

Maybe ActiveRecord should generate automatically a 
validates_numericality_of for each numeric field and include options for 
:only_integer for integer fields. When a user explicitly set the 
validation, it should override the automatic one...

That would be much more consistent in my opinion, regarding user''s
input...

Rodrigo.

> What I really think is that the current behaviour would also be
> unacceptable... I would prefer an exception to be generated when
> converting "12 34" to integer, instead of a result of 12 that
would be
> inconsistent with the user input...
Agreed.  It''s also supposed to be validating numericality, so if there
is a character in there which is not a number (a space) it should not
validate successfully.

All the posts to this point have suggested that a space is not part of
a numeric, so it would seem to make sense to back that claim up by
making it not validate.

TX

On Oct 27, 6:59 pm, Xavier Noria <f...@hashref.com>
wrote:
> In particular
> I don''t see why given
>
>     12                             678
>
> you one should pick 12678. I don''t see why you don''t pick
1000 out of
>
>     $1000
>
> Wouldn''t that be useful? What would a user could possibly mean
other
> than 1000? One could ask.
These are good points.

I''m not sure what kind of app and UI would cause someone to mistakedly
input
    12                             678
in a single field and expect to have "12" stored (and lose
"678"?),
but I suppose it could happen. That does seem far, far more edge case
than input like "1 234" though.

I don''t think it would make nearly as much sense to strip e.g.
"$" or
"%". If someone puts units in a numerical field, chances are the field
is intended for some other unit.

So while I think I get the point about arbitrariness, I still think
stripping whitespace uniquely stands out as being helpful without real
downsides (if you grant that the "12    678" example is unlikely).
Other characters may be ambiguous (spaces and commas), typos
(letters), intended as units etc and make more sense as validation
errors. But spaces are often used intentionally and are not treated
intuitively by Rails.

While there is something of a slippery slope here, I don''t see
practical downsides to removing spaces, in terms of breaking existing
UIs or code.

If the only real problem is that it''s arbitrary, and consistency with
to_i/to_f makes things easier for the programmer, note that this is
already not the case:

      def convert_number_column_value(value)
        if value == false
          0
        elsif value == true
          1
        elsif value.is_a?(String) && value.blank?
          nil
        else
          value
        end
      end

That''s what Rails does currently. Unlike to_i, it will turn a blank
string to nil, not zero, and turn bools into numbers. These changes
make some sense for an ORM. Making one further change to better handle
the general domain of non-programmer input makes sense to me.

On Oct 27, 3:07 pm, Ken Collins <k...@metaskills.net>
wrote:
> For instance, in my big day job app, I use this simple attribute
> stripper for string based columns.
>
> http://gist.github.com/219579
On a side note: I do something similar. You can lose the ".dup" since
".strip" will create a new String object, not modify "self"
in place.

On Oct 28, 12:15 am, Trejkaz <trej...@gmail.com>
wrote:
> All the posts to this point have suggested that a space is not part of
> a numeric, so it would seem to make sense to back that claim up by
> making it not validate.
I believe this is already the case when you use
validate_numericality_of.

If you use the monkeypatch from my first message, you don''t get those
errors as the spaces are removed before validation.

Henrik,
I don''t think Rails'' principal of least surprise was meant
necessarily
to extend to your end-users'' experience.  It applies to YOUR
experience.  Then, in turn, you are responsible for making your users''
experience unsurprising (amongst many other positive adjectives).

The first development framework that is able to make the *end-user''s*
experience unsurprising will be a smashing success -and is still
decades away.

As a developer, I would be surprised by having "1 234" turn into 1234
because I know that is not how Ruby works.

-Chris

On Oct 26, 8:48 am, Henrik N <hen...@nyh.se>
wrote:
> Thanks for the feedback, all.
>
> For most users, I imagine the suggested behavior follows the principle
> of least surprise and the current behavior does not. End users likely
> care less about consistency with to_i/to_f and more about the web app
> doing what they mean.
>
> On Oct 26, 12:53 pm, Amos King <amos.l.k...@gmail.com> wrote: