Re: RFC: Improving the rack parameter normalization

andy
Sat, 08 Aug 2009 01:06:45 -0000

Thanks for your comments, John.

Re the need for changing the parameter parser to achieve #1, I think
it''s absolutely necessary. I''m pretty convinced that the
existing
parser is simply incapable of retaining ordering of record structures
unless those record structures are populated via *identical* parameter
names. Anything you do to disambiguate parameter names will force the
parameter parser to treat those disambiguating chunks as keys in a
hash, and their relative order of appearance will thereby be lost. The
key thing I''ve done with my reimplementation is to come up with a
syntax that tells the parameter parser to omit certain "words" in the
parameter name (i.e. strings of non-bracket characters) from the final
structure, using those words solely to control allocations of new
hashes in the surrounding array. The syntax I''ve currently got for
this is an initial dot. I''m thinking of switching to double brackets
instead (e.g. "[[index]]" instead of "[.index]" because it
wouldn''t
preclude developers using dotted index values if they really want to.
(The corresponding argument to not take away developers'' ability to
use double-bracketed indexes is weaker because doing so with the
current parser accomplishes nothing - the double brackets are
processed identically as the same indexes with single bracketing.)

Also, keep in mind that the current parameter parser is actually
potentially dangerous, in that values from different subforms can end
up in the same record. My "quick edit" vs "full edit"
example is one
dangerous case, but another is when your subform collection is
actually polymorphic (the situation I''m working on right now and which
spawned all this). The current parser relies on seeing a repeated
attribute to know when to start filling in a new record, so
polymorphism in the subforms has the potential to screw things up! I
think my new parser would be able to completely reliably handle all
parameter names produced by rails'' form helpers, using a much more
restrictive syntax than is accepted by the current parser (that parser
is EXTREMELY lax and will parse totally screwy parameter names). With
the backward compatibility features, this shouldn''t affect non-rails
uses of rack.

Re :child_index, I didn''t actually introduce that - it''s part
of the
existing nested attributes support, I assume introduced as part of the
new accepts_nested_attributes_for feature. Normally you don''t need to
supply a :child_index value, and record position will be used
automatically in most cases. But here I want to use a value that will
trigger the proper parameter ordering.

Ultimately, I''m thinking that this sort of handling would be a better
default than what''s currently done - i.e. there''s really no
benefit
that I can see from having the records come back in the params
structure in an arbitrary order, and with this change it''d be quite
easy to make as-submitted order be a "sensible" default. Then imagine
altering acts_as_list so that the attributes= method would fix
positions when handed an array of records, and suddenly my controller
cruft could disappear, and this order maintenance would be in the
model, where it belongs, rather than in the controller or in
javascript.

So in summary, you''re right that the javascript approach could work
and would be simple, but I''d argue that the parameter parsing change
would open the door to a far more "sensible" default in any case. Then
I have the option of fixing positions with a one-liner in the
controller, and you could still use your javascript approach if you
prefer it. Eventually acts_as_list can handle it and we can both
retire our extra logic.

-andy

On Aug 7, 4:38 pm, John Trupiano <jtrupi...@gmail.com>
wrote:
> Hi Andy,
>
> I have in the past built an extension to the form_for helper family to
> accept a :element_id_prefix option that handles the "multiple html
elements
> with the same ID" problem.  It''s been sitting on my to-do
list for awhile to
> convert it to a rails patch and submit it for consideration.
>
> I like that you want to take advantage of the standards already defined for
> html form submission and POST data to achieve ordering, but your solution
> seems needlessly roundabout for the end goal.  You seem to actually be
> addressing two distinct issues here:
>
> 1) Clean up implementation of parameter parsing so that it maintains the
> order in the originally POSTed data string.
> 2) Introduce a :child_index value to fields_for to avoid the "multiple
html
> elements with the same ID" problem.
>
> In regards to #1, what you get by doing this is your ability to concisely
> auto-assign position values to the children objects in your controller
> (which still requires you to iterate over the passed in params).  This
seems
> to me to not be a necessarily common need.  So I''m inclined to
ask, what
> else do we get out of the redesign of the parameter parsing?  And do we
> really need to change the scheme for encoding field names to achieve this?
> Can we not achieve it by only changing the parsing code?
>
> #2 seems completely useful to me.  That said, I would also like it on the
> form_* family of functions as well.  If it''s a problem with
fields_for, it''s
> also a problem with form_for (and one I''ve obviously encountered
before).
>
> Stepping back a moment and looking at your specific problem, I would have
> simply solved it by outputting a hidden :position field as part of the
form,
> and then setting the :onUpdate parameter of sortable_element to set those
> hidden fields (only a couple of lines of javascript).  This in conjunction
> with a solution for "multiple html elements with the same ID"
would seem to
> completely serve your needs, and you wouldn''t need any additional
controller
> code to actually set the position field on the child elements.
>
> -John
>
> On Fri, Aug 7, 2009 at 12:57 PM, andy <andyrlo...@gmail.com> wrote:
>
> > I''ve got a slightly modified form of this working.
>
> > The rules for parameter names in my scheme are:
>
> > - may optionally begin with an unbracketed name
> > - thereafter must be a sequence of bracketed names
> > - any bracketed name starting with ''.'' must be
immediately followed by
> > one that does not
> > - the whole thing may optionally be terminated with
''[]''
>
> > A "bracketed name" in the above is just any string that
doesn''t
> > include ''['' or '']''
> > characters, surrounded by ''['' and
'']''
>
> > My replacement normalize_params implementation in rack/utils.rb checks
> > the parameter name it''s initially called with to see if it
meets these
> > criteria.
> > Moreover, I''m pretty certain that any name that follows the
rules but
> > does not
> > have any bracketed names beginning with ''.'' will
yield the same
> > results via
> > either the old or the new function, so what I''ve done should
be
> > backward
> > compatible.
>
> > The new scheme uses the dotted names to introduce arrays of hashes.
> > The
> > part of the name following the ''.'' is saved
discretely in the last
> > hash of that
> > array (under a key named ''.index''), and a new hash
is allocated
> > whenever
> > the current dotted name is different from the one that was most
> > recently
> > stashed. The stashed .index keys are all removed before the parameter
> > structure is finally returned, so they won''t screw up
attributes
> > setters.
>
> > This scheme means you don''t need to worry about using
''+xxx'' for the
> > first
> > parameter in a group, the rest of which will use
''.xxx'' instead, as
> > per my
> > original proposal.
>
> > With these changes in place (plus a small change to acts_as_list to
> > avoid
> > assigning a position value to a new record if one has already been
> > assigned), my original problem is very easily solved, as follows:
>
> > In my :task partial template, I use:
>
> >    <%= project_form.fields_for :tasks, task,
> >                :child_index=>".#{task.id ||
''REPLACE_ME''}" do |tf| %>
>
> > My "add a task" helper needs to replace
''REPLACE_ME'' with a unique
> > value via javascript (as is already common practice in these dynamic
> > multiform scenarios).
>
> > This yields parameter names in the subform like:
>
> >   project[tasks_attributes][.13][name]
>
> > Then in my projects controller, my update action is defined like this:
>
> >  def update
> >    params[:project][:tasks_attributes].each_with_index { |ta,i| ta
> > [:position] = i }
> >    @project = Project.find(params[:id])
> >    ...
> >  end
>
> > That''s it.  I''m actually thinking now that, since
it''s so
> > straightforward
> > to elect to use this feature (you just need to supply a :child_index
> > that
> > starts with a dot in the nested fields_for call), it may not require
> > any
> > changes to the form builders as I had originally expected.
>
> > I haven''t yet tried this with more deeply nested forms, but I
will do
> > so
> > shortly.
>
> > If anyone has any comments or recommendations, please post.
I''d like,
> > soon, to publish a patch for this to be incorporated into the rails
> > distro.
>
> > Thanks
>
> > -andy
>
> > On Aug 6, 11:06 pm, andy <andyrlo...@gmail.com> wrote:
> > > I have struggled much of the day today to do something that on
the
> > > surface seems like it should be a piece of cake. But either
it''s not,
> > > or I''m too dense to see the easy way to do it.
>
> > > I''m leaning toward modifying the rack parameter parser
to (a) make it
> > > far cleaner and more intelligible than it currently is (including
> > > documenting exactly what it does!), and (b) make it easy to do
what
> > > I''m after. I think I can do this in a mostly
backward-compatible
> > > fashion, and could provide a compatibility mode if needed for
anyone
> > > who needs the old behavior.
>
> > > I''d appreciate any feedback the experts may have before
I embark on
> > > this.
>
> > > Here''s a simplified version of what I''m trying
to do that ended up
> > > being (I think) impossible without resorting to some javascript
that
> > > really should not be required:
>
> > > Say I''ve got a nested model structure that looks like
this:
>
> > > class Project < ActiveRecord::Base
> > >   has_many :tasks
> > >   accepts_nested_parameters_for :tasks
> > > end
>
> > > class Task < ActiveRecord::Base
> > >   belongs_to :project
> > > end
>
> > > I''d like to be able create an edit form for :project
that includes a
> > > dynamically varying list of :task subforms. I''ll have a
delete button
> > > next to each :task subform, and an "Add a Task" button
at the bottom
> > > of the last. All fairly standard and not very hard, especially
with
> > > the new accepts_nested_parameters_for feature.
>
> > > But now here''s the kicker: I also want to use the
"sortable_element"
> > > helper so that the user can reorder the tasks - including newly
added
> > > tasks - within the project. In order to save the final ordering,
this
> > > means I need to be able to detect, from the request parameters,
the
> > > order that the :task subforms appeared in the overall form at the
time
> > > it was submitted. This should be easy, because browsers are
required
> > > to send form parameters in the order in which they appear at the
time
> > > of submission, and sortable_element works by physically
reordering the
> > > subforms in the overall DOM structure.
>
> > > The problem is that rack loses this ordering information, and
I''m
> > > pretty much convinced that it can''t be made to keep it.
You can ALMOST
> > > get this to work by specifying :index=>''''
and :child_index=>'''' in all
> > > your fields_for calls. But this results in illegal HTML (strictly
> > > speaking, though it works, at least in the browser I''ve
tried), and
> > > it''s also slightly risky. The illegal part comes from
the fact that
> > > you''ll end up with multiple input elements in your form
that have the
> > > same id value. E.g. a text field for task
''name'' would appear in
> > > every :task subform, and in each appearance it would carry the
same id
> > > value.  And not only would that be illegal, it would also make it
> > > impossible to have per-subform radio button groups. The radio
button
> > > input elements would all have the same name across all subforms,
so
> > > they''d all be linked, rather than only within a subform.
>
> > > Even without those problems, the strategy is dangerous because of
the
> > > way rack detects when it''s time to start a new hash,
rather than
> > > continuing to fill a current hash. This comes into play for a
> > > parameter with a name like
''project[tasks_attributes][][name]''.  The
> > > the empty brackets call for an array of hashes at that point in
the
> > > structure, and rack will look at whether ''name''
appears as a key in
> > > the current last element of that array. If so, it opens up a new
> > > element in the array, otherwise it adds to the current last
element.
>
> > > This mechanism will fail if not all your subforms offer the same
set
> > > of fields. E.g you might have separate buttons for "add
quick task"
> > > and "add task," both of which add the same sort of
object, but "add
> > > quick task" provides an abbreviated form for easier entry.
You''d have
> > > to add the other fields as hidden fields in the abbreviated
subform in
> > > order to be safe, and it''d probably be a hellish
debugging exercise to
> > > figure that out!
>
> > > OK, so here''s my idea:
>
> > > 1. When "[]" appears in a parameter name it always
indicates an array
> > > at that point in the structure. What fallows goes into a new
element
> > > at the end of the array, using logic equivalent to what is
currently
> > > provided by rack.
> > > 2. When "[.index]" appears, what follows will be be
added to the hash
> > > that is the current last element of the array. The leading dot
> > > prevents this index from itself being a hash key in what would
then
> > > have to be a containing hash, rather than a containing array,
>
> ...
>
>