Mechanize users - May 2013 - Mechanize #post preserving field order

Hi,

Is there anyway to preserve field order when using the post method?
I''ve
looked all through the source code and I can''t find a good way to do
this.
It becomes especially important when you have two of the same field names
and the server builds an array on the receiving end to process the request.

Let me know if anyone has a solution for this.

Thanks,
Jason
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20130504/80b2ad2a/attachment-0001.html>

On May 4, 2013, at 15:27, Jason Barnett <jason.w.barnett at gmail.com>
wrote:
> Is there anyway to preserve field order when using the post method?
I''ve looked all through the source code and I can''t find a
good way to do this. It becomes especially important when you have two of the
same field names and the server builds an array on the receiving end to process
the request.
Mechanize automatically preserves field order.  You don''t need to do
anything.

It looks like order gets preserved until one of the fields has two brackets,
i.e.:


SOURCE CODE:
-------------------------
@agent.post("#{primary_url}/", [
  [ ''rs'', ''get_desc'' ],
  [ ''rst'', '''' ],
  [ ''rsrnd'',
"#{Time.now.strftime(''%s%3N'')}" ],
  [ ''rsargs[]'', ''smime'' ]



LOG:
--------
D, [2013-05-05T07:39:56.335036 #3557] DEBUG -- : query:
"rst=&rs=get_desc&rsargs%5B0%5D=smime&rsrnd=1367764796334"




I''m not sure if this is a bug or what? It also seems to me that the
fields shouldn''t go through the whole CGI.escape process, but maybe
that''s an extremely unusual desire and I''ll just have to
rewrite the "Mechanize::Util#self.build_query_string" function for
this one time.

Tell me if I am way of base here.

--  
Jason Barnett


On Saturday, May 4, 2013 at 11:01 PM, Eric Hodel wrote:
> On May 4, 2013, at 15:27, Jason Barnett <jason.w.barnett at gmail.com
(mailto:jason.w.barnett at gmail.com)> wrote:
> > Is there anyway to preserve field order when using the post method?
I''ve looked all through the source code and I can''t find a
good way to do this. It becomes especially important when you have two of the
same field names and the server builds an array on the receiving end to process
the request.
>  
>  
> Mechanize automatically preserves field order. You don''t need to
do anything.
>  
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org (mailto:Mechanize-users at rubyforge.org)
> http://rubyforge.org/mailman/listinfo/mechanize-users
>  
>  

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20130505/4ba4437c/attachment.html>

To give you more information? I even confirmed that the fields are in the
correct order before calling "post_form(uri, form, headers)" at the
end of Mechanze#post method? Somewhere between there and the actual building of
the query it loses it''s order. I am unable to figure out how or what
this is happening. Code and samples below:


Modifications I made to code to try and track down:
https://gist.github.com/jasonwbarnett/e1a3baed7c6d4baa437a

Output: https://gist.github.com/jasonwbarnett/31f2da0d0d9e8012fd28


--  
Jason Barnett


On Sunday, May 5, 2013 at 7:51 AM, Jason Barnett wrote:
> It looks like order gets preserved until one of the fields has two
brackets, i.e.:
>  
>  
> SOURCE CODE:
> -------------------------
> @agent.post("#{primary_url}/", [
>   [ ''rs'', ''get_desc'' ],
>   [ ''rst'', '''' ],
>   [ ''rsrnd'',
"#{Time.now.strftime(''%s%3N'')}" ],
>   [ ''rsargs[]'', ''smime'' ]
>  
>  
>  
> LOG:
> --------
> D, [2013-05-05T07:39:56.335036 #3557] DEBUG -- : query:
"rst=&rs=get_desc&rsargs%5B0%5D=smime&rsrnd=1367764796334"
>  
>  
>  
>  
> I''m not sure if this is a bug or what? It also seems to me that
the fields shouldn''t go through the whole CGI.escape process, but maybe
that''s an extremely unusual desire and I''ll just have to
rewrite the "Mechanize::Util#self.build_query_string" function for
this one time.
>  
> Tell me if I am way of base here.
>  
> --  
> Jason Barnett
>  
>  
> On Saturday, May 4, 2013 at 11:01 PM, Eric Hodel wrote:
>  
> > On May 4, 2013, at 15:27, Jason Barnett <jason.w.barnett at
gmail.com (mailto:jason.w.barnett at gmail.com)> wrote:
> > > Is there anyway to preserve field order when using the post
method? I''ve looked all through the source code and I can''t
find a good way to do this. It becomes especially important when you have two of
the same field names and the server builds an array on the receiving end to
process the request.
> >  
> >  
> > Mechanize automatically preserves field order. You don''t need
to do anything.
> >  
> > _______________________________________________
> > Mechanize-users mailing list
> > Mechanize-users at rubyforge.org (mailto:Mechanize-users at
rubyforge.org)
> > http://rubyforge.org/mailman/listinfo/mechanize-users
> >  
> >  
> >  
>  
>  
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20130505/e7c4a3b3/attachment.html>

Hi Jason,

Can you open a bug report on Github with a script to reproduce this? I can
probably take a look tonight.

Godfrey


On Sun, May 5, 2013 at 5:28 AM, Jason Barnett <jason.w.barnett at
gmail.com>wrote:
>  To give you more information? I even confirmed that the fields are in the
> correct order before calling "post_form(uri, form, headers)" at
the end
> of Mechanze#post method? Somewhere between there and the actual building of
> the query it loses it''s order. I am unable to figure out how or
what this
> is happening. Code and samples below:
>
>
> Modifications I made to code to try and track down:
> https://gist.github.com/jasonwbarnett/e1a3baed7c6d4baa437a
>
> Output: https://gist.github.com/jasonwbarnett/31f2da0d0d9e8012fd28
>
>
> --
> Jason Barnett
>
> On Sunday, May 5, 2013 at 7:51 AM, Jason Barnett wrote:
>
>  It looks like order gets preserved until one of the fields has two
> brackets, i.e.:
>
>
> SOURCE CODE:
> -------------------------
> @agent.post("#{primary_url}/", [
>   [ ''rs'', ''get_desc'' ],
>   [ ''rst'', '''' ],
>   [ ''rsrnd'',
"#{Time.now.strftime(''%s%3N'')}" ],
>   [ ''rsargs[]'', ''smime'' ]
>
>
> LOG:
> --------
> D, [2013-05-05T07:39:56.335036 #3557] DEBUG -- : query:
>
"rst=&rs=get_desc&rsargs%5B0%5D=smime&rsrnd=1367764796334"
>
>
>
>
> I''m not sure if this is a bug or what? It also seems to me that
the fields
> shouldn''t go through the whole CGI.escape process, but maybe
that''s an
> extremely unusual desire and I''ll just have to rewrite the
"Mechanize::
> Util#self.build_query_string" function for this one time.
>
> Tell me if I am way of base here.
>
> --
> Jason Barnett
>
> On Saturday, May 4, 2013 at 11:01 PM, Eric Hodel wrote:
>
> On May 4, 2013, at 15:27, Jason Barnett <jason.w.barnett at
gmail.com> wrote:
>
> Is there anyway to preserve field order when using the post method?
I''ve
> looked all through the source code and I can''t find a good way to
do this.
> It becomes especially important when you have two of the same field names
> and the server builds an array on the receiving end to process the request.
>
>
> Mechanize automatically preserves field order. You don''t need to
do
> anything.
>
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>
>
>
>
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20130505/5077dbaf/attachment-0001.html>

I was able to yield the desired result and resolve the problem with the
following code changes:

https://gist.github.com/jasonwbarnett/ccf800a4a5f2f4a2ea81/revisions


The key was changing ''sort'' to ''reverse'' in
Mechanize::Form#build_query. Because I was able to fix things for my particular
use case, I didn''t bother digging into the proc_query and query.push in
Mechanize::Form#build_query. This may be a bug that needs to be resolved,
I''m not sure...

--  
Jason Barnett


On Sunday, May 5, 2013 at 8:28 AM, Jason Barnett wrote:
> To give you more information? I even confirmed that the fields are in the
correct order before calling "post_form(uri, form, headers)" at the
end of Mechanze#post method? Somewhere between there and the actual building of
the query it loses it''s order. I am unable to figure out how or what
this is happening. Code and samples below:
>  
>  
> Modifications I made to code to try and track down:
https://gist.github.com/jasonwbarnett/e1a3baed7c6d4baa437a
>  
> Output: https://gist.github.com/jasonwbarnett/31f2da0d0d9e8012fd28
>  
>  
> --  
> Jason Barnett
>  
>  
> On Sunday, May 5, 2013 at 7:51 AM, Jason Barnett wrote:
>  
> > It looks like order gets preserved until one of the fields has two
brackets, i.e.:
> >  
> >  
> > SOURCE CODE:
> > -------------------------
> > @agent.post("#{primary_url}/", [
> >   [ ''rs'', ''get_desc'' ],
> >   [ ''rst'', '''' ],
> >   [ ''rsrnd'',
"#{Time.now.strftime(''%s%3N'')}" ],
> >   [ ''rsargs[]'', ''smime'' ]
> >  
> >  
> >  
> > LOG:
> > --------
> > D, [2013-05-05T07:39:56.335036 #3557] DEBUG -- : query:
"rst=&rs=get_desc&rsargs%5B0%5D=smime&rsrnd=1367764796334"
> >  
> >  
> >  
> >  
> > I''m not sure if this is a bug or what? It also seems to me
that the fields shouldn''t go through the whole CGI.escape process, but
maybe that''s an extremely unusual desire and I''ll just have to
rewrite the "Mechanize::Util#self.build_query_string" function for
this one time.
> >  
> > Tell me if I am way of base here.
> >  
> > --  
> > Jason Barnett
> >  
> >  
> > On Saturday, May 4, 2013 at 11:01 PM, Eric Hodel wrote:
> >  
> > > On May 4, 2013, at 15:27, Jason Barnett <jason.w.barnett at
gmail.com (mailto:jason.w.barnett at gmail.com)> wrote:
> > > > Is there anyway to preserve field order when using the post
method? I''ve looked all through the source code and I can''t
find a good way to do this. It becomes especially important when you have two of
the same field names and the server builds an array on the receiving end to
process the request.
> > >  
> > >  
> > > Mechanize automatically preserves field order. You don''t
need to do anything.
> > >  
> > > _______________________________________________
> > > Mechanize-users mailing list
> > > Mechanize-users at rubyforge.org (mailto:Mechanize-users at
rubyforge.org)
> > > http://rubyforge.org/mailman/listinfo/mechanize-users
> > >  
> > >  
> > >  
> >  
> >  
>  
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20130505/3c300fca/attachment.html>

Posted bug report on Github:
https://github.com/sparklemotion/mechanize/issues/307

Thanks in advance!


On Sun, May 5, 2013 at 8:57 AM, Godfrey Chan <godfreykfc at gmail.com>
wrote:
> Hi Jason,
>
> Can you open a bug report on Github with a script to reproduce this? I can
> probably take a look tonight.
>
> Godfrey
>
>
> On Sun, May 5, 2013 at 5:28 AM, Jason Barnett <jason.w.barnett at
gmail.com>wrote:
>
>>  To give you more information? I even confirmed that the fields are in
>> the correct order before calling "post_form(uri, form,
headers)" at the
>> end of Mechanze#post method? Somewhere between there and the actual
>> building of the query it loses it''s order. I am unable to
figure out how or
>> what this is happening. Code and samples below:
>>
>>
>> Modifications I made to code to try and track down:
>> https://gist.github.com/jasonwbarnett/e1a3baed7c6d4baa437a
>>
>> Output: https://gist.github.com/jasonwbarnett/31f2da0d0d9e8012fd28
>>
>>
>> --
>> Jason Barnett
>>
>> On Sunday, May 5, 2013 at 7:51 AM, Jason Barnett wrote:
>>
>>  It looks like order gets preserved until one of the fields has two
>> brackets, i.e.:
>>
>>
>> SOURCE CODE:
>> -------------------------
>> @agent.post("#{primary_url}/", [
>>   [ ''rs'', ''get_desc'' ],
>>   [ ''rst'', '''' ],
>>   [ ''rsrnd'',
"#{Time.now.strftime(''%s%3N'')}" ],
>>   [ ''rsargs[]'', ''smime'' ]
>>
>>
>> LOG:
>> --------
>> D, [2013-05-05T07:39:56.335036 #3557] DEBUG -- : query:
>>
"rst=&rs=get_desc&rsargs%5B0%5D=smime&rsrnd=1367764796334"
>>
>>
>>
>>
>> I''m not sure if this is a bug or what? It also seems to me
that the
>> fields shouldn''t go through the whole CGI.escape process, but
maybe that''s
>> an extremely unusual desire and I''ll just have to rewrite the
"Mechanize
>> ::Util#self.build_query_string" function for this one time.
>>
>> Tell me if I am way of base here.
>>
>> --
>> Jason Barnett
>>
>> On Saturday, May 4, 2013 at 11:01 PM, Eric Hodel wrote:
>>
>> On May 4, 2013, at 15:27, Jason Barnett <jason.w.barnett at
gmail.com>
>> wrote:
>>
>> Is there anyway to preserve field order when using the post method?
I''ve
>> looked all through the source code and I can''t find a good way
to do this.
>> It becomes especially important when you have two of the same field
names
>> and the server builds an array on the receiving end to process the
request.
>>
>>
>> Mechanize automatically preserves field order. You don''t need
to do
>> anything.
>>
>> _______________________________________________
>> Mechanize-users mailing list
>> Mechanize-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mechanize-users
>>
>>
>>
>>
>> _______________________________________________
>> Mechanize-users mailing list
>> Mechanize-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mechanize-users
>>
>
>
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/mechanize-users/attachments/20130505/6691eb0f/attachment.html>