On Apr 20, 9:18 am, "Anush J."
<li...-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org>
wrote:> Hello guys
> After figuring out a way to load my db with info scraped of from a web
> page, the data on db is appended with "--- \n -" these
characters. I''m
> not sure what the problem is.. I did change my editor to xcode.
>
> I have attached my app''s
> - migration file
> - the method that is called to populate the database
> - the output on rails server
>
> You can see I''m specifically printing out the values like (eg:
> [item[0]]) before assigning it to dts.bug_id or any other db row for
> that matter. I can see the printed value is fine but after storing on
> the database they are being appended with those junk characters.
>
> Have anyone of you noticed this before. Am I doing something
> wrong..please help !!
You''ve got a bunch of string columns but you''re assigning
arrays to
them (such as [item[0]]), which forces activerecord to encode them as
yaml which is what causes the extra characters you are seeing - that
how yaml encodes arrays. If you did want to do this you should use the
serialize directive to inform activerecord that those columns should
be serialized/deserialized. If not, then just assign item[0] rather
than [item[0]]
Fred
Fred> Thanks in advance -- Anush
>
> class CreateBugs < ActiveRecord::Migration
> def self.up
> create_table :bugs do |t|
> t.string :bug_id
> t.string :bug_pri
> t.string :bug_sev
> t.string :bug_state
> t.string :bug_date
> t.string :bug_org
> t.string :bug_asg
> t.string :bug_sum
> t.string :bug_fcomment
> t.string :bug_fdate
> t.timestamps
> end
> end
>
> def self.down
> drop_table :bugs
> end
> end
>
> def scrap_dts
> puts "Inside def scrap_dts in import controller"
> agent = Mechanize.new
> agent.basic_auth(''anushj'',
''porfavor'')
> page >
agent.get(''http://hidden/query.cgi?action=search&name=anushj&origin=me&priority=...)
> table = agent.page.search(''//*[(@id >
"subreport_list")]//td'').map(&:text).map(&:strip)
>
> table = table.split(''Bug'')
> table.delete_if{|x| x.size == 0}
>
> table.each do |item|
> tmp = [item[0]]
> tmp = tmp.split(''-'')
> if Bug.find_by_bug_id(tmp[0]).nil?
> dts = Bug.new
> dts.bug_id = tmp[0]
> puts tmp[0]
> dts.bug_pri = [item[2]]
> puts [item[2]]
> dts.bug_sev = [item[4]]
> puts [item[4]]
> dts.bug_state = [item[5]]
> puts [item[5]]
> dts.bug_date = [item[6]]
> puts [item[6]]
> dts.bug_org = [item[9]]
> puts [item[9]]
> dts.bug_asg = [item[10]]
> puts [item[10]]
> dts.save
> puts "save record!"
> #dts_num = [item[0]]
> #puts dts_num
> end
> end
>
> end
>
> DB O/P while executing scrap_dts
>
> Inside def scrap_dts in import controller
> 106083-M30S
> P1
> S1
> New
> 2011/02/16
> anushj
> mmourier
> save record!
> 115412-M30S
> P1
> S1
> Closed: Verified
> 2011/04/05
> anushj
> anushj
> save record!
> 115762-M30S
> P1
> S1
> Closed: Verified
> 2011/04/11
> anushj
> anushj
> save record!
>
> Started GET "/" for 127.0.0.1 at 2011-04-19 23:37:33 -0700
> Processing by ImportController#index as HTML
> Bug Load (0.3ms) SELECT "bugs".* FROM "bugs" WHERE
"bugs"."bug_id" IN
> (''106083-M30S'') LIMIT 1
> Bug Load (0.2ms) SELECT "bugs"."id" FROM
"bugs" WHERE
> ("bugs"."bug_id" =
''["106083-M30S"]'') LIMIT 1
> AREL (0.4ms) INSERT INTO "bugs" ("bug_id",
"bug_pri", "bug_sev",
> "bug_state", "bug_date", "bug_org",
"bug_asg", "bug_sum",
> "bug_fcomment", "bug_fdate", "created_at",
"updated_at") VALUES (''---
> - 106083-M30S
> '', ''---
> - P1
> '', ''---
> - S1
> '', ''---
> - New
> '', ''---
> - 2011/02/16
> '', ''---
> - anushj
> '', ''---
> - mmourier
> '', NULL, NULL, NULL, ''2011-04-20
06:37:35.112929'', ''2011-04-20
> 06:37:35.112929'')
> Bug Load (0.3ms) SELECT "bugs".* FROM "bugs" WHERE
"bugs"."bug_id" IN
> (''115412-M30S'') LIMIT 1
> Bug Load (0.1ms) SELECT "bugs"."id" FROM
"bugs" WHERE
> ("bugs"."bug_id" =
''["115412-M30S"]'') LIMIT 1
> AREL (0.3ms) INSERT INTO "bugs" ("bug_id",
"bug_pri", "bug_sev",
> "bug_state", "bug_date", "bug_org",
"bug_asg", "bug_sum",
> "bug_fcomment", "bug_fdate", "created_at",
"updated_at") VALUES (''---
> - 115412-M30S
> '', ''---
> - P1
> '', ''---
> - S1
> '', ''---
> - "Closed: Verified"
> '', ''---
> - 2011/04/05
> '', ''---
> - anushj
> '', ''---
> - anushj
> '', NULL, NULL, NULL, ''2011-04-20
06:37:35.124746'', ''2011-04-20
> 06:37:35.124746'')
> Bug Load (0.2ms) SELECT "bugs".* FROM "bugs" WHERE
"bugs"."bug_id" IN
> (''115762-M30S'') LIMIT 1
> Bug Load (0.1ms) SELECT "bugs"."id" FROM
"bugs" WHERE
> ("bugs"."bug_id" =
''["115762-M30S"]'') LIMIT 1
> AREL (0.4ms) INSERT INTO "bugs" ("bug_id",
"bug_pri", "bug_sev",
> "bug_state", "bug_date", "bug_org",
"bug_asg", "bug_sum",
> "bug_fcomment", "bug_fdate", "created_at",
"updated_at") VALUES (''---
> - 115762-M30S
> '', ''---
> - P1
> '', ''---
> - S1
> '', ''---
> - "Closed: Verified"
> '', ''---
> - 2011/04/11
> '', ''---
> - anushj
> '', ''---
> - anushj
> '', NULL, NULL, NULL, ''2011-04-20
06:37:35.129591'', ''2011-04-20
> 06:37:35.129591'')
> Bug Load (3.9ms) SELECT "bugs".* FROM "bugs"
> SQL (0.2ms) SELECT COUNT(*) FROM "bugs"
> Rendered import/index.html.erb within layouts/application (46.3ms)
> Completed 200 OK in 1769ms (Views: 51.0ms | ActiveRecord: 6.2ms)
>
> --
> Posted viahttp://www.ruby-forum.com/.
--
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en.