I have a rails 3.0 application with complicated logic and was finding that changes to fix a bug would introduce another bug elsewhere. I needed an automatic regression test tool so I could quickly know if this happened. I am using cucumber for this. I know that I am not doing BDD or TDD, but that is beside the point. My initial set of scenarios was developed using capybara and seeding the database with test fixtures. Although it mostly worked, there were problems because it was not exercizing the javascript on my web page, so I switched to selenium. Now none of my scenarios worked. sqlite3 was complaining about the database being locked because it can only handle one request at a time. I tried switching to a mysql test database, but then the scenarios did not see the changes the application made to database. After much googling, I found that both of these problems were because selenium runs in a separate thread while capybara does not. The suggested solution for this was to change the database cleaner strategy from transaction to truncate. After this change, most of the scenarios ran, but for those using the scenario outline, only the first case would pass. The following cases all found an empty database. Truncate was deleting all the database records after the first case and not restoring it. After more googling I found I could set the database cleaner strategy to nil. Now all of my scenarios pass, but I have to be careful that no two scenarios use the same database records because database changes are not cleared between scenarios. I also have been able to go back to using sqlite3. Is there a better alternative? -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msg/rubyonrails-talk/-/6LGGdyyH_0UJ. For more options, visit https://groups.google.com/groups/opt_out.
andreo-FdlSlcb4kYpknbxzx/v8hQ@public.gmane.org
2013-Mar-14 09:10 UTC
Re: adventures with rails, cucumber, selenium, and a database
Without a single doubt, using factory_girl +database_cleaner gem. Are you using the test framework from rails? or Rspec? here is a good episode http://railscasts.com/episodes/275-how-i-test explaining how to integrate this. and probably ehre talks about database cleaner https://gist.github.com/docwhat/1190475. try to look up yourself some more information. your tests should be as isolated as possible, so your next test shouldnt depend on if the one before fails or passes and what does on the database. On Wednesday, 13 March 2013 15:14:53 UTC+1, jsnark wrote:> > I have a rails 3.0 application with complicated logic and was finding that > changes to fix a bug would introduce another bug elsewhere. I needed an > automatic regression test tool so I could quickly know if this happened. I > am using cucumber for this. I know that I am not doing BDD or TDD, but > that is beside the point. > > My initial set of scenarios was developed using capybara and seeding the > database with test fixtures. Although it mostly worked, there were > problems because it was not exercizing the javascript on my web page, so I > switched to selenium. Now none of my scenarios worked. sqlite3 was > complaining about the database being locked because it can only handle one > request at a time. I tried switching to a mysql test database, but then > the scenarios did not see the changes the application made to database. > After much googling, I found that both of these problems were because > selenium runs in a separate thread while capybara does not. The suggested > solution for this was to change the database cleaner strategy from > transaction to truncate. After this change, most of the scenarios ran, but > for those using the scenario outline, only the first case would pass. The > following cases all found an empty database. Truncate was deleting all the > database records after the first case and not restoring it. After more > googling I found I could set the database cleaner strategy to nil. Now all > of my scenarios pass, but I have to be careful that no two scenarios use > the same database records because database changes are not cleared between > scenarios. I also have been able to go back to using sqlite3. > > Is there a better alternative? > > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msg/rubyonrails-talk/-/Zf2-gyhIbP8J. For more options, visit https://groups.google.com/groups/opt_out.
jsnark
2013-Mar-14 17:46 UTC
Re: adventures with rails, cucumber, selenium, and a database
I run the tests using: $ bundle exec cucumber It is my understanding that cucumber is built on top of rspec. I am also using test/fixtures and not factory_girl. The reason is that I am rewriting a perl terminal interface tool as a web application. The legacy database tables go back 10 years. The data for any scenario is easily extracted from this database and converted into yaml. The typical scenario uses 50-100 rows from 10-30 tables. It would take forever to write all this data as factory_girl ruby code. It is very annoying that I had to resort to using the nil option for the database cleaner, but as I explained, neither transaction nor truncate works. On Thursday, March 14, 2013 5:10:22 AM UTC-4, and...-FdlSlcb4kYpknbxzx/v8hQ@public.gmane.org wrote:> > Without a single doubt, using factory_girl +database_cleaner gem. Are you > using the test framework from rails? or Rspec? > > here is a good episode http://railscasts.com/episodes/275-how-i-testexplaining how to integrate this. and probably ehre talks about database > cleaner https://gist.github.com/docwhat/1190475. try to look up yourself > some more information. > > your tests should be as isolated as possible, so your next test shouldnt > depend on if the one before fails or passes and what does on the database. > > On Wednesday, 13 March 2013 15:14:53 UTC+1, jsnark wrote: >> >> I have a rails 3.0 application with complicated logic and was finding >> that changes to fix a bug would introduce another bug elsewhere. I needed >> an automatic regression test tool so I could quickly know if this >> happened. I am using cucumber for this. I know that I am not doing BDD or >> TDD, but that is beside the point. >> >> My initial set of scenarios was developed using capybara and seeding the >> database with test fixtures. Although it mostly worked, there were >> problems because it was not exercizing the javascript on my web page, so I >> switched to selenium. Now none of my scenarios worked. sqlite3 was >> complaining about the database being locked because it can only handle one >> request at a time. I tried switching to a mysql test database, but then >> the scenarios did not see the changes the application made to database. >> After much googling, I found that both of these problems were because >> selenium runs in a separate thread while capybara does not. The suggested >> solution for this was to change the database cleaner strategy from >> transaction to truncate. After this change, most of the scenarios ran, but >> for those using the scenario outline, only the first case would pass. The >> following cases all found an empty database. Truncate was deleting all the >> database records after the first case and not restoring it. After more >> googling I found I could set the database cleaner strategy to nil. Now all >> of my scenarios pass, but I have to be careful that no two scenarios use >> the same database records because database changes are not cleared between >> scenarios. I also have been able to go back to using sqlite3. >> >> Is there a better alternative? >> >> >>-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msg/rubyonrails-talk/-/lGFHvqYjlRsJ. For more options, visit https://groups.google.com/groups/opt_out.
andreo-FdlSlcb4kYpknbxzx/v8hQ@public.gmane.org
2013-Mar-15 11:31 UTC
Re: adventures with rails, cucumber, selenium, and a database
So are you still using sqlite3 to run the tests? why didnt the tests see the difference on the database when using MySQL? On Thursday, 14 March 2013 18:46:23 UTC+1, jsnark wrote:> > I run the tests using: > > $ bundle exec cucumber > > It is my understanding that cucumber is built on top of rspec. > > I am also using test/fixtures and not factory_girl. The reason is that I > am rewriting a perl terminal interface tool as a web application. The > legacy database tables go back 10 years. The data for any scenario is > easily extracted from this database and converted into yaml. The typical > scenario uses 50-100 rows from 10-30 tables. It would take forever to > write all this data as factory_girl ruby code. > > It is very annoying that I had to resort to using the nil option for the > database cleaner, but as I explained, neither transaction nor truncate > works. > > On Thursday, March 14, 2013 5:10:22 AM UTC-4, and...-FdlSlcb4kYpknbxzx/v8hQ@public.gmane.org wrote: >> >> Without a single doubt, using factory_girl +database_cleaner gem. Are you >> using the test framework from rails? or Rspec? >> >> here is a good episode http://railscasts.com/episodes/275-how-i-testexplaining how to integrate this. and probably ehre talks about database >> cleaner https://gist.github.com/docwhat/1190475. try to look up yourself >> some more information. >> >> your tests should be as isolated as possible, so your next test shouldnt >> depend on if the one before fails or passes and what does on the database. >> >> On Wednesday, 13 March 2013 15:14:53 UTC+1, jsnark wrote: >>> >>> I have a rails 3.0 application with complicated logic and was finding >>> that changes to fix a bug would introduce another bug elsewhere. I needed >>> an automatic regression test tool so I could quickly know if this >>> happened. I am using cucumber for this. I know that I am not doing BDD or >>> TDD, but that is beside the point. >>> >>> My initial set of scenarios was developed using capybara and seeding the >>> database with test fixtures. Although it mostly worked, there were >>> problems because it was not exercizing the javascript on my web page, so I >>> switched to selenium. Now none of my scenarios worked. sqlite3 was >>> complaining about the database being locked because it can only handle one >>> request at a time. I tried switching to a mysql test database, but then >>> the scenarios did not see the changes the application made to database. >>> After much googling, I found that both of these problems were because >>> selenium runs in a separate thread while capybara does not. The suggested >>> solution for this was to change the database cleaner strategy from >>> transaction to truncate. After this change, most of the scenarios ran, but >>> for those using the scenario outline, only the first case would pass. The >>> following cases all found an empty database. Truncate was deleting all the >>> database records after the first case and not restoring it. After more >>> googling I found I could set the database cleaner strategy to nil. Now all >>> of my scenarios pass, but I have to be careful that no two scenarios use >>> the same database records because database changes are not cleared between >>> scenarios. I also have been able to go back to using sqlite3. >>> >>> Is there a better alternative? >>> >>> >>>-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msg/rubyonrails-talk/-/N5bnMw9aTQYJ. For more options, visit https://groups.google.com/groups/opt_out.
jsnark
2013-Mar-15 15:07 UTC
Re: adventures with rails, cucumber, selenium, and a database
The issue with cucumber, selenium and database transactions is described in: http://stackoverflow.com/questions/7511520/database-cleaner-transaction-with-cucumber-rails http://www.datatravels.com/technotes/2011/10/01/cucumber-capybara-with-selenium-and-truncated-fixt/ I also found suggested work-arounds at http://pastie.org/1745020 http://blog.thefrontiergroup.com.au/2009/10/database-transactions-in-cucumber-breaking-selenium/ but these do not work for me. On Friday, March 15, 2013 7:31:36 AM UTC-4, and...-FdlSlcb4kYpknbxzx/v8hQ@public.gmane.org wrote:> > So are you still using sqlite3 to run the tests? why didnt the tests see > the difference on the database when using MySQL? > > On Thursday, 14 March 2013 18:46:23 UTC+1, jsnark wrote: >> >> I run the tests using: >> >> $ bundle exec cucumber >> >> It is my understanding that cucumber is built on top of rspec. >> >> I am also using test/fixtures and not factory_girl. The reason is that I >> am rewriting a perl terminal interface tool as a web application. The >> legacy database tables go back 10 years. The data for any scenario is >> easily extracted from this database and converted into yaml. The typical >> scenario uses 50-100 rows from 10-30 tables. It would take forever to >> write all this data as factory_girl ruby code. >> >> It is very annoying that I had to resort to using the nil option for the >> database cleaner, but as I explained, neither transaction nor truncate >> works. >> >> On Thursday, March 14, 2013 5:10:22 AM UTC-4, and...-FdlSlcb4kYpknbxzx/v8hQ@public.gmane.org wrote: >>> >>> Without a single doubt, using factory_girl +database_cleaner gem. Are >>> you using the test framework from rails? or Rspec? >>> >>> here is a good episode http://railscasts.com/episodes/275-how-i-testexplaining how to integrate this. and probably ehre talks about database >>> cleaner https://gist.github.com/docwhat/1190475. try to look up >>> yourself some more information. >>> >>> your tests should be as isolated as possible, so your next test shouldnt >>> depend on if the one before fails or passes and what does on the database. >>> >>> On Wednesday, 13 March 2013 15:14:53 UTC+1, jsnark wrote: >>>> >>>> I have a rails 3.0 application with complicated logic and was finding >>>> that changes to fix a bug would introduce another bug elsewhere. I needed >>>> an automatic regression test tool so I could quickly know if this >>>> happened. I am using cucumber for this. I know that I am not doing BDD or >>>> TDD, but that is beside the point. >>>> >>>> My initial set of scenarios was developed using capybara and seeding >>>> the database with test fixtures. Although it mostly worked, there were >>>> problems because it was not exercizing the javascript on my web page, so I >>>> switched to selenium. Now none of my scenarios worked. sqlite3 was >>>> complaining about the database being locked because it can only handle one >>>> request at a time. I tried switching to a mysql test database, but then >>>> the scenarios did not see the changes the application made to database. >>>> After much googling, I found that both of these problems were because >>>> selenium runs in a separate thread while capybara does not. The suggested >>>> solution for this was to change the database cleaner strategy from >>>> transaction to truncate. After this change, most of the scenarios ran, but >>>> for those using the scenario outline, only the first case would pass. The >>>> following cases all found an empty database. Truncate was deleting all the >>>> database records after the first case and not restoring it. After more >>>> googling I found I could set the database cleaner strategy to nil. Now all >>>> of my scenarios pass, but I have to be careful that no two scenarios use >>>> the same database records because database changes are not cleared between >>>> scenarios. I also have been able to go back to using sqlite3. >>>> >>>> Is there a better alternative? >>>> >>>> >>>>-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msg/rubyonrails-talk/-/zAklCiywUe8J. For more options, visit https://groups.google.com/groups/opt_out.