I''m building a ruby wrapper for a SOAP service using savon and I''m running into some issues with testing and I would love some feedback. I have added some tests with the help of VCR, but unfortunately that doesn''t feel right given some of the constraints I''m running into. 1. The service provides no test mode or environment. 2. Putting the service into a known state is difficult because some objects are soft deleted. Attempting to save an object with the same unique identifier as a soft deleted object is a unique constraint violation. In order to test saving a new object in the system, an actual new object will need to be created in our production environment every time. 3. It is hard to simulate certain conditions because of #2. Other conditions like session timeouts are also difficult to simulate, because it requires actually sleeping in my spec. Sleeping in the spec is only necessary when not using VCR cassettes, once they are in place the sleep is no longer needed. 4. In the past when using VCR, I would be able to clear my spec/cassettes directory, run the tests again, and refresh what was there. Because of #2 this isn''t possible for any tests that persist new objects using the service. The only thing I can come up with is building a clone of the service using sinatra and provide a subset of the functionality I need for testing. I''m fairly certain this will allow me to get around all of the constraints I mentioned and allow me to stub out specific requests to simulate different conditions. Any feedback is greatly appreciated. Best, Michael Guterl
On Apr 11, 2012, at 7:08 AM, Michael Guterl wrote:> I''m building a ruby wrapper for a SOAP service using savon and I''m > running into some issues with testing and I would love some feedback. > I have added some tests with the help of VCR, but unfortunately that > doesn''t feel right given some of the constraints I''m running into. > > 1. The service provides no test mode or environment. > 2. Putting the service into a known state is difficult because some > objects are soft deleted. Attempting to save an object with the same > unique identifier as a soft deleted object is a unique constraint > violation. In order to test saving a new object in the system, an > actual new object will need to be created in our production > environment every time. > 3. It is hard to simulate certain conditions because of #2. Other > conditions like session timeouts are also difficult to simulate, > because it requires actually sleeping in my spec. Sleeping in the > spec is only necessary when not using VCR cassettes, once they are in > place the sleep is no longer needed. > 4. In the past when using VCR, I would be able to clear my > spec/cassettes directory, run the tests again, and refresh what was > there. Because of #2 this isn''t possible for any tests that persist > new objects using the service. > > The only thing I can come up with is building a clone of the service > using sinatra and provide a subset of the functionality I need for > testing. I''m fairly certain this will allow me to get around all of > the constraints I mentioned and allow me to stub out specific requests > to simulate different conditions. > > Any feedback is greatly appreciated. > > Best, > Michael Guterl > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-usersThey technically have no "sandbox", but could you open another production account for your own personal sandbox? Otherwise you''re dealing with some serious limitations.
On Fri, Apr 13, 2012 at 2:24 AM, Justin Ko <jko170 at gmail.com> wrote:> > On Apr 11, 2012, at 7:08 AM, Michael Guterl wrote: > >> I''m building a ruby wrapper for a SOAP service using savon and I''m >> running into some issues with testing and I would love some feedback. >> I have added some tests with the help of VCR, but unfortunately that >> doesn''t feel right given some of the constraints I''m running into. >> >> 1. The service provides no test mode or environment. >> 2. Putting the service into a known state is difficult because some >> objects are soft deleted. Attempting to save an object with the same >> unique identifier as a soft deleted object is a unique constraint >> violation. ?In order to test saving a new object in the system, an >> actual new object will need to be created in our production >> environment every time. >> 3. It is hard to simulate certain conditions because of #2. ?Other >> conditions like session timeouts are also difficult to simulate, >> because it requires actually sleeping in my spec. ?Sleeping in the >> spec is only necessary when not using VCR cassettes, once they are in >> place the sleep is no longer needed. >> 4. In the past when using VCR, I would be able to clear my >> spec/cassettes directory, run the tests again, and refresh what was >> there. ?Because of #2 this isn''t possible for any tests that persist >> new objects using the service. >> >> The only thing I can come up with is building a clone of the service >> using sinatra and provide a subset of the functionality I need for >> testing. ?I''m fairly certain this will allow me to get around all of >> the constraints I mentioned and allow me to stub out specific requests >> to simulate different conditions. >> >> Any feedback is greatly appreciated. >> >> Best, >> Michael Guterl >> _______________________________________________ >> rspec-users mailing list >> rspec-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rspec-users > > They technically have no "sandbox", but could you open another production account for your own personal sandbox? Otherwise you''re dealing with some serious limitations. >I could, but I''m still dealing with the constraint that records can never be deleted: #2. Also the fact that there is no way to simulate certain conditions is also painful. A good API should have a way for me to simulate an error, no? I''m getting by with little or no tests around this code, but I don''t think it will last for long...
Sent from my iPhone On Apr 15, 2012, at 4:04 PM, Michael Guterl <mguterl at gmail.com> wrote:> On Fri, Apr 13, 2012 at 2:24 AM, Justin Ko <jko170 at gmail.com> wrote: >> >> On Apr 11, 2012, at 7:08 AM, Michael Guterl wrote: >> >>> I''m building a ruby wrapper for a SOAP service using savon and I''m >>> running into some issues with testing and I would love some feedback. >>> I have added some tests with the help of VCR, but unfortunately that >>> doesn''t feel right given some of the constraints I''m running into. >>> >>> 1. The service provides no test mode or environment. >>> 2. Putting the service into a known state is difficult because some >>> objects are soft deleted. Attempting to save an object with the same >>> unique identifier as a soft deleted object is a unique constraint >>> violation. In order to test saving a new object in the system, an >>> actual new object will need to be created in our production >>> environment every time. >>> 3. It is hard to simulate certain conditions because of #2. Other >>> conditions like session timeouts are also difficult to simulate, >>> because it requires actually sleeping in my spec. Sleeping in the >>> spec is only necessary when not using VCR cassettes, once they are in >>> place the sleep is no longer needed. >>> 4. In the past when using VCR, I would be able to clear my >>> spec/cassettes directory, run the tests again, and refresh what was >>> there. Because of #2 this isn''t possible for any tests that persist >>> new objects using the service. >>> >>> The only thing I can come up with is building a clone of the service >>> using sinatra and provide a subset of the functionality I need for >>> testing. I''m fairly certain this will allow me to get around all of >>> the constraints I mentioned and allow me to stub out specific requests >>> to simulate different conditions. >>> >>> Any feedback is greatly appreciated. >>> >>> Best, >>> Michael Guterl >>> _______________________________________________ >>> rspec-users mailing list >>> rspec-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/rspec-users >> >> They technically have no "sandbox", but could you open another production account for your own personal sandbox? Otherwise you''re dealing with some serious limitations. >> > I could, but I''m still dealing with the constraint that records can > never be deleted: #2.So just pass a unique identifier each time. If the algorithm isn''t unique enough, have some retry logic.> Also the fact that there is no way to simulate > certain conditions is also painful. A good API should have a way for > me to simulate an error, no?Correct. I would drop this company if you can.> I''m getting by with little or no tests > around this code, but I don''t think it will last for long... > _______________________________________________ > rspec-users mailing list > rspec-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users