Camping list - Aug 2011 - Advice on strategy for maintaining state in Camping

Hi,



I need a little advice about maintaining state in Camping.



I use NaturallySpeaking voice recognition software for most of my work -- I
don''t have happy hands -- and I''ve been creating little Ruby
projects to
make it easier to do some things by voice. I''d like to build a UI for
them.
After some painful experiences with some windows-based UIs, I''d like to
try
using Camping ? it looks like fun, and I can use my HTML/CSS skills to make
something pretty.



For most of these little Ruby projects, I don''t have to store anything
in a
database because the data is already being stored elsewhere. For example,
I''m managing a team that''s building a Microsoft-based data
warehouse that
uses Excel pivot tables as a front-end, and pivot tables are hard to
manipulate using NaturallySpeaking. On my Camping site, I want to be able to
display a long list of the available pivot table fields so I can click on
them by voice.  I also want a text box so I can say, e.g., "show department
and unit by row, year by column" or "only show fields containing
committee."
So all I need to store is the info I''m using to manipulate the pivot
table:
 the connection to the Excel worksheet and a list of the available fields
that I grab from the Excel worksheet plus one or two properties about these
fields. I don''t need to save any of this info -- I grab it once at the
beginning of the session, and I don''t want to cache it because, for
example,
the list of fields will change depending on which data warehouse
"cube" I''m
connecting to.  I have other projects where the data is stored on a
website/application that I''m grabbing/manipulating using Web
services/APIs.
In short, I don''t need to permanently store any data, I just need to
maintain state for a relatively small number of objects.



What''s the easiest way to do this in Camping? Is there a way to cache
the
list of field objects? Or does it make more sense to store them in a
database and purge & refresh the data in the database each time I start up
the Camping site?



Thanks!

Anders



PS Maybe you''re thinking, "using Excel pivot tables as a front-end
to a data
warehouse??" It does sound bizarre, and I was pretty skeptical at first,
but
it actually works pretty well. Microsoft has put a fair amount of work into
turning Excel pivot tables into a pretty decent data warehouse front end.
And since you''re just using Excel, you get all the goodies are built
into
Excel. Not a good front-end if you are creating straightforward dashboards
that are totally locked down, but if you have a pretty broad range of fields
and you''re encouraging folks to slice and dice the data themselves, it
ends
up being easier than most of the other tools out there.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/camping-list/attachments/20110825/dc0e5a1c/attachment.html>

On Thu, Aug 25, 2011 at 15:25, Anders Schneiderman
<aschneiderman at gmail.com> wrote:
> Hi,
>
> I need a little advice about maintaining state in Camping.
>
Uh oh. Maintaining complex state is *always* a hassle, regardless of
language and framework?

The general way of maintaing state is using session cookies. It works like this:

  require ''camping''
  require ''camping/session''

  Camping.goes :App

  module App
    include Camping::Session
  end

  module App::Controllers
    class Index
      def get
        @state.counter = 0
        mab do
          p "Session ID = #{@state.session_id}"
          p ''Counter = 0''
          a ''Increment'', :href => R(Increment)
        end
      end
    end

    class Increment
      def get
        @state.counter += 1
        "Counter: #{@state.counter}"
      end
    end
  end

Notes:
- See also http://camping.rubyforge.org/api.html#class-Camping-Session
- Note that you can''t store more than ~3kB in sessions, so you
shouldn''t put
- `@state.session_id` is only available in Rack 1.3 (in previous
versions you have to generate it yourself if you need it)
- However, the current Camping release doesn''t work well with Rack 1.3
when it comes to session
- That''s fixed in the latest development version

I''m probably going to release a new version today or tomorrow. If you
can''t wait until then, just run this command:

  gem install camping --source http://bob.judofyr.net/gem
> I use NaturallySpeaking voice recognition software for most of my work -- I
> don''t have happy hands -- and I''ve been creating little
Ruby projects to
> make it easier to do some things by voice. I''d like to build a UI
for them.
> After some painful experiences with some windows-based UIs, I''d
like to try
> using Camping ? it looks like fun, and I can use my HTML/CSS skills to make
> something pretty.
Cool! Is easier to manage web apps than native apps using
NaturallySpeaking, or is it just the the native window-based UIs are
way too complex? I''ve never really optimized a web app for
accessibility (which is pretty terrible when I think about it)?
> For most of these little Ruby projects, I don''t have to store
anything in a
> database because the data is already being stored elsewhere. For example,
> I''m managing a team that''s building a Microsoft-based
data warehouse that
> uses?Excel pivot tables as a front-end, and pivot tables are hard to
> manipulate using NaturallySpeaking. On my Camping site, I want to be able
to
> display a long list of the available pivot table fields so I can click on
> them by voice.? I also want a text box so I can say, e.g., "show
department
> and unit by row, year by column" or "only show fields containing
committee."
> So all I need to store is the info I''m using to manipulate the
pivot table:
> ?the connection to the Excel worksheet and a list of the available fields
> that I grab from the Excel worksheet plus one or two properties about these
> fields. I don''t need to save any of this info -- I grab it once at
the
> beginning of the session, and I don''t want to cache it because,
for example,
> the list of fields will change depending on which data warehouse
"cube" I''m
> connecting to.? I have other projects where the data is stored on a
> website/application that I''m grabbing/manipulating using Web
services/APIs.
> In short, I don''t need to permanently store any data, I just need
to
> maintain state for a relatively small number of objects.
>
> What''s the easiest way to do this in Camping? Is there a way to
cache the
> list of field objects? Or does it make more sense to store them in a
> database and purge & refresh the data in the database each time I start
up
> the Camping site?
I think you''re going to save yourself a lot of problems if you can
download as much data and store it in a local database. It''s much
faster and easier to just have everything in the database. Then you
can easily query it for whatever you need. See
http://whywentcamping.com/Book:-Getting-Started#Modeling-the-world for
how to use ActiveRecord to set up the database/schema.

You can either download the data in a controller (e.g. when someone
hits the front page) or just at startup. If only you''re going to use
the app, it''s probably easiest to load it at startup. If you want to
load it at startup, you should probably use a SQLite memory database.
This keeps all the database in memory and you don''t have to clear it
everytime you start/stop your app.

Simply define a .create method (which the Camping server will call
when it starts up):

  def App.create
    App::Models::Base.establish_connection(:adapter =>
''sqlite3'',
:database => '':memory:'')
  end
> Thanks!
>
> Anders
Hopefully this should get you somewhat started.
>
> PS Maybe you''re thinking, "using Excel pivot tables as a
front-end to a data
> warehouse??" It does sound bizarre, and I was pretty skeptical at
first, but
> it actually works pretty well. Microsoft has put a fair amount of work into
> turning Excel pivot tables into a pretty decent data warehouse front end.
> And since you''re just using Excel, you get all the goodies are
built into
> Excel. Not a good front-end if you are creating straightforward dashboards
> that are totally locked down, but if you have a pretty broad range of
fields
> and you''re encouraging folks to slice and dice the data
themselves, it ends
> up being easier than most of the other tools out there.
I''ll have to admit that was exactly what I thought? But all that
really matters is that it works :-)

If I''m understanding your question correctly I think judicious use of
the @state instance variable will achieve what you''re looking for.
You''ll be able to store what you need and be able to access it from
request to request.

Another option would be to use sqlite in memory mode.

App::Models::Base.establish_connection(:adapter=>''sqlite3'',
:database=>'':memory:'')

You''ll gain the benefits of a database but you''ll be working
in memory
only, so nothing''s stored when your app is off.

Dave

On Thu, Aug 25, 2011 at 9:25 AM, Anders Schneiderman
<aschneiderman at gmail.com> wrote:
> Hi,
>
>
>
> I need a little advice about maintaining state in Camping.
>
>
>
> I use NaturallySpeaking voice recognition software for most of my work -- I
> don''t have happy hands -- and I''ve been creating little
Ruby projects to
> make it easier to do some things by voice. I''d like to build a UI
for them.
> After some painful experiences with some windows-based UIs, I''d
like to try
> using Camping ? it looks like fun, and I can use my HTML/CSS skills to make
> something pretty.
>
>
>
> For most of these little Ruby projects, I don''t have to store
anything in a
> database because the data is already being stored elsewhere. For example,
> I''m managing a team that''s building a Microsoft-based
data warehouse that
> uses?Excel pivot tables as a front-end, and pivot tables are hard to
> manipulate using NaturallySpeaking. On my Camping site, I want to be able
to
> display a long list of the available pivot table fields so I can click on
> them by voice.? I also want a text box so I can say, e.g., "show
department
> and unit by row, year by column" or "only show fields containing
committee."
> So all I need to store is the info I''m using to manipulate the
pivot table:
> ?the connection to the Excel worksheet and a list of the available fields
> that I grab from the Excel worksheet plus one or two properties about these
> fields. I don''t need to save any of this info -- I grab it once at
the
> beginning of the session, and I don''t want to cache it because,
for example,
> the list of fields will change depending on which data warehouse
"cube" I''m
> connecting to.? I have other projects where the data is stored on a
> website/application that I''m grabbing/manipulating using Web
services/APIs.
> In short, I don''t need to permanently store any data, I just need
to
> maintain state for a relatively small number of objects.
>
>
>
> What''s the easiest way to do this in Camping? Is there a way to
cache the
> list of field objects? Or does it make more sense to store them in a
> database and purge & refresh the data in the database each time I start
up
> the Camping site?
>
>
>
> Thanks!
>
> Anders
>
>
>
> PS Maybe you''re thinking, "using Excel pivot tables as a
front-end to a data
> warehouse??" It does sound bizarre, and I was pretty skeptical at
first, but
> it actually works pretty well. Microsoft has put a fair amount of work into
> turning Excel pivot tables into a pretty decent data warehouse front end.
> And since you''re just using Excel, you get all the goodies are
built into
> Excel. Not a good front-end if you are creating straightforward dashboards
> that are totally locked down, but if you have a pretty broad range of
fields
> and you''re encouraging folks to slice and dice the data
themselves, it ends
> up being easier than most of the other tools out there.
>
> _______________________________________________
> Camping-list mailing list
> Camping-list at rubyforge.org
> http://rubyforge.org/mailman/listinfo/camping-list
>


-- 
Dave

Thank you so much Magnus and David for your speedy advice!
Magnus, I think you''re right a SQLlight database seems like the best
way to
go.
>Cool! Is easier to manage web apps than native apps using
>NaturallySpeaking, or is it just the the native window-based UIs are
>way too complex? I''ve never really optimized a web app for
>accessibility (which is pretty terrible when I think about it)?
It''s a bit of both. NaturallySpeaking tries to make their software as
Web-friendly as possible, so, for example, if I display the fields I want to
be able to choose as a bunch of hyperlinks in a on the page, I can click any
of them by voice as I could any link on a webpage.  With wxruby, that''s
not
the case.  And since I''ve done a lot of HTML/CSS work in past jobs, I
can
bang it out a lot faster than learning wxruby or some other UI ? and
it''s a
lot easier to build something that has a little style to  it.  :)



Thanks very much!

Anders
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/camping-list/attachments/20110825/50ccaa64/attachment.html>

Hold up guys. This is a little web app for just you to use? not multiple users?

Depending on how you deploy camping you can just stick some stuff in some class
variables if you just need them in one controller, or even global variables if
you want them in many places. Then all you''d need to do is boot a local
copy with The Camping Server and do your things. The objects will just stay in
ruby''s memory because unlike cgi apps or things like PHP, ruby web apps
don''t flush their global scope reload on every request.

Wouldn''t that be the ridiculously easy and straight forward way to
solve this?


?
Jenna

On 26/08/2011, at 7:33 AM, Anders Schneiderman wrote:
> Thank you so much Magnus and David for your speedy advice!
> 
> Magnus, I think you''re right a SQLlight database seems like the
best way to go.
> 
> >Cool! Is easier to manage web apps than native apps using
> >NaturallySpeaking, or is it just the the native window-based UIs are
> >way too complex? I''ve never really optimized a web app for
> >accessibility (which is pretty terrible when I think about it)?
> It''s a bit of both. NaturallySpeaking tries to make their software
as Web-friendly as possible, so, for example, if I display the fields I want to
be able to choose as a bunch of hyperlinks in a on the page, I can click any of
them by voice as I could any link on a webpage.  With wxruby, that''s
not  the case.  And since I''ve done a lot of HTML/CSS work in past
jobs, I can bang it out a lot faster than learning wxruby or some other UI ? and
it''s a lot easier to build something that has a little style to  it. 
:)
>  
> Thanks very much!
> Anders
> _______________________________________________
> Camping-list mailing list
> Camping-list at rubyforge.org
> http://rubyforge.org/mailman/listinfo/camping-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/camping-list/attachments/20110826/b9dbaf42/attachment.html>

> a SQLlight database seems like the best way to go
given your initial scenario of read-only Excel files, i disagree. why plumb data
from Excel into SQLite and from SQLite into a web UI via several layers of Ruby
code, themselves distanced from underlying datastores via ORM libraries, when
you can specialized .XLS oriented tools oriented to non-coder (or quasi-coder)

i dont have anything to do with Cambridge Semantics, but if i had a copy of
Excel or Windows i''d check it..

http://www.cambridgesemantics.com/products/anzo_for_excel

if massaging the data or filtering it one could try Refine:

https://code.google.com/p/google-refine/

if it was me, i usually use xls2txt and an on-the-fly RDF (to JSON/Hash in RAM)
conversion and skip the SQL or "non-programmer" tools which mean well
but never do quite what you want (or require excessive gymnastics vs a few lines
of Ruby/Haskell)

http://gitorious.org/element/element/blobs/master/ruby/W/csv.rb

also Camping may be *just* what you are looking for..

Depending on how you deploy camping you can just stick some stuff in some class
variables if you just need them in one controller, or even global variables if
you want them in many places. Then all you''d need to do is boot a local
copy with The Camping Server and do your things. The objects will just stay in
ruby''s memory because unlike cgi apps or things like PHP, ruby web apps
don''t flush their global scope reload on every request.

Wouldn''t that be the ridiculously easy and straight forward way to
solve this?

Thanks, Jenna!  Sounds good.

Since I''m new to camping, can you tell me where I would add the global
objects?

Anders
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/camping-list/attachments/20110827/ab8a5442/attachment.html>

There are a few places you could stash one.

If you have some class somewhere, say, MyThinger, you could declare a little
accessor like this:

class MyThinger
	class << self
		attr_accessor :things
	end
end

Then you would be able to write MyThinger.things and use it like a variable,
getting and setting it to whatever stuff you''d like to keep around.

The other simpler way to do it would be to just use a Global Variable. In ruby,
a global variable is any variable which begins with a $. So, you could use
$things, and it''d stick around. It''s usually considered bad
practice to use them, but if you choose a descriptive name nothing else is
likely to want to use, there really is no justification for such fears.


?
Jenna


On 28/08/2011, at 5:30 AM, Anders Schneiderman wrote:
> Depending on how you deploy camping you can just stick some stuff in some
class variables if you just need them in one controller, or even global
variables if you want them in many places. Then all you''d need to do is
boot a local copy with The Camping Server and do your things. The objects will
just stay in ruby''s memory because unlike cgi apps or things like PHP,
ruby web apps don''t flush their global scope reload on every request.
> 
> Wouldn''t that be the ridiculously easy and straight forward way to
solve this?
> 
> Thanks, Jenna!  Sounds good.
> 
> Since I''m new to camping, can you tell me where I would add the
global objects?
> 
> Anders
> _______________________________________________
> Camping-list mailing list
> Camping-list at rubyforge.org
> http://rubyforge.org/mailman/listinfo/camping-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/camping-list/attachments/20110828/0c31525e/attachment-0001.html>

Anders - if you get this working, do let us know (with any relevant  
links). It''d be good to show the world... I worked for some time on  
disability/technology/art projects, and I know at least one large UK  
network of users that would be interested - DaveE
> Hi,
>
> I need a little advice about maintaining state in Camping.
>
> I use NaturallySpeaking voice recognition software for most of my  
> work -- I don''t have happy hands -- and I''ve been
creating little
> Ruby projects to make it easier to do some things by voice. I''d
like
> to build a UI for them.  After some painful experiences with some  
> windows-based UIs, I''d like to try using Camping ? it looks like  
> fun, and I can use my HTML/CSS skills to make something pretty.
>
> For most of these little Ruby projects, I don''t have to store  
> anything in a database because the data is already being stored  
> elsewhere. For example, I''m managing a team that''s
building a
> Microsoft-based data warehouse that uses Excel pivot tables as a  
> front-end, and pivot tables are hard to manipulate using  
> NaturallySpeaking. On my Camping site, I want to be able to display  
> a long list of the available pivot table fields so I can click on  
> them by voice.  I also want a text box so I can say, e.g., "show  
> department and unit by row, year by column" or "only show fields
> containing committee." So all I need to store is the info I''m
using
> to manipulate the pivot table:  the connection to the Excel  
> worksheet and a list of the available fields that I grab from the  
> Excel worksheet plus one or two properties about these fields. I  
> don''t need to save any of this info -- I grab it once at the  
> beginning of the session, and I don''t want to cache it because,
for
> example, the list of fields will change depending on which data  
> warehouse "cube" I''m connecting to.  I have other
projects where the
> data is stored on a website/application that I''m grabbing/ 
> manipulating using Web services/APIs. In short, I don''t need to  
> permanently store any data, I just need to maintain state for a  
> relatively small number of objects.
>
> What''s the easiest way to do this in Camping? Is there a way to  
> cache the list of field objects? Or does it make more sense to store  
> them in a database and purge & refresh the data in the database each  
> time I start up the Camping site?
>
> Thanks!
> Anders
>
> PS Maybe you''re thinking, "using Excel pivot tables as a
front-end
> to a data warehouse??" It does sound bizarre, and I was pretty  
> skeptical at first, but it actually works pretty well. Microsoft has  
> put a fair amount of work into turning Excel pivot tables into a  
> pretty decent data warehouse front end.  And since you''re just
using
> Excel, you get all the goodies are built into Excel. Not a good  
> front-end if you are creating straightforward dashboards that are  
> totally locked down, but if you have a pretty broad range of fields  
> and you''re encouraging folks to slice and dice the data
themselves,
> it ends up being easier than most of the other tools out there.
> _______________________________________________
> Camping-list mailing list
> Camping-list at rubyforge.org
> http://rubyforge.org/mailman/listinfo/camping-list