Mongrel users - Aug 2007 - e c30ac536947f7330943f8de9c33f70ef2d5994e7

e, elemental, is a stack for the data web

there are 4 components: earth, air, fire, water

earth is a pure-ruby RDF triple-store, with a fs backend. no dependencies on 3rd
party databases, just add filesystem (tm). theres also a ram backend built with
the Mongrel URI-classifier trie as the primitive datastructure. from these two
it should be easy to extrapolate how to write a memcached/hadoop backend, or
whatever is your cup of tea. ive found reiser4 works quite well.. the main
inspiration behind the store design is git.

the API for the store should be familiar to users of jQuery.. as with one class
wrapping a DOM node, you get one class wrapping a RDF resource. the resource can
be an actual file on the filesystem - eg throw a dir of jpgs off your cam into a
web dir and start annotating them.. literals are any valid JSON data, and
resource versioning is available.

a JSON query engine is included, which queries like this:

  {  type:Type,
    ''type='':''http://weblog.com/story'',
     date:Modified,
    ''abs?'':dc+''terms/abstract'',
    ''title?'':dc+''elements/1.1/title'',
    ''content?'':sioc+''content'',
     limit:7,
     offset:0,
     order: {p:''date'',d:''desc''}}

as well as plain keyword search of literals. the underlying indices are visible
in and implemented using the store itself.

designed with optimizability /  hackability in mind, to invent the
partitioning/distribution to meet your needs, adjust the editing method to send
all edits to an Utu chan or RDF-over-JSON-over-Jabber to viewers, mirror data to
a beastly SPARQL engine, etc.

  http://whats-your.name/e/e.tar.gz


water is the mongrel layer. it provides a default HTML view for crawlers, links,
dillo, and pays attention to the headers to export RDF to tools that want it
like tabulator, triplr, virtuoso. everything is cached for both server
(serialized resource) and client (if-modified/etag). the base URI is mapped to
the root of the server for your domain.

embedded in the oldschool HTML is RDFa which is used to bootstrap a base
javascript GUI aka ''fire''. you''ll likely want to
customize this for your app, although i do plan on providing a more complete set
of base tools - graphical query editor and some sample scripts as a seperate
package for things like blogs, wikis, bug tracs. facilitating GUIs is a small
set of jQuery extensions, ''air'' to get form fields
auto-watched for changes and updating triples on the server, JS DOM creation +
bindings (closures and js vars instead of jQuery selectors to avoid the
fragility and traversal, if you want). ive tested the base gUI and proven it can
at least load and make edits in firefox2/3, opera, ie(s4linux), and
webkit(qt/gdk), but not this week.

theres no global lock preventing concurrency, no class vars and only a couple
instance vars (could make these go away, but this isnt haskell) . so far the
only thing somewhat slow is ruby''s sort_by. taking about 0.1 second for
30-50 mb of blog post data sorting on date. query results can be cached, but im
going to investigate the performance and mem-hoggage of a RAM store for the
index before rewriting everything in Lua.

the roadmap is add a few missing features: recursive queries (eg, get the 10
newest blog posts, and the 10 most recent comments for each of these), and
OpenID-based accounts (local-only is supported currently). is there a
mongrel-openID gem?

oh yeah, docs/tutorials will come some day, maybe..

On Thu, 16 Aug 2007 05:18:59 -0400
cr <_ at whats-your.name> wrote:
> e, elemental, is a stack for the data web
> 
> there are 4 components: earth, air, fire, water
Super duper cool.

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

Carmen,

Could you cook up an introduction to RDF via e? I''m a bit behind the
curve here. More the idea of RDF and its usage in e than the e code
itself.

Evan

On 8/16/07, cr <_ at whats-your.name> wrote:
> e, elemental, is a stack for the data web
>
> there are 4 components: earth, air, fire, water
>
> earth is a pure-ruby RDF triple-store, with a fs backend. no dependencies
on 3rd party databases, just add filesystem (tm). theres also a ram backend
built with the Mongrel URI-classifier trie as the primitive datastructure. from
these two it should be easy to extrapolate how to write a memcached/hadoop
backend, or whatever is your cup of tea. ive found reiser4 works quite well..
the main inspiration behind the store design is git.
>
> the API for the store should be familiar to users of jQuery.. as with one
class wrapping a DOM node, you get one class wrapping a RDF resource. the
resource can be an actual file on the filesystem - eg throw a dir of jpgs off
your cam into a web dir and start annotating them.. literals are any valid JSON
data, and resource versioning is available.
>
> a JSON query engine is included, which queries like this:
>
>   {  type:Type,
>     ''type='':''http://weblog.com/story'',
>      date:Modified,
>     ''abs?'':dc+''terms/abstract'',
>     ''title?'':dc+''elements/1.1/title'',
>     ''content?'':sioc+''content'',
>      limit:7,
>      offset:0,
>      order: {p:''date'',d:''desc''}}
>
> as well as plain keyword search of literals. the underlying indices are
visible in and implemented using the store itself.
>
> designed with optimizability /  hackability in mind, to invent the
partitioning/distribution to meet your needs, adjust the editing method to send
all edits to an Utu chan or RDF-over-JSON-over-Jabber to viewers, mirror data to
a beastly SPARQL engine, etc.
>
>   http://whats-your.name/e/e.tar.gz
>
>
> water is the mongrel layer. it provides a default HTML view for crawlers,
links, dillo, and pays attention to the headers to export RDF to tools that want
it like tabulator, triplr, virtuoso. everything is cached for both server
(serialized resource) and client (if-modified/etag). the base URI is mapped to
the root of the server for your domain.
>
> embedded in the oldschool HTML is RDFa which is used to bootstrap a base
javascript GUI aka ''fire''. you''ll likely want to
customize this for your app, although i do plan on providing a more complete set
of base tools - graphical query editor and some sample scripts as a seperate
package for things like blogs, wikis, bug tracs. facilitating GUIs is a small
set of jQuery extensions, ''air'' to get form fields
auto-watched for changes and updating triples on the server, JS DOM creation +
bindings (closures and js vars instead of jQuery selectors to avoid the
fragility and traversal, if you want). ive tested the base gUI and proven it can
at least load and make edits in firefox2/3, opera, ie(s4linux), and
webkit(qt/gdk), but not this week.
>
> theres no global lock preventing concurrency, no class vars and only a
couple instance vars (could make these go away, but this isnt haskell) . so far
the only thing somewhat slow is ruby''s sort_by. taking about 0.1 second
for 30-50 mb of blog post data sorting on date. query results can be cached, but
im going to investigate the performance and mem-hoggage of a RAM store for the
index before rewriting everything in Lua.
>
> the roadmap is add a few missing features: recursive queries (eg, get the
10 newest blog posts, and the 10 most recent comments for each of these), and
OpenID-based accounts (local-only is supported currently). is there a
mongrel-openID gem?
>
> oh yeah, docs/tutorials will come some day, maybe..
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>

-- 
Evan Weaver
Cloudburst, LLC

On Aug 18, 2007, at 18:33 , Evan Weaver wrote:
> Carmen,
>
> Could you cook up an introduction to RDF via e? I''m a bit behind
the
> curve here. More the idea of RDF and its usage in e than the e code
> itself.
>
> Evan
> <snip>
Carmen,

That would be greatly appreciated.

Thanks!

   ~Wayne

s///g
Wayne E. Seguin
Sr. Systems Architect & Systems Administrator

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://rubyforge.org/pipermail/mongrel-users/attachments/20070818/9334a23f/attachment.html