[Eventmachine-talk] DistribuStream released

Tony Arcieri
Thu, 18 Oct 2007 15:57:29 -0000

I''m glad to announce the first initial public release of
DistribuStream:

http://distribustream.org/

DistribuStream is a peercasting system, facilitating streaming media over a
managed peer-to-peer network.  You can think of it as being similar to
BitTorrent, but employing progressive downloads so media can be consumed in
on-demand streams (and it''s not written in Python! :).  It was
developed by ClickCaster,
Inc. <http://www.clickcaster.com/> through the University of Colorado
Computer Science Program.  The whole project is written in Ruby and uses
EventMachine as the basis of both the client and server.  It''s
available as
a gem (gem install distribustream) and you can find a basic writeup on how
to use it here:

http://www.clickcaster.com/items/distribustream-0-1-0-released

This release targets primarily developers as it has some pretty major issues
that need to get resolved before it''s generally usable.  The most
pressing
is the client implementation, which currently stores the entire file in RAM
before writing it out to an IO object.  This needs to be changed to a disk
buffer, and / or combined with the protocol''s ability to
"unprovide" certain
file segments from the peer network, so the client can store a manageable
moving window of data.

In terms of implementation details that would nice to get resolved, the
present implementation uses a hodgepodge of both EM and threaded
components.  While the underlying protocol (the Peer Distributed Transfer
Protocol, see pdtp.org) is implemented using EventMachine, all of the
peer-to-peer file transfers utilize the regular threaded Mongrel (i.e. not
Kirk''s Mongrel) and the regular (and abominable) Net::HTTP client.

I would very much like to eliminate all of the threaded components and
running everything inside the Reactor loop with the EM HttpClient and either
Kirk''s Mongrel or Francis'' em_httpserver.  In earlier
implementations where
we tried this we ran into inexplicable stalling inside of the Reactor
itself, and because of that the team decided to go with the hybrid Reactor /
threaded approach.  Now that the project is open source I''d like to
start
looking into this again, and would greatly appreciate whatever help anyone
can offer.

I''ll be announcing it to ruby-talk in a few days or so, but thought
I''d
announce it here first and get some feedback.

--
Tony Arcieri
ClickCaster, Inc.
tony at clickcaster.com
-------------- next part --------------
An HTML attachment was scrubbed...