Swfdec - Jul 2008 - [Gnash-dev] client command-line or library for streaming audio and video - does it exist?

On Mon, Jul 21, 2008 at 9:13 PM, Rob Savoye <rob at welcomehome.org>
wrote:
> Luke Kenneth Casson Leighton wrote:
>>
>> development of gnash and swf-dec is the existence of a command-line
client
>> and/or library for uploading (and downloading) AMF-formatted streams to
>> and
>> from a flash media server such as red5.
>> does such a client - or library - exist, that anyone knows of, in any
>> language -
>> c, java or python?
>
>  Gnash has utility/rtmpget, which is a command line utility for
> downloading/uploading of video files via RTMP. It's written in C++, and
uses
> other Gnash internal libraries for most of the heavy lifting. Right now to
> use RTMP anything, use the 'rtmp' branch of Gnash instead of trunk.

 excellent - ahhh, i loveit!!  code in a source code repository which is 40%
commented out ha ha.  i'm delighted to see this kind of hand-crafted stuff,
brings back real fond memories of writing one HUNDRED thousand lines of
hand-crafted marshalling msrpc code, for samba tng, ten years ago.

whilst i realise and fully appreciate that hand-marshalling of RPC
packets is initially far far easier, and you get a better "feel" for
what's really going on, on-the-wire...
.. might i make a suggestion? please do consider looking at this:

   http://oser.cvs.sourceforge.net/oser/exchange5.5/exploration/test/mapi_lib/

this was where i got to on my second major network-reverse-engineering
project.

it might not immediately look relevant, until you look in the
"aparser" and the
"templates" directories.

in "aparser", you will find an awk-based IDL compiler (don't
laugh!  it works!).
 it's a generic IDL compiler with a lot of quirks and a very fussy syntax
(IDL
input has to be one struct per line; no forward-referencing etc etc. the usual
things you'd expect of a compiler thrown together in a few days).

in "templates", you will find ... whoops, there are two separate
templates
directories: one for the server and one for the client:
http://oser.cvs.sourceforge.net/oser/exchange5.5/exploration/test/mapi_mta/ is
the other one.  duplicated copies of aparser yes i know i know - i was in
a hurry :)

you will find two separate sets of templates, and from the mapi IDL file i was
reverse-engineering, one set of templates generated the "stub server"
code and
the other set generated the "client" code.

it was crude - and effective.

after doing _so much_ network-based reverse-engineering, and having done _three
years_ of hand-crafting of RPC packets, when i tackled MAPI i found it
_infinitely_ preferable - and quicker - to develop the IDL compiler to "do
what
i wanted" than it was to slog through thousands of lines of hand-crafted
code.

_even_ when i didn't know what the specification of the protocol i was
reverse-engineering.

the nice thing about aparser - which was originally written by andrew tridgell -
was that it's short enough, and in few enough lines, for even a non-awk
programmer like myself to "get to grips" with it, and hack in the
features that
i needed, to support marshalling of MAPI, when aparser had originally been
designed to do SMB and MSRPC.

i always intended to do a third template set: an ethereal (now wireshark)
"sniffer" set.

i cannot overemphasise enough how vital it is that you consider doing the
client _and_ the server _at the same time_ (and, even better, the sniffer
as well).

and also to do - day after day, every single time you do any tests - as
many cross-system bootstrap matrices as you can possibly find.  in this
rather weird multi-free-software reverse-engineering effort, that means
_four_  no .. _five_ clients and five servers to do the per-packet
bootstrapping on:

* pyamf/tape/rtmp client + server (when they exist) 
* red5 (server only? no client?)
* gnash (rtmpget + cygnal)
* swf-dec (client only?)
* Adobe FMS and Actionscript

it's _vital_ that you do your client and your server in lock-step, one
packet at a time.  sniff "adobe-to-adobe", get one client packet; code
it up in your client.  sniff "yourclient-to-adobe", receive the
hexdump
in your client and implement it in your SERVER as well as your client.
sniff "adobe-to-yourserver" and send back the hexdump you just
received
from the previous test.  if you "got it right" the adobe client will
send you the NEXT sequence in the packet chain, and you've completed
one entire loop.  repeat.  repeat.  repeat.  one packet at a time,
four steps per packet.

actually ... six steps, if you include implementing the network sniffer
as well.

oh - as a double-check, you try "yourclient-to-yourserver".  and also
any interop tests (against other clients and other servers).  ironically,
i often found that, following this method, my own hand-crafted client code
vs my own hand-crafted server code would SUCCEED against the server and
client i was network-reverse-engineering but FAIL when talking to each
other!  [the perils of hand-crafting RPC code that is auto-generated by
an IDL compiler...]

in following this procedure, you can see why it is utterly vital to
start using an IDL compiler almost _immediately_.  with _six_ sets of
near-identical code to write, using a template-based compiler to
auto-generate the code is an absolute godsend.

aparser isn't "efficient", in terms of code compactness - but it
doesn't
matter. what it will get you is _some_ code - consistent code that you
can RELY on, until the understanding is deep enough to be able to write
a better (optimised) compiler.

plus, if all projects utilise the same IDL specification and compiler
technology, all projects go much much faster, benefitting from sharing
of development via a formal language.

lastly, i leave you with this: in the precursor version of aparser, i
developed an smb server and matching smb client library (called cliffs)
which were actually auto-generated almost directly from paul leach's
CIFS specification document!  the CIFS.txt file spec contains
"structs"
which i simply cut/paste out of the document, tidied up, and then made
aparser's syntax understand them :)

l.

oh - just one other thing: the prs_xxxx functions which you will
see in the mapi code have a "debug" mode.  consequently, as the data
is being marshalled (or unmarshalled), the auto-generated code
automatically outputs a human-readable dump of the data.

so you get sniffing and decoding - like ethereal/wireshark - "for
free".

and, i made sure that the templates performed "indentation" of
structs.  so, when you identified a struct correctly, it would be
tab-indented.

i read the wiki page on SOL on gnashdev - the lovely comments about
how to use gdb to do binary dumps - it looks like _real_ hard work :)

make life easier for yourself: integrate a tab-indented debug-dump into
the marshalling code.  that way, when things "go wrong", and you
encounter
some packets which don't "fit", it's blindingly obvious,
because the
indentation goes absolutely nuts.

l.