Greetings.
In looking through the mail archives, I don't see any mention of
parallelism of the vorbis code (particularly the encoder). I was
wondering if anyone was looking into this at all? Either with threads,
and/or on some kind of parallel machine (e.g., a Beowulf cluster)? I see
very little mention of threads in the source code, which leads me to
believe that they are only there for their respective frameworks (e.g.,
the xmms plugin).
I ask because I would probably be willing to help in this department. I
am a maintainer of one of the freeware implementations of the Message
passing Interface (MPI) -- LAM/MPI. As such, I know a bit about parallel
programming, both with threads and with MPI. Indeed, several months ago I
did a naieve implemenation of parallelizing the bladeenc MP3 encoder
(bladeenc was the first encoder that I found in a web search). I don't
claim to understand much of the MP3 process (indeed, one of the
assumptions that I used about the MP3 encoding process in parallelizing
bladeenc turned out later to be false. Oops. :-), but the parallelism
does seem to work, and does give mostly linear speedup as you add more
CPUs.
So if this group is willing, I can help out (yes, with code :-) with the
parallelism -- although I will need help understanding the algorithms,
etc. I give the following URLs as reference:
http://www.mpi.nd.edu/~jsquyres/bladeenc/
- The parallel version of the bladeenc encoder, with lots of
explanations
http://www.mpi.nd.edu/lam/
- The main LAM/MPI web site
http://www.mpi-forum.org/
- The main MPI web site (contains the specs, etc.)
Thanks for your time.
(I've read some of the vorbis sites and the archives of this list, but
I'm
still a newbie here -- forgive me if I asked stupid things or if this list
isn't the appropriate place for this post :-)
{+} Jeff Squyres
{+} squyres@cse.nd.edu
{+} Perpetual Obsessive Notre Dame Student Craving Utter Madness
{+} "I came to ND for 4 years and ended up staying for a decade"
--- >8 ----
List archives: http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
At 10:23 2000/08/16 -0400, you wrote:>In looking through the mail archives, I don't see any mention of >parallelism of the vorbis code (particularly the encoder). I was >wondering if anyone was looking into this at all? Either with threads, >and/or on some kind of parallel machine (e.g., a Beowulf cluster)? I see >very little mention of threads in the source code, which leads me to >believe that they are only there for their respective frameworks (e.g., >the xmms plugin).I knew there was something I was forgetting! I was going to ask about this. It would help greatly for mass-encoding. If a web-radio shop wanted to mass-encode a whole bunch of CD's, they could setup a Mosix NOW, or a Beowulf cluster, and take great advantage of it (I want to do this for my CD collection, too. Got some spare 486's kicking around. 8-) ). I'm more of a user than a developper, but I think this is a fabulous idea. I've got a few questions about this.. Is the coding required for parallelism detrimental in for single-CPU uses? If so, could it be enabled/disabled with a command-line switch, or would that add too-much overhead (like a FAT binary for MacOS when switching from 0x0 to PPC)? Thanks! Adam Toronto, Ontario, Canada --- >8 ---- List archives: http://www.xiph.org/archives/ Ogg project homepage: http://www.xiph.org/ogg/
> Greetings. > > In looking through the mail archives, I don't see any mention of > parallelism of the vorbis code (particularly the encoder). I was > wondering if anyone was looking into this at all? Either with threads,Hi, What would be the advantages of a parallel encoder? If you need the extra speed, you're probably encoding lots of tracks, so just start a bunch of encoders at once? The present vorbis example encoder (not optimized) is better than realtime, so that's no problem either. I guess I miss the point of parallellizing it... Ciao, Segher --- >8 ---- List archives: http://www.xiph.org/archives/ Ogg project homepage: http://www.xiph.org/ogg/
At 10:23 AM 8/16/00 -0400, you wrote:>Greetings. > >In looking through the mail archives, I don't see any mention of >parallelism of the vorbis code (particularly the encoder). I was >wondering if anyone was looking into this at all? Either with threads, >and/or on some kind of parallel machine (e.g., a Beowulf cluster)? I see >very little mention of threads in the source code, which leads me to >believe that they are only there for their respective frameworks (e.g., >the xmms plugin).I looked at it (in the context of an encoder, but without modification of the library, or at least not significant modification. I can't see anywhere where it would be practical and useful (there is one bit that would naturally split over two cpus, but you'd have one cpu doing 95% of the work - it's not worth it). If you made significant internal modifications to the library, it might become practical, but I'm not sure - more importantly, I can't see why it would be useful. If encoding a single file, the encoder is fast enough (i.e. better than realtime on a fairly recent cpu). If encoding multiple files, you can spawn multiple encoders. However, it might be more user complexity than is desired to have to explicitly start multiple encoders. In this case, doing one thread per file (up to some limit - you might have a dual cpu machine, and thus use a hypothetical -p 2 option) might be useful. If anyone thinks they'd actually use this, let me know (offlist). If I get more than one or two answers of 'yes', I'll try and add support for this to oggenc (oggenc is written such that the addition of threading shouldn't cause any additional problems, so this should be fairly easy). Michael --- >8 ---- List archives: http://www.xiph.org/archives/ Ogg project homepage: http://www.xiph.org/ogg/
>That's why, if you see the MP3 ripping/encoding programs, that the encoding >is always a forked child, because it takes MUCH more time than ripping >does. It's also much more CPU intensive.Actually, in my ripper (for Win32) I've got ripping and encoding both off in their own threads, to allow the GUI to still be responsive. I suspect that others do as well. ________________________________________________________________________ Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com --- >8 ---- List archives: http://www.xiph.org/archives/ Ogg project homepage: http://www.xiph.org/ogg/
Some people asked why would anyone use a parallel vorbis encoder. It is fast enough already, faster than real time ! My answer to this is, that there is no such thing as a too fast program ! So it encodes at 2x realtime, that is it encodes 60 minutes of audio in 30 minutes. Now wouldn't it be much better if it only took 30 seconds ? Anything that takes more than a second is too slow :-) David Balazic --- >8 ---- List archives: http://www.xiph.org/archives/ Ogg project homepage: http://www.xiph.org/ogg/
Again, majordomo rejects messages with 'which' on a line by itself
(it's an
admin command).
Monty
------- Forwarded Message
From: "Timothy Wayper" <timmy@r3.co.nz>
To: <vorbis-dev@xiph.org>
References: <Pine.GSO.4.21.0008180843130.758-100000@general2.asu.edu>
Subject: Re: [vorbis-dev] Parallelism
Date: Fri, 18 Aug 2000 13:02:46 -0500
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2314.1300
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
Note: I've since found that David Burnett has already got vorbis supported
under BeOS in the form of a plugin for CL-Amp and a couple of command line
tools. (Thanks David!)
> Are you thinking GUI or command line utility? In the command line
department,> oggenc should work, or be very easy to fix. As for playback, I'm still
trying> to beat ogg123 into shape. Next I need to fix byte ordering, but after
that I> would love to have someone do a BeOS driver for the libao output library
we're> using.
Ultimately, I would like to see vorbis supported as media add-on under BeOS,
allowing any application to encode and decode vorbis streams as easily WAV's
and MP3's are currently supported :-) (And without the royalties that force
Be to only include MP3 encoding in the Pro version of BeOS.) The user justs
selects Ogg as their output format, the application plugs the appropriate
nodes together, and away we go! I'm sure Be themselves would love this level
of integration.
Having an BeOS driver for libao would certainly be possible, too.
> That brings me to a general question I have about BeOS: ogg123 and libao
are> both C. Can audio output on a BeOS computer be done with C code, or is
C++> required? If C++ is required, what are the issues involved in integrating
C> and C++ code? (I have no experience in this area.)
The use of C++ is is required to play audio under BeOS (except for a system
beep function).
Interfacing the code itself should be fairly easy, there's no reason the C++
BeOS code can't have a pure C API. Where there might be complications is in
dealing with the Be application model. POSIX code can be directly compiled
to command-line apps under BeOS, but interfacing to the App server requires
a C++ application object to be created... but anyway, I'm sure us BeOS guys
can sort out these issues.
Cheers,
Tim W.
------- End of Forwarded Message
--- >8 ----
List archives: http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
At 21:04 18/08/00 -0400, you wrote: Hello,> > Ripping can be *much* faster than encoding. Logically speaking, you have > > a producer (or producers) and a consumer (or consumers). The ripper(s) > > produce the input, and the encoder(s) consume the input. Since one side > > is clearly faster than the other, I don't see why trying to speed up the > > other side is a Bad Thing. > >I disagree. The fastest commercially available ripping driver is the >Plextor Ultraplex. On my desktop I can encode (into MP3, there is no >strong reason to expect that vorbis will not be someday comparable to lame >in speed) faster then I can rip on that drive.WOW!! Can you encode/compress at 20x +/- ??? What CPU(s) do you have ? --- >8 ---- List archives: http://www.xiph.org/archives/ Ogg project homepage: http://www.xiph.org/ogg/