Vorbis dev - Jun 2003 - ogg multithread encoding

Hello!

I'm working on a program which purpose is to compress a single wav file
using many processes. I know that my solution may not look excellent but in my
opinion it's simple to implement it in a program.

What I've managed to do is:
-split single wav file into many parts (with appropriate headers),
-create process for each part,
-each process executes 'oggenc' program.

It seemed to be easy at the beggining to merge all small .ogg files later. But
it looks to be the most difficult task.
Do you know maybe other solution to this problem (or a simple way to merge these
ogg files)?

Thanks in advance
Krzysztof Wayda

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'vorbis-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

On Wed 18 June 2003 15:05, torque wrote:
> Hello!
>
> I'm working on a program which purpose is to compress a single
> wav file using many processes. I know that my solution may not
> look excellent but in my opinion it's simple to implement it in a
> program.
>
> What I've managed to do is:
> -split single wav file into many parts (with appropriate
> headers), -create process for each part,
> -each process executes 'oggenc' program.
>
> It seemed to be easy at the beggining to merge all small .ogg
> files later. But it looks to be the most difficult task. Do you
> know maybe other solution to this problem (or a simple way to
> merge these ogg files)?
Ehm, simply concatenate them together? Ofcourse you have some 
overhead with the headers being repeated, but it will yield a valid 
Ogg file...

Lourens

-- 
GPG public key: http://home.student.utwente.nl/l.e.veen/lourens.key

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'vorbis-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.

I am new to Vorbis and not very familiar with its encoder yet. However,
based on my previous experience with other codecs including MP3 and
MPEG-4, there are two ways you can resolve this problem.

The first method would be suitable for a workstation environment where
you don't have to manage the system resources and have a mature kernel
with multi-threading and sync support. The input data (in this case the
wav file) should not be split into separate wav files with different
headers as you have done. You create a master thread which manages the
encoding and this has the responsibility to read the wav header and
manage the encoding parameters. The master thread parses the wav file
and feeds chunks of it to each of the other threads. The chunk size
should ideally equal the number of samples in a Vorbis frame or the size
of the analysis window. Each of the slave threads then encodes the input
frame data and returns the output data to the master thread. This of
course requires that you have a separate API for encoding just a single
frame. Given that this is an open-source venture you can just extract
the functionality you want from the main encode loop to create a
suitable API.

If you are doing this for an embedded platform with multiple processors
where you have to manage the memory and code layout, etc, it makes sense
to split a codec functionally and have each thread or processor do a
part of the encoding and hand-off to the next thread in the chain for
processing.

Santosh

<p>-----Original Message-----
From: Michael Smith [mailto:msmith@xiph.org] 
Sent: Thursday, June 19, 2003 5:25 AM
To: vorbis-dev@xiph.org
Subject: Re: [vorbis-dev] ogg multithread encoding

On Thursday 19 June 2003 01:28, torque wrote:
> >Ehm, simply concatenate them together? Ofcourse you have some
> >overhead with the headers being repeated, but it will yield a valid
> >Ogg file...
>
> OK, it works:) Actually, it seems to be a vaild .ogg file. But what
about
> those headers? In XMMS, Winamp etc. the time is not properly
displayed. I
> think, that maybe I could leave only the first header (but I don't
know
> exactly how many bytes is it) and update certain values in it somehow.
What
> do you think about it?
You _could_ do this, but it's difficult. There are quite a few special
cases 
you'd need to get right. You should read the spec (ogg spec, and the
relevant 
bits on embedding into ogg and the headers of the vorbis spec) if you
want to 
do this.

You could also come to the realisation (as have other people who have
looked 
into doing this) that a parallel vorbis encoder really isn't that
useful, 
given the encoder speed relative to current (or even somewhat old) cpu 
speeds.

Mike

--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'vorbis-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.
--- >8 ----
List archives:  http://www.xiph.org/archives/
Ogg project homepage: http://www.xiph.org/ogg/
To unsubscribe from this list, send a message to
'vorbis-dev-request@xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is
needed.
Unsubscribe messages sent to the list will be ignored/filtered.