theora - Oct 2006 - Splitting Theora encoding into multiple tasks for Thoggen

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello all,

I have been thinking about multiple core processors and how Thoggen
(http://www.thoggen.net) encodes DVDs to Ogg-Theora files.

I was intending to file a feature request that basically suggested
dividing the total size of the target DVD by the number of logical CPUs
and encoding each chunk of DVD as a separate task so that encoding could
take place simultaneously.

This would result in a number of separate Ogg-Theora files that would
then need to be concatenated together. It is my understanding that this
would be fairly straight-forward, but wanted to check it with this list
first. Can anyone see any issue with this from a Theora point of view?
Am I correct that a file produced in this way should not appear any
different to the end user watching the file?

Thanks for your help,

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFRVZE1AKNiGq1QC0RAr0aAKDnUa1KkjVi1SHsMcwEKq+ey04sBQCgr1wj
BgO8cz/292N/T0PDr8BV9E4=x7co
-----END PGP SIGNATURE-----

>>>>> "Aaron" == Aaron Whitehouse
<lists@whitehouse.org.nz> writes:
> Hello all, I have been thinking about multiple core processors and how
> Thoggen (http://www.thoggen.net) encodes DVDs to Ogg-Theora files.
> I was intending to file a feature request that basically suggested
> dividing the total size of the target DVD by the number of logical
> CPUs and encoding each chunk of DVD as a separate task so that
> encoding could take place simultaneously.
> This would result in a number of separate Ogg-Theora files that would
> then need to be concatenated together. It is my understanding that
> this would be fairly straight-forward, but wanted to check it with
> this list first. Can anyone see any issue with this from a Theora
> point of view?  Am I correct that a file produced in this way should
> not appear any different to the end user watching the file?
If you just cat complete theora/vorbis streams together, the end user
will have to use a software that is capable of working with such kind of
concatenated streams.  Mplayer, AFAIR is not capable.  I also don't know
whether other capable players (VNC, Helix?) will correctly display
time-stamps for the parts of video encoded.

Well you could try to encode multiple parts of the DVD into
theora-streams without vorbis audio. Then you can throw away the header
packet for the second and following parts, adjust the granulepos, stream
serial number and EOS/BOS fields.  And you will in theory get a
contiguous theora stream.  Playback software shouldn't be able to
distinguish the stream from the streams that you currently produce.

But you will have to encode the vorbis audio in one pass.  Since
play-time for vorbis fragments will differ from play time of
theora-fragments (frame-time), you won't be quite able to concatenate
two vorbis-streams in a way that keeps sync with a contiguous theora
stream.

David
-- 
GnuPG public key: http://user.cs.tu-berlin.de/~dvdkhlng/dk.gpg
Fingerprint: B17A DC95 D293 657B 4205  D016 7DEF 5323 C174 7D40

On Mon, Oct 30, 2006 at 02:32:52PM +1300, Aaron Whitehouse wrote:
> This would result in a number of separate Ogg-Theora files that would
> then need to be concatenated together. It is my understanding that this
> would be fairly straight-forward, but wanted to check it with this list
> first. Can anyone see any issue with this from a Theora point of view?
David is correct that there are bugs in a number of popular players' 
handling of concatenated streams, although this is required by the spec.

You can help us encourage them to fix that (if nothing else, it's an 
obvious way to do chapters) but in the meantime, since you're in control 
of the encoding you can also work around the issue. Assuming each 
portion is encoded with the same parameters, you can strip the headers 
from subsequent pieces and remux the stream into a single segment.

I believe Andraz Tori has already written such a thing for cinelerra. 
You might see if you can borrow from his code.

Keep us posted! :)

 -r