ogg dev - Mar 2013 - can you suggest on extending ogg as short-clip container and the make of its tool?

On 6 March 2013 18:26,  <gildororonar at mail-on.us>
wrote:
>
> Quoting "Benjamin Schwartz" <ben at bemasc.net>:
>>
>> I'm not sure how you reached this conclusion, but I think you
should
>> revisit it.  I think everyone, including you, will be a lot happier if
you
>> store each sound effect clip in its own file.  This is a common
practice
>> for sound effects and samples in ogg.
>
> No it is not a common practise. In fact I fail to find a single
> computer game using separate file for each context / sound effect
> short clips.
>
> I will start naming a few that I studied in the last a few days.
> Samurai Sprit PC edition, Fallout 3 and Skyrim, L.A. Norie. Most
> common practise is to enclose them in something indexable. For
> example, lots of Bethesda games uses an container format (seems
> non-compressing) they invented themselves, called ".bsa". I'd
like to
> say using each game vendor's own archive format is more likely the
> standard practise.
>
> There are a few strong reasons to do it: the clips are often as short
> as 4kB, in a 64kB cluster file system it is a waste of more than 90%
> space. If they are installed on FAT32 the FS chokes more.
I agree with you that it is practical to archive many sound effect
files into an indexable container, and
I've also heard that this is common in the game industry.

I agree with Ben that a file containing a single Ogg bitstream is a
poor choice of container for that use case.

My understanding is that it is common to have the data for each sound
effect in a standalone Ogg bitstream,
and to pack these Ogg bitstreams into a single file in a different
container which has been designed for
random access.

This index could be as simple as concatenating the sound effect files
and providing a separate
index file mapping filename to (start byteoffset, length in bytes). I
can't see any advantage to
modifying the encoder to use larger Ogg pages, and anything involving
bisection search to find the
data is unnecessarily slower and more complex than just providing an
index of byte offsets.

Conrad.

Conrad Parker <conrad at metadecks.org> wrote:
> My understanding is that it is common to have the data for each sound
> effect in a standalone Ogg bitstream,
> and to pack these Ogg bitstreams into a single file in a different
> container which has been designed for
> random access.
> This index could be as simple as concatenating the sound effect files
> and providing a separate
> index file mapping filename to (start byteoffset, length in bytes).
Then what would be that container format? Which is technically  
suitable and inline with the spirit of xiph?

I checked 4 formats without luck.

I checked GNU Tar format and Microsoft CAB format, both lack index,  
thus requiring bisection search.

I checked ISO format and .cfs file formats, both have good indexes but  
a lot more complicated than necessary, requiring a whole library to  
access the file instead of a few for-loops in C.

I heared Microsoft AVI format has an index, but not sure if it can be  
used to its advantage in this use case.

If the index file is a text file with 3 fields: file name, start  
offset, length, then you cannot avoid using bisection method anyway,  
and the number of bisection search do not reduce, the advantage being  
a smaller file to be operated.



-------------------------------------------------

VFEmail.net - http://www.vfemail.net
$14.95 ONETIME Lifetime accounts with Privacy Features!  
15GB disk! No bandwidth quotas!
Commercial and Bulk Mail Options!

Conrad Parker <conrad at metadecks.org> wrote:
> I agree with you that it is practical to archive many sound effect
> files into an indexable container, and
> I've also heard that this is common in the game industry.

I guess Ben is right that if you only look at opensource computer games, they
are using each file for a single sound effect. I guess the reason is, well,
consider the size of opensource game sector, first, opensource games are
probably not part of the game industry, and the sound effect they use is so few
compare to commercial game products, so their storage problem is also a small
one that are usually ignored.

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/ogg-dev/attachments/20130307/f1509be3/attachment.htm

On 7 March 2013 10:26,  <gildororonar at mail-on.us> wrote:
> If the index file is a text file with 3 fields: file name, start
> offset, length, then you cannot avoid using bisection method anyway,
> and the number of bisection search do not reduce, the advantage being
> a smaller file to be operated.
>
I don't follow this, why would you still need a search if you have an
index? Unless you mean now you need to search the index, which is a
much smaller file that you would typically just parse into some memory
structure before using.

On the general issue of combining short samples, this does keep coming
up. So either: 1. it's a bad, but commonly occurring solution to the
requirement in which case it might be useful to have a 'standard'
better alternative to propose, or 2. it's a good solution in which
case thinking about how it might be best done with ogg could be
useful.
For the simple file proliferation issue, just chain small streams
together and have a parallel index file to the start of each stream is
a workable solution, that could be improved a bit by 'encapsulating'
it (basically add a non-ogg header+index before the ogg).

-- 
imalone
http://ibmalone.blogspot.co.uk