Lots of comments throughout this one... On Jan 7, 2011, at 15:28, Declan Kelly wrote:> On Fri, Jan 07, 2011 at 02:22:51PM -0800, brianw at sounds.wa.com wrote: >> However, you should be aware that many modern producers use software >> to create their music, and when the software stores sound clips in >> MP3 format, what you end up with is music that sometimes looks like >> MP3. > > I recently bought the double-CD "Influence" remaster by The Art Of > Noise > and some rarer tracks were sourced from MP3 because that was all their > archivist could find. Most of the reissue was direct from analogue > tapes > so this wasn't a quick "shovelware" reissue job.Very interesting. I've purchased many CD titles with reissued material where they clearly did not have access to anything but vinyl for some tracks. It was weird to hear the needle drop, with rumble throughout, especially since I has the original CD versions of the same tracks in my collection. Also, the early days of Warp Records involved many artists who sent in demoes on analog tape, and there is obvious wow and flutter on these tracks, especially at the beginning before the tape was properly tensioned. A while back, when HD audio was quite rare, I purchased some titles that were clearly low quality. Some 24/88.2 and 24/96 material had nothing but noise above 20 kHz, but the audio content was clearly from CD sources. I suppose the thought was that there was less noise in the audible range below 20 kHz. Certain "HD" titles were atrocious, with obvious indications that the source was 44.1 kHz and the upper frequencies were clearly nothing more than aliased frequencies! What's worse is that the aliased titles were samples on an HD audio site. They clearly didn't get my money.>> it just has to do with the software that was used to create the music >> originally. > > A friend of mine recorded his band's last album on DCC in the mid > 1990s > and released it on CD. It sounds horrible; the lossy compression of > DCC > is even worse than MiniDisc's ATRAC. I'm sure this CD would fail most > FFT quality tests, as literally everyone who heard it (not just people > with "golden ears" or good sound systems) complained about the > quality.I have also worked with mastering of a compilation CD including some material recorded and mixed on DCC. What's interesting in this one case is that the "music" was in the "noise" genre, and the DCC was used as an effect. It could not handle the content and therefore ending up changing it. What I found most interesting was that I had hired a professional studio in Seattle, and the owner actually stuck his head in the room for this one track. He'd heard a lot of interesting music in the local scene, but nothing that messed up.>> Nine Inch Nails who provide FLAC files. > > The initial free online release of NIN's The Slip had a problem > with the > 24-bit version: it wasn't the full 24/96. Turns out it was a genuine > oversight by the mastering lab (who used the 24/96 to master the > vinyl), > and the true 24/96 files were reissued less than a week later. > So even with the artist fully behind 24-bit FLAC, and doing almost all > of the work in their own studio, things can still go wrong.Thanks! That's interesting to note. I think that I ended up with the true 24/96 files, but I am curious: How do you tell whether you have the full 24/96 or not? I have written software which can detect 16-bit audio samples stored in a 24-bit file format. Frequency analysis makes it obvious whether the content extends above 20 kHz. But I just want to make sure that there isn't another technique that I'm missing.>> Some audio turns out smaller with ALAC, other audio turns out >> smaller with FLAC. Overall, the average performance is identical. > > I've been trying Flake (an alternative FLAC encoder; all it does is > encode) recently and while it goes way faster than the reference FLAC > encoder in terms of speed, the first encode I tried with it ended up > larger than with the reference encoder, at all compression levels. > The compression levels in Flake go up to 11, but past 9 or 10 there is > no guarantee of full compatibility for playback.The whole picture is a bit inconsistent. If Flake is only an encoder, and compression levels above 8 are not guaranteed to be compatible, then what's the purpose? If Flake cannot decode, then what good is it to create a file that no other decoder can handle?>> Many tools run the >> command-line flac utility behind the scenes. Others use the FLAC >> library directly. > > About a month ago I was setting up FLAC support for a friend on a > Windows PC. Almost every GUI front end (based on the latest available > download) that I tried was using an out of date version of the FLAC > binary or library, and had the default compression set to -6 or lower. > There is no excuse for not cranking up the compression level on any > modern computer: surely you want to get the files as small as > possible?Based on similar experiences, I refuse to use anything but the official FLAC command-line for all of my work. The GUI front ends sometimes look convenient, but I don't trust the quality. Besides, many of the GUI front ends are actually quite horrible from a UI perspective, so they seem to have no redeeming qualities.>> I doubt there would be any professional >> interest in changing things just for the sake of change or "newness." > > The only "new" thing I want in FLAC is a fix for a minor bug in the > way > the command line tool treats the end of line. If long filenames > push the > "percent complete" past the screen width and you are encoding or > testing > a lot of files, you can end up with the longer file names having 20 or > more lines on the screen output, and scrolling the previous files off > the screen. My workaround is to use a longer scrollback in whatever > terminal I'm using, or use GNU Screen to get scrollback on a text > console.Ok, despite the fact that I said FLAC should not be changed, I actually agree with you that the command-line could use some improvements and bug fixes. Every time I see the long filenames issue, I worry that there is a problem, until I remember that this is a known issue and just ignore it. One of my goals would be to add support for the Apple CAF format. This would actually serve a very important need. My FLAC recorder can create files that are up to 4 GB compressed, which is literally too large when uncompressed to WAV or AIFF. Usually, I'm dealing with long recordings of multiple performances, so I can just tell FLAC to grab the first hour, or the last hour, and that's small enough to fit within the 2 GB or 4 GB limit. In contrast, CAF has no limit on file length, so I could theoretically uncompressed a 4 GB FLAC into a 9 GB CAF without any problem. I can't decide whether FLAC-to-CAF (and CAF-to-FLAC) should be developed into a new program based on the FLAC library, perhaps with a GUI, or if it should be incorporated into the standard flac command- line tool. At the very least, if I ever get this working standalone, I would eventually want to roll the feature into the flac command- line for others to use. Anyway, the flac command-line should be improved, even though the file format and library are probably just fine being left alone. Brian Willoughby Sound Consulting
> The whole picture is a bit inconsistent. If Flake is only an > encoder, and compression levels above 8 are not guaranteed to be > compatible, then what's the purpose? If Flake cannot decode, then > what good is it to create a file that no other decoder can handle?Compression levels above 8 for Flake just don't conform to the 'subset' defined in FLAC, meaning that some implementations may not play them. They play fine on libFLAC-based players on desktop platforms, though.
On Fri, Jan 07, 2011 at 05:11:26PM -0800, brianw at sounds.wa.com wrote:> Lots of comments throughout this one...And I'm going to cherry-pick a few replies as it's getting late.> What I found most interesting was that I had > hired a professional studio in Seattle, and the owner actually stuck > his head in the room for this one track. He'd heard a lot of > interesting music in the local scene, but nothing that messed up.My friend's band was some kind of Metal, so you might expect most of their fans to be nearly deaf, or unable to tell the difference. But most of them did. All their previous stuff was recorded and mixed (and then released) on analogue cassette, and sounded way better. DCC was still pretty new at the time, so he chose to "upgrade" the portastudio to DCC instead of MiniDisc, because DCC was backwards compatible with cassette. [NIN 24/96]> Thanks! That's interesting to note. I think that I ended up with > the true 24/96 files, but I am curious: How do you tell whether you > have the full 24/96 or not?Extract to WAV, do a hex dump, and look for repeated 0x00 bytes. Someone on the hydrogenaudio forums did that, reported it on the NIN forums, and Reznor got the reissued 24/96 FLAC'd and seeded on tracker.nin.com in a couple of days.> 16-bit audio samples stored in a 24-bit file format. Frequency > analysis makes it obvious whether the content extends above 20 kHz.Google for that hydrogenaudio thread: Reznor made one post on it, and mentioned that the recording was done at both 24/96 (Lavry) and 24/192 (Apogee) on all songs, and they chose whichever they preferred at mix time.> If Flake is only an > encoder, and compression levels above 8 are not guaranteed to be > compatible, then what's the purpose? If Flake cannot decode, then > what good is it to create a file that no other decoder can handle?Because it (hopefully) produces compatible FLAC files that can be decompressed by any FLAC decoder out there. It's still at version 0.11 and is intended as an alternative to the reference encoder. I knew about it, but didn't try it until it popped up in my Ubuntu package manager when I was searching for something else. http://flake-enc.sourceforge.net The extra stuff it does is variable block size encoding (which is legal according to the spec, just not implemented in the reference encoder. If you try the higher compression levels there is a warning that the files created might not work with all FLAC decoders. I can understand working on an alternative encoder first, because if it is to remain true to format, any decoder should always be able to work on what it creates. The metaflac tool will display what encoder was used to create any FLAC file. I was wrong about it going up to 11 - it actually goes up to 12. But like the reference encoder, those levels are just shortcuts to more complex options (prediction type, block size, etc). My first test was a 16-minute WAV ripped from CD, encoded at all the compression levels that Flake has, and then tested using flac -t (and they all passed). However they were all bigger than what flac -8 produces, so I have done no more testing. I really need to throw more varied music at it, then load it all up on a microSD card and see if any hardware players cannot deal with the higher compression levels. But if there isn't an appreciable reduction in size then I won't be changing over from the reference encoder.> Every time I see the long filenames > issue, I worry that there is a problem, until I remember that this is > a known issue and just ignore it.Thanks, I thought it was just me...> Anyway, the flac command-line should be improved, even though the > file format and library are probably just fine being left alone.The format should definitely remain frozen: that was one of the founding principles of the ARJ compressor, after widespread confusion when PKZIP 2.0 was released (older versions of PKUNZIP could not deal with the new compression methods). -- -Dec. ---
> I was wrong about it going up to 11 - it actually goes up to 12.Too bad. I thought for a minute there that it goes up to eleven because... "Well, it's one louder, isn't it? It's not ten. You see, most blokes, you know, will be playing at ten. You're on ten here, all the way up, all the way up, all the way up, you're on ten on your guitar. Where can you go from there? Where? " Oh well. - BW On Fri, Jan 7, 2011 at 9:08 PM, Declan Kelly <flac-dev at groov.ie> wrote:> On Fri, Jan 07, 2011 at 05:11:26PM -0800, brianw at sounds.wa.com wrote: > >> Lots of comments throughout this one... > > And I'm going to cherry-pick a few replies as it's getting late. > >> What I found most interesting was that I had >> hired a professional studio in Seattle, and the owner actually stuck >> his head in the room for this one track. ?He'd heard a lot of >> interesting music in the local scene, but nothing that messed up. > > My friend's band was some kind of Metal, so you might expect most of > their fans to be nearly deaf, or unable to tell the difference. But most > of them did. All their previous stuff was recorded and mixed (and then > released) on analogue cassette, and sounded way better. DCC was still > pretty new at the time, so he chose to "upgrade" the portastudio to DCC > instead of MiniDisc, because DCC was backwards compatible with cassette. > > > [NIN 24/96] > >> Thanks! ?That's interesting to note. ?I think that I ended up with >> the true 24/96 files, but I am curious: How do you tell whether you >> have the full 24/96 or not? > > Extract to WAV, do a hex dump, and look for repeated 0x00 bytes. Someone > on the hydrogenaudio forums did that, reported it on the NIN forums, and > Reznor got the reissued 24/96 FLAC'd and seeded on tracker.nin.com in a > couple of days. > >> 16-bit audio samples stored in a 24-bit file format. ?Frequency >> analysis makes it obvious whether the content extends above 20 kHz. > > Google for that hydrogenaudio thread: Reznor made one post on it, and > mentioned that the recording was done at both 24/96 (Lavry) and 24/192 > (Apogee) on all songs, and they chose whichever they preferred at mix > time. > > >> If Flake is only an >> encoder, and compression levels above 8 are not guaranteed to be >> compatible, then what's the purpose? ?If Flake cannot decode, then >> what good is it to create a file that no other decoder can handle? > > Because it (hopefully) produces compatible FLAC files that can be > decompressed by any FLAC decoder out there. It's still at version 0.11 > and is intended as an alternative to the reference encoder. I knew about > it, but didn't try it until it popped up in my Ubuntu package manager > when I was searching for something else. > > http://flake-enc.sourceforge.net > > The extra stuff it does is variable block size encoding (which is legal > according to the spec, just not implemented in the reference encoder. > If you try the higher compression levels there is a warning that the > files created might not work with all FLAC decoders. > > I can understand working on an alternative encoder first, because if it > is to remain true to format, any decoder should always be able to work > on what it creates. The metaflac tool will display what encoder was used > to create any FLAC file. > > I was wrong about it going up to 11 - it actually goes up to 12. > But like the reference encoder, those levels are just shortcuts to more > complex options (prediction type, block size, etc). My first test was a > 16-minute WAV ripped from CD, encoded at all the compression levels that > Flake has, and then tested using flac -t (and they all passed). However > they were all bigger than what flac -8 produces, so I have done no more > testing. > > I really need to throw more varied music at it, then load it all up on a > microSD card and see if any hardware players cannot deal with the higher > compression levels. > But if there isn't an appreciable reduction in size then I won't be > changing over from the reference encoder. > > >> Every time I see the long filenames >> issue, I worry that there is a problem, until I remember that this is >> a known issue and just ignore it. > > Thanks, I thought it was just me... > > >> Anyway, the flac command-line should be improved, even though the >> file format and library are probably just fine being left alone. > > The format should definitely remain frozen: that was one of the founding > principles of the ARJ compressor, after widespread confusion when PKZIP > 2.0 was released (older versions of PKUNZIP could not deal with the new > compression methods). > > -- > -Dec. > --- > _______________________________________________ > Flac-dev mailing list > Flac-dev at xiph.org > http://lists.xiph.org/mailman/listinfo/flac-dev >
On Jan 7, 2011, at 18:08, Declan Kelly wrote:> On Fri, Jan 07, 2011 at 05:11:26PM -0800, brianw at sounds.wa.com wrote: > [NIN 24/96] > >> Thanks! That's interesting to note. I think that I ended up with >> the true 24/96 files, but I am curious: How do you tell whether you >> have the full 24/96 or not? > > Extract to WAV, do a hex dump, and look for repeated 0x00 bytes. > Someone > on the hydrogenaudio forums did that, reported it on the NIN > forums, and > Reznor got the reissued 24/96 FLAC'd and seeded on tracker.nin.com > in a > couple of days.My 16-bit detector does exactly that, except that it only looks for 0x00 in the lowest 8 bits of each sample. I used to use hexdump, but didn't trust my eyes when scanning manually. With a program, there aren't any false positives for 0x00 bytes in other positions. What it does is scan until the first sample is found with something in the lowest 8 bits, and then reports the file as true 24-bit and quits early. If it scans the entire file without finding any 24-bit values, it gives the sad news that it's really 16-bit samples disguised as 24. By the way, I had to special-case 0x00800001 and treat it as 16-bit. I don't know whether it was the MOTU 896HD or Logic, but something was creating that one value in the midst of an otherwise 16-bit pure file. But there are tens of millions of other 24-bit values, so ignoring that one won't create a false report.>> 16-bit audio samples stored in a 24-bit file format. Frequency >> analysis makes it obvious whether the content extends above 20 kHz. > > Google for that hydrogenaudio thread: Reznor made one post on it, and > mentioned that the recording was done at both 24/96 (Lavry) and 24/192 > (Apogee) on all songs, and they chose whichever they preferred at mix > time.Thanks. Brian
On Fri, 2011-01-07 at 17:11 -0800, Brian Willoughby wrote:> Ok, despite the fact that I said FLAC should not be changed, I > actually agree with you that the command-line could use some > improvements and bug fixes. Every time I see the long filenames > issue, I worry that there is a problem, until I remember that this is > a known issue and just ignore it. > > One of my goals would be to add support for the Apple CAF format. > This would actually serve a very important need. My FLAC recorder can > create files that are up to 4 GB compressed, which is literally too > large when uncompressed to WAV or AIFF. Usually, I'm dealing with > long recordings of multiple performances, so I can just tell FLAC to > grab the first hour, or the last hour, and that's small enough to fit > within the 2 GB or 4 GB limit. In contrast, CAF has no limit on file > length, so I could theoretically uncompressed a 4 GB FLAC into a 9 GB > CAF without any problem. > > I can't decide whether FLAC-to-CAF (and CAF-to-FLAC) should be > developed into a new program based on the FLAC library, perhaps with > a GUI, or if it should be incorporated into the standard flac command- > line tool. At the very least, if I ever get this working standalone, > I would eventually want to roll the feature into the flac command- > line for others to use. > > Anyway, the flac command-line should be improved, even though the > file format and library are probably just fine being left alone.The sndfile-convert utility which comes with libsndfile can almost certainly be used to do this (either to CAF or RIFF64). It's less than ideal as an encoder because your control over the FLAC settings is rather limited, but it should decode fine. If you want a proper two-way tool that is I think the way I'd do it - by writing a (command line) converter that links to reference libflac and to libsndfile for the file I/O (unless you wanted to be Mac-only and used the system quicktime interfaces for CAF), and can provide whatever options you want. Alternatively, it looks like the sf_command interface in libsndfile could be used to pass through more encoder settings to the FLAC support in libsndfile (which uses the reference libflac to do the work), which would make sndfile-convert capable of the job http://www.mega-nerd.com/libsndfile/command.html This would probably be a lower effort (certainly less reinvention) way to do the job. Richard