The High-Resolution Audio Challenge
Recent posts have challenged previous studies trying to establish whether high-resolution audio is perceptible. There’s also a lot of conversation about the same topic on a closed professional audio engineering list I belong to. I found Ethan Winer’s “high-resolution” examples lacking and Meyer and Moran’s study seriously lacking because of the content used. Not surprisingly, both of these gentlemen argued that I need to conduct a study of my own. While I was on the CEA audio board, I submitted a proposal to have them conduct a rigorous study. They weren’t interested although they did use some of my files for a comparison marketing survey. The participants listened through the files in a glass walled, conference room on a small scale system and not surprisingly, they couldn’t detect any differences.
I read a lot of comments by FB users saying they can easily detect the difference between and CD and high-res file. Or that HD music is “dramatically” better than CD quality. I doubt these claims.
What would a true test look like? Is the core question whether human hearing can consciously detect a high-res file from a downconverted version of the same file? We already know that there is greater fidelity (fidelity meaning more closely matching the source feed) in a high-resolution PCM file. The plots that I’ve provided in the previous few articles clearly shows the presence of ultrasonics and increased dynamic range. But can any listener with a good system listen to a standard-resolution file and a high-resolution file of the same recording and perceive any differences? It’s important that the only difference between the two files be the reduced frequency response AND the lower dynamic range. To keep the comparison fair, both files have to be level matched and otherwise be identical — same file size, same sample rate, and word length.
So here we go. I’ve uploaded 6 different tracks to my FTP site (1.22 gigabytes) in a folder called:
_Hi-Res_vs_Redbook CD Survey June 2018.
To get the credentials for the FTP site, please click on the FREE HD-AUDIO banner on the right sidebar or click here.
My process for preparing these files is as follows: I opened the original 96 kHz/24-bit PCM files into Adobe Audition. To create the Redbook version, I downconverted the high-resolution master to 44.1 /16-bits using triangular dither and noise shaping (which I’ve found to be the most neutral conversion process _ you may disagree. I open to other processes.). Then I upconverted the Redbook version to a new 96 kHz/24-bit PCM file. I analyzed the two versions to make sure the volumes and frequency plots are the same (except for the steep cutoff at 22.050 kHz) and the dynamic range was reduced as a result of the bit reduction. Take a look at the following illustrations:
This is the analysis for the original file. Notice the extended frequency response, the smooth ultrasonics in the spectrum, and the extreme dynamic range. [Click to enlarge]
Here’s the downconverted version at Redbook specifications. The hard cut at 22.050 is present, the spectra has a flat cutoff, and the dynamic range is 50-60 dB less! [Click to enlarge].
So I invite any reader to download these files (you may have to wait for the server to become available) and do your own listening comparison. Obviously, it’s pointless to cheat by doing your own analysis with Audition, Audacity, or Sound Mirror. What’s the point of doing this if you cheat? There 6 different selections labelled A or B. Simply let me know which of the two you believe is the high-resolution version and which is the Redbook downconverted one. I’ll tabulate the results after a few weeks and see what we get. It should be interesting. It would be helpful if you let me know about your system (headphones vs speakers), you listening environment, age, and level of training. I would also ask you to affirm in your email that you have not attempted to circumvent the protocol laid out above and that you have only listened to the files in making your judgement. Thanks.
Once you’re ready to submit your results, please visit The High-Resolution Challenge Results Page. You’ll be asked to fill in some fields on a form. Please don’t post these files elsewhere or post your picks. It would be great to get 100 people to participate. I’ll give a free AIX Records sampler to the first 10 people that send in their guesses (free shipping too!)
This is casual test and in no way is meant to be definitive. But using materials that have better fidelity than Redbook CDs might at least remove my primary complaint against previous studies. I look forward to hearing from those willing to participate. Thanks and have fun.
Mark
I would be willing to support a crowd sourced study. Keep up the scientific research.
Cheers
Drew Wilson Toronto Canada
Hear Hear!!!
I have a lot of high-res; both LPCM (24/96) and SACDs as well as DSD recordings I’ve made myself. I’m not convinced that any High-Res sounds better than Red Book due to the higher sample rate or bit depth. I believe that well produced Red Book can sound just as good (and in some cases better) than commercially available High-Res. Here’s an illustration of what I mean:
JVC has produced some carefully remastered Red Book CDs using a process that they call XRCD. They are expensive, but they sound really good. A case in point – I have a remastered RCA Victor Red Seal from 1955 of Prokofiev’s Lt. Kiji ballet in XRCD. I also have BMG’s SACD of the same title from the same performance and master tape. The XRCD sounds so much better than the SACD that it’s hard to believe that they are the same performance from the same master!
It seems to me that a carefully mastered and produced Red Book CD can sound better than a mediocre produced high-res release of the same material. The production is everything, while the high-res processes seem to be mostly icing on the cake. If every record producer did as good a job as JVC does with their XRCD process, high-resolution formats would not be necessary.
The XRCDs are very, very good CDs…but they are still CDs and therefore aren’t capable of providing real high-res music. But I’m with you that most — virtually all — recordings can be successfully delivered on CDs. The Prokofieff you refer to would have been made from an analog master, of course. If you transferred the master to SACD and to CD, they should sound identical. The fidelity of both formats is far greater than analog tape. I suspect the products you refer to were mastered from different sources.
Mark,
It is my opinion that perhaps the greatest audible difference between “high-res” and Redbook audio is due to the filter employed in the playback of the Redbook format. A dramatic example can be heard by listening to the three different antialiasing filter characteristics that can be selected in recent Mark Levinson products that incorporate Precision Link DACs, such as the No.519 Media Player, No.585 Integrated Amplifier and the No.526 Preamplifier. You can choose between “Fast,” which has a steep roll-off, offering the best measured performance, “Slow,” which has a gentler roll-off of high frequencies and exhibits less pre- and post-ringing on transients, and “Mininum Phase,” which has a steep roll-off of high frequencies, but unlike the other two filters, only exhibits post-ringing on transients. The fact that there are very obvious audible differences between these three options is what causes me to conclude that there is a clear advantage to sample rates that move any filter artifacts well-into the ultrasonic range. Clearly, one can also conclude that any comparison between high-res and Redbook quality must take into account the type of antialising filter in use, and the artifacts arising thereof.
Thanks Kevin…I think you’re right. That’s why this Hi-Res Challenge doesn’t really matter to me. I want the added bandwidth as a safety margin.
Regarding Kevin’s comments, does Audition describe the type of low pass filter used in the down sampling step?
Yes, it does.
I love the intent and the methodology of your test. To me it seems a no brainer, when you listen to older analogue music upsampled to higher res. The tape hiss and background noise are so evident. Dynamic range non existent.
I think the test is so dependent on the audition stereo that is being used in the test.
Is there back ground ambient noise in the room? my back ground in a small city urban area in a old house is 47-53 db.
I am listening and testing on a pair of Klipshorns, one of the highest sensitivity speakers over the last 70 years. When I put you AIX 5.1 discs on my system it is so evident, as there is no, not a lick of background noise or sound until the music passages start. I set my system for a average of 95db. The dynamic range jumps out of your disc like a lightning bolt. Most of the modern tracks recorded in the loudness wars era are ~10-15 db higher from a base level. Should it not be easy to sample your music profile and compare it to regular CD, Flac, and lesser music on a studio system like you have and perform a real time comparison??
It is one thing to perform testing on a wave form scope, but in real life on a good sound system, the Dynamic range and soundstage particularly when I go to 5.1 with your material is so discernible. Asking people, even audiophiles to listen when the parameters are not preset, I think is a stretch. Just as you have expounded on to a great extent. I listen to vinyl and I am dumbfounded on how any audiophile who has evolved can say that it sounds anything like your true 5.1 24/96 content??? Looking forward to the sample material
Great! I will give it a go.
Also, I am delighted to see that the LUFS is set to nearly -20 dB for these files. If only that was more common!
Mark,
great stuff! I totally agree that the right way to do the HiRes – RedBook comparison is to start off with material that’s truly HiRes, not some transfer from an old analogue master. The UK magazine HiFiNews publishes reviews of HiRes releases every month and thus reveals that a very large proportion of those releases constitute upsamples of 44.1/ 48 KHz material. I believe that, apart from some notable exceptions, the music industry is not really interested in audio quality. It is much more interested in reselling its back catalogue in infinite variations.
An example of a positive exception is the 40th anniversary edition of Supertramp’s Crime of the Century (A&M 0600753307885), published in 2014. Clearly, the remastering was done with a lot of tlc and it’s shipped in beautiful packaging. If all rereleases were of this quality I’d be a very happy bunny, indeed. Still, probably due to the clean remastering, it is very obvious this was recorded on 70’s analogue tape equipment, especially in the more ‘intense’ parts of the songs. You can transcribe them to DSD or HiRes PCM, but that won’t make any difference.
Now, I just need to think about tackling your challenge, as I have just my NAIM UnitiLite as the one device capable of playing HiRes files – need to think how to get your files there – probably a USB stick. Thanks for the challenge!
Cheers,
Hans Sas.
I have a question and a suggestion:
Question:
The link leads to the contact page. Is that correct? Should I just write you an e-mail to get the credentials to the server?
Suggestion:
Edit your post here and ask participants to download the free program Foobar as well as its separate ABX plugin and then do an ABX test of the files, preferably with 16 trials, and then have them save and send you the log it produces. If they get a really good result, please ask them to take the test again to avoid any false positives.
This is a much better way to determine if anybody can hear a difference than have them guess which file is which. Not to sound too paranoid, but it is after all possible for people to lie and say they didn’t check the files in an audio editor when in fact they actually did – some hi-res advocates are not the most honest people it seems (case in point: most refuse to take a blind test).
This is a terrific idea. I’m sorry I didn’t think of it although I am aware of Foobar and the ABX plugin. I’ll post an update today and make that suggestion.
Great :-)!
So is it correct that I would need to send an e-mail through the contact page to get access to the server?
I downloaded two file sets and brought them into Sound Forge. The file identities were immediately obvious just by looking at their spectrum. Nearly 80 people responded to my test, and many of them admitted they looked at an FFT. So your test here is fatally flawed because there’s no way to know who looked and who only listened.
To anyone who lives near me in New Milford, Connecticut (near Danbury), you are invited to arrange a visit and I’ll play Mark’s files through either or both of my two very high fidelity systems while you listen blind. That avoids any chance of cheating, and then we can report here what happened. This is the only way to know if people can tell HD from CD, short of Mark doing what I did to add fake ultrasonic content to defeat cheating.
Ethan, I’ve acknowledged that this survey is not a rigor study. I asked participants to confirm that they only listened before making their choices and will trust that they did so. Messing around with the ultrasonics is not a good idea IMHO. I’d rather trust people to report. What is there to gain by cheating?
I agree there’s nothing for your benefit to be gained by cheating, but people will anyway if only to impress you with their submission. But the core principle you are testing here is flawed. Nobody can hear ultrasonics. That’s what the “ultra” part means. :->) So why does this topic warrant endless interest and investigation? It was determined more than 50 years ago that people can’t hear when ultrasonics are present then removed. The famous Oohashi study was easily refuted when the test was repeated with extra care to avoid IM artifacts aliasing down into the audible range. How many more tests must be done before people will stop fretting over silly stuff that doesn’t matter at all, and start focusing on what really does matter? For example, loudspeaker distortion and beaming, and of course room acoustics which VERY FEW audiophiles even know about let alone address.
I do believe that the people that read my blog and appreciate what I have to offer on this topic don’t need to “impress” me. You insist that history has already determined that ultrasonics have no impact — maybe they aren’t perceptible. But it’s simply too easy to capture, reproduce, and distribute audio that matches the entire frequency output of a musical performance, I would argue why not?
You ask “why not?” Besides bandwidth and storage costs, which are never free, all of the HD content I see costs more than normal audio. So there’s three reasons for you.
Look, for me the issue is keeping the facts straight and avoiding unscientific nonsense to become accepted as true. Fake news, if you will. It just makes society dumber. The fact is nobody can tell HD versus CD, and that’s the real point. for me.
The bandwidth and storage of 96 kHz/24-bit PCM is trivial these days. The additional cost of “high-resolution” audio is market driven and thus not a constant. It could cost the same as standard-resolution. My point has always been that the stuff being offered as high-resolution is not worth the added storage and bandwidth — and certainly not worth the added costs. You don’t want to acknowledge that pre and post ringing are moved out of the audio band, LPFs filters are less harsh, fidelity to the content present in the room during performances is increased, dynamic range matches real world hearing, and maybe there are unknown benefits that we haven’t yet identified. It’s just too easy to not do it for me. You’re welcome to disagree and as I’ve said in practical terms high-resolution is meaningless because the industry is using it for marketing purposes alone.
And that’s why I’ve always respected your opinion on this topic, Mark, because you say “well, I can’t hear the difference, nor does it seem that anybody else can, but if the content is in the original signal, I want to include it in the product I sell to my customers”.
There are intellectual reasons why I choose certain products over others as well. An obvious example is that I might like the sound of tube amp XYZ over solid state amp XZY, but I don’t feel comfortable paying a lot of money for distortion and an uneven frequency response (the latter I can probably make myself with an equalizer, which will be much cheaper).
So, I think it’s perfectly fair to choose A over B for intellectual reasons, as long as that’s stated, which it is in your case :-).
Personally, I still choose CD quality due to cost, accessibility (most of my music is only available on CD) and disc space (and, obviously, because I can’t hear a difference), but I have no problem paying for hi-res if that’s all that’s available (as with your catalogue).
So, let’s just do the test (preferably ABX as I suggested yesterday) and see what it leads to, and then we’ll hopefully all be the wiser :-).
Hi Mark, I applaud you for doing this and lets hope it garners more interest…
The realisation is that – as has now been admitted with nutritional studies – the only way to get accurate data is in a ‘closed ward’ study. In other words the participants are effectively ‘contained’ within a controlled environment so they don’t sneak in candy bars, you get the point in hope! Human beings lie, often without realising they are doing so and the end result is studies that are fatally flawed.