The HD-Audio Challenge Results
In the middle of June, I posted a blog titled the HD-Audio Challenge. Readers were encouraged to download six pairs of music tracks, do some careful listening to determine which was the original 96/24 master, and which was a downconversion to 44.1/16 — CD specifications. Many authorities — including myself — question the value of “hi-res audio” files — at least the ones that are offered by the likes of HDtracks, Qobuz, SuperHiRez, and Tidal. My contention has always been that if you start with a standard-resolution master — typically a recording that was done using analog tape machines — and merely digitize it at 96 or 192 kHz and 24-bits, the end result won’t sound any better then the original. And these new transfers might sound worse because many times the original masters are lost or damaged. Mastering houses are forced to use safety copies or cassette masters instead of the actual master.
But what about new recordings? The AIX Records catalog contains over 80 newly recorded masters captured at 96 kH/24-bit in PCM format. The six tracks that were offered for analysis in the HD-Audio challenge were drawn from the AIX Records catalog. High-resolution audio advocates — and marketing types — believe that adding another octave(s) by moving sampling rates above 44.1/48 kHz and increasing the potential dynamic range by using 24-bits instead of 16 produces an audible difference — an increase in fidelity. I had more than a few readers tell it would be easy — that they would have no problem telling them apart. One of my favorites came from another audio professional who claimed he moved up to 192 kHz because “96 kHz is just too brittle and harsh”. Interestingly, he failed to identify the high-resolution versions from the CD ones, so there you go.
My tracks are much better examples for this type of comparison because the original masters actually possess more fidelity than most commercially available masters and CDs! Others have offered “hi-res audio” vs “CD spec audio for comparison but they used tracks provided by the major record labels or “so-called” high-resolution download sites — the famous Meyer and Moran study fell into this unfortunate trap. The tracks on SACDs/DVD-As and HD download sites are almost ALL standard-resolution recordings and thus make terrible candidates for study.
The HD-Audio challenge is not a rigorous study. Even if some participants managed to select all 6 high-res tracks correctly, there is a reasonable chance they just got lucky. I acknowledge that possibility. Almost 400 people contacted me asking for the credentials and just less than 100 responded. I haven’t yet had a chance to go through all of the responses but I thought it was time to reveal the answers. I’ll post a statistical breakdown soon.
The high-resolution, original 96 kHz/24-bit masters are:
1. A
2. B
3. B
4. A
5. A
6. A
I’ll start doing some analysis and see how people did. But my recollection is that very few people got them all right, most were about 50/50, and a large number responded by saying they simply couldn’t tell them apart. I fall into the latter category.
You gain some (additional) respect from me. Someone wiling to change their mind or viewpoint based upon a test like this. To admit you couldn’t tell them apart when you are known far and wide as an advocate for genuine hi-rez recordings. Someone who isn’t hard headed about their position especially one so closely associated with your work.
A man of depth, and good character. Bravo!
Hi Mark,
Sad that only 100 out of 400 responded – I hope more do so, I wonder if people were surprised by their findings and a little reluctant to post their results?
Mark, if you couldn’t hear a difference, maybe you just need better, more “revealing” wires, like the really nice ones use at audio demos….
LOL Yep, I’ve been told that I’m too old, my system isn’t up to the task, or that I have a bias.
I am not going to support or question the hypothesis in any way. People can decide for themselves. I will say that it would be important to know whether or not a person has an ability to hear a difference if it is indeed present. A hearing test, prior to the challenge, would be appropriate and would better support the results on either side.
Also, due to the way the brain processes information, someone may not notice a difference right off (or may only notice when it is pointed out to them). This applies to both visual and auditory information.
In my experience, it often takes me time to notice subtle differences. To wit, when the Pono Player was released I took the HiRes (Pono) challenge. My excitement turned to disappointment because I was not hearing a whole lot of difference. But I forced myself to listen to the same snippet of a track over and over and that is when I started noticing some differences. That said, I do not know if that would be in the spirit of the challenge.
Blogs like this one do much to restore my confidence in the world of audio, it demonstrates real integrity to publicly accept that the results of a test like that, including admitting you yourself failed to identify the high res tracks, which run counter to your pre-existing belief and position. In a perfect world this would be the norm, unfortunately it seldom seems to be so.
My own view (and this is not limited to audio equipment) is that whether or not measurement demonstrates an improvement is less important than whether or not you can discern an improvement in performance and whether it meets your expectations for performance. That is not to dispute the value of measurement and I’m very much a supporter of the objectivist approach to audio equipment but in my own field I sometimes see people becoming fixated on measured differences that are irrelevant for end users.
I must admit I’ve always thought that the value of real high res recordings lay more in the attention and efforts to get it right by the recording and mastering engineers rather than sample rate etc but that is just my personal opinion.
Thanks John. I think you’re right about the care used in the making of high-resolution or audiophile recordings.
I’ll do the ABX tests that I promised soon – I’ve simply had (and still have) too many things on my to-do list.
A bit late to the party but I got 2 correct (1&3) and could not tell any difference with 6. I have to say, though, that the quality of the recordings was very high indeed.
Thanks Steven.
I haven’t tried these recordings yet, but I can seldom hear a difference on studio recordings. But, I usually can on recordings that capture a lot of reflections from the room…recordings that capture a strong sense of the space in which they were recorded. In my opinion, this also holds true in a some 1950’s recordings. So, as long as the discussion continues on just that extra octave of spectrum alone, I fear we never will consistently hear it – and lose hi-rez forever to the trolls who, for motives unclear, consistently and with vitriol bash our hobby on the web. I hypothesize that the FIR filters required to use low sampling rates (those at or near Nyquist) wreck the arrival time off all those reflections and the waveshape of the non-sinusoidal artifacts. I think we need to focus on BOTH time domain and frequency domain. Not even Fourier himself could prove that sinusoids make up “ALL” sounds. Nonetheless, thanks for the work – and keep up the good fight… both against fraudulent claims of hi-rez and against the staunchly anti-hi-rez!
The Main reason for the small benefits off HD audio can be explained by an analogy to video.
When you are talking video resolution and compare to audio many people make the mistake of assuming DVD is a proper analogy for CD just because they are reasonable contemporary. In reality CD is more of 1080p and HD audio should be compared to 4K ultra. There is a small benefit of going up to 4K from 1080p sometimes but in the most viewing cases not. DVD is more of LP