A | B Listening Tests
The NPR Music “How well can you hear audio quality?” quiz made me start thinking about some very fundamental issues associated with master quality audio (we didn’t get to listen to any high-resolution audio during their evaluations). One of the most critical is whether listening to audio in bigger bit buckets matters at all? Are we simply fooling ourselves when we start talking about all of the benefits of better quality sound? Will it really matter whether Tidal or Apple make available higher fidelity streams? Should equipment manufacturers and content providers get behind tests like the one held on the NPR Music page?
How about the whole concept of comparing sample A against sample B? I know the major magazines and plenty of other respected writers reject the A | B listening tests. They’re not ready to risk the embarrassment of getting things wrong. Or perhaps the methodology is flawed and producing skewed results. I failed to correctly identify half of the NPR examples…but I’m not embarrassed or troubled. As I think everyone except Neil Young will acknowledge, if there are differences between a well produced 320 kbps MPS file, a CD-Audio disc, and a high-resolution audio file, they are very subtle and without training and excellent equipment very hard to tell apart.
I’m a believer in high-resolution audio. I’m personally convinced that new recordings produced using 96 kHz/24-bit PCM and great care during the production process can eclipse standard resolution formats including analog tape and certainly vinyl LPs. Even if it is shown that fidelity is not dramatically affected by format, sample rates, word lengths, etc., I would still opt to record at 96 kHz/24-bits for the simple reason that it is easy to do and provides an increase in the potential fidelity of the recording (and playback). But that’s my limit. More and more designers and manufacturers of high-end equipment are going ahead with every higher sample rates and longer words.
Does anyone honestly believe that DAC with 384 kHz/32-bit PCM capabilities are going to improve their audio experience? And why stop there? Let’s just keep going and jump straight to 1576 kHz/48-bits! Bigger numbers are always better, right?
Here’s a little secret for all of you upgrading to ultra high-resolution audio playback systems. There are no commercially available records of pop/rock/country/urban or other genres that were recording at rates higher than 192 kHz/24-bits and virtually all new releases are being captured at 48 kHz/24-bits. And that’s not going to change anytime soon…if ever.
I’m shocked when I hear colleagues and friends utterly reject using direct comparisons of two different audio formats as “a useless exercise that will only result in consumers doubting the validity of high-resolution audio”. Sure it will. If there’s no audible improvement in using 384 kHz/32-bit PCM vs. 320 kbps MP3s, then a lot of business will be lost. As I learned from a major cable representative some moths ago, “telling the truth about high-resolution audio is bad for commerce”.
So all we have to do is continue to talk about all of the benefits of so-called “high-resolution audio” and jump on the gravy train. Stand by folks…you haven’t seen anything yet.
Don’t be too quick to give up on such tests. I got 6 of 6 for the non-compressed file, and 5 of 6 picking the 128 mp3 which means 5 of 6 picking the higher rate mp3.
I have begun to think average audiophiles don’t wish to put in the effort to do good blind testing. It results in them sometimes claiming sonic equivalence when they shouldn’t. As well as disbelieving sonic equivalence when it has been reached. I believe through placebo and expectation bias they find it easy to hear such seemingly real differences in tiny things. When doing ABX testing the experience is so different, and even moderately discernible differences are a real chore to hear vs knowing thru sighting comparisons what is what, they reject the experience altogether. Sighted differences are as plain as the nose on your face, and suddenly become perhaps impossible which simply creates cognitive dissonance too great for most to ignore. It has become a tough nut to crack.
I think properly implemented A|B testing can work. However, in the overall scheme of things the differences are subtle and perhaps not important to the general music listening public.
Oh, I very much know good AB testing can work. I am less optimistic about things like the NPR comparison and many others available for people. That is the sense where I think it is easy to hear sighted, and hard to hear blind, and the audio public will just dismiss blind as somehow missing something.
“you haven’t seen anything yet”
I’ll be waiting on pins and needles. LOL
There’s actually much more to this than A-B comparisons. A system generally reveals its qualities over time rather than in brief sound bites, in my view. The recent NPR test completely ignores this vital point and, not at all surprisingly, very experienced ears, mine included, have sometimes failed to spot the differences that are there, but which usually unveil themselves over the course of a track or album.
As for the numbers game, we can confidently expect that to continue to rise on the basis that it looks impressive to unwitting customers, regardless of what difference it actually makes.
As a daily reader of your writings Mark, I’m becoming increasingly disillusioned at the direction high res is headed. The recent Tidal launch debacle has created a lot of animosity that is being expressed in tests and other editorial that seeks to prove that high res is the latest in a long line of audio snake oil.
A year or so ago we all thought that Apple was making it’s entry into the lossless streaming arena, but this week’s Music announcement, and the low bit rate that will accompany it, suggest an increasing perception that there simply isn’t an appetite or even a need to move things forward, other than among the tiny niche of audiophiles that, I’m afraid, are of little interest to big business.
To think it all looked so bright not so long ago.
I’ve written previously that I experience a difference over many hours of working with CD quality files vs. High-Res stuff. When it comes to MP3s, I don’t have enough listening time with them.
The average listener is attached to low-fi equipment and cares little about the purity of a recording’s tonal structure and sadly any attempt to change that will fall on deaf ears. As a once bitten audio buff with Golden ears and my own pursuits in my youth of high end hardware I now find my hearing at 62 isn’t what it used to be. Surprise! Neil Young is older and looks it. How he could even promote a high fidelity format and tell the difference at his age is beyond me, when I can no longer do so myself.
I’m right there with you.
You should read this article (if you have not yet) — http://www.avclub.com/article/four-columbia-house-insiders-explain-shady-math-be-219964
“What we had in the ’90s was… what another very famous, huge record executive [said to me] in a very, very hilarious way. We went to his Fifth Avenue townhouse, gorgeous space, and he said—maybe I’ll give it away if I can do his accent properly—but he said [Affects accent.], “You know what this is? This right here? It’s stupid money. It’s CD money. That’s the kind of money that made dumb people feel smart.” You have the biggest fucking markup in retail history, and somehow in one winter, the music business—with Phillips leading the way—said, “Hey, that $7.99 album you love? Guess what? Your lucky day: You get to buy it for $18.99, it’s going to sound worse, and you have to buy fucking pieces of equipment.”
Same as it ever was…
Thanks…I watched about 15 minutes but will return.
Thank you for this extremely sensible commentary. I think spot on. I particularly like your comment “As I think everyone except Neil Young will acknowledge, if there are differences between a well produced 320 kbps MPS file, a CD-Audio disc, and a high-resolution audio file, they are very subtle and without training and excellent equipment very hard to tell apart.” and honestly I think there are more than just Neil Young out there, some of the people are so fervent in their belief that the differences are so noticeable even on the most inexpensive of systems. My system costs a lot of money (not obnoxiously so, but reasonable enough) and your comment hits it right on the nose for me.
Thank you.
Thanks Larry. It’s difficult for me to hear music industry executives dismissing things that are at the core of new audio hardware and software.
As an otologist, I can only applaud your article. I see so many people with less than perfect hearing that I consider this to be the new norm. There is no headphone or speaker with a perfect flat line frequency response. THD of 1% represents the just noticeable difference of the human ear. So when manufacturers and audio critics trot out specs that are beyond the response of the human ears claiming they hear illusory sensations, I suspect there findings & bias. Where are the double blind studies to prove these claims?