Research That I Can’t Talk About
I’ve done a preliminary read through a couple pdf documents that are purportedly supposed inform…both qualitatively and quantitatively… the companies behind the well-funded HRA study. I’m prevented from saying anything specific about the results or the information drawn from the online and in person components of the study. But I can say that it was more disappointing than I would have imagined. In a rush to get the study published last week, the documents were not made available to any experts (or maybe it was just me) involved in supplying the hardware and software components to the study…or anyone else that I know of. Apparently no one with the appropriate amount of technical knowledge read the final document prior to its publication. I’ve written and presented a lot of academic papers over my life and they are never released or published with numerous people facts checking and editing. Very sloppy.
There was simply no way that any productive information could be drawn from the actual listening tests. I knew that going in. The results confirmed what the famous Meyer and Moran study found back in 2006…although for different reasons and with different source materials. Unlike Meyer and Moran, the three tracks used in this study were all bona fide high-resolution tracks from my own catalog. I produced two versions of three different genres of music…in stereo. The first was the original 96 kHz/24-bit PCM file and the second one was a “lossy” MP3 encoded at exactly the same level. Groups of three individuals sat in the evaluation room (a very poor acoustic space…a glass walled board room) and listened through speakers and headphones.
My lack of surprise at the lack of any findings on the perceptibility turned to astonishment when I skimmed through the final report. The entire study refers to this as a comparison of “standard-resolution” audio vs. “high-resolution” audio. The study wasn’t design to compare “standard resolution” files with anything. Unlike the samples of the same files that I put on my own FTP that did have CD spec versions, I delivered only lossy MP3 files to people at the study (actually they were 96 kHz/24-bits files containing the MP3s). I know I tend to be a little harsh on the uninitiated but this is beyond my level of tolerance. If you were a big research firm that has been hired to do serious research for a major organization, I would expect accurate information on the basics. These people seemed to know nothing about the subject they were researching and didn’t bother to seek professional assistance.
As recipients of this study thumb those the pages of the reports, they’re going to going to say, “well that wasn’t too bad…I’m not sure if I could tell the difference between “standard-resolution”, CD Specification audio (or analog tape or vinyl LPs) and a high-resolution file. But it’s actually a lot worse then that. There weren’t any “standard-resolution” files present at the evaluations…only lossy MP3s and High-Resolution files.
I expected disappointment. The single biggest question that continues to loom over the whole HRA initiative is whether humans react differently to CD resolution or Hi-Res. I think it’s critically important to determine this once and for all. I’m going on a full court press in 2015. Let determine whether all of this is hype/spin or whether there is better sonic future ahead of us.
A bunch of academic institutions including MIT would do science the right way and have more credibility than the industry or other interest groups.
Funny, I’m wearing my new MIT T-Shirt this morning…my son finished grad school there last June. I’m convinced I could pull this off and am going to start the process of acquiring the equipment and expertise.
The funniest thing is that WAV/AIFF/AU are restricted to 4 GB file size & W64 format does not actually work.
I use w64 all the time, works fine for me.
Indeed, Mark, it is most disappointing to witness just how this whole issue of a “standard” of high resolution audio is being handled, defined and promoted.
I have a keen interest in high quality audio recording and reproduction. As an older guy, my hearing is less than ideal, but I know and appreciate well-recorded and replayed material when I listen.
So much of today’s popular music is “over-produced” and compressed to within an inch of its life. It would seem, to me, to be almost impossible to hear much difference between an MP3 and a 24/96 file when played through the average portable device and ear buds.
As an owner and user of very high quality digital interfaces and microphones, etc., it often makes me smile to scan the articles and ads in audio lifestyle magazines and observe the investments required/recommended to achieve “audio nirvana” in one’s listening room.
I could rabbit on about recording techniques and production values, etc. at length. But the bottom line is the integrity and provenance of the recording itself. Along with that is the experience, expectations and “training” of the listener. In an attempt to convey a realistic representation of the musical event, consideration must be given to exactly how much of the available sound information is retained… the more subtle cues, for example, of space and sound transients which have captured by the engineer.
I agree with you: DSD is not the answer. High accuracy PCM at 24/96 minimum is required to capture sufficient detail to convey any sense of “being there”.
Keep up the good work: don’t be discouraged by sloppy and money-oriented practices. Education of music lovers is the thing. There will always be a group of people who desire and aim for excellence in music reproduction and performance. Mediocrity sucks.
A.S. I solely proposed a digital audio format which is far higher quality sound than had been touting for on this site.
1} 16-bit recordings with perforce limited 120 dB pressure peaks are perfectly reconstructed with noise shaping & there is no need in either 24-bit and 120 dB SNR equipment
2} The reason why CD does not feature subtleties is in anti-alias filter. This is mostly why 192 kHz sounds perceivably better. Upsampling can partly solve the whole problem.
Who says lossy MP3s are not ‘standard resolution audio’? Has the term been defined and agreed upon? lol
I do. CDs, vinyl LPs, and analog tape are very good but still standard resolution…which we’ve been producing for many decades. Lossy, compressed files are below that. I refer to them as “reduced resolution”. And High-Resolution is anything above 96 kHz/24-bits as the JAS has it.
I was meaning to point out the futility and frustration of dealing with scientifically undefined terms…
Hmm, hate to say it but it almost sounds like the researchers provided study results that they thought the people that hired them wanted to hear. Maybe the researcher weren’t ignorant, rather they had that specific goal in mind and to achieve the goal they had to ignore some facts.
There are lots of other problems in the reports that point to a lack of knowledge. They said the samples were played at 17 dB and 22 dB…if that was the actual sound pressure level, nobody would have heard anything.
Glad it is the school and not the cable manufacturer although the cost of some of their wire is probably as high as the tuition! Congrats to you and your son, it is quite an achievement.
Yes, very proud parents…he’s a smart one. Snagged one of 5 Fulbights this years out of 1000 applicants too. He in Mexico City doing his project. Thanks Joe.
Having given thousands of demonstrations that evinced the listener’s aural acuity, I have come to only one firm conclusion; virtually anyone if guided properly will hear the difference between a source containing significant amounts of transient intermodulation distortion products and one which has vanishingly low levels of same. As you say Mark, that’s simply comparing awful sound to excellent; that’s not the comparison of concern.
As for the audible difference between 16/44 done well and 24/96 similarly well executed, I would cheerfully predict that trained listeners would have little problem , but the control would have to be a good recording of carefully chosen music, identical playback output stages and no stress on the listeners due to test circumstances.
For a wider perspective, I personally think most folks would hear a difference between 16/44 and 24/96 but would need some ear/listening training first. Folks such as you and I who have used our ears critically for thousands of hours do in fact have greater awareness and wider angle perception than folks who have never gotten involved w/ hi-fi listening. A neurophysiologist who recently visited my shop made that statement to me in no uncertain terms. This all at least partially explains some of the bizarre results that so-called ‘controlled tests’ often generate; too many variables in this equation, results all over the place.Hearing is believing, simply put.
It’s important to do the rigorous test…I’m not yet convinced.
Here is some food for thought and/or further experiment: http://www.davidgriesinger.com/intermod.ppt
It’s a few years old, but the analysis is driven by physics and physiology, which I don’t think will have changed in the mean time.
It’s a long Power Point presentation. I’ll read through…but I know the paper they depend on.
Craig,
Regarding the effect of stress on aural acuity, stress may actually improve the listener’s ability. We evolved to be able to accurately analyse audible cues while stressed. You’re wandering through the forest, thinking of lunch, when you hear a twig snap. Is it someone else thinking of lunch, or is it someone thinking of having you for lunch? Successfully analysing subtle audible clues will make the difference between eating and being eaten.
I do accept that stress from external sources should be minimised (comfortable chair etc). The listener should be able to give their full attention to the task rather than being distracted.
Hey Mark,
This research into the audibility of high resolution audio reminds me of the Pepsi Challenge. As detailed in Malcolm Gladwell’s book “Blink”, Pepsi’s blind sip tests in the early ‘80s convinced Coca-Cola to bring out a new formula of Coke that bombed in the marketplace. Why? The blind sip test favored Pepsi’s sweeter formula, but the experience of drinking the whole bottle (or can) ended up favoring Coke’s original formula.
Is a double-blind listening comparison the audio equivalent of a cola sip test? Maybe it takes an extended, relaxed listening experience (drinking the whole bottle) to appreciate the qualitative difference of high resolution audio recordings. I know that’s been my experience.
Allen Farmelo has some interesting thoughts on this in his article “The Possibility Of Subconscious Auditory Effects In Audioworkers”. http://www.tapeop.com/blog/2015/01/14/subconscious-auditory-effects/
Food for thought.
Thanks Russ…I agree there’s a lot more to evaluating this than what’s been attempted.
Another parallel to draw from the Pepsi challenge- I had a friend that worked for one of the soda manufacturing giants. People would get on film of the challenge and disappear so Pepsi couldn’t get a signed release thereby hijacking the events. It reminds me of Marks constant battle of companies and people hijacking definitive standards for HRA.
You really can’t trust anything that promoted or marketed. Not surprised.