E-Mail 'HD-Audio Challenge Part II' To A Friend

Email a copy of 'HD-Audio Challenge Part II' to a friend

* Required Field

Separate multiple entries with a comma. Maximum 5 entries.

Separate multiple entries with a comma. Maximum 5 entries.

E-Mail Image Verification

Loading ... Loading ...

26 thoughts on “HD-Audio Challenge Part II

  • August 16, 2018 at 5:15 pm

    IMO using DSD for resolution comparison test is problematic. I can tell difference between DSD and PCM. Apparently DSD encoding or decoding or both introduces some coloring that does make it sound less natural. I have the same problem with vinyl records due to mechanical nature or reproduction. I personally prefer transparent and neutral sound. One of downsides of CAS I attended in July was that most systems for some reason played back vinyl records which made difficult to estimate actual sound quality.

    • August 17, 2018 at 7:44 am

      If you can tell the difference between a DSD and PCM recording that came from the same source (microphones of analog tapes) then you’re better than the people that participated in a study of exactly that. The researchers determined that no one could tell the difference in that study. High-resolution PCM is simply the format of choice because engineers can work with it and it produces “transparent and neutral sound”. Trade show vendors (and magazines/websites) like to hype the latest trends regardless of whether they deliver improved fidelity of not. Take vinyl, MQA, DSD etc. as examples.

  • August 16, 2018 at 6:52 pm

    No need to publish this, but I only get 79 when I add up the responses. Might want to check the count again.

    • August 17, 2018 at 7:45 am

      There were some that had a mixed bag of “no choice” and selections. I only counted those that were complete.

  • August 17, 2018 at 2:47 am

    The reason why your recordings in a CD resolution sound so good obvisiously is, that they were originally made in Highres (24/96).
    This just shows that good stuff will keep on sounding good even in 16/44.1!

    • August 17, 2018 at 7:46 am

      Exactly the point of recording using high-resolution during production.

  • August 17, 2018 at 3:59 am

    As a matter of pure chance, if the files were truly indistinguishable, you would expect about half of the folks to be able to guess half of the six pairs correctly. Adding up the 79 (not 83) responses in your list, 38 folks were able to guess 3 or more correctly, and 41 folks were able to guess less than three correctly. I’d say that’s within the margin of error.

    I was one of those who couldn’t hear any real difference. I initially put it down to doing this with midrange quality headphones (AKG K240 Studios – the current version of the phones you’ll see musicians wearing on old album covers – the ones with two black spring wires arching high over their heads) on a Dell laptop with a soundcard claiming 96/24 quality – or to my being 68 2/3 years old – though I’ve always had better than average high frequency hearing (I had to trade in my old Yamaha electric piano for a Casio because the Yamaha didn’t have the overtones fade out – too little chip memory for that – so they simply recirculated as long as I held down the sustain pedal, causing a traffic jam in the hight frequencies), but unless the whole test population is Boomers like me, I think this test is pretty conclusive.

    It’s still worth buying AIX’s records because of the musical and production quality and “stage perspective” surround mixes, however.

    I don’t begrudge some commercial records their compressed dynamic range – like Steely Dan’s “infinite mix” approach that lets you hear every layer clearly – it’s like the “heightened reality” of sci-fi thrillers. (The DVD-A of “Gaucho” – also a stage perspective mix – is a masterpiece.) Sometimes you just want something a bit more exciting than everyday life. That’s why the Motown performers wore sequin-covered clothing – so they’d glitter in the spotlights.

    • August 17, 2018 at 7:48 am

      Thanks Phil. I enjoy the Steely Dan and other highly produced and tastefully mastered albums. It important to recognize that those tracks use only very few bits, however.

  • August 17, 2018 at 7:01 am

    Hi Mark,

    Regardless of the high definition debate you have confirmed how many people are quite prepared to show you they haven’t got a clue what they’re talking about….

    • August 17, 2018 at 7:50 am

      Good morning John. As I’ve stated repeatedly, the music industry is a business. I saw that during my time on the CEA audio board. The group of manufacturers — and the organization — has absolutely no interest in the truth or fidelity. They only cared about maximizing profits for their members. But that shouldn’t come as a surprise.

  • August 17, 2018 at 7:30 am

    “How many people got 0 correct or couldn’t tell any difference? 17”

    Could you split that out into how many tried and scored 0 versus how many said “no difference that they could hear”?

    Many thanks.

    • August 17, 2018 at 7:54 am

      There were 17 individuals that replied “no choice” for all 6 selections. Then there were a number of submissions that had one or two selections and the rest “NC”.

  • August 17, 2018 at 11:13 am

    You really shouldn’t group the no choice with the 0 of 6. Because had this been a forced choice you would expect the no choice people to average 3 of 6 correct. No difference/no choice isn’t the same as scoring zero. It would equate with getting half right.

    In any case your results are essentially random.

  • August 17, 2018 at 3:05 pm

    If I’ve done the maths correctly, the statistical outcome if people were guessing would be as follows, with deviation of the actual outcome indicated:

    6: 1.6%, 1.2/79, +0.8
    5: 9.4%, 7.4/79, +2.6
    4: 23%, 19/79, -6
    3: 31%, 25/79, -12
    2: 23%, 19/79, -1
    1: 9.4%, 7.4/79, -1.4
    0: 1.6%, 1.2/79, ?

    Since 6 incorrect responses are not reported separately from those who did not make a selection, we can assume that had they simply guessed, they would have followed the statistical distribution. These 17 entries should thus be distributed as 0.3, 1.6, 4.0, 5.3, 4.0, 1.6, 0.3. This gives an amended tally of 2.3, 11.6, 17, 18.3, 22, 7.6, 0.3. The deviation from the expected value here is +1.1, +4.2, -2, -6.7, +3, +0.2, -0.9. There is a shift towards correct responses, though there is also an elevated number of 2 correct. Given the sample size, deviations of this magnitude are probably normal.

    My conclusion is that at most 16 people were honest enough (with themselves) to say they couldn’t hear a difference.

    • August 17, 2018 at 5:58 pm

      Thanks for this additional analysis.

      • August 17, 2018 at 5:59 pm

        I went back and took another look. I’m not sure how to handle people the had “NC” on some a a couple of selections. There were 17 people that picked “NC” for all of them.

        • August 18, 2018 at 2:47 pm

          If nobody replied with all wrong, it’s better to leave those 17 out entirely. The expected distribution of the remaining 62 then becomes:

          6: 0.97, +1.0
          5: 5.8, +4.2
          4: 15, -2
          3: 19, -6
          2: 15, +3
          1: 5.8, +0.2
          0: 0.97, -0.97

          There are many ways to spin this. For instance, one might say that the number of people (12) getting at least 5 correct exceeded the expected value (6.8) by 76%. Clear proof that high-res is audible! However, if we look at how many got 3 or more correct, the number falls short of the expected by 6.6%. High-res is obviously not worthwhile! Statistics, the liar’s best friend.

  • August 18, 2018 at 3:05 am

    I’m interested in the disparity between the number of people downloading the files but not then providing a response. My feeling is many did believing that it would be a simple process then finding embarrassment at not being able to discern any difference – possibly with very expensive gear?

    • August 18, 2018 at 10:11 am

      I’ll try to nudge those that tried but didn’t respond.

    • August 18, 2018 at 1:19 pm

      Having put up listening tests on large forums a few times, you never get much response. The response here is actually huge compared to most. I’ve had files that get downloaded several hundred times, and usually there are between 1 and 2 dozen responses. I’ve seen similar results from others who post such files.

      While it could be from a variety of reasons I tend to think it is because people try them and don’t hear the differences they expected to hear. The higher responses have been when I offer some degradation of files in several steps and at levels where some of the files are obviously different to anyone. Even then responses seem to come in rather quickly at first before stopping and few in total respond.

      • August 19, 2018 at 3:01 pm

        I have a feeling people are simply not willing to admit they can’t tell the difference because they don’t want to feel inadequate in some way particularly if they have invested heavily in kit. I can’t tell the difference but as I’ve commented before I definitely believe that the higher sampling rates have an ‘ease’ to them if you are listening for prolonged periods of time – this plus the engineering advantages Mark has mentioned many times still makes me an advocate for Hi definition recording and playback.

  • August 19, 2018 at 11:55 pm

    I got only one right – 2! Still I was not sure if I can hear the difference and most likely I was just guessing.

  • August 23, 2018 at 2:56 am

    As promised, I tried an ABX test.
    Here’s the log from the first song:

    foo_abx 2.0 report
    foobar2000 v1.3.7
    2018-08-21 21:27:30

    File A: Tune_1_A.wav
    SHA1: a172a18acd31bde18e70254654eb3d6a62a98869
    File B: Tune_1_B.wav
    SHA1: 23140e6544890298339f2a1de731f972a0285283

    DS : Højttalere (CA USB Audio)
    Crossfading: YES

    21:27:30 : Test started.
    21:38:15 : 00/01
    21:39:02 : 00/02
    21:40:15 : 00/03
    21:41:35 : 01/04
    21:45:51 : 02/05
    21:47:17 : 02/06
    21:48:20 : 03/07
    21:49:10 : 03/08
    22:00:04 : 03/09
    22:01:08 : 04/10
    22:02:23 : 04/11
    22:04:27 : 05/12
    22:06:39 : 06/13
    22:07:30 : 06/14
    22:08:32 : 06/15
    22:09:46 : 07/16
    22:09:46 : Test finished.

    Total: 7/16
    Probability that you were guessing: 77.3%

    — signature —

    And here’s the log from the third song:

    foo_abx 2.0 report
    foobar2000 v1.3.7
    2018-08-22 20:11:00

    File A: Tune_3_A.wav
    SHA1: e4db0c5771607dd8cf1a2b629cb7dcbff593ef37
    File B: Tune_3_B.wav
    SHA1: 990af2551ff3a9fa1a6af0ed5b1773705464866d

    DS : Højttalere (CA USB Audio)
    Crossfading: YES

    20:11:00 : Test started.
    20:15:29 : 00/01
    20:17:03 : 00/02
    20:18:47 : 00/03
    20:20:18 : 00/04
    20:22:19 : 01/05
    20:23:38 : 02/06
    20:24:58 : 03/07
    20:26:42 : 04/08
    20:28:12 : 05/09
    20:29:28 : 06/10
    20:31:01 : 06/11
    20:32:19 : 06/12
    20:33:36 : 06/13
    20:35:45 : 06/14
    20:36:48 : 06/15
    20:39:45 : 07/16
    20:39:45 : Test finished.

    Total: 7/16
    Probability that you were guessing: 77.3%

    — signature —

    I had actually planned to first do an ABX test of all six songs through my speakers and then afterwards through headphones, but in the end I only ABX’ed these two songs through my speakers, as I actually thought I could hear a small difference between the two files on those particular songs. For the four others I couldn’t hear a difference, so I didn’t even try to ABX them. But as you can see from my results, I couldn’t hear a difference between the two songs I ABX’ed either.
    I think it’s likely that I heard some volume level difference in the recording between A and B that seemed like a difference in sound quality, meaning at, say, 1:20 the musician was a little bit closer to the microphone, or that particular part of the song was a little bit louder than it was at 1:30 when I switched from A to B. So the “difference” I heard was in both files – in the recording and not in the resolution. I hope this makes sense.
    So, now the number of people who couldn’t hear a difference has gone up to 18.

    But Mark, I have to compliment you for being honest about all of this and essentially saying “well, I hoped people would be able to hear a difference, but maybe I was wrong”.

  • August 23, 2018 at 6:13 pm

    Myself and 3 other audio friends all came up with results which pretty well indicated that our selections were as good as random, whilst 3 of the 4 of us selected 4 correctly none of us agreed with each other in our results so statistically I believe this would classify as random.
    However what I would like to add to the discussion is that I added an extra question to the mix and that was ; Do you think that the two files actually sound different from each other ? regardless of whether we could pick the Hi Res version or not ?
    Overwhelmingly we agreed that the files sounded different to each other with 3 of the 4 of us thinking that perhaps only 1 of the 6 may have been the same file ( i.e. it sounds the same ). By the way we did this blind from each other so we could not influence each other and we just wrote down our results without discussing them until we finished.

    My conclusion from this is that there is a sufficient difference in sound that we can perceive that there is a difference between CD and Hi Res , we just can’t tell which one is Hi Res !

    Maybe it is just that our sound systems are not good enough to allow us to discern ?

    I wonder how many other people thought the files sound different to each other and how many thought that they sounded the same ?

  • September 25, 2018 at 11:06 pm

    Sort of a technical question. If the output of the computer is set to 96/24 and each of the versions is played with that setting. Assuming bitperfect transmission of the file to the DAC, will the simply ignore the filler on the lower res file?

    • September 27, 2018 at 12:15 am

      The sample rate of both files is identical.

Comments are closed.