A common view is that blind tests are flawed.
http://www.stereophile.com/features/113 -
" But when you have taken part in a number of these blind tests and experienced how two amplifiers you know from personal experience to sound extremely different can still fail to be identified under blind conditions, then perhaps an alternative hypothesis is called for: that the very procedure of a blind listening test can conceal small but real subjective differences" - John Atkinson. Stereophile
http://www.avguide.com/forums/blind-listening-tests-are-flawed-editorial
"The answer is that blind listening tests fundamentally distort the listening process and are worthless in determining the audibility of a certain phenomenon." - Robert Harley. The Absolute Sound.
The basis of the flaw argument is that what was clearly heard when sighted is no longer so when listening blind, so therefore the blind test is faulty. I think that there is an alternative explanation.
There are those who argue sighted tests are wrong as well. Many cable sceptics claim people cannot really be hearing a difference as blind tests find that there is none. I think that is also wrong and that there is an alternative explanation.
That explanation is touched upon in Sean Olive's blog on sighted vs blind testing - Audio Musings, Sean Olive, Harman International
http://seanolive.blogspot.com/2009/04/dishonesty-of-sighted-audio-product.html
This is about speakers, but it shows the different results that you get with sighted and blind tests and I think that also applies to cables. Here are the results of the different tests
What that shows is when sighted people find bigger differences than when blind. It does not show that either sighted or blind tests are flawed. That in my opinion is an inaccurate conclusion. Instead of a mistake in either type of test what we see is a different result and we should then be asking why is that happening? Before that the same result happens with cables.
Sighted testing
There are hundreds if not thousands of sighted tests of audiophile cables on the Internet. Go onto any hifi forum or get a hifi magazine and you will find them. For example
http://www.whathifi.com/review/silver-high-breed-metaphor-2
"This new name interconnect, to us, is a lively listen with a wide sound. It's decent for the price, and the supple midrange and bass work well.
But it suffers from a vicious top end, while production subtleties are stampeded over by a lack of cohesion. To pep up a dull system, this might be ideal; those with treble heavy kit should steer well clear." - What Hifi on the Silver High Breed Metaphor 2 interconnect.
http://www.whathifi.com/review/audioquest-cinnamon-usb
"This well-made Audioquest cable has a solid feel to it. Our sample is the 1.5m; 0.75 and 3m versions are also available.
Pairing our Chord Chordette DAC with the Cinnamon, we found that the sound of our set-up gained smoothness and a sense of authority.
Our kit served up a sweet-sounding treble, and well-integrated midrange, while the bass was sure-footed with nice weight to its delivery.
We’d like to hear more zip in dynamics, and more subtlety with tricky timing, but this Audioquest is worth checking out." - What Hifi on the Audioquest Cinnamon USB cable.
But whilst cable sceptics and believers argue about whether cables affect sound quality, cable believers also argue amongst themselves about what is the best sounding cable. One person will report a bright treble and another a dull treble.One person will report a 'night and day' difference whereas another reports a subtle difference. Then some will say they can hear no difference at all.
Then the 'Golden Ears' and 'Better Kit' arguments start. Claims are made that if you cannot hear a differences then there is something wrong with your ears or the rest of your hifi is not good enough and has insufficient detail for the cable to work. I cannot find any proof to back either argument up, but I will not dismiss them, I will offer an alternative reason to what is happening.
Sighted testing has also resulted in a clam that cables made out of silver wire sounds brighter than copper. But there is so much inconsistency in the results of sighted testing. Indeed none of it can be considered objective at all. All sighted testing is based on personal experience and opinion.
In 2011 when What Hifi was challenged by a forum member MaxFlynn through the Press Complaints Commission that it was deceiving its readers by reviewing cables as they really do make a difference, the result was that the complaint was dismissed as What Hifi stated that all of its reviews were just subjective opinion.
I my opinion too many reviews, if not the vast majority are presented as if they are objective and such and such a cable is better sounding than others. If cables really did do what they claim to do then why are the results not consistent and why, as in Part 2 are cable makers not able to show how construction is connected to sound quality?
Blind Comparison testing
I think that many people mix up the different types of blind testing and so here I want to show how the different types of blind test yield different, but consistent results.
What Hifi have been running a series of articles called The Big Question in the magazine where forum members are invited to the test centre and take part in a series of blind tests. These are the best examples of blind comparison tests I can find. The listeners are asked their opinions on what they hear and are aware switching is taking place, but they cannot see exactly what is happening.
What Hifi The Big Question on cables. Sept 2009
From the Sept 2009 issue. Three forum members were invited to WHF and blind tested where they though the kit (Roksan, Cyrus, Spendor) was being changed, but instead the cables were. The same three tracks were used throughout.
The kit started out with the cheapest cables WHF could find and no one liked it saying it sounded flat and dull. Then a Lindy mains conditioner and Copperline Alpha power cords were introduced and the sound improved. The IC was changed to some Atlas Equators and two out the three tracks were said to have improved with better bass and detail.
Last the 60p per metre speaker cable was changed for £6 per metre Chord Carnival Sliverscreen. Again, changes were noticed, but they were not big. Various swaps took place after that which confirmed the above, that the power cords made the biggest difference. When the test was revealed the participants were surprised to say the least!
What Hifi, Blind Test of HDMI cables, July 2010
Another What Hifi test of three forum members who are unaware that the change being made is with three HDMI cables. As far as they know equipment could be being changed. The cables are a freebie, a Chord costing £75 and a QED costing £150. Throughout the test all three struggle to find any difference, but are more confident that there is a difference in the sound rather than the picture. They preferred the freebie cable over the Chord one and found it to be as good as the most expensive QED.
An evening spent comparing Nordost ICs and speaker cables. AVForums June 2006
Further to the above ipod experiment, a report from a member of the AVForums and his experience of sighted and blind listening tests at a dealers.
http://www.avforums.com/forums/interconnects-speaker-cables-switches/351773-evening-comparing-nordost-interconnects-speaker-cable.html
The conclusion comparing the tests
"And here's what I heard.
1. All the cables sounded subtly different with one exception.
2. Differences were less apparent with some music than others
3. My assessment and experiences "blind" were different to my experiences "sighted""
What you have here is that there are still differences being reported, but they are not night and day anymore and cheap is as likely to do well as expensive. That is the same as when speakers were blind comparison tested at Harman International and the following DAC blind test at Stereo Mojo;
Like the other blind comparison as opposed to ABX tests this one found the cheapest and most expensive DAC in the final, with only a hairs width between the two in terms of sound.
http://www.stereomojo.com/Stereomojo%20Six%20DAC%20Shootout.htm/StereomojoSixDACShootout.htm
The 'night and day' differences have disappeared. This is consistent with all blind comparison tests that I can find.
ABX testing
There are numerous ABX tests of cables. Here are a few;
AV Science forum, Observations of a controlled cable test Nov 2007
A blind test between Monster cables and Opus MM, which as far as I can find is $33,000 worth of cable
http://www.avsforum.com/avs-vb/showthread.php?t=941184
but the owner of the very high end kit and cables was unable to tell the difference.
Sound & Vision. Article by Tom Nousaine with 3 Blind Tests of speaker cables. c1995
http://www.nousaine.com/pdfs/Wired%20Wisdom.pdf
All three are fails by the listeners using their own hifi systems and with their choice of track, volume and time.
AV Science Forum, Monster vs Opus cables. 2002
Not particularly rigorous as in there were not enough tests, but as the poster states "And to cut to the chase, Mike could not identify the Monster from the Opus MM with any accuracy (nor the reverse, which also would have been a positive result if he had been consistently wrong) using our testing methodology. We stopped the test a little less than halfway through, I think we got through 8 A/Bs before we gave up."
http://www.avsforum.com/avs-vb/showthread.php?t=941184
HiFi Wigwam, The Great Cable debate. Power cable ABX test Oct 2005.
This is a very well done large scale ABX test. A similar set up to Head-fi where four mains cables including 2 kettle leads (stock power cords that had come with hifi products), an audiophile one, a DIY one and a tester CD were sent out forum members. The results were inconclusive to say the least, for example;
The kettle lead was C. There were 23 answers :
4 said that the kettle lead was A
6 said that it was B
8 said that it was C
5 said that they didn't know.
http://www.hifiwigwam.com/showthread.php?654-The-Great-Cable-Debate&highlight=blind+test
Conclusion
There is a definite correlation between the type of test and the result. A sighted test finds the biggest differences, blind comparison reduces those differences and ABX testing finds no differences at all. So no one type of test is any better or more accurate than any other and no type of test is flawed as all produce consistent results. That is important where figuring out how cables can sound different to some people some of the time.
No comments:
Post a Comment