Audio equipment testing
Audio equipment testing is the measurement of audio quality through objective and/or subjective means. The results of such tests are published in journals, magazines, whitepapers, websites, and in other media.
Those who test and evaluate equipment can be roughly divided into two groups: "Objectivists", who believe that all perceivable differences in audio equipment can be explained scientifically through measurement and double-blind listening tests; and the "Subjectivists", who believe that the human ear is capable of hearing details and differences that cannot be directly measured.
Summary of Objective versus Subjective Audiophiles, in general:
- Both agree that measurements are not a substitute for listening tests.
- Both agree that different audio components may have different sound qualities.
- Disagree that subjective listeners can overcome placebo and confirmation bias in non-blind listening tests.
- Disagree about whether perceived sound quality can be measured through objective means.
Objectivists
- Objectivists point out that properly conducted and interpreted double-blind tests fail to support subjectivists' claims of significant or even subtle sonic differences between devices in cases where measurements predict that there should be no sonic differences in normal music listening.
- Objectivists feel that subjectivists often lack engineering training, technical knowledge, and objective credentials, but nevertheless make authoritative claims about product performance.
- Objectivists are likely to stress the importance of accounting for the influence of placebo and confirmation bias in subjective listening tests .
- Objectivists reject arguments that are based on accepted physical principles but applied to circumstances where they are irrelevant. For instance, the skin effect, which relates the efficiency of cables to the frequency transmitted, is often applied to audio frequencies where it is insignificant .
- Objectivists believe that subjectivists' preferences are often driven by gullibility and fashion—e.g., the late eighties' vogue for marking the edges of CDs with a green felt marker or suspending cables above the floor on small racks—and bear no relation to well-known laws of physics.
- Objectivists claim that subjectivists often reject attempts to notate differences in sound using objective measurements, despite the evidence of their effectiveness.
- Because measured audio distortion is higher in electromechanical components such as microphones, turntables, tonearms, phono cartridges, and loudspeakers than in purely electronic components such as preamplifiers and power amplifiers, objectivists generally do not accept that very subtle differences in the latter can have an appreciable effect on the perceived quality of reproduced sound.
Subjectivists
Subjectivist Harry Pearson, long-time editor of the audiophile magazine The Absolute Sound has stated:
"We believe that the sound of music, unamplified, occurring in a real space is a philosophic absolute against which we may judge the performance of devices designed to reproduce music."
- Subjectivists rely on demonstrations and comparisons, but believe there are problems in applying double-blind methods to comparisons of audio devices. They believe that a relaxing environment and sufficient time measured in days or weeks is necessary for the discriminating ear to do its work.
- Subjectivists believe that careful individual listening is an appropriate tool for discovering the true worth of a device or treatment, and will generally acquire equipment that suits their own listening or style preferences as opposed to measurable equipment performance.
- Subjectivists are likely to de-emphasize or ignore the potential impact of placebo and confirmation bias on subjective listening tests.
Subjectivists claim that experienced listeners can be relied upon for valid advice on how equipment sounds. British Hi-fi critic, Martin Colloms, writes that "the ability to assess sound quality is not a gift, nor is it the feature of a hyperactive imagination; it is simply a learned skill", which can be acquired by example, education and practice. In any event, the eventual purchase decision will be made by the end-user, whose "perception is reality" and can be influenced by factors other than the equipment's actual performance.
Opposing viewpoints
The most significant difference between the two groups is that subjectivists claim there is a limit to what can be tested using objective measurements, while objectivists believe that since blind testing is the gold standard of all science, perceived sound quality should not be exempt from objective measurements.Objectivists tend to see the subjectivists as irrational and prone to gullibility, while subjectivists often dismiss objectivists as simple "meter men" who lack a nuanced appreciation of sound.
Although the debate can be heated in certain quarters, both groups seek optimal listening experiences, and in some cases, the findings of one group has informed the other.
Objectivists argue that vacuum-tube amplifiers often exhibit lower-fidelity than solid-state designs, and that in addition to their substantially higher total harmonic distortion level, they require rebiasing, tend to be less reliable, less powerful, generate more heat, and are usually more expensive.
Subjectivists are often argue that while tube electronics are less linear than solid-state electronics at high-signal levels, they are much more linear at low-signal levels and that many musical signals spend much of their time at these low levels.
However, in cases where the subjectivists' claims can be verified by objective measure, it could not be considered a strictly "subjective" position. A more literally subjectivist argument is that listeners may be able to discern pleasurable benefits from tube equipment that would escape any attempt at objective measurement.
Objectivists claim that digital sound can have higher fidelity than analog sound because it lacks clicks, pops, wow, flutter, audio feedback, degradation, generational loss, and rumble, has a higher signal-to-noise ratio, has a wider dynamic range, has less total harmonic distortion, and has a flatter and more extended frequency response.
Subjectivists have argued that the process of converting a bit-stream to an analog waveform requires heavy filtering to remove spurious high-frequency information and that it should be expected that such filtering should involve some signal degradation and a large amount of phase shift in the passband. They point out that commonly used consumer-grade digital-to-analog converters exhibit poor linearity at low levels.
Both of these problems have since been verified by objective measure, and were then addressed by such solutions as digital filtering, oversampling, and the use of DACs operating at 20-bit resolution.
Today, many objectivists and subjectivists may agree that a preference for analog formats is often rooted in pleasing and familiar distortions and artifacts. At the same time, members of both camps might also agree that the best historical analog formats have often displayed great levels of audio fidelity as well.
Musician Neil Young is a harsh critic of the sound of the original CD format, but has expressed approval for the sound of higher definition formats such as SACD, which he believes has a greater safety margin between its ideal behavior and the theoretical limits of human hearing.
However, many independent listening tests and mathematical studies have shown that Young's assertions are basically untrue. Regardless, in 2011, Neil Young filed paperwork to trademark his own developing high-resolution audio player.
Objectivists and Subjectivist were once at odds about the importance of total harmonic distortion. Both groups now seem to agree that the distribution of harmonic distortion can be important in perceived sound quality. This is another case where subjective opinion was eventually verified by objective measure, and taken into account by objectivist engineers.
Although this is no longer technically a point of contention, objectivists may be more likely to stress the importance of reducing total harmonic distortion in a system, while subjectivists may more often stress the importance on creating a more pleasing distribution of harmonic distortion.
Difficulty of testing
It is difficult, but very important, to match sound levels before comparing systems, as minute increases in loudness—more than 0.15 dB or 0.1 dB—have been demonstrated to cause perceived improvements in sound quality.Listening tests are subjected to many variables, and results are notoriously unreliable. Thomas Edison, for example, showed that large audiences responded favorably when presented both live performances by artists and reproductions by his recording system, which today would be regarded as primitive in quality.
Similarly, results of component evaluation between various listeners or even the same listener under different circumstances cannot be easily replicated or standardized.
Similarly, the acoustic behavior of the listening room—the interaction between loudspeakers and the room's acoustics—and the interaction between an electromechanical device and an electronic device are subjected to many more variables than between electronic components. Thus the "difference" in sound quality between amplifiers is actually the ability of an amplifier to interface well with loudspeakers or a lucky combination of loudspeaker, amplifier, and room that works well together.
The introduction of switching apparatus, with either metal connection or electronic processing, may, some believe, obscure the differences between the two signal sources being tested.
Skepticism advocate James Randi has offered a $1 million prize to subjectivists who can prove their most dubious claims through scientific blind testing. The prize remains unclaimed.