The Magnitude Fallacy

Humans are worriers. It’s what we do. Some more than others, of course – my late Italian grandmother could have won an Olympic gold medal in it – but there are legitimate scientific underpinnings to this. Statistically speaking, humans are not good at estimating probabilities or risk. In his book Innumeracy, author John Allen Paulos sheds light onto the curious reality, demonstrating how humans tend to overestimate the risk of uncommon causes of death, and underestimate the risk of common causes (people tend to worry far more about the shark attack or the plane crash than simple heart disease, but guess which is by far the most likely to kill you?).

This extends even to skilled professionals – Paulos points to a study (and I was able to find several more recent studies that came to the same conclusions) that show that “most doctors’ assessments of the risks of various operations, procedures, and medications (even in their own specialties) were way off the mark, often by several orders of magnitude.”

It should be no surprise, then, that we in the pro audio industry are equally affected by humankind’s innate inability to accurately grasp the magnitudes of problems. (You knew that’s where I was going with this, didn’t you?) Our own field is rife with examples of things that matter – but maybe not as much as we might think. In fact, I’ve coined a term for these cases where we grossly overestimate the severity or importance of a given concept – The Magnitude Fallacy (I have not yet ascended to the professional heights required to name such a concept after myself).

Put simply, the Magnitude Fallacy states that some things are absolutely real, verifiable, occurrences, but are not nearly as important or critical as we might tend to think. Situations for which the proper response is “yes, but don’t worry about it.” After rolling this concept around in my head for some time, I have become more attuned to examples popping up throughout conversations with colleagues and discussions on audio forums.

One example from my own discipline of sound system alignment – I have, more than once recently, encountered the statement that, when sending both L/R and Subwoofer mixes out of the mixing console, the mixes must leave the console via the same level of “bus hierarchy” – that is, both Mix buses or both Matrices, never one or the other. The logic is that this prevents the mixes from leaving the console at different times, and the difference will disrupt the sound system’s main-subwoofer alignment. Now the first part is perfectly true, given that the console in question does not have built-in phase compensation – for consoles that don’t, there can indeed be a several-sample difference in when signals from Mains or Groups vs Matrices arrive at the console outputs – see the SynAudCon Digital Mixer Study for examples. Let’s put aside the geometric realities of a main-sub alignment – a concept that, itself, suffers from the Magnitude Fallacy in many typically arrangements - and focus on the potential risk caused by this time offset leaving the console.

If we assume the system was already aligned “perfectly” as one desires – and then we have unequal processing path lengths in the console – how much does this disrupt the alignment? Some investigation reveals that the path length difference between Mains, Groups and Matrices is on the order of 0.125 ms (Yamaha QL) to 0.333 ms (DiGiCo SD series @ 48 kHz). This gives us a feel for order of magnitude – let’s assuming a sound system that crosses over from mains to subs around 80 Hz, a relative offset in the main and sub signals of 0.333 ms results in a phase offset of 9.5 degrees, and disrupts the summation by about 0.03 dB (yes, you read that correctly). Let’s assume for the sake of argument that we manage to display truly stunning levels of incompetence by having our subwoofer signal take an entire extra trip through the mixing console before it gets to the PA (let’s call it 0.85ms of latency, for an X32). We’re still less than 25 degrees of phase offset away from perfection, and our summation has suffered by less than 0.2 dB as a result. Suffice it to say you have other problems to deal with in such a situation. With parallel paths running willy nilly through the console, the mix would probably not sound very good – but not because the mains and sub timing was disrupted by 40 samples.

Perhaps a less esoteric example is in order. Among folks taking their first steps in sound system analysis work, there is a pervasive belief that the measurement mics being used must have the accompanying response correction files loaded into the analyzer, while experienced professional systems engineers seldom seem to bother with such. What have the pros learned to justify the apparent lack of concern?

FIGURE 1 - Four measurement mics from different manufacturers, no correction files

Figure 1 shows the result of placing four different measurement microphones made by four different manufacturers in close vicinity and measuring the same source – no correction files loaded. Even though the cost of the mics in use spans an order of magnitude, and the exercise includes both one of the least and one of the most expensive models on the market – the traces are virtually indistinguishable, especially below 4 kHz. In fact this “mic compare” test is a “best practice” way to start a multi-mic measurement session – confirm that all mics are giving the same answer, none of them is broken, and get to work.

FIGURE 2 - TOP: Same position with and without correction curve.

BOTTOM: Two positions separated by 12 inches.

Figure 2 makes this even more clear. The top pane shows two measurements taken from the same mic position: with (red) and without (black) the correction curve loaded into the analyzer. As we can see – for a measurement microphone of reputable quality, the correction curve really isn’t bringing much to the table. The bottom of FIGURE 2 shows the same “without” measurement as before, compared to another measurement taken 12 inches away (purple). The reality is the natural seat to seat variations in acoustic response absolutely swamp the minute deviations corrected by a mic correction file. Although it's easy to see why these files might seem to be of paramount importance to an interested but inexperienced person, the realities of the situation mean that, for the way we use these microphones, the correction file doesn’t offer much benefit besides the not-so-obvious one (proof that the manufacturer measured the microphone’s response and confirmed that it was to spec, and not broken).

Once you are looking out for such examples of the Magnitude Fallacy, you will notice them popping up all the time. One particularly fun one I saw recently – a contractor was planning to “double up” on speaker cable to lower the line loss, and a well-meaning colleague cautioned them against this practice, because if the two cable lengths were not cut to exactly the same length, phase issues could result in cancellation. To cause a cancellation at 20 kHz, the cable lengths would need to be unmatched by a bit over 4 miles. So if you did a couple extra wraps around the outside of the building with one of the pairs, and then terminated back at the loudspeaker, you still run no risk of path length differences causing audio problems (although you would probably be fired for unrelated reasons).

To me, the fact that these are real concerns that are voiced by working professionals is perfect evidence of the Magnitude Fallacy at work. The concerns are based in real, valid science and audio engineering principles. It’s just that the effects are on such a small order that they don’t warrant a high placement on the Totem Pole of Concern, and these seemingly concerning pieces of information are readily assuaged with addition of context. Thus, a dose of reality is advised in such matters – we must consider the scale of the effect as well. The fact that a certain phenomenon or scientific principle exists does not mean it is likely to be a cause of concern in a practical situation. In other words – asking yourself “how much sleep should I lose over this?” is a healthy exercise in perspective – let’s not overlook the real mountains that require our attention because we’re focused on the molehills.