012 / SOUND
Voice spectrum
Any sound breaks down into simple sine waves of different frequencies. The browser does it with a fast Fourier transform dozens of times a second. Turn on the microphone and say "ah", whistle or play a note. In the synthesizer, build a sound yourself from harmonics.
Sound is processed in your browser and is not sent anywhere.
Theory behind the experimentHow a sound splits into sines, what harmonics are, and why you cannot know frequency and time precisely at oncePlain explanation, equations and how the model works
In plain words
Sound is a fast wobble of air pressure. A microphone turns it into a number that changes 48 thousand times a second. Plot that number and you get a wave: for a whistle it looks like a smooth sine, for a voice like a jagged line.
Fourier showed that any such wave can be built from ordinary sines of different frequencies and loudness. The Fourier transform does the reverse: it takes a piece of the wave and finds how much of each sine is in it. That is the spectrum. Frequency runs along the bottom, loudness goes up.
If a sound repeats with frequency , like a sung note or a plucked string, it can only contain the frequencies and so on. These are harmonics. The first sets the pitch, and how loud the others are largely sets the timbre. That is why "ah" and "oo" on the same note sound different: the fundamental stays, the harmonics change.
Computing the spectrum head-on is slow. The fast Fourier transform (FFT) does the same job more cleverly and hundreds of times faster. That is how a browser can redo the spectrum dozens of times a second, even on a phone.
Every spectrum has a price. To tell two close frequencies apart you have to listen for a long time. To catch a new sound quickly you have to listen briefly. You cannot have both: it is an uncertainty relation, just like in quantum mechanics, only for sound.
Try it yourself
- Turn on the microphone and whistle. The spectrum shows one narrow line: a whistle is almost a pure sine.
- Sing "ah". A comb of evenly spaced lines appears. The numbered dashed lines mark the harmonics .
- Keep the note and switch from "ah" to "oo". The first line stays put, the others change in loudness.
- Clap your hands. A clap is short and holds almost every frequency at once: the waterfall gets a horizontal stripe across the whole screen.
- Switch the FFT window from "Fast" to "Sharp". The lines get thinner, but the waterfall starts to lag behind the sound.
- Sing a note and press "Rebuild this sound in the synth". The synth plays your harmonics as pure sines. Similar, but without breath and the live onset.
- In the synth pick "Square", then "Saw", then "Triangle". Compare the harmonic bars with the formulas below.
- On the saw press "Drop the minus signs". The waveform changes a lot, yet the spectrum stays and the sound is nearly the same: the ear hears how loud the harmonics are and barely hears their shift.
Physics and equations
Fourier series. A periodic sound with frequency can be written as a sum of harmonics:
Here is the amplitude of harmonic . In general each harmonic also has its own phase. For the three classic waves the coefficients are known exactly:
The square and the triangle have no even harmonics at all. The triangle's amplitudes fall as , so it sounds softer than the saw, whose amplitudes fall as .
Discrete transform. A computer does not see a continuous wave but samples taken at rate (usually 48,000 per second). Their spectrum is
The highest frequency that can be recorded is , 24 kHz. This is the Nyquist limit. A direct sum needs about multiplications, the Cooley-Tukey algorithm (FFT) about . For that is 67 million against 106 thousand.
Window. A piece of sound is cut off at its edges, and the cut itself creates false frequencies all over the spectrum. So the edges are faded out with a window function. The browser uses the Blackman window:
It strongly suppresses the false tails but makes each line a few bins wide. The level in the spectrum is
These are decibels relative to the microphone's maximum, not sound-meter loudness. A sine at the largest possible amplitude reads about −13.6 dB after the Blackman window.
Uncertainty. A window of samples lasts , and neighbouring frequencies in the spectrum are apart:
For that is 171 ms and 5.9 Hz. The grid step is not the limit of accuracy. A single peak can be located between bins, but two tones closer than a few steps merge into one.
Phase and hearing. The loudness of a wave depends only on the harmonic amplitudes, not on their signs or shifts:
So flipping the sign of a harmonic changes the waveform but not the spectrum. The ear, too, mostly hears only the amplitudes. This is the observation of Ohm and Helmholtz, though it is not absolute: abrupt phase changes can sometimes be heard.
How the model works
The spectrum is computed by the browser itself with the built-in Web Audio AnalyserNode: Blackman window, FFT of , or samples, level in decibels. Between frames the magnitudes are smoothed:
For verification, an FFT module written in TypeScript reproduces the browser's computation. Tests check it against the direct sum, and the level of a pure sine against the theoretical −13.6 dB.
The pitch is found from the harmonics. For each candidate peak the sum is taken of how far it and the peaks at stand above the local noise level. The candidate with the largest sum wins. The exact peak position is refined with a parabola through three neighbouring bins:
A note is shown only if it holds for four frames in a row within half a semitone. Tests check that loud white noise gives no notes, and that a voice in room noise is found to better than 1 Hz.
The frequency axis is logarithmic, from 20 Hz to 20 kHz, the way the ear hears. The waterfall adds 60 rows per second of real time and keeps the last 6 seconds. The synth adds up 16 harmonics and sets the volume by the RMS value, so every waveform sounds equally loud.
The site asks the browser to switch off noise suppression and auto gain, because noise suppression erases steady notes over time. On some phones the built-in microphone does not work without that processing, and then the spectrum may be slightly altered. Sound is processed only in your browser and is not sent anywhere.
Sources and further reading
- J. W. Cooley, J. W. Tukey. An algorithm for the machine calculation of complex Fourier series. Math. Comp. 19, 297 (1965). doi ↗
- F. J. Harris. On the use of windows for harmonic analysis with the discrete Fourier transform. Proc. IEEE 66, 51 (1978). doi ↗
- H. von Helmholtz. On the Sensations of Tone (1863, English translation 1885). archive.org ↗
- W3C. Web Audio API: AnalyserNode, FFT windowing and smoothing over time. w3.org ↗