Audio in the Cloud · Instrument Lab · iteration 9
Does it sound recorded?
The same notes twice: new, the drop-tuned rock band of the newest page, a distorted guitar through a high-gain amp, an electric bass, a drum kit and a choir, measured from recordings and still nothing but code; and recorded, those instruments themselves, note by note. Each phrase is one instrument. Switch between the two while one plays.
The same notes three ways: today's, the strings the A-Pop page played before its sections (still there as its classic strings); new, built from measurements of a real violin, cello and double bass and still nothing but code; and recorded, those players themselves, note by note. One player a part. Switch between them while one plays.
The same notes twice: new, the brass and woodwinds the Marching page plays, measured from recordings of real players and still nothing but code; and recorded, those players themselves, note by note. Each phrase is one instrument. Switch between the two while one plays.
The same notes on three nylon-string guitars: today's, the plucked-string sound the Bossa page plays now; new, rebuilt from measurements of a real classical guitar and still nothing but code; and recorded, that guitar itself. Switch between them while one plays.
The same notes on three pianos: today's, the synthesized piano the genre pages play now; new, rebuilt from measurements of a real concert grand and still nothing but code; and recorded, that grand itself. Switch between them while one plays.
Listen
Starting…
Same notes, same timing, matched loudness. The switch is instant and the versions stay in step, so the only thing that changes is the instrument. Keys: space plays and stops, and the number on a version switches to it.
Piano radio
Background music in eight styles, written for the new piano and played on it live as you listen. Code plays the timing, the touch and the pedal too, and nothing here is recorded: every note is built on your device.
Blind test
The names are hidden and the phrase is picked at random. Switch between X and Y as often as you like, then answer.
How it's made
Each new instrument is a formula, and every number in it is measured from the recording. Nothing is sampled: your device works the sound out from these.
Strings
y(t) = ∑h Ah(e, v) cos(2πh ∫f0 dt + φh) + bow(t)
Ah in dB = Lh + sh · D(e) + wh · v(t)
- h
- the harmonic: 1 is the note itself, 2 an octave up, 3 an octave and a fifth, and so on.
- Lh, φh
- each harmonic's level and where it peaks against the fundamental, measured on every recorded violin, cello and double bass note.
- e(t)
- how hard the bow digs in, moment by moment: the violin speaks in about 40 ms, the cello and the bass more slowly, and the string rings on a moment when the bow leaves it.
- D(e), sh
- bowing harder makes the note louder by D, and the upper harmonics grow faster than the first (sh), so louder is brighter.
- v(t), wh
- the vibrato swings the pitch v cents either way, five or six times a second, and each harmonic slides along the wooden body's resonances, so its level swings too: wh dB per cent, up on one harmonic, down on the next, measured on the recordings. That shimmer is what makes strings sound like strings.
- bow(t)
- noise with the recorded bows' own spectrum, measured between the harmonics octave by octave.
- section
- a section adds up six such players, each a few cents off and with a vibrato of their own, so the harmonics wobble 4 to 6 dB as the players drift in and out of step, as the recorded sections' do. Each player is heard by two microphones a fraction of a millisecond apart, which keeps it wide, and its levels Lh come from the recorded sections, which sound darker and smoother than one player up close.
- spiccato
- a short note, the bow thrown at the string and off it: e jumps over the held level (about 5 dB when soft, 2 when loud), stays a moment (60 to 80 ms), and the string rings down on its own (to a third in 145 ms soft, 240 loud), with a burst of low, dull bow noise as it lands. Six numbers at each strength, fitted to a recorded cello section's spiccato.
Brass and woodwinds
y(t) = ∑h Ah(e) cos(2πh ∫f0 dt + φh) + breath(t)
Ah(e) in dB = Lh + sh · D(e)
- h
- the harmonic: 1 is the note itself, 2 an octave up, 3 an octave and a fifth, and so on.
- Lh, φh
- each harmonic's level and where it peaks against the fundamental, which shapes the wave. Measured on every recorded note over the first 1.5 seconds, the part of a note a phrase uses.
- e(t)
- how hard the note is blown, moment by moment: it rises in 30 to 70 ms, overshoots a little on the brass, and swells or fades as the music asks.
- D(e), sh
- blowing harder makes the note louder by D, and the upper harmonics grow up to three times faster than the first (sh), so louder is brighter.
- f0(t)
- the pitch, which starts a little flat on a brass attack, wanders a few cents, glides on a slur and, on the flute, sings vibrato.
- breath(t)
- noise with the recorded players' own breath spectrum, measured between the harmonics octave by octave.
Piano
y(t) = ∑k ak [(1 − r) e−t/τ1 + r e−t/τ2] sin(2πfkt + φk)
fk = k f1 √(1 + Bk2)
- ak
- each overtone's level at the strike, measured from the recorded grand every three semitones, 27 notes in all.
- τ1, τ2, r
- a struck string fades in two stages: a fast one as the soundboard drinks its energy (τ1), then a quiet aftersound that rings for seconds (τ2, a share r), where the unison's strings beat slowly. Measured by register and overtone.
- B
- how stiff the strings are, so overtones run sharp, more and more up the keyboard; the octaves are stretched as a tuner stretches them.
- touch
- a harder hit is louder and much brighter: one filter per note, fitted to the recording, up to 30 dB more at 10 kHz on a fortissimo.
- and
- the hammer's knock, the damper's thud, and two microphones that hear the whole soundboard.
Rock band
y(t) = amp(∑k ak e−r(fk)t sin(2πfk w(t) + φk) + hiss(t))
- ak
- each overtone's level as the pick leaves the string: notches where the pick and the two pickups sit along it, the pickup's resonance, and the pick's own softness. The guitar's overall tone was then fitted to recorded distorted guitars: 5.5 dB apart on average, from 13 at first.
- r(f)
- how fast each overtone fades, by its frequency, measured on recorded clean electric guitars: the low ones ring for seconds, the top dies in a fraction of one. A palm on the strings makes every overtone die faster, the higher the faster.
- w(t)
- the pitch, as time warped: a hard pick starts a slack, drop-tuned string sharp, it settles, and it wanders a cent or two.
- amp
- a boost into two tube-like stages (soft clipping that leans to one side, so even harmonics too), a tone stack, a power stage, and a 4×12 cabinet as an impulse response.
- bass
- the same string, picked or plucked with a finger (which slaps it on the frets a little), through a clean path and a gritty one: 6.2 dB from three recorded basses, from 22.
- kit
- drumheads as inharmonic modes that bend down in pitch as the head relaxes, the snare's wires buzzing in step with its bottom head while that head rings on under them, cymbals as hundreds of bronze modes down to their low hum, and the room the recorded kit was in: 8.4 dB from it, from 26 at first.
- choir
- each singer one wave whose overtones follow the vowel measured from recorded choirs (1.5 dB apart on "ah", 4.5 on "oo"), three to a part, each with a vibrato of their own that starts late, a scoop into the note and a slow drift, and breath under them.
Nylon guitar
y(t) = ∑k ak e−t/τk (1 + mk sin 2πβkt) sin(2πfkt + φk)
fk = k f1 √(1 + Bk2)
- ak, τk
- each overtone's starting level and how fast it fades, measured note by note on a recorded classical guitar from B1 to A♯5.
- mk, βk
- no overtone fades smoothly: each wobbles by about a decibel, two or three times a second, as the string's two ways of swinging beat.
- B
- how stiff the string is, so its overtones run a little sharp.
- body
- the top's resonance near 190 Hz swallows overtones that land on it, and letting go of the string knocks the body with a short, woody thud.
- strings
- six of them: a new note on a string stops the one it was playing, while the others ring on.
The yardstick
gap = meant, b | Lnew(t, b) − Lrec(t, b) |
- t, b
- the phrase cut into 46 ms moments and 25 third-octave bands from 63 Hz to 16 kHz.
- L
- the level in each (dB), after both versions are matched in loudness and lined up on their first note. Anything 50 dB under the loudest moment counts as silence, and only where one of them sounds counts.
- 0 dB
- would be the recording itself. About 1 dB in a band is the smallest change most people hear.
How close each version has come
The gap between each version and the recording, averaged over the instrument's phrases here. Lower is closer; the brass line is the recording. Hover or tap a point for each phrase.
The numbers
New in this pass
- The kit, second pass. The snare had a low boom under it, a ring at about 137 Hz the recorded snare doesn't have. Its ring now sits where the recorded one's does, on the bottom head at about 220 Hz, and the rest of it dies as fast as the recording's: 20 dB down in 68 ms, where it took 182. The hi-hats and cymbals now reach down into the low end the recorded ones carry. Hit by hit, the kit went from 10.5 dB to 8.4 apart; hear it on the Drum groove.
- A band. A distorted guitar tuned down to drop C through a high-gain amp, an electric bass, a drum kit and a choir, for a page of early-2000s heavy rock. Like the rest of the lab, every note is worked out on your device, in slices, so the page stays responsive while it builds.
- Measured against recordings. Each instrument was fitted to recordings of the real thing, piece by piece and band by band: the guitar from 13 dB to 5.5 apart, the bass from 22 to 6.2, the kit from 26 to 8.4, the choir to 1.5 ("ah") and 4.5 ("oo").
- The kit's room. The recorded kit's close microphones come mixed with room microphones, so the library's kit now has a room of its own, a short, darkening tail every drum and cymbal is sent to.
- And for the page: synth pads, strings, pulses, a lead, a sub bass, a pluck and a bell, and a DJ's decks: scratches written out stroke by stroke (baby, chirp, transformer, flare, scribble, tear), with sounds made from the choir's own vowel, plus risers, a reversed cymbal and vinyl crackle. Those have no recording here to stand against.
What's left
- Palm mutes have no recording to measure against: the chugs are the recorded clean mutes' fade rates, through the amp.
- Power chords here are single recorded notes played together, each through its own amp, where a real guitar's three strings share one, so the recording's chords are cleaner than a real one's.
- The kick keeps more of its beater's click than the recorded kick, on purpose: it has to cut through high-gain guitars. And the pedalled hi-hat still dies about twice as fast as the recorded one.
New in this pass
- Short notes. Spiccato, the bow thrown at the string and off it, measured from a real cello section's: soft, a note jumps about 5 dB over the same note held, loud about 2, speaks in 65 to 80 ms, rings down on its own, and lands with a burst of low, dull bow noise. The A-Pop page's stabs and runs are played this way now. Hear it on the Spiccato phrase.
- Sections. Violins, violas, cellos and basses as sections of several players, the way a pop song's strings play, built from recordings of a real violin section and a real cello section at two strengths. One player up close is 10 to 25 dB brighter in its upper harmonics than a section across a room, and the new section is as wide as the recorded ones, where the strings before were nearly mono. Soft notes swell in over about a second, loud ones bite.
- Strings, measured. A violin, a cello and a double bass, every harmonic's level and the shape of each note's wave measured from recordings of real players, 59 notes in all, and played by the same engine as the brass and woodwinds: a bowed note is a steady tone, as a blown one is.
- The shimmer. What a bowed string has that a wind doesn't is its wooden body. As the vibrato swings the pitch, every harmonic slides along the body's resonances and its level swings with it, up on one and down on the next: on the recorded cello by 1 to 10 dB. Each note here swings as its recording does.
- The bow. Its noise under every note is the recorded bows' own, measured between the harmonics, and it bites a little as a note starts. A string rings on a moment when the bow leaves it, longer on the bigger instruments.
- A viola too, worked out from the violin, a little bigger. Nobody recorded one here.
Next
- Your ears decide the next pass. Known so far: the solo players were recorded at one strength, so how their tone brightens with force is taken from the sections; the recorded double bass has no vibrato, so the bass's shimmer is borrowed from the cello an octave up; nobody recorded a viola or a bass section, so those are worked out from their neighbours.
- Pizzicato on the A-Pop page is now a section: six plucked strings, a little apart in time and tuning, fitted to a recording of a plucked section (a soundfont's samples). It measures 4 dB from it, where the one string it replaces was 12. The page plays everything else on the sections, its held parts a section a part and its stabs and runs spiccato, with its classic strings a switch away. Only the cellos' spiccato was recorded: the violins' and violas' are worked out from it.
- The upright bass can be built too: the double bass recorded here is bowed, but the same soundfont has a plucked upright. The Bossa page's upright is now built from that recording's measurements, overtone by overtone, and measures 2 dB from it (the old one was 16).
New in this pass
- Measured where music plays. The recorded players held each note for five seconds, and a long note sags as the breath runs down: by two seconds the recorded trumpet is 5 dB quieter and much darker. Each note is now measured over its first 1.5 seconds, the part a phrase uses, instead of over the whole sagging tail. The trumpet was the most affected: its upper harmonics were up to 10 dB too dark, and now sit within a decibel or two of the recording.
- Attacks to match. Measured that way, a tongued trumpet, trombone or tuba note overshoots the level it settles to by about 2 to 4 dB, not 4 to 10, so the brass attacks are gentler.
- A yardstick. Every version so far is now scored against the recordings with the same measure (How it's made, above): the brass and woodwinds went from 4.7 to 4.4 to 3.9 dB, and the trumpet's phrase from 6.2 to 3.9.
From the last pass
- Breath, measured. Under every held note is the breath of the recorded players themselves: what is left between the harmonics of each recorded note, octave by octave.
- Six instruments to hear against the real thing. The trumpet, trombone, tuba, flute, clarinet and alto sax each play a phrase next to recordings of real players, note for note, at the strength the recordings were made at.
Already there
- Measured, not guessed. Every harmonic's level and the shape of each note's wave come from recordings of real players, note by note across each instrument's range.
- Louder is brighter. Each note is three waves, soft, medium and loud, crossfaded as it is blown harder, so an attack blooms from dark to bright as a real one does.
- Players, not oscillators. A brass note starts a little flat and settles, every pitch wanders a few cents, the flute sings vibrato, and a section is several players a few cents and milliseconds apart.
- Still pure code. The Marching page plays a whole band of them live, worked out as it goes.
Next
- Your ears decide the next pass. Spot the recording shows where they still give themselves away. Known so far: the first tenth of a second of each note is where most of the gap now is (the recorded tuba's attacks are brighter, the clarinet's harmonics come in later); each recording has one strength, so how the tone brightens with force is a slope rather than a measurement; the falls, shakes and the tongue's puff are built from how players describe them; and the mellophone, baritone, piccolo and tenor sax, which nobody recorded here, are worked out from their nearest relatives.
- Then recordings at several strengths, to measure how each instrument brightens as it is blown harder.
New in this pass
- A guitar, measured. Every overtone's starting level and how fast it fades, how stiff each string is and how the body pulls on it all come from a recorded classical guitar, measured note by note from B1 to A#5 and checked against it the same way.
- Alive as it rings. On the recording no overtone fades smoothly: each wobbles by about a decibel, two or three times a second, as the string's two ways of swinging beat against each other. These do too, and a fretted note's pitch drifts a few cents as the finger presses.
- The pluck and the body. The trebles lose a little in the first tenth of a second, the top's resonance near 190 Hz swallows the overtones that land on it (G3's fundamental is nearly gone in half a second), and letting go of the string knocks the body with a short, woody thud, as on the recording, without a rumble underneath that a real body would not give off.
- Six strings. A new note on a string stops the one it was playing while the others ring on, so arpeggios and pinched chords ring the way a guitarist's do.
- Still pure code. The guitar adds about 25 KB to the library; your browser builds all 88 of its takes in about a second and a half.
Next
- Your ears decide the next pass. The blind test shows where the new guitar still gives itself away. Known so far: the recording has one strength of pluck, so how the tone changes with force is a guess, the open strings ringing along in sympathy (the halo round a real guitar's notes) are not there yet, nor is the small room the recording was made in.
- Then the Bossa page switches over to the new guitar.
- Then upright bass, strings and drums, measured the same way.
New in the piano's last pass
- Open, not phasey. Above the bass, every overtone now reaches the two microphones at its own level and phase, as the recording's do, so the top end sounds wide and open instead of hollow between the speakers.
- A truer strike. The hammer's thump and the key coming back up are rebuilt from the recording: a low knock in the case that rings for a few tenths of a second, and a soft thud on release.
- Fuller, more even bass. Low notes get the faint high overtones the recording has, and every key now starts at full strength (on some keys the strings' parts partly cancelled, leaving the fundamental several dB weak).
- A steadier treble. The top octave fades as fast as the real one, and no note strays far from its neighbours.
Already there
- Measured, not guessed. Every overtone's level, tuning and two-stage fade (the fast prompt sound, then the quiet aftersound) comes from measurements of a real concert grand, taken every three semitones across the keyboard.
- Touch changes the tone. Playing harder makes it brighter the way the recording does, not only louder.
- Still pure code. About 35 KB of JavaScript builds every note in your browser, with no samples and no network.
Next
- Your ears decide the next pass. The blind test shows where the new piano still gives itself away. Known so far: loud chords carry a little more deep thump than the recording, and the very top of each strike (above 8 kHz) is softer.
- Then the band, measured the same way: the nylon guitar is here now; upright bass, strings and drums next.
- Then the pages switch over to the shared instruments one at a time, so every page gets better together.