Browser Audio Latency

Link post

Any time you are working with digital audio, you have to handle latency: how long does it take from when the audio enters the system until it leaves again? Special purpose professional-level audio hardware generally has been optimized for extremely low latency, often adding as little as 2ms. Extreme low latency, however, has costs: scheduling must be very predictable, power consumption is higher, audio processing algorithms can look at fewer samples, and you have much less recovery time before causing an audible glitch.

Web browsers are in an interesting position, where they have to work with whatever audio hardware the computer happens to have, and need to handle a very wide range of use cases.

I’ve recently been working on two musical projects where I care a lot about latency, but in different ways:

Bass Whistle JS: a bass synthesizer that you control by whistling. To feel natural and responsive, it needs the lowest possible latency.
Bucket Brigade: a platform for making music with others over the internet. It doesn’t need low latency, but it does need to know precisely how much there is so it can compensate.

But how much latency is there? We can measure it: send a beep to the speaker, check how much time passes until we detect the beep in the samples coming from the microphone (example worklet). On my 2016 15″ MacBook running OSX 11.1, if I do:

  var audioCtx = new window.AudioContext();
  var micStream =
    await navigator.mediaDevices.getUserMedia(
      {audio: true});

I measure ~67ms end-to-end latency in Chrome, and ~55ms in Firefox. Instead of using default settings, though, we could tune for low latency:

  var audioCtx = new window.AudioContext({
    latencyHint: 0,
  });
  var micStream =
    await navigator.mediaDevices.getUserMedia({
      audio: {
         latency: 0,
         echoCancellation: false,
         noiseSuppression: false,
         autoGainControl: false,
      }});

Note that we’re asking AudioContext for an output latency of ‘0’ and not interactive. While MDN documents interactive as “try to use the lowest possible and reliable latency it can”, ‘0’ is even lower. We are also turning off all of the smart processing options, which would add latency.

This gives me ~19ms in Chrome and ~14ms in Firefox. Try it yourself.

In the other direction, for Bucket Brigade, I tune for high latency with the hope that it will be completely consistent:

  var audioCtx = new window.AudioContext({
    latencyHint: ‘playback’,
  });
  var micStream =
    await navigator.mediaDevices.getUserMedia({
      audio: {
         echoCancellation: false,
         noiseSuppression: true,
         autoGainControl: false,
      }});

Setting high latency values on getUserMedia doesn’t seem to have an effect in either Chrome or Firefox, so I’ve left it off.

One of the most awkward aspects of Bucket Brigade today, which also would apply to anything else that lets you record multitrack audio in the browser, is that you have to measure latency. There is not yet any way to ask a browser how much end-to-end latency there is. There’s input latency, MediaTrackSettings.latency, only implemented in Chrome, but in my testing it always returns 0.01 (10ms) even when I switch to a higher latency microphone. There’s output latency, AudioContext.outputLatency, only implemented in Firefox, but in my testing it returns implausibly low numbers, less than half of the end-to-end latency.

There are some situations in which measurement is going to be the best you can do: Bluetooth headsets are notorious for reporting much lower latencies than they actually implement. That is beyond the control of the browser, however, and giving developers the number the devices claim would still be a big improvement. Even if you can’t completely trust the number, if it changes then you know you need to re-run measurement.