Yes, with similar accuracy (45% vs 68% in this low-res study) to instantaneous phoneme decoding:
Meanwhile, the area 55b arrays, and the dorsal 55b array in particular, appeared to encode the longer units of language, short sentences and sentences (i.e., those with contextual information), much better than phonemes and words, especially during the reading phase (Figure 5B).
The translation accuracy and precision in that study is quite unimpressive; as I mentioned though, resolution makes an enormous difference:
Enabled by these high-resolution recordings, our study participant—who can no longer speak intelligibly owing to amyotrophic lateral sclerosis—achieved a 9.1% word error rate on a 50-word vocabulary (2.7 times fewer errors than the previous state-of-the-art speech BCI2) and a 23.8% word error rate on a 125,000-word vocabulary (the first successful demonstration, to our knowledge, of large-vocabulary decoding). Our participant’s attempted speech was decoded at 62 words per minute, which is 3.4 times as fast as the previous record
This is with 256 total electrode channels. The tech I’m proposing has about a million times this resolution.
The linked study’s 68% accuracy figure is on an exercise predicting which one of ten ~4 word phrases the subject has been cued to speak.
I find it unreasonable to call it “the most pessimistic” way things could go when you extrapolate that to “We will be able to read any novel improvised sentence out of people’s brains faster than they can speak them.” I can imagine a scenario much more pessimistic than that.
I see (agree), was misreading the decoder architecture. Will amend this post when I get back to my laptop.
The original study had two different architectures; one decoded phonemes and matched to the nearest of 50 words, while the other was not phonemic, matching only ~10 phrases. I completely missed this architectural gap for the first 20 minutes after Ninety-Three’s second response, which I score myself a D on metacognition and C+ on outcomes.
Phonemic decoding seems to scale extremely well; 50 words → 125,000 words only doubles error rate. I expect anticipatory / semantic decoding to scale worse, but not extremely poorly.
I agree that 68% and 45% accuracy are terrible, especially on a 50 word vocabulary. The 68% figure was to contextualize the 45% anticipatory accuracy; to show that anticipation doesn’t cause a dramatic accuracy hit, per your original comment.
Then we see that improved methods at 256-electrode resolution (second study) brings accuracy up to ~76% at 125,000 word vocabulary.
So what I’m extrapolating from this is that, given 2023 SOTA, anticipatory accuracy on ~125,000 word decoding should be ~60-75%. I don’t see why having even a mere hundred times the resolution should get less than 95% accuracy on priors?
Also, the study had a small *test* set, but that’s not the same as *training* on only 10 phrases. Very different statements about underlying capacity.
Yes, with similar accuracy (45% vs 68% in this low-res study) to instantaneous phoneme decoding:The translation accuracy and precision in that study is quite unimpressive; as I mentioned though, resolution makes an enormous difference:
This is with 256 total electrode channels. The tech I’m proposing has about a million times this resolution.
The linked study’s 68% accuracy figure is on an exercise predicting which one of ten ~4 word phrases the subject has been cued to speak.
I find it unreasonable to call it “the most pessimistic” way things could go when you extrapolate that to “We will be able to read any novel improvised sentence out of people’s brains faster than they can speak them.” I can imagine a scenario much more pessimistic than that.
I see (agree), was misreading the decoder architecture. Will amend this post when I get back to my laptop.
The original study had two different architectures; one decoded phonemes and matched to the nearest of 50 words, while the other was not phonemic, matching only ~10 phrases. I completely missed this architectural gap for the first 20 minutes after Ninety-Three’s second response, which I score myself a D on metacognition and C+ on outcomes.
Phonemic decoding seems to scale extremely well; 50 words → 125,000 words only doubles error rate. I expect anticipatory / semantic decoding to scale worse, but not extremely poorly.
I agree that 68% and 45% accuracy are terrible, especially on a 50 word vocabulary. The 68% figure was to contextualize the 45% anticipatory accuracy; to show that anticipation doesn’t cause a dramatic accuracy hit, per your original comment.
Then we see that improved methods at 256-electrode resolution (second study) brings accuracy up to ~76% at 125,000 word vocabulary.
So what I’m extrapolating from this is that, given 2023 SOTA, anticipatory accuracy on ~125,000 word decoding should be ~60-75%. I don’t see why having even a mere hundred times the resolution should get less than 95% accuracy on priors?
Also, the study had a small *test* set, but that’s not the same as *training* on only 10 phrases. Very different statements about underlying capacity.