{"id":24932,"date":"2025-11-21T21:03:00","date_gmt":"2025-11-21T21:03:00","guid":{"rendered":"https:\/\/blackrockneurotech.com\/?p=24932"},"modified":"2025-12-22T22:16:16","modified_gmt":"2025-12-22T22:16:16","slug":"decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface","status":"publish","type":"post","link":"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/","title":{"rendered":"Decoding Inner Speech with an End-to-End Brain-to-Text Neural Interface"},"content":{"rendered":"\n<p><strong>Authors: <\/strong>Yizi Zhang, Linyang He, Chaofei Fan, Tingkai Liu, Han Yu, Trung Le, Jingyuan Li, Scott Linderman, Lea Duncker, Francis R Willett, Nima Mesgarani, Liam Paninsk<\/p>\n\n\n\n<p><strong>Abstract:<\/strong> Speech brain\u2013computer interfaces (BCIs) aim to restore communication for people with paralysis by translating neural activity into text. Most systems use cascaded frameworks that decode phonemes before assembling sentences with ann-gram language model (LM), preventing joint optimization of all stages simultaneously. Here, we introduce an end-to-end BraIn-to-Text (BIT) framework that translates neural activity into coherent sentences using a single differentiable neural network. Central to our approach is a cross-task, cross-species pretrained neural encoder, whose representations transfer to both attempted and imagined<br>speech. In a cascaded setting with an n-gram LM, the pretrained encoder establishes a new state-of-the-art (SOTA) on the Brain-to-Text \u201924 and \u201925 benchmarks. Integrated end-to-end with audio large language models (LLMs) and trained with contrastive learning for cross-modal alignment, BIT reduces the word error rate (WER) of the prior end-to-end method from 24.69% to 10.22%. Notably, we find<br>that small-scale audio-LLMs markedly improve end-to-end decoding. Beyond record-setting performance, BIT aligns attempted and imagined speech embeddings to enable cross-task generalization. Altogether, our approach advances the integration of large, diverse neural datasets, paving the way for an end-to-end decoding framework that supports seamless, differentiable optimization.<\/p>\n\n\n\n<p>Read the pre-print <a href=\"https:\/\/arxiv.org\/pdf\/2511.21740\">here<\/a>. <\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Authors: Yizi Zhang, Linyang He, Chaofei Fan, Tingkai Liu, Han Yu, Trung Le, Jingyuan Li, Scott Linderman, Lea Duncker, Francis [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-24932","post","type-post","status-publish","format-standard","hentry","category-blog"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.0 (Yoast SEO v27.0) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Decoding Inner Speech with an End-to-End Brain-to-Text Neural Interface | Blog | Blackrock Neurotech<\/title>\n<meta name=\"description\" content=\"Authors: Yizi Zhang, Linyang He, Chaofei Fan, Tingkai Liu, Han Yu, Trung Le, Jingyuan Li, Scott Linderman, Lea Duncker, Francis R Willett, Nima Mesgarani,\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Decoding Inner Speech with an End-to-End Brain-to-Text Neural Interface | Blog | Blackrock Neurotech\" \/>\n<meta property=\"og:description\" content=\"Authors: Yizi Zhang, Linyang He, Chaofei Fan, Tingkai Liu, Han Yu, Trung Le, Jingyuan Li, Scott Linderman, Lea Duncker, Francis R Willett, Nima Mesgarani,\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/\" \/>\n<meta property=\"og:site_name\" content=\"Blackrock Neurotech\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/blackrockneuro\/\" \/>\n<meta property=\"article:published_time\" content=\"2025-11-21T21:03:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-12-22T22:16:16+00:00\" \/>\n<meta name=\"author\" content=\"Jessica Nani\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@blackrockneuro_\" \/>\n<meta name=\"twitter:site\" content=\"@blackrockneuro_\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/\"},\"author\":{\"name\":\"Jessica Nani\",\"@id\":\"https:\/\/blackrockneurotech.com\/#\/schema\/person\/a0cf54e55265f6a8733613617e13c34f\"},\"headline\":\"Decoding Inner Speech with an End-to-End Brain-to-Text Neural Interface\",\"datePublished\":\"2025-11-21T21:03:00+00:00\",\"dateModified\":\"2025-12-22T22:16:16+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/\"},\"wordCount\":220,\"publisher\":{\"@id\":\"https:\/\/blackrockneurotech.com\/#organization\"},\"articleSection\":[\"Blog\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/\",\"url\":\"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/\",\"name\":\"Decoding Inner Speech with an End-to-End Brain-to-Text Neural Interface | Blog | Blackrock Neurotech\",\"isPartOf\":{\"@id\":\"https:\/\/blackrockneurotech.com\/#website\"},\"datePublished\":\"2025-11-21T21:03:00+00:00\",\"dateModified\":\"2025-12-22T22:16:16+00:00\",\"description\":\"Authors: Yizi Zhang, Linyang He, Chaofei Fan, Tingkai Liu, Han Yu, Trung Le, Jingyuan Li, Scott Linderman, Lea Duncker, Francis R Willett, Nima Mesgarani,\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blackrockneurotech.com\/#website\",\"url\":\"https:\/\/blackrockneurotech.com\/\",\"name\":\"Blackrock Neurotech\",\"description\":\"A neurotech company.\",\"publisher\":{\"@id\":\"https:\/\/blackrockneurotech.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blackrockneurotech.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/blackrockneurotech.com\/#organization\",\"name\":\"Blackrock Neurotech\",\"url\":\"https:\/\/blackrockneurotech.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blackrockneurotech.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/blackrockneurotech.com\/wp-content\/uploads\/2023\/06\/Blackrock-Neurotech.jpg\",\"contentUrl\":\"https:\/\/blackrockneurotech.com\/wp-content\/uploads\/2023\/06\/Blackrock-Neurotech.jpg\",\"width\":200,\"height\":200,\"caption\":\"Blackrock Neurotech\"},\"image\":{\"@id\":\"https:\/\/blackrockneurotech.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/blackrockneuro\/\",\"https:\/\/x.com\/blackrockneuro_\",\"https:\/\/www.instagram.com\/blackrockneuro\/\",\"https:\/\/www.youtube.com\/@BlackrockNeurotech\",\"https:\/\/www.linkedin.com\/company\/blackrock-neurotech\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/blackrockneurotech.com\/#\/schema\/person\/a0cf54e55265f6a8733613617e13c34f\",\"name\":\"Jessica Nani\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blackrockneurotech.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/2c6181502c42d607cea39d2556e980782443c972282ab6155b09180246ff922b?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/2c6181502c42d607cea39d2556e980782443c972282ab6155b09180246ff922b?s=96&d=mm&r=g\",\"caption\":\"Jessica Nani\"},\"url\":\"https:\/\/blackrockneurotech.com\/insights\/author\/blackrock-jessn\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Decoding Inner Speech with an End-to-End Brain-to-Text Neural Interface | Blog | Blackrock Neurotech","description":"Authors: Yizi Zhang, Linyang He, Chaofei Fan, Tingkai Liu, Han Yu, Trung Le, Jingyuan Li, Scott Linderman, Lea Duncker, Francis R Willett, Nima Mesgarani,","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/","og_locale":"en_US","og_type":"article","og_title":"Decoding Inner Speech with an End-to-End Brain-to-Text Neural Interface | Blog | Blackrock Neurotech","og_description":"Authors: Yizi Zhang, Linyang He, Chaofei Fan, Tingkai Liu, Han Yu, Trung Le, Jingyuan Li, Scott Linderman, Lea Duncker, Francis R Willett, Nima Mesgarani,","og_url":"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/","og_site_name":"Blackrock Neurotech","article_publisher":"https:\/\/www.facebook.com\/blackrockneuro\/","article_published_time":"2025-11-21T21:03:00+00:00","article_modified_time":"2025-12-22T22:16:16+00:00","author":"Jessica Nani","twitter_card":"summary_large_image","twitter_creator":"@blackrockneuro_","twitter_site":"@blackrockneuro_","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/#article","isPartOf":{"@id":"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/"},"author":{"name":"Jessica Nani","@id":"https:\/\/blackrockneurotech.com\/#\/schema\/person\/a0cf54e55265f6a8733613617e13c34f"},"headline":"Decoding Inner Speech with an End-to-End Brain-to-Text Neural Interface","datePublished":"2025-11-21T21:03:00+00:00","dateModified":"2025-12-22T22:16:16+00:00","mainEntityOfPage":{"@id":"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/"},"wordCount":220,"publisher":{"@id":"https:\/\/blackrockneurotech.com\/#organization"},"articleSection":["Blog"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/","url":"https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/","name":"Decoding Inner Speech with an End-to-End Brain-to-Text Neural Interface | Blog | Blackrock Neurotech","isPartOf":{"@id":"https:\/\/blackrockneurotech.com\/#website"},"datePublished":"2025-11-21T21:03:00+00:00","dateModified":"2025-12-22T22:16:16+00:00","description":"Authors: Yizi Zhang, Linyang He, Chaofei Fan, Tingkai Liu, Han Yu, Trung Le, Jingyuan Li, Scott Linderman, Lea Duncker, Francis R Willett, Nima Mesgarani,","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blackrockneurotech.com\/insights\/decoding-inner-speech-with-an-end-to-end-brain-to-text-neural-interface\/"]}]},{"@type":"WebSite","@id":"https:\/\/blackrockneurotech.com\/#website","url":"https:\/\/blackrockneurotech.com\/","name":"Blackrock Neurotech","description":"A neurotech company.","publisher":{"@id":"https:\/\/blackrockneurotech.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blackrockneurotech.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/blackrockneurotech.com\/#organization","name":"Blackrock Neurotech","url":"https:\/\/blackrockneurotech.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blackrockneurotech.com\/#\/schema\/logo\/image\/","url":"https:\/\/blackrockneurotech.com\/wp-content\/uploads\/2023\/06\/Blackrock-Neurotech.jpg","contentUrl":"https:\/\/blackrockneurotech.com\/wp-content\/uploads\/2023\/06\/Blackrock-Neurotech.jpg","width":200,"height":200,"caption":"Blackrock Neurotech"},"image":{"@id":"https:\/\/blackrockneurotech.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/blackrockneuro\/","https:\/\/x.com\/blackrockneuro_","https:\/\/www.instagram.com\/blackrockneuro\/","https:\/\/www.youtube.com\/@BlackrockNeurotech","https:\/\/www.linkedin.com\/company\/blackrock-neurotech\/"]},{"@type":"Person","@id":"https:\/\/blackrockneurotech.com\/#\/schema\/person\/a0cf54e55265f6a8733613617e13c34f","name":"Jessica Nani","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blackrockneurotech.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/2c6181502c42d607cea39d2556e980782443c972282ab6155b09180246ff922b?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/2c6181502c42d607cea39d2556e980782443c972282ab6155b09180246ff922b?s=96&d=mm&r=g","caption":"Jessica Nani"},"url":"https:\/\/blackrockneurotech.com\/insights\/author\/blackrock-jessn\/"}]}},"_links":{"self":[{"href":"https:\/\/blackrockneurotech.com\/wp-json\/wp\/v2\/posts\/24932","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blackrockneurotech.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blackrockneurotech.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blackrockneurotech.com\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/blackrockneurotech.com\/wp-json\/wp\/v2\/comments?post=24932"}],"version-history":[{"count":1,"href":"https:\/\/blackrockneurotech.com\/wp-json\/wp\/v2\/posts\/24932\/revisions"}],"predecessor-version":[{"id":24933,"href":"https:\/\/blackrockneurotech.com\/wp-json\/wp\/v2\/posts\/24932\/revisions\/24933"}],"wp:attachment":[{"href":"https:\/\/blackrockneurotech.com\/wp-json\/wp\/v2\/media?parent=24932"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blackrockneurotech.com\/wp-json\/wp\/v2\/categories?post=24932"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blackrockneurotech.com\/wp-json\/wp\/v2\/tags?post=24932"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}