瑞士日内瓦大学Reidar Riveland课题组发现，自然语言指令诱导神经元网络的组合泛化。2024年3月18日，《自然—神经科学》杂志在线发表了这项成果。
研究人员利用自然语言处理技术的进步，创建了一个基于语言指令的泛化神经模型。模型通过一组常见的心理物理任务进行训练，并接收由预训练语言模型嵌入的指令。这个最佳模型可以完全根据语言指令执行以前从未见过的任务，平均正确率达到 83%（即零射击学习）。研究人员发现，语言可以为感觉运动表征提供支架，使相互关联任务的活动与指令的语义表征具有共同的几何形状，从而使语言能够在未知环境中提示练习技能的正确构成。
研究人员展示了该模型如何仅利用运动反馈，就能生成对其所识别的新任务的语言描述，从而指导伙伴模型执行任务。该模型提供了几项可通过实验检验的预测，概述了语言信息必须如何表示才能促进人脑中灵活而普遍的认知。
据了解，人类的一项基本认知功能是解释语言指令，以便在没有明确任务经验的情况下完成新任务。然而，人们对用于完成这一任务的神经计算仍然知之甚少。
附：英文原文
Title: Natural language instructions induce compositional generalization in networks of neurons
Author: Riveland, Reidar, Pouget, Alexandre
Issue&Volume: 2024-03-18
Abstract: A fundamental human cognitive feat is to interpret linguistic instructions in order to perform novel tasks without explicit task experience. Yet, the neural computations that might be used to accomplish this remain poorly understood. We use advances in natural language processing to create a neural model of generalization based on linguistic instructions. Models are trained on a set of common psychophysical tasks, and receive instructions embedded by a pretrained language model. Our best models can perform a previously unseen task with an average performance of 83% correct based solely on linguistic instructions (that is, zero-shot learning). We found that language scaffolds sensorimotor representations such that activity for interrelated tasks shares a common geometry with the semantic representations of instructions, allowing language to cue the proper composition of practiced skills in unseen settings. We show how this model generates a linguistic description of a novel task it has identified using only motor feedback, which can subsequently guide a partner model to perform the task. Our models offer several experimentally testable predictions outlining how linguistic information must be represented to facilitate flexible and general cognition in the human brain.
DOI: 10.1038/s41593-024-01607-5
Source: https://www.nature.com/articles/s41593-024-01607-5
Nature Neuroscience：《自然—神经科学》，创刊于1998年。隶属于施普林格·自然出版集团，最新IF：28.771
官方网址：https://www.nature.com/neuro/
投稿链接：https://mts-nn.nature.com/cgi-bin/main.plex
本期文章：《自然—神经科学》：Online/在线发表
