大工至善|大学至真分享 http://blog.sciencenet.cn/u/lcj2212916

博文

[转载]【计算机科学】【1996】从已训练神经网络中提取可理解模型

已有 1403 次阅读 2020-10-2 22:08 |系统分类:科研笔记|文章来源:转载

本文为美国威斯康星大学麦迪逊分校(作者:Mark W. Craven)的博士论文,共211页。

 

尽管神经网络已经被用于在许多实际问题领域中开发高精度的分类器,但众所周知,它们所学习的模型很难理解。本文研究从已训练的神经网络中提取可理解模型的任务,从而减轻这一局限性。本文的主要贡献是一种新的算法,克服了以往方法的局限性,提出了一种从训练网络中提取可理解模型的新方法。该算法称为Trepan将任务视为归纳学习问题。给定一个经过训练的网络或任何其他学习模型,Trepan使用查询来归纳一个决策树,该树近似于模型所表示的函数。与以前在这方面的工作不同,Trepan广泛适用于大型网络和具有高维输入空间的问题。

 

本论文介绍了一个实验,通过将其应用于单个网络和在分类、回归和强化学习领域训练的神经网络的集合来评估Trepan。这些实验表明,Trepan能够提取出可理解的决策树,同时保持对各自网络的高保真度。在神经网络比传统决策树算法具有更高预测精度的问题域中,用Trepan方法提取的树也显示出更高的精度,但在复杂度方面与直接从训练数据中学习的树相比是相当的。

 

本文的第二个贡献是一种叫做BBP的算法,它构造性地诱导简单的神经网络。该算法的动机与Trepan相似:在神经网络具有特别适当的归纳偏差的问题域中学习可理解的模型。BBP算法是基于一种假设提升方法,它可以学习连接相对较少的感知器。该算法提供了一个吸引人的组合优点:为相当自然的目标函数类提供了可学习性保证;在各种问题域中提供了良好的预测精度;构造了语法简单的模型,从而促进了人类对所学知识的理解。这些算法提供了一种机制来提高对训练过的神经网络所学知识的理解。

 

Although neural networks have been used todevelop highly accurate classifiers in numerous real-world problem domains, themodels they learn are notoriously difficult to understand. This thesisinvestigates the task of extracting comprehensible models from trained neuralnetworks, thereby alleviating this limitation. The primary contribution of thethesis is an algorithm that overcomes the significant limitations of previousmethods by taking a novel approach to the task of extracting comprehensiblemodels from trained networks. This algorithm, called Trepan, views the task asan inductive learning problem. Given a trained network, or any other learnedmodel, Trepan uses queries to induce a decision tree that approximates thefunction represented by the model. Unlike previous work in this area, Trepan isbroadly applicable as well as scalable to large networks and problems withhigh-dimensional input spaces. The thesis presents experiments that evaluateTrepan by applying it to individual networks and to ensembles of neuralnetworks trained in classification, regression, and reinforcement-learningdomains. These experiments demonstrate that Trepan is able to extract decisiontrees that are comprehensible, yet maintain high levels of fidelity to theirrespective networks. In problem domains in which neural networks providesuperior predictive accuracy to conventional decision tree algorithms, thetrees extracted by Trepan also exhibit superior accuracy, but are comparable interms of complexity, to the trees learned directly from the training data. Asecondary contribution of this thesis is an algorithm, called BBP, thatconstructively induces simple neural networks. The motivation underlying thisalgorithm is similar to that for Trepan: to learn comprehensible models inproblem domains in which neural networks have an especially appropriateinductive bias. The BBP algorithm, which is based on a hypothesis-boostingmethod, learns perceptrons that have relatively few connections. This algorithmprovides an appealing combination of strengths: it provides learnabilityguarantees for a fairly natural class of target functions; it provides goodpredictive accuracy in a variety of problem domains; and it constructssyntactically simple models, thereby facilitating human comprehension of whatit has learned. These algorithms provide mechanisms for improving theunderstanding of what a trained neural network has learned.

 

1. 引言

2. 项目背景

3. TREPAN算法

4. TREPAN经验评估

5. TREPAN解析评估

6. MOFN-SWS算法:用于提取M-of-N准则的本地算法

7. 基于提升的感知学习算法

8. 其他相关工作

9. 结论

附录A TREPAN提取的表征树


更多精彩文章请关注公众号:205328s611i1aqxbbgxv19.jpg




https://wap.sciencenet.cn/blog-69686-1252991.html

上一篇:[转载]【电子技术】【2014.11】基于STM32F429的CMOS教学摄像机
下一篇:[转载]【统计学】【2015.09】基于状态空间模型的时间序列预测与插值
收藏 IP: 112.31.16.*| 热度|

0

该博文允许注册用户评论 请点击登录 评论 (0 个评论)

数据加载中...

Archiver|手机版|科学网 ( 京ICP备07017567号-12 )

GMT+8, 2024-3-29 02:05

Powered by ScienceNet.cn

Copyright © 2007- 中国科学报社

返回顶部