| 引用本文: | 刘建伟,王新坦.神经网络结构搜索方法综述[J].控制理论与应用,2023,40(1):12~31.[点击复制] |
| Liu Jian-Wei,Wang Xin-Tan.A review of neural network structure search method[J].Control Theory and Technology,2023,40(1):12~31.[点击复制] |
|
| 神经网络结构搜索方法综述 |
| A review of neural network structure search method |
| 摘要点击 1774 全文点击 451 投稿时间：2021-06-30 修订日期：2022-12-12 |
| 查看全文 查看/发表评论 下载PDF阅读器 |
| DOI编号 10.7641/CTA.2022.10568 |
| 2023,40(1):12-31 |
| 中文关键词 神经网络结构搜索 搜索空间 搜索策略 性能评估策略 |
| 英文关键词 neural network structure search search space search strategy performance evaluation strategy |
| 基金项目 中国石油大学(北京)科研基金项目(2462020YXZZ023)资助. |
|
| 中文摘要 |
| 如今, 深度学习广泛地应用于生活、工作中的各个方面, 给我们带来了极大的便利. 在此背景下, 需要设计
针对不同任务的神经网络结构, 满足不同的需求. 但是, 人工设计神经网络结构需要专业的知识, 进行大量的实验.
因此, 神经网络结构搜索算法的研究显得极为重要. 神经网络结构搜索(NAS)是自动深度学习(AutoDL) 过程中的一
个基本步骤, 对深度学习的发展与应用有着重要的影响. 早期, 一些神经网络结构搜索算法虽然搜索到了性能优越
的神经网络结构, 但是需要大量的计算资源且搜索效率低下. 因此, 研究人员探索了多种设计神经网络结构的算法,
也提出了许多减少计算资源、提高搜索效率的方法. 本文首先简要介绍了神经网络结构的搜索空间, 其次对神经网
络结构搜索算法进行了全面的分类汇总、分析, 主要包括随机搜索算法、进化算法、强化学习、基于梯度下降的方
法、基于顺序模型的优化算法, 再其次探索并总结了提高神经网络结构搜索效率的方法, 最后探讨了目前神经网络
结构搜索工作中存在的问题以及未来的研究方向. |
| 英文摘要 |
| Nowadays, deep learning is widely used in all aspects of life and work, which brings us great convenience.
In this context, we need to design neural network structure for different tasks to meet different needs. However, manually
design of neural network structure needs professional knowledge and a lot of experiments. Therefore, the research of
neural network structure search algorithm is very important. Neural network structure search (NAS) is a basic step in the
process of automatic deep learning (AutoDL), which has an important impact on the development and application of deep
learning. In the early stage, although some neural network structure search algorithms have found excellent neural network
structure, they need a lot of computing resources and search efficiency is low. Therefore, researchers have explored a
variety of neural network structure design algorithms, and also proposed many methods to reduce computing resources and
improve search efficiency. Firstly, we briefly introduce the search space of neural network structure. Secondly, we make a
comprehensive classification, summary and analysis of neural network structure search algorithms, including random search
algorithm, evolutionary algorithm, reinforcement learning, gradient descent based method and optimization algorithm based
on sequential model. Thirdly, we explore and summarize methods to improve the efficiency of neural network structure
search. Finally, we discuss the problems existing in the current neural network structure search work and the future research
direction. |
|
|
|
|
|