| This Paper:Browse 1052 Download 604 |
码上扫一扫！ |
| Modeling robotic operations controlled by natural language |
| Y.Cheng,J.Bao,Y.Jia,Z.Deng,Z.Sun,S.Bi,C.Li,N.Xi |
|
|
| (Department of Electrical and Computer Engineering, Michigan State University, East Lansing, MI, 48823, U.S.A.) |
|
| 摘要: |
| There are multiple ways to control a robotic system. Most of them require the users to have prior knowledge about robots or get trained before using them. Natural language based control attracts increasing attention due to its versatility and less requirements for users. Since natural language instructions from users cannot be understood by the robots directly, the linguistic input has to be processed into a formal representation which captures the task specification and removes the ambiguity inherent in natural language. For most of existing natural language controlled robotic system, they assume the given language instructions are already in correct orders. However, it is very likely for untrained users to give commands in a mixed order based on their direct observation and intuitive thinking. Simply following the order of the commands can lead to failures of tasks. To provide a remedy for the problem, we propose a novel framework named dependency relation matrix (DRM) to model and organize the semantic information extracted from language input, in order to figure out an executable sequence of subtasks for later execution. In addition, the proposed approach projects abstract language input and detailed sensory information into the same space, and uses the difference between the goal specification and temporal status of the task under implementation to monitor the progress of task execution. In this paper, we describe the DRM framework in detail, and illustrate the utility of this approach with experiment results. |
| 关键词: Natural language control, task planning, human-machine interaction |
| DOI： |
|
| 基金项目:This work was partially supported by the National Science Foundation (Nos. CNS-1320561, IIS-1208390), the U.S. Army Research Laboratory, and the U. S. Army Research Office (Nos. W911NF-11-D-0001, W911NF-09-1-0321, W911NF-10-1-0358, W911NF-14-1-0327, W911NF-16-1-0572). |
|
| Modeling robotic operations controlled by natural language |
| Y. Cheng,J. Bao,Y. Jia,Z. Deng,Z. Sun,S. Bi,C. Li,N. Xi |
| (Department of Electrical and Computer Engineering, Michigan State University, East Lansing, MI, 48823, U.S.A.; Department of Industrial and Manufacturing Systems Engineering, The University of Hong Kong, Hong Kong;Department of Automotive Engineering, Clemson University, Greenville, South Carolina, 29607, U.S.A.) |
| Abstract: |
| There are multiple ways to control a robotic system. Most of them require the users to have prior knowledge about robots or get trained before using them. Natural language based control attracts increasing attention due to its versatility and less requirements for users. Since natural language instructions from users cannot be understood by the robots directly, the linguistic input has to be processed into a formal representation which captures the task specification and removes the ambiguity inherent in natural language. For most of existing natural language controlled robotic system, they assume the given language instructions are already in correct orders. However, it is very likely for untrained users to give commands in a mixed order based on their direct observation and intuitive thinking. Simply following the order of the commands can lead to failures of tasks. To provide a remedy for the problem, we propose a novel framework named dependency relation matrix (DRM) to model and organize the semantic information extracted from language input, in order to figure out an executable sequence of subtasks for later execution. In addition, the proposed approach projects abstract language input and detailed sensory information into the same space, and uses the difference between the goal specification and temporal status of the task under implementation to monitor the progress of task execution. In this paper, we describe the DRM framework in detail, and illustrate the utility of this approach with experiment results. |
| Key words: Natural language control, task planning, human-machine interaction |
