neural networks research group
areas
people
projects
demos
publications
software/data
Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog (2018)
Jesse Thomason
,
Aishwarya Padmakumar
,
Jivko Sinapov
,
Nick Walker
,
Yuqian Jiang
,
Harel Yedidsion
,
Justin Hart
,
Peter Stone
, and
Raymond J. Mooney
In this work, we present methods for parsing natural language to underlying meanings, and using robotic sensors to create multi-modal models of perceptual concepts. We combine these steps towards language understanding into a holistic agent for jointly improving parsing and perception on a robotic platform through human-robot dialog. We train and evaluate this agent on Amazon Mechanical Turk, then demonstrate it on a robotic platform initialized from that conversational data. Our experiments show that improving both parsing and perception components from conversations improves communication quality and human ratings of the agent.
View:
PDF
Citation:
In
Late-breaking Track at the SIGDIAL Special Session on Physically Situated Dialogue (RoboDIAL-18)
, Melbourne, Australia, July 2018.
Bibtex:
@inproceedings{thomason:robodial18, title={Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog}, author={Jesse Thomason and Aishwarya Padmakumar and Jivko Sinapov and Nick Walker and Yuqian Jiang and Harel Yedidsion and Justin Hart and Peter Stone and Raymond J. Mooney}, booktitle={Late-breaking Track at the SIGDIAL Special Session on Physically Situated Dialogue (RoboDIAL-18)}, month={July}, address={Melbourne, Australia}, url="http://nn.cs.utexas.edu/pub-view.php?PubID=127720", year={2018} }
People
Justin Hart
hart [at] cs utexas edu
Yuqian Jiang
Raymond J. Mooney
mooney [at] cs utexas edu
Aishwarya Padmakumar
aish [at] cs utexas edu
Jivko Sinapov
jsinapov [at] cs utexas edu
Peter Stone
pstone [at] cs utexas edu
Jesse Thomason
thomason DOT jesse AT gmail
Nick Walker
nswalker [at] cs uw edu
Harel Yedidsion
harel [at] cs utexas edu
Areas of Interest
Active Learning
Language and Robotics
Learning for Semantic Parsing
Semi-Supervised Learning