Robot control policies learned in simulation do not often transfer well to the real world. Many existing solutions to this sim-to-real problem, such as the Grounded Action Transformation (GAT) algorithm, seek to correct for--or ground--these differences by matching the simulator tothe real world. However, the efficacy of these approaches is limited if they do not explicitly account for stochasticity inthe target environment. In this work, we analyze the prob-lems associated with grounding a deterministic simulator in astochastic real world environment, and we present examples where GAT fails to transfer a good policy due to stochastic transitions in the target domain. In response, we introduce the Stochastic Grounded Action Transformation (SGAT) algorithm, which models this stochasticity when grounding the simulator. We find experimentally--for both simulated and physical target domains--that SGAT can find policies that are robust to stochasticity in the target domain

PDF

In IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS 2020), Las Vegas, NV, USA, October 2020.

@inproceedings{IROS20-Desai, title={Stochastic Grounded Action Transformation for Robot Learning in Simulation}, author={Siddharth Desai and Haresh Karnan and Josiah P. Hanna and Garrett Warnell and Peter Stone}, booktitle={IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS 2020)}, month={October}, address={Las Vegas, NV, USA}, url="http://nn.cs.utexas.edu/?IROS20-Desai", year={2020} }