ChatGPT for Robotics: Design Principles and Model Abilities

Sai Vemprala; Rogerio Bonatti; Arthur Bucker; Ashish Kapoor

← Том 1, № 3 (2026): Восприятие, робототехника и вычислительные системы

Ғылыми мақала

ChatGPT for Robotics: Design Principles and Model Abilities

Sai VempralaORCID

Scaled Foundations, Kirkland, WA, USA

Rogerio BonattiORCID

Microsoft (United States), United States

Arthur Bucker

Carnegie Mellon University, United States

Ashish KapoorORCID

Scaled Foundations, Kirkland, WA, USA

Жарияланған күні: 28 тамыз 2026 ж.Түскен күні: 1 қаңтар 2024 ж.

Аңдатпа

This paper presents an experimental study regarding the use of OpenAI’s ChatGPT [1] for robotics applications. We outline a strategy that combines design principles for prompt engineering and the creation of a high-level function library which allows ChatGPT to adapt to different robotics tasks, simulators, and form factors. We focus our evaluations on the effectiveness of different prompt engineering techniques and dialog strategies towards the execution of various types of robotics tasks. We explore ChatGPT’s ability to use free-form dialog, parse XML tags, and to synthesize code, in addition to the use of task-specific prompting functions and closed-loop reasoning through dialogues. Our study encompasses a range of tasks within the robotics domain, from basic logical, geometrical, and mathematical reasoning all the way to complex domains such as aerial navigation, manipulation, and embodied agents. We show that ChatGPT can be effective at solving several of such tasks, while allowing users to interact with it primarily via natural language instructions. In addition to these studies, we introduce an open-sourced research tool calledPromptCraft, which contains a platform where researchers can collaboratively upload and vote on examples of good prompting schemes for robotics applications, as well as a sample robotics simulator with ChatGPT integration, making it easier for users to get started with using ChatGPT for robotics. Videos and blog: aka.ms/ChatGPT-Robotics PromptCraft, AirSim-ChatGPT code: https://github.com/microsoft/PromptCraft-Robotics.

Источник и лицензия. Статья открытого доступа, впервые опубликована как «ChatGPT for Robotics: Design Principles and Model Abilities» в журнале IEEE Access, 2024. DOI: 10.1109/access.2024.3387941. © Sai Vemprala, Rogerio Bonatti, Arthur Bucker, Ashish Kapoor. Воспроизведена без изменений по лицензии Creative Commons Attribution 4.0.

Кілт сөздер

Robotics Artificial intelligence Computer science Dialog box Human–computer interaction Robot Robotic paradigms Upload

Әдебиеттер тізімі

Nilsson, N. (1996). Artificial intelligence: A modern approach. Artificial Intelligence. https://doi.org/10.1016/0004-3702(96)00007-0
Shah, S., Dey, D., Lovett, C. et al. (2017). AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles. Springer proceedings in advanced robotics. https://doi.org/10.1007/978-3-319-67361-5_40
Tellex, S., Gopalan, N., Kress‐Gazit, H. et al. (2020). Robots That Use Language. Annual Review of Control Robotics and Autonomous Systems. https://doi.org/10.1146/annurev-control-101119-071628
Arkin, J., Park, D., Roy, S. et al. (2020). Multimodal estimation and communication of latent semantic knowledge for robust execution of robot instructions. The International Journal of Robotics Research. https://doi.org/10.1177/0278364920917755
Walter, M., Patki, S., Daniele, A. et al. (2022). Language Understanding for Field and Service Robots in a Priori Unknown Environments. Field Robotics. https://doi.org/10.55417/fr.2022040
Chen, M., Tworek, J., Jun, H. et al. (2021). Evaluating Large Language Models Trained on Code. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2107.03374
Brohan, A., Brown, N. et al. (2022). Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2204.01691

ChatGPT for Robotics: Design Principles and Model Abilities

Аңдатпа

Кілт сөздер

Дәйексөз келтіру

Әдебиеттер тізімі