CS 6751 / MAE 6730

Introduction to Robotic Mobile Manipulation

Course Staff

  • Instructor: Prof. Ross Knepper (rak@cs.cornell.edu)
    • Office hours: Upson 571 on Mondays 1-5pm; Wednesday 3-6pm; Friday 1:30-4pm


Mobile manipulation is the ability for a robot to interact physically and with versatility in the world. It is one of the greatest technical challenges in robotics, due primarily to the interplay of uncertainty about the world and clutter within it. As robots become integrated into complex human environments, mobile manipulation is increasingly necessary. Robotic mobile manipulation will enable applications like personal assistant robots in the home and factory worker robots in advanced manufacturing. This course covers the fundamental theory, concepts, and systems of mobile manipulation, including both software and hardware. Topics we will cover this semester include statistical techniques like machine learning and particle filters, combined task and motion planning, human-robot interaction, and introspection. The course features a semester-long project in which each student must deliver a component that functions with several other students’ components to form a working robotic system. The scope of possible components is quite broad and extends beyond traditional robotics issues into other aspects of CS. This course is offered to prepare a student for Ph.D. research in robotics.

Note that for the CS breadth requirement, this course counts as the AI area with the systems research style.

Course Aims and Outcomes

Aims. By the end of this course, I will be prepared to understand and contribute to the robotics
research literature, especially as it pertains to mobile manipulation.

Learning Outcomes.  After this course, I will:

  • be able to read and apply research papers on mobile manipulation topics.
  • understand fundamental theory forming the basis of six robotics disciplines: kinematics,
    dynamics, controls, grasping, planning, and human-robot interaction.
  • know principles of robotic systems design, and be able to analyze trade-offs in such designs.
  • be able to integrate a system of several components.
  • know how to approach problems in mobile manipulation.


Graduate standing or permission of the instructor. Undergraduates should have taken a previous robotics course such as MAE 4180 or CS 4752. A background in mathematics is required, especially linear algebra (e.g. MATH 4310) and probability (e.g. MATH 4720). Proficiency in C++ or Python is required.


  • Class participation: 10%
    • Class discussion: 8%
    • Filling out course evaluation survey: 2%
  • Project: 60%
    • Research content: 40%
    • Written report: 10%
    • Oral report: 10%
  • Paper presentation: 30%
    • Students will present an important robotics research paper to the class.

Optional Textbooks (none required)

  • Alonzo Kelly, Mobile Robotics: Mathematics, Models and Methods, Cambridge University Press, 2013.
  • Steven M. Lavalle, Planning Algorithms, Cambridge University Press, 2006.
    • Free online: http://planning.cs.uiuc.edu/book.html
  • Matthew T. Mason, Mechanics of Manipulation, MIT Press 2001.
  • Sciavicco and Siciliano, Modelling and Control of Robot Manipulators, Springer 2000.
  • Sebastian Thrun, Wolfram Burgard, and Dieter Fox, Probabilistic Robotics, MIT Press, 2005.


Day Date In class Due in class/Event
1. Thu 1/26 Orientation First day of class
2. Tue 1/31
3. Thu 2/2  Deadline to form groups (end of class)
4.  Tue 2/7
5. Thu 2/9 Present project proposals  Project proposals due
6. Tue 2/14
7. Thu 2/16
Tue 2/21 No class (February break)
8. Thu 2/23  Present project status Implementation milestone 1 due
9. Tue 2/28
10. Thu 3/2
11. Tue 3/7 No presentation; Work on projects Prof. Knepper out of town
12. Thu 3/9 No presentation; Work on projects Prof. Knepper out of town
13. Tue 3/14 Project paper outline due
14. Thu 3/16  Present project status Implementation milestone 2 due
15. Tue 3/21
16. Thu 3/23
17. Tue 3/28 Present related work Project paper related work section draft due
18. Thu 3/30 No presentation; Work on projects Prof. Knepper out of town
 Tue 4/4 No class (spring break)
Thu 4/6 No class (spring break)
19. Tue 4/11
20. Thu 4/13  Present project status Implementation milestone 3 due
21. Tue 4/18
22. Thu 4/20 Project paper core technical sections draft due
23. Tue 4/25
24. Thu 4/27
25. Tue 5/2  Rough draft of completed project paper due (can exclude experimental results)
26. Thu 5/4 Project demos
27. Tue 5/9 Project demos  Last day of class
 Mon 5/15  Final paper due

List of recommended papers

These papers are suitable for presentation in class.  The list is not exclusive.

Motion Planning

  • Berenson, Srinivasa, and Kuffner. “Task space regions: A framework for pose-constrained manipulation planning.” International Journal of Robotics Research, 2011. [PDF]
  • Dogar and Srinivasa, “A framework for push-grasping in clutter.” Robotics: Science and Systems, 2011. [PDF]
  • LaValle and Kuffner. “Randomized kinodynamic planning.” International Journal of Robotics Research, 2001. [PDF]
  • Karaman, and Frazzoli, “Sampling-based algorithms for optimal motion planning.” International Journal of Robotics Research, 2011. [PDF]
  • Stilman and Kuffner, “Navigation among movable obstacles: Real-time reasoning in complex environments”, International Journal of Humanoid Robotics, 2005. [PDF]
  • Hauser, Bretl, Harada, and Jean-Claude Latombe, “Using motion primitives in probabilistic sample-based planning for humanoid robots”, Workshop on the Algorithmic foundation of robotics VII, 2008. [PDF]


  • Tedrake, Manchester, Tobenkin, and Roberts. “LQR-trees: Feedback motion planning via sums-of-squares verification.” International Journal of Robotics Research, 2010. [PDF]
  • Brock and Khatib, “Elastic strips: A framework for motion generation in human environments”, International Journal of Robotics Research, 2002. [PDF]
  • Ayanian and Kumar, “Decentralized feedback controllers for multiagent teams in environments with obstacles”, IEEE Transactions on Robotics, 2010. [PDF]

Computer Vision

  • Collet, Martinez, and Srinivasa, “The MOPED framework: Object recognition and pose estimation for manipulation”, International Journal of Robotics Research, 2011. [PDF]
  • Saxena, Chung, and Ng, “Learning depth from single monocular images”, Neural Information Processing Systems (NIPS), 2005. [PDF]

Human-Robot Interaction

  • Tellex, Kollar, Dickerson, Walter, Banerjee, Teller, and Nicholas Roy, “Understanding natural language commands for robotic navigation and mobile manipulation”, AAAI, 2011. [PDF]
  • Kemp, Edsinger, and Torres-Jara, “Challenges for robot manipulation in human environments”, IEEE Robotics and Automation Magazine, 2007. [PDF]
  • Cakmak, Chao, and Thomaz, “Designing interactions for robot active learners”, IEEE Transactions on Autonomous Mental Development, 2010. [PDF]

Grasping and Manipulation

  • Rodriguez, Mason, and Steve Ferry, “From Caging to Grasping”, International Journal of Robotics Research, 2012. [PDF]
  • Gabiccini, Bicchi, Prattichizzo, and Malvezzi. “On the role of hand synergies in the optimal choice of grasping forces.” Autonomous Robots, 2011. [PDF]
  • Dov Katz, Yuri Pyuro, and Oliver Brock, “Learning to manipulate articulated objects in unstructured environments using a grounded relational representation,” Robotics: Science and Systems, 2009. [PDF]


Comments are closed