GSoC 2023: Dynamic wrench application through the GUI

General
,
1

GSoC 2023: Dynamic wrench application through the GUI

Organization: Open Robotics
Mentors: Addisu Taddese (@azeey), Dharini Dutia (@quarkytale)
Student: Henrique Barros Oliveira (GitHub, LinkedIn)
Link to GSoC project: https://summerofcode.withgoogle.com/programs/2023/projects/pGR7C6Xq

Hello everyone,

This summer, I was selected to work on Gazebo as part of the Google Summer of Code 2023 program. My project focused on developing tools to enable applying forces and torques to objects within Gazebo’s simulation environment through the GUI.

Under the guidance of my wonderful mentors, Addisu Taddese and Dharini Dutia, I wrote C++ code for the gz-sim library, as well as documentation in the form of tutorials. Over the summer, I opened 11 Pull Requests, out of which 9 have already been merged at the time of writing, and my contributions will be available in the next Garden and Harmonic releases.

About me

I’m a fifth-year Mechatronics Engineering student from the University of São Paulo, Brazil, currently enrolled in a double degree exchange program at CentraleSupélec, in France. I’m passionate about robotics, computer science and control systems. Previously, I spent three years working with autonomous drones at the student team Skyrats, where I was introduced to ROS and Gazebo and developed a desire to contribute to open source. GSoC ‘23 was my first time seriously contributing to open source, and it was an amazing experience.

About the project

Previously in Gazebo, it was complicated for users to exert external forces and torques to objects during the simulation. Moving objects was already possible through the Transform Control plugin, which enables dragging models to a new pose through the GUI. This is useful for repositioning models during the simulation, but has the drawback of not interacting with the dynamics of the simulation, such as gravity and collision with other models. Also, introducing dynamic perturbations, such as pushing a pendulum or exerting an external effort to a robot, was previously impossible without the user having to write a plugin for that purpose.

As a result, this project had the goal of developing tools that would allow users to easily apply wrenches to models through the GUI. This was done through the implementation of two new GUI plugins that can be loaded into Gazebo: Mouse Drag and Apply Force Torque. More detailed tutorials for both can be found in this PR.

Mouse Drag plugin

The Mouse Drag plugin allows the user to drag objects around in the scene with the mouse cursor by applying forces and/or torques to links during the simulation. This will enable use cases such as easily moving objects in the scene, emulating human-robot interaction, and perturbing control systems. This plugin provides two dragging modes: translation and rotation.

The translation mode, activated by Ctrl+Right-clicking and dragging a link in the scene, allows the user to pull the object towards the mouse position. Through an option in the interface, it can be specified if it should be pulled from the link’s center of mass or from the point of the initial click on the link.

panda

The rotation mode, activated by Ctrl+Left-clicking and dragging a link in the scene, allows the user to pull the object towards a desired orientation, only applying a torque to it.

Rotation

Apply Force and Torque plugin

The Apply Force Torque plugin is an implementation of the “Apply Force and Torque” interface previously present in Gazebo Classic. This plugin allows users to apply an instantaneous force and/or torque to any link in the simulation, with precise control of its amplitude and direction.

Force

Torque

Beyond specifying the XYZ components of the force and torque, the user may also scale the corresponding vector by modifying its magnitude field, which keeps the direction unchanged. Additionally, clicking on the visualization on the scene toggles a rotation tool (demonstrated below), which can be used to change the force or torque’s direction without changing its magnitude.

RotationTool

Pull Requests

The following Pull Requests correspond to the work developed during this project, including the plugins described above as well as other smaller features and fixes:

Challenges and implementation

Here I’ve listed some of the parts of the implementation I found interesting or challenging. If anyone would like to discuss any of them, feel free to reach out.

  • Sending force and torque commands from the GUI to the simulation running on the server.
  • Handling the two threads running on the GUI, with some functions needing to be called exclusively from the rendering thread.
  • Interpreting mouse motion into a target pose for the link in the Mouse Drag plugin.
  • Calculating the forces and torques using a spring-damper system with constant stiffness and critical damping in Mouse Drag.

Conclusion and future work

Considering the initial goal of providing tools for dynamic wrench application through the GUI, this project was successful in developing plugins for various use cases.

In the future, it would be interesting to see more automated tests for such GUI plugins. I started some in gz-sim#2085, but didn’t get to finish them.

The team was very receptive and I received a lot of great feedback during the project. I loved the open source workflow and the sense of community that comes from it. I want to give my special thanks to my mentors, Addisu Taddese and Dharini Dutia, for their guidance and for making this experience so enjoyable.

Thanks,
Henrique Barros Oliveira

9 Likes
2

Congratulations on your great work and thank you for this contribution!

Well, a general question:

You probably had to go through a lot of code digging in the beginning, to learn the concepts and the structure of the code? E.g. you mention the GUI threads, and that you can only call some functions exclusively from the rendering thread. Such knowledge must be acquired by every aspiring Gazebo developer, yet unfortunately there currently is very little documentation on these topics.

So: would you maybe have any resources from during your onboarding period that you can share?
I am really thinking of anything, e.g.:

  • Notes you took during some expanatory meeting on the code (even if simply hand-written),
  • Or maybe a recording from a video call during which you were explained some part of the code,
  • Maybe a small document you wrote about some part of the code,
  • Discussions on Gazebo Answers or Robotic Stack Exchange,
  • Etc.

This would for sure be helpful for everyone trying to write their first plugins.

I think if everyone in the ROS/Gazebo community had more of a reflex to disseminate this kind of information, that this could thoroughly improve the availability of documentation, in a low-effort way…

3 Likes
3

This is great, I have been looking for something for ages :grin:

Is there a way to visualize such arrows when I am sending forces/torques not from the GUI, but from code / topics ?

4

@torydebra, there’s some ongoing at https://github.com/gazebosim/gz-sim/pull/1898 that you can take a look at, but it’s still under review.