In a previous post, Taking the AR4 Further: From Gazebo to Isaac Sim, we showed how we brought the AR4 robot from Gazebo into Isaac Sim, because every robot deserves a glow-up in RTX. Now, we’re going one step further: using our ROS 2 and MoveIt integrations to turn the AR4 into a chess-playing machine. Is it a grandmaster yet? Not quite. But it definitely knows how to move its pieces.
In this post, we’ll explain how we combined Isaac Sim, ROS 2, and the MoveIt API to design and orchestrate a fully simulated robot that can play chess like a pro.
What’s the Point?
The main goal was simple: show that the AR4 can manipulate multiple objects across different positions. At the same time, we wanted to highlight how straightforward it is to hook Isaac Sim with ROS 2. The combination gives you simulations that are not just realistic, but also extremely practical for testing real-world robotic behavior.
Previously, we set up the AR4 with ROS 2 and MoveIt inside Isaac Sim, but the robot was basically just hanging out, not doing much. Now, we’re putting it to work on a concrete (and fun) task: playing chess.
How we did it
To get our robot ready for chess, we started by building a dockerized environment based on Isaac Sim 4.2.0. This setup bundled everything we needed: Isaac Sim, ROS 2, MoveIt, and all the custom code. Having everything in a container made it easy to keep the environment clean, reproducible, and shareable.
Scene set up
We used the Isaac Sim Python API to spawn both the AR4 robot and the chessboard models in the same scene. This is handled by an entrypoint script that runs automatically with the ROS 2 launch file. This script also configures the simulation environment, enabling the ROS 2 extension and updating the simulation. This setup ensures that every time the simulation starts, the robot and chessboard are placed correctly, the ROS 2 bridge is up and the simulation ready to interact with the rest of the ROS 2 nodes.
ROS 2 and MoveIt integration
The integration with ROS 2 and MoveIt has already been set up in a previous post, where we connected the AR4 to MoveIt using the topic_based_ros2_control plugin. Since the MoveIt configuration was reused from our Gazebo setup, the AR4 was ready to move in Isaac Sim with minimal effort. For this project, we built on top of that by using the MoveIt C++ API to send planning and execution commands to the robot specifying target poses and letting MoveIt handle the motion planning behind the scenes.
Arm control and behavior trees
The logic to perform the required actions was built using a behavior tree, implemented with the BehaviorTree.CPP library. Behavior trees define a way to structure robot behavior, especially when tasks involve multiple steps, fallbacks, and the occasional need to recover from failure.
For this particular case, the tree handled the full execution pipeline:
- Open gripper
- Pick up a piece
- Move to target position
- Place piece
- Close gripper
- Move to Safe Positions in between different nodes
Each action lived in its own node, keeping the flow modular and easy to extend.
The tree followed a sequence-fallback pattern. The main sequence tried to execute the full pick-and-place pipeline. If any step failed the fallback node was started. This triggered a recovery sequence where the robot moved to a known safe home position and reset its internal state, ready to try again. The structure of the behavior tree is shown in the figure below.
Chess pieces interaction
Finally, to interact with the chess pieces, and to know their exact positions in the scene, we used the Omniverse and Isaac Sim Python API. This allowed us to query each piece transform directly from the simulation world and update it dynamically as moves were made.
The Grandmaster in Action
In the following video, you can see the AR4 in action, taking orders from a user via ROS 2 actions. Think of it as a very obedient chess assistant: it won’t question your strategy, but it will happily move your pieces exactly where you tell it to.
Watch the AR4 in Isaac Sim
Why Does It Matter?
This project isn’t about teaching robots to win tournaments (yet). It’s about showcasing what becomes possible when you combine:
- ROS 2: a powerful and flexible robotics middleware
- MoveIt: motion planning made simple
- Isaac Sim: hyper-realistic environments with seamless ROS 2 integration
Together, these tools create an environment where testing robotic behaviors feels less like debugging and more like experimenting. And yes, sometimes that experimenting even involves teaching a robot to play chess.
Let’s Keep in Touch
If you’re working on something similar or just want to connect, reach out to us at contact@ekumenlabs.com, and follow us on our social channels to stay up to date with future releases and projects.