Announcing the MCAP Storage Plugin for ROS 2

Record and visualize your MCAP data with ROS 2 tooling

Esther Weon ·

James Smith ·

4 min read

Published October 12, 2022

Announcing the MCAP Storage Plugin for ROS 2

We announced the MCAP container file format earlier this year to help robotics teams record their pub/sub data using a serialization-agnostic file format. No matter your data format – ROS 1, ROS 2, Protobuf, JSON, or more – MCAP helps you record your robots doing what they do.

But to make MCAP truly powerful as a general-purpose data format, we needed support from across the industry. PlotJuggler adding support for MCAP was an exciting step in this direction.

Today, we’re thrilled to announce that we are yet another step closer to that goal with rosbag2_storage_mcap, a ROS 2 storage plugin for recording robotics data to MCAP files.

MCAP support in ROS 2

We designed MCAP to address a number of shortcomings we saw in existing recording formats. It is write-optimized, supports multiple message serialization formats, easy to recover, and fully self-contained. MCAP files are also efficient to access over a network, allowing you to request the data you need without downloading the entire file.

To make it easier for ROS 2 users to incorporate MCAP into their existing data workflows, we decided to create a ROS 2 plugin to support recording, reading, converting, and playing back data using MCAP.

Recording to MCAP

We leveraged rosbag2’s pluggable storage architecture to create the ROS 2 MCAP plugin. rosbag2_storage_mcap behaves exactly like the existing sqlite plugin does, but produces self-contained MCAP files that include all the definitions needed to decode custom messages.

To install the plugin:

$ sudo apt install ros-$ROS_DISTRO-rosbag2-storage-mcap

To start recording data, run your ROS nodes in one Terminal window, and then run your MCAP recorder in another:

$ ros2 bag record -s mcap –all

If you run a recording process on your robot, you can easily update your launch command to record with MCAP:

<launch>
  <executable cmd="ros2 bag record -a -s mcap" output="screen" />
</launch>

Reading from MCAP

The rosbag2_storage_mcap plugin works seamlessly with the existing rosbag2_py API for reading messages out of bags, so your rosbag2 scripts will continue to work with minimal changes. Just specify “mcap” as your storage ID when opening a bag file:

reader.open(
  rosbag2_py.StorageOptions(uri=input_bag, storage_id="mcap"),
  rosbag2_py.ConverterOptions(
    input_serialization_format="cdr", output_serialization_format="cdr"
  ),
)

Alternatively, you can use the mcap-ros2-support python library to programmatically read ROS 2 messages from recorded MCAP files. This library doesn’t require setting up a ROS 2 environment or any other ROS dependencies.

Converting SQLite files

We’ve added support for converting existing .db3 files to .mcap in the rosbag2 cli:

$ echo “compression: Zstd” > mcap_storage_options.yaml
$ cat << EOF > to_mcap.yaml
  output_bags:
  - uri: output_bag
    storage_id: mcap
    storage_config_uri: mcap_storage_options.yaml
    all: true
  EOF
$ ros2 bag convert  -i existing_data.db3 -o to_mcap.yaml

Playback

With the mcap storage plugin, you can play back the ROS 2 messages easily. Use the --regex argument to filter topics for playback:

$ ros2 bag play my.mcap –regex “.*CAMERA.*”

When recording, you can customize the behavior of the plugin to suit your robot’s resource constraints:

$ echo “{‘noCRC’: true, ‘noChunking’: true}” > storage_config.yaml
$ ros2 bag record -s mcap –all –storage-config-file storage_config.yaml

MCAP vs. SQLite

So how exactly does MCAP stack up against SQLite? We compared the two formats on performance and general usability, and discovered some interesting findings.

Throughput and memory usage

We used an NVIDIA Jetson Nano to test differences in throughput and memory usage when recording data with the MCAP and SQLite plugins.

We found that in all configurations and for all message sizes, the MCAP plugin consistently yielded a statistically significant advantage in throughput:

Throughput comparison graph Regardless of message size, the MCAP plugin exhibited much higher throughput than the SQLite plugin when recording data.

We also found that MCAP was better equipped to adapt to recording data in different scenarios. The plugin is designed so that you can optionally disable many convenient, but non-essential, features when recording on resource-constrained robots – like message indexing, CRCs for data integrity, etc.

Data recovery

MCAP’s design gives better guarantees than the default SQLite storage plugin around data recoverability. Since the ROS 2 storage plugin records SQLite files in an unsafe writing mode, it can’t make any guarantees about the integrity of its data when recording is interrupted.

MCAP, on the other hand, has an append-only design. This means that even if recording is interrupted, MCAP can guarantee that only the last recorded message has any danger of being corrupted.

Third-party integrations

Because MCAP files are completely self-contained, data visualization tools like Foxglove Studio and PlotJuggler are better equipped to visualize them. For one, these third-party apps can decode your ROS 2 data’s custom messages – even if they don’t have access to your ROS workspace, or even if those custom message definitions have changed since recording.

Learning more about MCAP

When we created MCAP, we wanted to support teams in building all kinds of robots – regardless of language, framework, or resource constraints. With MCAP, you can use append-only writing to safeguard data, read data at lightning-fast speeds, and store disparate data more efficiently.

To learn more about the history and benefits of MCAP, Foxglove engineer John Hurliman will be giving a talk on “MCAP: A Next-Generation File Format for ROS Recording” at this year’s ROSCon! Be sure to check his presentation out on October 21 – and come by our booth to say hello!

To ask questions or share feedback on our MCAP roadmap, you can join our Slack community or contact us directly. You can also check out the MCAP GitHub repo to contribute to our open source project or read our docs.