Simplify robotics development by managing data from the command line
In robotics development, importing data from your robots and sharing it across your organization efficiently is critical. Foxglove offers a variety of tools to streamline this process – the web UI allows anyone with a browser to manage team data from a central repository; the Python library allows developers to write scripts that retrieve and manipulate data from this same repository. While these are popular options, the Foxglove CLI provides yet another lightweight, yet powerful, alternative.
In this tutorial, we will explore how to leverage the Foxglove CLI to import and export data, so that your team can spend less time wrangling data and more time building robots.
Unlike the web UI, the Foxglove CLI provides a reliably consistent interface that developers can rely on to interact with data. And unlike the Python library, the CLI provides developers with the direct feedback of terminal commands – no need to create a file, install the relevant packages, and run the script.
The CLI becomes particularly useful in scenarios where automation and scripting play a crucial role. For instance, if you are dealing with setting up large-scale data processing tasks or integrating Foxglove into a broader automation pipeline, the CLI can be a powerful option.
To get started, create a free Foxglove account – this will allow you to import data to the Foxglove cloud.
Next, install the latest release of the Foxglove CLI for your platform, and use it to sign in to your Foxglove account from the command line:
$ foxglove auth login
language-bash
Alternatively, you can configure the CLI to use an API key.
Once you’re logged in, you’re ready to start importing and exporting data!
Foxglove supports importing ROS 1 bag files and MCAP files.
If you’re already using ROS 2 .db3 files, you can convert them to the MCAP format with the MCAP CLI. Starting from the ROS 2 Iron Irwini release, rosbag2 now records data directly to MCAP by default, for easy analysis with third-party tools like Foxglove.
Once you have data files to import, use the CLI to create the devices (i.e. robots) on which they were recorded:
$ foxglove devices add --name RobotA
Device created: dev_flm75pLkfzUBX2DH
$ foxglove devices add --name RobotB
Device created: dev_Wm1gvryKJmREqnVT
language-bash
If you create more than one device, you can retrieve a list of all their names and IDs for easy reference:
$ foxglove devices list
| ID | NAME | CREATED AT | UPDATED AT |
| --------------------- | —------ | --------------------- | --------------------- |
| dev_flm75pLkfzUBX2DH | RobotA | 2023-12-10T17:20:55Z | 2023-12-10T17:20:55Z |
| dev_Wm1gvryKJmREqnVT | RobotB | 2023-12-10T17:22:55Z | 2023-12-10T17:22:55Z |
language-bash
Next, it's time to import a local file associated with one of these newly created devices! If you don't have a file to import, download this example MCAP file to continue with the tutorial:
$ curl -X GET https://assets.foxglove.dev/NuScenes-v1.0-mini-scene-0061-df24c12.mcap -o recording.mcap
language-bash
Next, associate your local file to the device that recorded it:
# For MCAP file recorded by RobotA
$ foxglove data import /path/to/recording.mcap --device-name RobotA
# For ROS 1 file recorded by RobotB
$ foxglove data import /path/to/recording.bag --device-name RobotB
language-bash
To verify that you’ve successfully imported these data files to the Foxglove cloud, retrieve a list of all imported data:
$ foxglove recordings list
-[ RECORD 1 ]--+----------------------------
Recording ID | 5m88YzSVGPVBDax2
Path | recording.bag
Size | 67.1 MiB
Message Count | 1606
Created At | 2023-12-10T19:06:55.425Z
Imported At | 2023-12-10T19:06:55.425Z
Start | 2018-07-25T03:28:47.604844Z
End | 2018-07-25T03:29:06.797517Z
Import Status | complete
Site ID | aCQa2Ks3AyNvQo4H
Site Name | Foxglove Cloud
Edge Site ID |
Edge Site Name |
Device ID | dev_Wm1gvryKJmREqnVT
Device Name | RobotB
Metadata |
-[ RECORD 2 ]--+----------------------------
Recording ID | 5m88YzSVGPVBDax2
Path | recording.mcap
Size | 488.5 MiB
Message Count | 33886
Created At | 2023-12-10T18:06:55.425Z
Imported At | 2023-12-10T18:06:55.425Z
Start | 2018-06-24T03:28:47.604844Z
End | 2018-06-24T03:29:06.797517Z
Import Status | complete
Site ID | aCQa2Ks3AyNvQo4H
Site Name | Foxglove Cloud
Edge Site ID |
Edge Site Name |
Device ID | dev_flm75pLkfzUBX2DH
Device Name | RobotA
Metadata |
language-bash
If you don’t see your recordings, they may still be listed as pending imports:
$ foxglove pending-imports list
-[ RECORD 1 ]-+-------------------------------------
Created At | 2023-12-10T18:06:55.425Z
Imported At | 2023-12-10T18:06:55.425Z
Org ID | 1329pe9f-cf63-92ec-7e19-93db76a7103b
Filename | recording.mcap
Pipeline stage| ingestion
Request ID | 9831ab13-31e9-450f-bfb0-2250d7c6d2f6
Device ID | dev_flm75pLkfzUBX2DH
Device name |
Import ID |
Site ID |
Status | ready for indexing
Error |
...
language-bash
Now that you have imported data, you can export meaningful segments of it for downstream sharing and analysis.
Query your imported data by device ID, recording time range, and list of topics. For example, if you wanted to calibrate the sensors of a robot that crashed into a wall, pull just its camera and LIDAR messages around the time of the incident. If you don’t provide a list of topics, the CLI will return all topics for the device within the time range.
Once you’ve identified the data you want to export, you can output it as an MCAP file (default), as a ROS 1 bag file (--output-format bag1), or directly to the console (--output-format json):
# MCAP files
$ foxglove data export --device-id dev_flm75pLkfzUBX2DH \
--start 2021-01-01T00:00:00Z --end 2022-01-01T00:00:00Z \
--topics /gps/fix,/gps/fix_velocity > output.mcap
# ROS 1 bag files
$ foxglove data export --device-id dev_flm75pLkfzUBX2DH \
--start 2021-01-01T00:00:00Z --end 2022-01-01T00:00:00Z \
--output-format bag1 \
--topics /gps/fix,/gps/fix_velocity > output.bag
language-bash
Next, let's verify that you’ve exported the correct data.
For MCAP files, download the MCAP CLI, and run mcap info on your output file:
$ mcap info output.mcap
library: mcap go #(devel); fg-data-platform-db07abe7
profile: ros1
messages: 6728
duration: 5m39.304931438s
start: 2021-03-22T08:03:38.473036858-07:00 (1616425418.473036858)
end: 2021-03-22T08:09:17.777968296-07:00 (1616425757.777968296)
compression:
lz4: [1/1 chunks] (86.05%)
channels:
(1) /gps/fix 3364 msgs (9.91 Hz) : sensor_msgs/NavSatFix [ros1msg]
(2) /gps/fix_velocity 3364 msgs (9.91 Hz) : geometry_msgs/TwistWithCovarianceStamped [ros1msg]
attachments: 0
language-bash
For ROS 1 bag files, download rosbag to run similar commands:
$ rosbag reindex output.bag
$ rosbag info output.bag
path: output.bag
version: 2.0
duration: 5:39s (339s)
start: Mar 22 2021 08:03:38.47 (1616425418.47)
end: Mar 22 2021 08:09:17.78 (1616425757.78)
size: 328.6 KB
messages: 6728
compression: lz4 [1/1 chunks; 12.87%]
uncompressed: 1.8 MB @ 5.5 KB/s
compressed: 240.0 KB @ 0.7 KB/s (12.87%)
types: geometry_msgs/TwistWithCovarianceStamped [b00b6ce36bf21f646151de97da2c485c]
sensor_msgs/NavSatFix [7f6e605ad1e52d05162190ff17be80b6]
topics: /gps/fix 3364 msgs : sensor_msgs/NavSatFix
/gps/fix_velocity 3364 msgs : geometry_msgs/TwistWithCovarianceStamped
language-bash
You can also drag and drop output ROS 1 bag and MCAP files into Foxglove to visualize their playback – we recommend adding a Raw Messages panel to view the raw output of one of the topics you exported.
Knowing how to import and export data is just the beginning. To navigate your growing collection of data recordings, you can annotate them with events. Events denote points of interest, and can occur instantaneously – like for a robot collision – or over a range of time – like the time a robot spends over a certain speed.
To create an event, you must provide it with a device-id and time (start and end – these values should be identical for instantaneous events) – colon-separated key-value pairs for metadata is optional:
# Instantaneous event
$ foxglove events add --device-id dev_flm75pLkfzUBX2DH \
--start 2021-01-02T00:00:00Z --end 2021-01-02T00:00:00Z \
--metadata collision:true
Created event: evt_N6doUtPYh8i7iZxf
# Time range event
$ foxglove events add --device-id dev_Wm1gvryKJmREqnVT \
--start 2021-01-02T00:00:00Z --end 2021-01-02T00:05:00Z \
--metadata speeding:true --metadata mph:60
Created event: evt_idMGJImlICYP4dcy
language-bash
Retrieve a list of all events to ensure your event was created successfully:
$ foxglove events list
| ID | DEVICE ID | START | END | CREATED AT | UPDATED AT | METADATA |
| -------------------- | -------------------- | -------------------- | -------------------- | ------------------------ | ------------------------ | ------------------------------ |
| evt_N6doUtPYh8i7iZxf | dev_flm75pLkfzUBX2DH | 2021-01-02T00:00:00Z | 2021-01-02T00:00:00Z | 2023-12-13T13:22:44.263Z | 2023-12-13T13:22:44.263Z | {"collision":"true"} |
| evt_idMGJImlICYP4dcy | dev_Wm1gvryKJmREqnVT | 2021-01-02T00:00:00Z | 2021-01-02T00:05:00Z | 2023-12-13T13:26:37.080Z | 2023-12-13T13:26:37.080Z | {"speeding":"true","mph":"60"} |
language-bash
The Foxglove CLI unlocks new ways to interact with your robots’ data and simplifies your team’s development workflows. Its consistent interface for creating, exporting, and retrieving resources makes it a valuable addition to every roboticist’s toolbox – especially when they want to write scripts to automate their workflows.
Check out the docs to learn how you can leverage the CLI for other development workflows – like viewing pending imports, publishing Foxglove extensions, and more.
