We developed Foxglove Data Platform to help you manage large volumes of robotics data, and Foxglove Studio to help you visually explore that data. But while these two pillars of data management and visualization are critical components of robotics development, we have much bigger plans to help you organize and explore data.

In addition to storing and viewing data, we want to help you identify the most interesting minutes – or seconds! – of that data. We want to help you cut through the noise and analyze large amounts of data more efficiently.

To equip our users with this level of granular control, we built Foxglove events. With events, you can now add metadata-rich annotations to your robotics data recordings – via either Foxglove Studio or Data Platform.

Creating events

In Data Platform

After importing a data recording into Foxglove Data Platform, navigate to the recording’s device page in Foxglove console and click “Create event”.

Each event you create must have a timestamp, a duration, and any number of metadata keys and values. A duration of 0 would indicate an instantaneous event (e.g. “the robot stopped”), while a duration of anything greater than 0 would indicate a time range (e.g. “the robot is going up a hill”).

New event in Foxglove Data Platform

You can then go to your “Events” page and see your newly created event.

In Studio

Navigate to the Foxglove console’s “Recordings” page and select a data recording to “View in Studio”. The Foxglove Studio window that opens with your Data Platform data source will display a bookmark icon on the playback bar.

Foxglove Studio playback bar

Seek to a point of interest in your data, then click that bookmark icon to add an event to your data at that point on the timeline.

Create event in Foxglove Studio

These events created from the Studio interface will also persist to your Foxglove console account’s “Events” page.

Viewing events

In Data Platform

Head over to the “Events” page for a list of all created events.

Foxglove console's "Events" page

In Studio

Open the Data source sidebar, and select the “Events” tab for a list of events associated with your current data source.

When streaming data from Foxglove Data Platform, Foxglove Studio’s playback bar will display events using blue bars. Hovering on a blue bar will display that event’s metadata in a tooltip:

If the Data source sidebar is open to your list of events, hovering on an event will also highlight the corresponding event metadata in that sidebar pane.

Hovering on event in Foxglove Studio

All this makes it much easier to analyze what happened during a particular event.

Searching events

In Data Platform

You can use the filter controls on the “Events” page to help you find events. You can filter by device name, time, or metadata keys and values:

Foxglove Data Platform's event search

The metadata search field uses a specific syntax to filter down your events:

weather:rain – Find events with a certain metadata key and corresponding value

weather:rain weather:fog – Finds events with weather that matches both rain AND fog

weather:rain,fog – Finds events with weather that matches rain OR fog

“error” – Finds events with either a key or value tagged “error”

Read more in the docs here.

In Studio

Open the Data source sidebar and click on the “Events” tab to see all your events. The search input field at the top of the tab uses the same query syntax supported in Foxglove Data Platform.

Type a query to filter events by metadata keys and values and find events of interest.

The future of Foxglove events

Whether you’re annotating when your rover encountered an obstacle, when your self-driving car noticed a stop sign, or when your warehouse robot successfully completed a task, Foxglove events can help you find the needle in the haystack. Instead of each team member navigating hours of recordings to find an important event, your entire team can collaborate on annotating your data and immediately revisit the moments that matter.

Check out the Studio and Data Platform docs for more information on how to incorporate events into your existing workflows. To ask questions or share feedback on our Foxglove event roadmap, you can join our Slack community or contact us directly.


One of the most powerful features of Foxglove Studio is the ability to pause, seek, and step through your robotics recordings. Anyone that’s had to debug a robot’s behavior from a log file knows how much these capabilities streamline the development cycle and speed up the process of diagnosing complex issues.

But robotics data comes with unique challenges. A typical robotics log file can contain dozens of disparate data streams, all occurring at different rates – this makes it especially challenging to get the data you need when you need it. In response to these challenges, we’ve made some recent changes in Foxglove Studio to improve the experience of pausing, seeking, and stepping through this complex data – whether you’re loading files locally or from Foxglove Data Platform.

The challenges

Unlike a video file with a single video channel and 2 to 5 audio channels, robot data consists of dozens, if not hundreds, of channels. Each can occur at different rates – you might have sensor data at 10hz or 30hz, odometry data at 50hz or 100hz, and map data at 0.01hz or even slower! Across all this, you may even have a transform system publishing a firehouse of transform data with no defined rate.

Unlocking accurate analysis

Despite these challenges, we want to pause, seek, and step through our data as easily as a video! We’ve made some recent changes in Studio to improve this experience – whether you are loading files locally or from Data Platform.

Message lookback

Let’s say you open a local MCAP or ROS bag file that’s 30 minutes long and want to jump to some time 12 minutes into the recording. If Studio loaded only the messages at the time you jumped to, there’s a high likelihood that every panel would be blank! Why? It’s unlikely that every topic you are visualizing has a message at exactly the time you jumped to; most will have messages sometime before or sometime after.

In Studio, we now perform a “message lookback” on your data, to find the most recent message for each topic you want to visualize. When you seek 12 minutes into the recording, you’ll see the latest 100hz odom message in the Raw Messages panel, the most recent camera image in the Image panel, and an occupancy map in the 3D panel, even if these messages occurred minutes apart. In ROS, this is referred to as “latching”, and must be explicitly enabled on topics that publish infrequently. With message lookback, Foxglove Studio automatically displays all topics as if they were latched.

Using the example dataset below with messages arriving at different times across topics, assume we seek to the red line. To properly display data across each of our visualization panels, we need to load message data from different times within the file.

Message publishing frequency

You can see the effect message lookback has on the user experience when seeking. If we only loaded the most recent 0.1 seconds (or even 1 second!), we could omit important visual information that might not occur again in our data.

Before implementing message lookback, Foxglove Studio panels are missing critical information when a user seeks around a data source.

After implementing message lookback, Foxglove Studio panels accurately display all data as soon as a user seeks to a new timestamp.

Faster time to first message

Whether loading data from local files or a remote server, we want Studio to feel responsive, and display your message data as soon as it is available. We recently optimized how we load data, to more efficiently display it in visualization panels.

Foxglove Studio has a concept of a “player”, which is responsible for fetching messages from a data source and delivering them to each visualization panel. Originally, Studio was designed to load messages from a local or remote data source in batches, with a specific start and end time for each batch. Panels needed to wait until an entire batch finished loading before rendering anything.

Now, we’ve updated the players to handle data preloading and playback separately and with different techniques. This architecture allows us to load message data for playback much more quickly. It also allows us to write adapters for different data sources using a common interface, while still leveraging each data source's specific properties for loading messages.

More efficient preloading

Most panels in Studio display the most recent message on a topic. However, some visualizations benefit from displaying messages across the entire playback time range. The Plot and Map panels are examples of this behavior: a common workflow is opening a plot panel to plot a signal over the entire time range of data — this allows for quickly scanning for anomalies. For a map panel, it allows seeing the GPS location of the robot across the entire time range.

Preloading

But preloading data in this manner also brings benefits to the 3D panel, even though it primarily visualizes the most recent message on each topic (e.g. the latest point cloud). Complex robots often have many coordinate frames — for example, each joint of a robot arm might be its own frame, or a self-driving car might have a frame for each of its cameras as well as a frame for the street map. To render a single 3D scene containing visual elements from different coordinate frames, the panel needs to figure out how to transform all elements to a single coordinate space. We use a database of transform messages to perform these calculations.

Since transform messages accumulate and update over time, if the database only had the latest message, or was missing transforms between coordinate frames, you might see an incorrect or partially rendered scene. Instead, the 3D panel is now able to preload all the transform messages from the entire time range, to correctly place all elements in the scene even when the user rapidly seeks across time.

Preloading data is powered by the same iterator-based message reading described above, allowing us to re-use message loading patterns, and enabling the preloading layer to handle caching appropriately.

Looking forward

Visualizing data is at the heart of understanding what a robotics system did and why it did it; rarely is there a one-size-fits all solution. At Foxglove we are constantly evolving our products to help roboticists gain insight into their systems in new and powerful ways, and to help bring robots to market faster.

Join our Slack community to ask any questions, or reach out to us directly with any feedback.


Last week, we had our first ever Foxglove Launch Week – an action-packed week of exciting new product announcements and Foxglove news!

Here's a recap of all our updates:

Foxglove

We raised $15M for our Series A!

Foxglove is officially SOC 2 compliant

Foxglove Studio

We introduced Studio's new 3D panel – and recorded our first Foxglove podcast episode about it!

CTO Roman Shtylman does a deep dive on how data loading in Studio can streamline your debugging workflows

Foxglove Data Platform

We're offering reduced pricing for Foxglove Teams

We introduced Foxglove events, a way to annotate your robotics data with rich metadata

MCAP

We have a new MCAP storage plugin for ROS 2

Foxglove engineer John Hurliman will be giving a talk on "MCAP: A Next-Generation File Format for ROS Recording" at this year's ROSCon

We hope you had as much fun as we did!

If you have questions about any of our announcements (or if you have any feedback for our next Launch Week, coming soon!), you can message us on Twitter or join our Slack community.


Foxglove Launch Week (Fall 2022)

Foxglove

Foxglove Studio

Foxglove Data Platform

MCAP

Read more:

Ready to get started?Download today on Linux, Windows, or macOS.