Announcing FlatBuffers Support in Foxglove

Analyze your FlatBuffers data with Foxglove Studio and Data Platform

Sam Nosenzo ·

Esther Weon ·

6 min read

Published December 12, 2022

Announcing FlatBuffers Support in Foxglove

We’re thrilled to announce that Foxglove now offers FlatBuffers support. FlatBuffers is a cross-platform, data serialization library that compiles data to a strongly typed binary format. It is optimized for transmission, storage, and speed, making it a popular data format for applications where performance is critical – such as gaming, mobile, and robotics.

You can now store your FlatBuffers data in MCAP files, upload these files to Foxglove Data Platform for analysis, and even visualize them in Foxglove for playback and debugging.

The benefits of FlatBuffers

FlatBuffers streamlines reading by allowing you to access your serialized data without iterating through it to parse or unpack it. Foxglove Studio leverages these benefits when reading in FlatBuffers-encoded byte arrays, as it uses a slice into the message buffer to access the data it needs.

FlatBuffers is also memory-efficient when writing to MCAP files. While its serialized data output is on par with Protobuf, it has a much smaller memory footprint than JSON.

While using FlatBuffers for your robotics data can provide many of these benefits, choosing a data serialization format is ultimately a choice between tradeoffs that depend on your specific constraints and application. Be sure to check out the official FlatBuffers documentation for performance benchmarks and more information.

Using FlatBuffers schemas with Foxglove Studio

In addition to FlatBuffers support in Foxglove Studio, we’ve added FlatBuffers schema files to the @foxglove/schemas repo. These schema files are available to import or copy into your own projects, so you can read or write FlatBuffers messages that adhere to schemas supported by Foxglove Studio visualizations.

To get started with using these new schemas, let’s walk through how we would use FlatBuffers to write foxglove.Grid messages to an MCAP file, so we can play it back in Foxglove Studio for debugging and analysis.

Generating the binary FlatBuffers schema

First, let’s copy the Grid.fbs file and its dependencies from @foxglove/schemas into a project folder. Then, let’s use the FlatBuffers schema compiler to compile our Grid.fbs file’s dependencies and generate a binary Grid.bfbs file (“binary FlatBuffers schema”):

$ flatc -b --schema -o <OUTPUT_PATH> <PATH_TO_FBS_FOLDER>

The output .bfbs file makes sure that all the tools needed to decode the schema are self-contained in that file. We would repeat this step of generating .bfbs files for every schema we want to register to an MCAP channel.

Generating the TypeScript classes

To construct our serialized message buffers, we must also generate TypeScript files from our original Grid.fbs file:

$ flatc --ts -o <OUTPUT_PATH> <PATH_TO_FBS_FOLDER>

Creating an MCAP writer with a registered channel

We’re finally ready to start writing foxglove.Grid messages to an MCAP file!

Let’s create a file that imports McapWriter from the @mcap/core TypeScript library. We’ll be using this FileHandleWritable class to specify our output MCAP file (flatbuffer.mcap):

import { McapWriter } from "@mcap/core";

// …

const fileHandle = await open("flatbuffer.mcap", "w");
const fileHandleWritable = new FileHandleWritable(fileHandle);

const mcapFile = new McapWriter({
  writable: fileHandleWritable,
  useStatistics: false,
  useChunks: true,
  useChunkIndex: true,
});

Let’s reference our binary FlatBuffers schema file to create a buffer for our messages:

const binaryGridSchemaBuffer = fs.readFileSync(<PATH_TO_GENERATED_GRID_BFBS_FILE>);

Next, we’ll register an MCAP channel with this grid schema:

const gridSchemaId = await mcapFile.registerSchema({
  name: "foxglove.Grid",
  encoding: "flatbuffer",
  data: binaryGridSchemaBuffer,
});

const gridChannelId = await mcapFile.registerChannel({
  schemaId: gridSchemaId,
  topic: "grid",
  messageEncoding: "flatbuffer",
  metadata: new Map(),
});

Writing FlatBuffers messages to the buffer

Let’s import the Builder from the flatbuffers npm package to write our serialized message data to the buffer – reference the official TypeScript FlatBuffers tutorial for more information on this step.

For our example, we’ll be using a makeNewGrid helper function to construct a grid object conforming to our Grid TypeScript definition, and a buildGridMessage helper function to actually write this object to a buffer:

import { Builder } from "flatbuffers";

// …

const grid = makeNewGrid();

const gridBuilder = new Builder();
const fbGrid = buildGridMessage(gridBuilder, grid as Grid);

We’ll see that buildGridMessage constructs the message depth-first, as required by FlatBuffers – for example, before it can buildPose to add the grid’s pose data to the buffer, it must first buildVector3 for the pose’s position and buildQuaternion for the pose’s orientation to add those primitive values to the buffer. In short, all sub-schema data needs to be added to the buffer before we can call FbPose.start:

function buildPose(...args) {
  const pos = buildVector3(...args);
  const quat = buildQuaternion(...args);

  FbPose.start(builder); // FbPose is the generated TypeScript class for the `Pose` FlatBuffers schema
  // …
}

const pose = buildPose(...args);

After writing all non-primitive fields to the buffer, we start the Grid data and include each field’s buffer index in the actual grid message (e.g. addPose):

const pose = buildPose(builder, json.pose);
// ... build other Grid fields to the buffer
FbGrid.start(builder); // initiate the Grid message
FbGrid.addPose(builder, pose); // Adds pose’s buffer index to the grid message

Once we’ve added all grid fields to the buffer, we need to finish the builder with our grid message’s index. This makes the Grid our buffer’s root schema – when reading the buffer, our code can rely on the fact that the entire buffer conforms to this schema:

gridBuilder.finish(fbGrid);

We can now add our message data as a UINT8 array it to our registered MCAP channel:

await mcapFile.addMessage({
    channelId: gridChannelId,
    sequence: 0,
    publishTime: //message publish time,
    logTime: // message log time,
    data: gridBuilder.asUint8Array(),
});

And there we have it! Running this file will write FlatBuffers-encoded foxglove.Grid messages to an MCAP file. For more details, check out the full example here.

Learn more

We want to make it easy for all robotics teams – regardless of language or framework – take full advantage of our tools. Now that FlatBuffers have full support across the Foxglove ecosystem, we hope people can start to reap the full benefits of this powerful format and load their data into Foxglove Studio more easily.

To get started with writing FlatBuffers data in different languages, check out the official FlatBuffers tutorial guide here. To explore all Foxglove schemas, you can check out our docs or the @foxglove/schemas GitHub repo. As always, join our Slack community to ask questions, or reach out to us directly with any feedback.

Special thanks to James Kuszmaul for the contributions and help along the way.