Use 2D annotations and 3D markers to add context to your camera images
With Foxglove's new Image panel, you can now add useful context to your robots’ raw and compressed camera images with 2D annotations and 3D markers. The original Image panel – now renamed to “Image (Legacy)” in the app – is now deprecated and will be removed in a future release.
When reviewing your sensor images, you may want to verify that your autonomy stack is working as expected. For example, if your team develops self-driving cars, you may want to verify that your perception models are correctly classifying objects in its world (e.g. “car”, “bus”, “pedestrian”, etc.), and that your planning models are accurately differentiating between them to help the car navigate the world.
With 2D annotation and 3D marker topics, you can now add this debugging context to the image directly.
2D annotations come in three forms – text labels, circles, and points. To display them, publish a topic with ImageAnnotations messages. To toggle them on, select the ones you’d like to display in the panel settings’ “Image annotations” section:
With 2D annotations, you can label detected objects with text, or highlight relevant areas of a scene with circles, or delineate boundaries with points.
To display 3D markers, publish topics with any of the 3D panel’s supported message types – these can range from cubes and poses to point clouds and occupancy grids. First set a “Calibration” topic in the Image panel settings – this determines the camera’s field-of-view, and is used to transform 3D markers into the image’s 2D space. Then select the 3D markers you’d like to superimpose on the image in the “Topics” section:
Superimpose bounding boxes, maps, and other 3D markers on your annotated camera images, for even richer context on what your robot is experiencing.
If you save these changes in a layout, you’ve instantly added useful context for your whole team to reference when debugging your robotics data!
By including additional context in your robots’ camera feed images, you can use the new Image panel to accelerate your robotics team’s usual visualization and debugging workflows.
For more information on 2D annotations and 3D markers, you can check out our Image panel docs. You can also join our Discord community or contact us directly with any questions.
