3D

Display markers, entities, camera images, meshes, URDF models, and more in a 3D scene.

Introduction

By publishing specifically formatted messages to topics, you can create complex 3D scenes from primitive shapes, grids, point clouds, and other renderables.

Open the Panel settings tab in the app sidebar or click the settings icon (⚙) in the panel toolbar to configure the panel.

3d panel

Concepts

The 3D panel renders different types of visualizations into a common scene. There are a few general concepts to understand to correctly visualize your robot data.

Transforms

A robotics system produces many messages describing its observations of the world around it. These messages may exist in one or more coordinate frames. For example a lidar sensor might report readings relative to its frame of reference while an IMU will produce readings relative to another. Both of these might exist on different parts of the robot.

Transforms define the spacial relationships between two coordinate frames at some time. Transforms define this relationship between two frames often refering to one as the child and the other as a parent.

In the 3D panel, the Display frame identifies the viewport coordinate frame. To render objects into the scene, there must exist a transform path from the object's coordinate frame to the display frame.

Topics

Each topic matching a supported visualization is displayable in the scene. Under each topic is a set of settings depending on the type of visualization. These configure additional aspects of how the visualization appears in the scene. You can toggle individual topics.

Supported Messages

The 3D panel can visualize an assortment of different messages.

To visualize a topic, the messages on that topic must conform to one of the known message schemas listed below.

Transforms

frameworkschema
ROS 1tf/tfMessage
ROS 1tf2_msgs/TFMessage
ROS 2tf2_msgs/msg/TFMessage
Customfoxglove.FrameTransform

Scene Entity

A scene entity is a collection of primitive shapes (cubes, spheres, text, meshes, lines, etc). Use scene entities to display anything from a basic bounding box to a complex 3d decision tree or a road network.

You can add multiple scene entities using a SceneUpdate message.

frameworkschema
Customfoxglove.SceneUpdate
Customfoxglove.SceneEntity

ROS Markers

The 3D panel supports ROS Markers. Similar to SceneEntities, these Marker messages describe primitive shapes or meshes. ROS users with existing Marker messages can visualize them in the 3D scene with no changes to their data.

frameworkschema
ROS 1visualization_msgs/Marker
ROS 2visualization_msgs/msg/Marker
ROS 1visualization_msgs/MarkerArray
ROS 2visualization_msgs/msg/MarkerArray

Mesh Markers

For markers with a mesh_resource field, following URL schemes are supported:

  • http(s)://
  • package:// (Desktop app only)
  • file:// (Desktop app only)

And the following file formats:

glTF (.glb)

This is the preferred format, as it enjoys the best performance of all supported file types.

Binary glTF files bundle all required assets into a single file, with support for embedded meshes, compression, and the same physically-based material system used in Studio. As a result, your model should appear in Studio similarly to how it appears in other 3D programs.

STL (.stl)

STL files are well supported in Studio, but lack some of glTF's visualization features. The main advantage to STL is the ability to share the same files between your hardware manufacturing process and robot visualization tooling.

STL was designed for 3D printing and CAD applications, and does not include materials or hierarchies of meshes. While they can be represented in a binary encoding, STL files are commonly represented with ASCII characters, which leads to larger files.

COLLADA (.dae)

As a predecessor to glTF, COLLADA has a similar feature set. With that said, it does have larger XML-based files, no compression, and additional processing overhead.

There is a bug in RViz where the up-axis metadata is ignored, resulting in incorrect orientations for many .dae files in ROS environments. To work around this, the 3D panel has a Ignore COLLADA <up_axis> setting to toggle between observing the <up-axis> tag or ignoring it like RViz.

Wavefront OBJ (.obj)

OBJ is a simple ASCII format predating all other supported formats. It has large file sizes, no material support, no mesh hierarchies, no compression, and additional processing overhead.

Material support was added to the OBJ format as separate .mtl files, which Studio does not read.

ROS Geometry Msgs

PolygonStamped

PolygonStamped messages are displayed as a line between all the points.

frameworkschema
ROS 1geometry_msgs/PolygonStamped
ROS 2geometry_msgs/msg/PolygonStamped

Path

An array of timestamped poses in a named coordinate frame, denoting an object's path through space.

frameworkschema
ROS 1nav_msgs/Path
ROS 2nav_msgs/msg/Path
Customfoxglove.PosesInFrame

Point Cloud

A collection of N-dimensional points, which may contain additional fields with information like normals, intensity, etc.

pointcloud

frameworkschema
ROS 1sensor_msgs/PointCloud2
ROS 2sensor_msgs/msg/PointCloud2
Customfoxglove.PointCloud

Settings

fielddescription
Point sizeSize of each rendered point
Point shapeShape of each rendered point
Decay timeDuration of time (in sec) that each point stays rendered
Color modeOne of:
Flat: solid color
Color map: pre-defined color palette
Gradient: smooth transition between two custom colors
BGR (packed): sensor_msgs/PointCloud2 only; use embedded color from each point's rgb field (see below)
BGRA (packed): sensor_msgs/PointCloud2 only; use embedded color from each point's rgba field (see below)
RGBA (separate fields): foxglove.PointCloud only; use embedded color from each point's red, green, blue, and alpha fields (see below)
Flat colorOnly shown if "Color mode" is set to "Flat"; hex code for color of each point
Color byOnly shown if "Color mode" is set to any value but "Flat"; numeric field in message used for coloring logic
Color mapOnly shown if "Color mode" is set to "Color map"; "Turbo" (Google) or "Rainbow" (RViz); for mapping "Color by" field values to colors
OpacityOnly shown if "Color mode" is set to "Color map" or "BGR (packed)"; sets alpha value for all points
Value minOnly shown if "Color mode" is set to any value but "Flat"; minimum value used to normalize incoming points' "Color by" field values
Value maxOnly shown if "Color mode" is set to any value but "Flat"; maximum value used to normalize incoming points' "Color by" field values

RGBA color modes

When using the "BGR (packed)", "BGRA (packed)", and "RGBA (separate fields)" color modes, your point cloud message must contain certain fields to display color information for each point.

RGBA (separate fields)

For foxglove.PointCloud messages, each point can contain color information in four separate fields, named red, green, blue, and alpha, of any numeric type:

  • Floating-point values — 0–1 range
  • Unsigned integer values — Maximum possible range (e.g. 0–255 for a UINT8 field)
  • Signed integer values-max to max (e.g. −127 to 127 for an INT8 field; a value of −128 is treated as identical to −127)
BGR (packed) and BGRA (packed)

For sensor_msgs/PointCloud2 messages, each point can contain color information in a single field named rgb or rgba:

  • Must use a 4-byte type from sensor_msgs/PointField (UINT32, value 6, is recommended)
  • Each red, green, blue, and alpha value is represented by one byte in the 0–255 range
  • Bytes must be packed in [0xBB, 0xGG, 0xRR, 0xAA] order (i.e. (0xAA << 24) | (0xRR << 16) | (0xGG << 8) | 0xBB in little-endian order). This order is compatible with RViz.

If using the "BGR" mode, the alpha value must still be present, but is ignored.

Grid

A way to represent 2D grid of data in color.

grid

frameworkschema
ROS 1nav_msgs/OccupancyGrid
ROS 2nav_msgs/msg/OccupancyGrid
Customfoxglove.Grid

foxglove.Grid settings

fielddescription
Color modeOne of:
Flat: solid color
Color map: pre-defined color palette
Gradient: smooth transition between two custom colors
RGBA (separate fields): use embedded color from each point's red, green, blue, and alpha fields (see below)
Flat colorOnly shown if "Color mode" is set to "Flat"; hex code for color of each point
Color byOnly shown if "Color mode" is set to any value but "Flat"; numeric field in message used for coloring logic
Color mapOnly shown if "Color mode" is set to "Color map"; "Turbo" (Google) or "Rainbow" (RViz); for mapping "Color by" field values to colors
OpacityOnly shown if "Color mode" is set to "Color map" or "BGR (packed)"; sets alpha value for all points
Value minOnly shown if "Color mode" is set to any value but "Flat"; minimum value used to normalize incoming points' "Color by" field values
Value maxOnly shown if "Color mode" is set to any value but "Flat"; maximum value used to normalize incoming points' "Color by" field values
Frame lock"On" means the grid is locked to the frame specified by its frame_id, and will move as that frame's transforms change. "Off" means the grid is relative to the fixed frame and will not move after it is first displayed.
RGBA (separate fields) color mode

Each cell can contain color information in four separate fields, named red, green, blue, and alpha, of any numeric type:

  • Floating-point values — 0–1 range
  • Unsigned integer values — Maximum possible range (e.g. 0–255 for a UINT8 field)
  • Signed integer values-max to max (e.g. −127 to 127 for an INT8 field; a value of −128 is treated as identical to −127)
fielddescription
Min colorColor corresponding to minimum cell value (0)
Max colorColor corresponding to maximum cell value (100). Note that cells with value exactly 100 are displayed as fully transparent.
Unknown colorColor corresponding to unknown cell value (−1)
Fallback colorColor corresponding to cell values that fall outside the range from −1 to 100
Frame lock"On" means the grid is locked to the frame specified by its frame_id, and will move as that frame's transforms change. "Off" means the grid is relative to the fixed frame and will not move after it is first displayed.

Image

The 3D panel can display image data in the 3D scene. The image is displayed using the corresponding CameraCalibration or CameraInfo messages.

frameworkschema
ROS 1sensor_msgs/Image
ROS 2sensor_msgs/msg/Image
ROS 1sensor_msgs/CompressedImage
ROS 2sensor_msgs/msg/CompressedImage
Customfoxglove.RawImage
Customfoxglove.CompressedImage

Camera Field-of-view

frameworkschema
ROS 1sensor_msgs/CameraInfo
ROS 2sensor_msgs/msg/CameraInfo
Customfoxglove.CameraCalibration

Laser Scan

A single scan from a planar laser range-finder.

frameworkschema
ROS 1sensor_msgs/LaserScan
ROS 2sensor_msgs/msg/LaserScan
Customfoxglove.LaserScan

Pose

Poses in a named coordinate frame.

frameworkschema
ROS 1geometry_msgs/PoseArray
ROS 2geometry_msgs/msg/PoseArray
ROS 1geometry_msgs/PoseStamped
ROS 2geometry_msgs/msg/PoseStamped
Customfoxglove.PosesInFrame

Velodyne Scan

Velodyne LIDAR scan packets. These messages come from the velodyne ROS driver or the Velodyne connector built into Studio (desktop app).

frameworkschema
ROS 1velodyne_msgs/VelodyneScan
ROS 2velodyne_msgs/msg/VelodyneScan

Settings

Frame

Select your 3D scene's display frame and follow mode.

fielddescription
Display frameConfigures the coordinate frame for rendering the scene.

To render a scene element, you need a transform from its frame to the selected display frame. If its frame is the display frame, no additional transform is needed.
Follow modeConfigures the viewport following behavior.
  • "Pose" - Follows the display frame's position and orientation
  • "Position" - Follows the display frame's position only
  • "Fixed" - Does not update viewport

Scene

Configure generic rendering properties and viewport properties.

fielddescription
Render statsDisplay rendering performance statistics in panel corner.
BackgroundScene's background color
Label scaleScale of rendered labels
Ignore COLLADA <up_axis>Ignore the <up_axis> tag in COLLADA files.
Mesh up axisDirection of “up” when loading meshes (STL, OBJ) without orientation info ("Y-up", "Z-up").

View

Configure the camera settings.

fielddescription
DistanceDistance of camera from viewport center (i.e. zoom level)
PerspectiveToggles between 3D ("on") and 2D ("off") view of the scene. The 2D view orients the camera towards z-down.
Target Xx-coordinate of viewport center
Yy-coordinate of viewport center
Zz-coordinate of viewport center
ThetaCamera's theta
PhiCamera's phi
Y-Axis FOVCamera's field of vision on y-axis
NearCamera frustum near plane
FarCamera frustum far plane

Transforms

Configure how to visualize transform frames and which transforms to show. Visualizing a transform tree can help debug why elements are not rendering where expected.

transforms

Use the top-level menu to toggle all transform visualizations.

Settings

fielddescription
EditableToggles in-app editing of transform frames. When "on", you can update the Translation and Rotation values to tweak transform frames, see the impact on the scene, and debug how you may want to adjust the frames on your robot. When turned "Off", will revert to original translation and rotation values.
LabelsToggle the display of transform labels.
Axis scaleScale of transform axis (displayed as an arrow marker)
Line widthWidth of transform (displayed as a line marker)
Line colorColor of transform (displayed as a line marker)
Enable preloadingPreload transform messages from the data source.

Topics

The Topics section configures the topics displayed in the scene.

Any topic with messages matching one of the visualization formats will show up in the topics list. Each supported format has format specific visualization settings. These settings will show up under the topic.

Use the top-level menu to toggle all topics. You can toggle any individual topic by click the eye icon next to the topic name.

Custom Layers

Custom layers allow you to add additional visual elements not from your data source.

Use the top-level menu to add additional custom layers.

Add Grid

A grid is 2D square with a fixed size and number of divisions. You can create any numbers of grids. Grids can be rendered relative to the display frame or any other transform frame.

grid

Use the grid layer settings to change the frame, size, color, divisions, and other properties of the grid.

Add Unified Robot Description Format (URDF)

URDF robot models are loaded automatically if your data source supports parameters (i.e. a native ROS 1 or ROS 2 connection) and the /robot_description parameter is set to valid URDF XML.

You can add additional URDF models with a custom layer. Use the layer URL setting to configure the source of the model.

Valid urls start with: http(s)://, package:// (desktop app only), or file:// (desktop app only).

Publish

Configure click-to-publish behavior for the 3D panel.

fielddescription
TypeType of message to publish.
TopicTopic to publish message to – only possible with a ROS data source with publish support.

User interactions

Click any object in the scene to display its relevant details in the selected object popup.

3d panel clicked marker

Selecting an object will set a $selected_id variable to the clicked objects id value if it has one.

The panel controls on the right can be used to do the following:

  • Select toggle object select mode
  • 3D Toggle between 3D and 2D views of the scene
  • Measure measure the distance between two points

Shortcuts

To move the camera:

  • w – Forward
  • a – Left
  • s – Backward
  • d – Right
  • z, or Scroll up – Zoom in
  • x, or Scroll down – Zoom out
  • Drag – Parallel to the ground. Will disengage “follow” mode, if enabled.
  • Right-click + drag – Pan and rotate. Dragging horizontally rotates around the world's z-axis; dragging vertically pans around the x and y axes
  • Shift + other action – Adjusts all values to be 1/10 of baseline values; allows for more precise movements and adjustments