Spotlight: Stefan Novak of a.i. solutions on Taking Foxglove Studio to Outer Space

Writing custom panels in Foxglove Studio using the extensions API

Esther Weon ·

8 min read

Published November 5, 2021

Space situational awareness simulation with satellites and space debris modelled and rendered using a.i. solutions’ FreeFlyer software.

As a software engineering lead at a.i. solutions, Stefan Novak has helped clients like NASA and the Air Force plan hundreds of successful excursions into outer space. His team in particular focuses on FreeFlyer, a software application that provides support for space mission design, analysis, and operations. Its customizable interfaces have a lot in common with Foxglove Studio – making it no surprise that Stefan used Studio earlier this year to build custom visualizations for upcoming space missions.

We had the pleasure of sitting down with Stefan to discuss the projects he’s working on, and how Foxglove Studio has helped his team plan for the next satellite launch.

How would you describe your role and work at a.i. solutions?

I work on a small team at a.i. solutions that develops FreeFlyer, our commercial space mission design software. FreeFlyer has been used on 225+ space missions, supporting everything from early mission design to mission operations. At its core, it is a high-fidelity orbital simulation engine with a powerful IDE built around it. It’s used by researchers and startups alike, with organizations like NASA, the Air Force, and NOAA using it to perform the complex analysis required for space mission operations.

For some context on what our work entails, my first project on the team was implementing terrain modeling to compute high-fidelity line-of-sight calculations. This lets us determine when regions of terrain are visible or not visible from a spacecraft, particularly when a spacecraft sets below the horizon in a mountainous environment.

One of my favorite moments at work is watching a satellite that used FreeFlyer to plan its trajectory and compute its maneuvers finally launch! There are a lot of interesting computer science problems to work on, while also being able to contribute to future space missions.

How did you first become interested in robotics?

L: My first robot, built using a radio-controlled toy chassis. R: An interactive Flash site I created to demonstrate how the control system worked.

During a senior design project for my undergraduate degree in mechanical engineering, my team designed and built an operational self-driving car that would follow another radio-controlled car using optical navigation. I developed the steering and engine control systems and implemented the software system. My favorite part of the project was creating an interactive 2D simulation of the steering algorithm in Macromedia Flash!

I was also exposed to robotics working as a Test Officer for the U.S. Army, my first job out of college. I got to ride around in the back of M3A3 Bradley tanks while testing on-board software. I even got to test a self-driving Humvee. During that time, I was able to attend the DARPA Urban Challenge in 2007 and see a variety of self-driving car platforms in-person.

What drew you to leveraging robotics for space missions?

globe Hypothetical large-scale satellite constellation and coverage map for the continental US, modeled and rendered using FreeFlyer.

There is no shortage of challenging engineering problems in the space domain – your work can contribute to advancements in earth science, space weather, global communications networks, and even exploration of the solar system. Satellites generate a lot of telemetry data which is then processed and visualized in ways similar to modern robotics. I’ve found that the Robot Operating System (ROS) is a great fit for handling the “data plumbing” of telemetry within a satelllite ground system or mission operations environment. It's exciting to take open source technology that was built for the ground, and use it for space. I think there’s a lot of potential in leveraging open source robotics software for supporting space missions!

What does a typical day for you look like? What projects are you working on now?

Currently, I'm developing tools that can be used for space missions where a satellite will be approaching and docking with an older satellite that needs to be refueled and serviced. The new satellite would have optical, lidar, and infrared sensors to detect the older satellite and estimate its pose (i.e. its position and orientation).

space mission Simulation of an approach and rendezvous with the International Space Station, modeled and rendered using FreeFlyer.

While on the project, I realized that this is similar to problems that self-driving car companies have to solve. I started researching the tools used in that domain, which is how I came across ROS and Foxglove Studio. While FreeFlyer is useful for simulating telemetry data, it is less geared towards managing that data. Foxglove Studio, on the other hand, is designed to integrate with ROS, making it easy to manage telemetry data and leverage built-in message types that map directly to the problem I was working on. Its UI is also expressly designed around easily choosing a telemetry topic to investigate, visualizing it with rich 3D scenes, and debugging it with interactive controls.

How did Foxglove Studio help achieve your goals?

We wanted to overlay a wireframe of a satellite’s CAD model on top of an optical image taken by another satellite. The wireframe needed to be positioned and aligned on top of the image using the satellite’s estimated pose. If there was an error in the on-board pose estimate, we want to see that visually - the wireframe wouldn’t be aligned with the satellite in the image underneath. This would give the satellite operators a warning that the on-board pose estimation algorithm wasn’t converging on the right solution, and the operators would be able to send a command up to the satellite to reseed its pose estimation algorithm.

To do this, I developed a custom Foxglove panel that renders camera images and pose telemetry data together in a Babylon.js scene, with the image as the background and the wireframe in the foreground. I implemented this data visualization using the Foxglove Studio Extensions Manager, a developer-friendly command-line tool for creating new extensions and building, packaging, and installing them.

custom extension Foxglove Studio dashboard displaying images and telemetry simulated by FreeFlyer.

I found the Foxglove Extensions API very well designed. It makes it easy to publish and subscribe to ROS messages within your custom panel, and to use those incoming messages to update React UI components. I was able to render data from multiple ROS topics, and bundle up additional Javascript dependencies into my extension (in my case, Babylon.js). You can also embed any dynamic HTML or Javascript into a Foxglove panel pretty easily! It’s really cool that Foxglove provides the tooling to do this and that I was able to build this custom visualization without having to fork Foxglove.

What do you have planned next?

I’ve just submitted an abstract to speak at the Ground System Architectures Workshop, an annual gathering to talk about software architectures for satellite mission control centers. In it, I argue that satellite ground systems have a lot in common with autonomous vehicles. Just like self-driving cars, modern spacecraft generate large amounts of live telemetry data, use sensors to locate targets, and continuously update their models of the world.

If my abstract is accepted, I’ll be sharing the lessons I learned from leveraging free and open source tools used in the self-driving car domain (like ROS and Foxglove Studio!) for satellite rendezvous and proximity operations (RPO). It will be interesting to see how the same tools that catalyzed such exciting progress in autonomous vehicle development could be just as relevant to the space community.

Check out the talk I presented at GSAW last year here: Cloud-based Processing of Flight Dynamics Data Pipelines.

What advice would you share with anyone interested in getting into space missions?

There are a lot of different avenues to be a part of the space industry, depending on your passions! Within the space industry, there are opportunities for people with backgrounds in anything – from robotics and hardware design, to modeling and simulation, to software and quality testing. It’s also a pretty small and tight-knit industry, and I love connecting people to opportunities that are on my radar.

The Atlantis Space Shuttle, on display here at the Kennedy Space Center near Cape Canaveral, FL, has inspired awe and wonder in generations of onlookers.

If you’re interested in learning more, you can follow my updates on LinkedIn and Instagram. I love learning what types of problems people enjoy working on, and helping them translate that to opportunities in the space industry, so please feel free to connect with me!