Overview

To accelerate development of their Fury foundation-model, Scout AI relies on Foxglove as the unified interface for operating robots, monitoring live missions, debugging issues, and collecting high-quality training data. Foxglove gives their operations, systems, and AI teams a single tool for understanding robot behavior. Whether live in the field or offline in replay, it allows them to iterate faster and deploy more capable models across various robot domains.

Key Results 

1. Tightened their train–deploy–diagnose–data loop
2. Unified operations, AI, and systems engineering around a single observability tool
3. Avoided building and maintaining another bespoke visualization stack

‍“The goal of the company is to iterate and build these models, but also the whole system… The faster you can get visualization and diagnostics, the faster you can be productive.” Collin Otis, Co-Founder & CTO, Scout AI

The Challenge: Accelerate iteration on an end-to-end learned model.

Scout AI is building Fury, a foundation model for military robotics that can reason over real-time sensor data and natural-language instructions, then coordinate a variety of air, ground, and maritime systems. At previous companies, the team lived through years of custom tooling: QT-based desktop visualizers on Linux, bespoke JavaScript/web visualizers backed by homegrown frameworks, internal plugins, and dashboards that were powerful but brittle. Those tools took significant engineering time to build and extend, and all would eventually become blockers that distracted from the core development focus.

1. Night: retrain models with the latest data
2. Morning: run new missions (both in real life and simulated) using the latest version of Fury
3. Afternoon: review data, commands, and outcomes
4. Repeat

This quick, iterative approach creates several requirements:

Tight train–deploy–diagnose–data loop

Because it’s an end-to-end learned system, diagnostics and data collection are two sides of the same coin. The team needs to spot model failures and simultaneously identify new data snippets (“trims”) to feed into the next training run.

Multi-modal observability

They must reason over video, text, 3D trajectories, time-series telemetry, and human intent (prompts, spoken instructions) at once.

High-stakes operational constraints

Systems are deployed in safety-critical environments. As Collin puts it, “It’s a very abusive environment, not just on hardware, but on software… people’s lives are at stake… your stuff has to work.”

From the beginning, Scout AI knew they needed an out-of-the-box, extensible interface that didn’t require them to build and maintain custom tooling. To train and deploy Fury, Scout AI runs robots five days a week at Forge, their unmanned systems training center near Paso Robles, California. Former Army and Special Operations personnel run realistic missions across their different vehicles, continuously generating data and stress-testing the model in both real world and simulated environments. They chose Foxglove for operating, debugging, and observing these systems to enable a fast iteration loop.

‍“Building a UI is an impediment. If you have to build that before you can build your core product, that’s a pain. Even when we tried to build really good UIs, they eventually became cumbersome. Foxglove allows us to focus on building our core product.” Collin Otis, Co-Founder & CTO, Scout AI

‍

The Solution: Consolidating distributed teams and disparate workflows with Foxglove’s unified robotics platform.

When starting development at Scout AI, the team already knew they needed:

• A flexible, off-the-shelf visualization surface they wouldn’t have to rebuild every year
• Deep ROS and robotics support, with modern web-based collaboration
• The ability to customize and extend behavior without owning all the visualization infrastructure themselves

Operating and monitoring live systems
Scout AI relies on Foxglove as the primary interface for running its robots. A custom Foxglove extension allows operators and engineers to view system state, monitor fleet status, start and stop logging, and trigger missions, both from a tablet at the Forge training center, or anywhere in the world remotely. Live WebSocket connections stream telemetry, video, and commands in real time.

Foxglove also serves as Scout AI’s fleet operations console. During data-collection runs, test flights, and field demonstrations, operations staff and engineers share the same layouts including control plots, system health, and 3D state, giving teams a unified view across air and ground systems.

Debugging through logging and replay
Every mission is logged onboard and uploaded to an S3 bucket. For investigations, engineers slice logs, convert them to MCAP, and replay them directly through Foxglove. This lets them inspect synchronized video, telemetry, controls, trajectories, and model outputs without recreating scenarios in the field.

Collecting training data with custom operator panels
Scout AI built a specialized panel in Foxglove to drive its data-collection workflows. Operators issue natural-language instructions or demonstrations and drop waypoints on a map; Foxglove packages the instruction, map context, and sensor streams into a prompt bundle that is logged for training.

Operators can also stream microphone audio through Foxglove. The audio is logged, transcribed, and incorporated into model training, capturing “stream-of-consciousness” human reasoning paired with robot behavior.

Annotation workflows and training-data quality
AI technicians and operators use Foxglove to mark start/stop times, edit descriptions, and verify collected data. The AI team monitors data quality through the same interface, allowing them to focus on model architecture while maintaining high-integrity training datasets.

Demos for customers and stakeholders
Scout AI's everyday interface also functions as its demo platform. With Foxglove, teams can show live missions to defense customers, OEM partners, investors, and congressional staff, providing a clear, intuitive view of real-world autonomous operations.

Looking ahead

Scout AI is working toward one million agents running Fury across air, land, and sea. As they scale the diversity of platforms will grow, the complexity of multi-agent coordination will increase, and the importance of fast, reliable, shared observability will only become more critical

Foxglove is the backbone of how Scout AI operates, observes, and iterates on its systems today — from Forge’s training missions to remote operations and AI research. As they expand their fleet and deployments, Foxglove looks forward to continuing being the chosen tool to close the loop between data, models, and real-world autonomy.

‍

‍