How we built Foxglove Automations, and what comes next
Bringing AI-driven tooling closer to your robotics data
Foxglove Automations run pre-defined agent prompts on your data in response to triggers like imported recordings, updated events, scheduled jobs, API calls, and Slack requests. They work with Foxglove’s powerful and ever-expanding suite of tools, sitting in the same infrastructure as your data. They can review your data, annotate with typed events, add episodes to Datasets, and interact with external services to file tickets and upload reports where your team works.
Automations are available on Foxglove-hosted and Bring Your Own Storage (BYOS) projects, on Pro and Enterprise plans.
A motivating example
An engineer is collecting instances of “missed grasps” for a robotic arm in order to improve their control system. The arm has sensors which detect when the fingers are all the way closed, and when this happens while trying to grab something it’s a reasonable signal. However, some of these may be false positives – for example when picking up a piece of paper or cloth – and would need to be carefully filtered out using a variety of signals from the logs.
In a separate technical walkthrough, we illustrate in detail how Automations can process these recordings and events as they’re imported into Foxglove. There, we define an “annotator” Automation which annotates events when it detects missed grasps. These “missed grasp” events in turn kick off a second “reviewer” Automation that looks at the camera views and gripper telemetry around the missed grasp and files an issue with evidence attached and an initial analysis of the problem.
According to the gripper signal, the fingers closed all the way, indicating a possible miss. However upon reviewing the image data, gripper signal, and control system state together the agent realized that the gripper had in fact closed on something thin and updated the ticket with that new information. This determination is not something that can be easily expressed as a deterministic evaluation, and is better expressed in natural language so the AI can decide for itself based on context whether the miss was real or not.
Evolution of the product concept
Automations grew out of a problem we kept hearing from robotics teams: their robots generate more data than anyone can fully review. Triage queues keep growing, most of what gets flagged turns out to be a false positive, and hiring more reviewers doesn’t scale.
The underlying use case is not actually new. Foxglove’s great strength has always been that it makes triaging robot data much faster and more insightful, and this has always helped teams build better robots. That said, someone still has to do the triage, and fleets of modern robots now generate more data than any team of humans can hope to review. AI is starting to change that. Agents can take the first pass through new data, and engineers can spend their time reviewing what the agents find. So to start, we planned on building a triage product with carefully managed decision trees, image-model checks, and a review inbox.
Meanwhile, our design partners were describing similar needs in other parts of their work, outside of triage. There was often a queue, a repeatable decision or analysis task, and a tracking system like Jira that lives outside Foxglove. We saw this pattern in detection on ingest, enriching tickets with evidence, repeatable fleet digests, and gates that accept or reject episodes for ML training datasets. A triage-specific system would not have served these scenarios well, so we generalized to a prompt-and-trigger mechanism which can support a wider range of automated tasks, including triage. We still plan to build improved tooling and functionality to better support each individual use case, but we wanted to start with a broader foundation.
Starting with the fundamentals
Earlier this year we released the AI agent sidebar and began instrumenting Foxglove with MCP tooling for agentic use. In broad strokes, Automation runs are implemented as chats without a human in the loop, using a harness and system prompt that is mostly shared with the agent sidebar. On top of this we added triggers that kick off automation runs, and input prompts that take context from the triggers. We also built tooling specifically for Automations, including a headless Foxglove visualizer that runs in a sandboxed VM so the agent can render layouts and screenshot panels to check its work. Outputs are primarily handled as side effects such as creating events, drafting Datasets, and making updates in external systems such as Jira and Linear. Automations can both generate open-ended reports and make decisions and categorizations.
Like the AI agent sidebar, automations have as-needed access to the same powerful suite of tools that a robotics engineer using Foxglove has access to. They can read data, interact with events, run user scripts, and use headless instances of the Foxglove app for deep analysis. An Automation run performs actions as its owner, in one project, right beside the data on Foxglove-hosted and BYOS projects. Runs are billed as normal AI usage, and run data is processed by Anthropic under our terms of service.
Creating Automations today
As with any LLM prompt, writing an effective Automation is a lot like creating a guide for a new teammate. The clearer the instructions and the smaller the steps, the better the results.
Some tips:
- It helps to tell the agent what it’s looking at. Topic names rarely explain themselves, so a sentence or two about which topics and fields matter (and what normal looks like) goes a long way. You can load details from external MCP services to help populate this kind of context. The less the agent has to re-discover each run, the faster and cheaper each run will be.
- Prepare a reference layout with appropriate user scripts and panels supporting the task, and mention the layout in the prompt. For example, if your answer relies on a rolling windowed average, compute this as a derived topic in a layout so the agent can read the answer deterministically. If you want the agent to interpret particular camera angles or 3D views, configure those panels in advance for the task at hand.
- Narrowly focused Automations do better than broad, ambitious ones. Decompose complex tasks so each step answers one clear and well-defined question. When a job has several steps, chain Automations with each other. For example, a first Automation might flag candidate events, and a second verify them.
- Add screenshots and images to help the agent distinguish common patterns.
We recommend using built-in AI agent sidebar or your favorite MCP client to improve your prompt and reference layout. After a few runs, ask in the agent sidebar to review what the agent did and suggest improvements to the prompt, like which topics to check first or what thresholds matter. Try to clearly describe what a positive and a negative case look like. Visualize the data. Get the task to work on a single instance, and then run it against known positive and negative examples to see how it generalizes. Have your AI client read past runs. It can point out where the agent struggled and what additional context to add so the agent doesn’t have to search for it.
Where things are headed
Modern LLMs allow you to articulate sophisticated policies about how to interpret real-world data using natural language, making them remarkably expressive and flexible. That said, running raw data directly through the models is not an effective or efficient way to process the data. We already provide a wide range of deterministic tools to the automation, and over time we plan to progressively push more work into sandboxed code, richer MCP tools, FoxQL queries, and sub-agents running smaller decision models so that your automations will get cheaper, faster, and more repeatable while retaining the same expressive power.
We’d love to hear how Automations is working for you, and how we can make it better. Start with the technical walkthrough or the Automations docs, and join our community to share feedback.


