
As an embedded software engineer at the [Woods Hole Oceanographic Institution (WHOI)](https://whoi.edu/), Ryan Govostes helps researchers across the institution on R&D projects capture and interpret scientific data. He stumbled upon Foxglove Studio earlier this year in his quest for flexible visualizations, and has been using the tool ever since to accelerate development for his teammates – whether that’s by building them custom tools or surfacing meaningful patterns in complex data. We discuss the exciting projects he’s contributed to, and what lies on the horizon for him.

**Tell us a little bit about yourself!** **How did you first become interested in robotics?**

My background is actually in information security and has nothing to do with robotics. I spent the first part of my career in Silicon Valley, focusing on developing technologies to defend software against hackers.

A few years ago, I visited the Computer History Museum to attend a symposium on the Antikythera Mechanism, this amazing ancient analog computer that was discovered on a shipwreck in the Aegean Sea. One of the presenters was Brendan Foley, an archaeologist then at WHOI, who was involved in a new survey of the shipwreck. His team was using underwater robots to search for artifacts missed by Jacques Cousteau decades earlier. I just thought it was the coolest thing I’d ever heard of.

What really stuck with me [from this presentation](https://www.youtube.com/watch?v=bYxwnQZndTM&t=1448s) was that Brendan estimates there are 750,000 ancient shipwrecks sitting at the bottom of the Mediterranean – the number we’ve found is a minute fraction of that. I joined WHOI a few years later, hoping to develop technologies for making the next discoveries at scale.

**What do you find compelling about leveraging robots specifically for ocean exploration?**

I’d never really thought about how little of the ocean we have explored below the surface. There are so many constraints with human exploration – like how do you safely investigate what’s happening to the ocean during a hurricane? Or monitor a site over years to understand the effects of climate change?

Robots have plenty of their own challenges, but they have amazing capabilities. For example, we have gliders that can collect and transmit data back to shore via satellite for six months at a time. They are completely game-changing for our understanding of the ocean.

![Remus 6000](/images/blog/spotlight-ryan-govostes-software-engineer-at-WHOI/remus.jpg)

_The REMUS 6000 can go to depths of 6,000 meters and spend nearly a full day exploring. (© Woods Hole Oceanographic Institution, M. Purcell)_

**What do you find especially exciting about your work? How would you characterize its impact?**

It’s exciting to be building technologies for exploring new frontiers. The upgraded [HOV _Alvin_](https://www.whoi.edu/what-we-do/explore/underwater-vehicles/hov-alvin) will allow scientists to study 98% of the ocean floor in person.Some researchers are even thinking about using robots to explore oceans in other worlds, like Jupiter’s moon Europa.

![HOV Alvin](/images/blog/spotlight-ryan-govostes-software-engineer-at-WHOI/alvin.jpg)

_The HOV_ Alvin _will be going out for sea trials at depths of 6,500m later this year. (© Woods Hole Oceanographic Institution, L. Lamar)_

There’s an incredible urgency to understanding how a warming ocean will affect life on Earth, both above and below the surface. We are in the first year of the [United Nations Decade of Ocean Science for Sustainable Development](https://www.oceandecade.org), a concerted effort among scientists, policy makers, and consumers to ensure the preservation of our oceans for future generations. It’s an especially important time for software engineers to contribute to ocean science – to data pipelines, automation, machine learning, and so on.

**What does a typical day for you look like? What projects are you working on now, and what are some of the interesting development challenges you’ve encountered?**

There’s something new to learn every day. Since I lend programming expertise to researchers across the Institution, I might be collaborating with a biologist to monitor for harmful algal blooms one week, then calibrating a camera system for an underwater vehicle the next. I’ve mostly gotten over my reluctance to ask incredibly basic questions – the other day I asked why an autonomous underwater vehicle (AUV) was so heavy, forgetting that underwater vehicles aren’t supposed to float.

Most recently, I’ve been working with a team on a new propulsion technology called [Asymmetric Propulsion](https://armadamarinerobotics.com/technology/). Traditionally, an AUV has a propeller to generate thrust and some other mechanism like fins to control steering. But by varying the velocity of the propeller over the course of a rotation, my collaborators realized they could generate thrust and steer with a single propeller blade – similar to how you can steer a kayak by paddling harder to one side. This enables vehicles that are more maneuverable and reliable, with fewer points of failure.

![Asymmetric Propulsion](/images/blog/spotlight-ryan-govostes-software-engineer-at-WHOI/asymmetric-propulsion.png)

_An AUV executing a turn to port with a single propeller blade, using Asymmetric Propulsion. (© Woods Hole Oceanographic Institution, J. Kaeli, R. Littlefield, F. Jaffre)_

This was my first major robotics project – and one of my first times using ROS – so there were plenty of challenges. In ROS, I think there’s an illusion that it’s easy to develop a robot, because you can choose from a large number of available packages and just glue components together. In reality, I needed to spend time becoming familiar with each package and understanding the benefits or drawbacks it could introduce to the finished system.

Meanwhile, my collaborators were eager to test and study the propulsion system. I had to be very strategic about the order in which I wrote the code, so I didn’t disrupt their work. One of the first things I did was jury-rig a tool using the web browser Gamepad API, plus a couple of socat processes to relay commands to the motor controller over Wi-Fi. It allowed my teammates to drive the vehicle around on a test tank, while I developed target-tracking capabilities. To learn more about this whole software development process, you can check out our talk for the [OCEANS 2021 conference](https://global21.oceansconference.org/) this week.

![motor controller](/images/blog/spotlight-ryan-govostes-software-engineer-at-WHOI/motor-controller.png)
_The superimposed red line above shows the vehicle's position over time, as researchers navigate it around a test tank. (© Woods Hole Oceanographic Institution, R. Govostes)_

**It seems like you support a pretty wide range of workflows for different teams across the institution. What kind of tooling do you use when developing solutions for these various workflows?**

There’s a terrific talk by Bret Victor called “[Media For Thinking the Unthinkable](http://worrydream.com/MediaForThinkingTheUnthinkable/)”, where he argues that the representation of a complex system is how we think about that system – to build powerful new systems, we need powerful new representations. With the appropriate visualizations, you can recognize patterns and understand underlying relationships that reveal more about the larger system.

This is what really excited me about using Foxglove Studio to help our team understand Asymmetric Propulsion, where we have this non-obvious relationship between the propeller’s angular velocity and the AUV’s steering behavior. Everyone on the team comes from a different discipline and has different areas of expertise. We have a brilliant mechanical engineer who designed this AUV and has gotten its buoyancy and trim perfectly dialed in, but it’s not his job to know arcane ROS command line invocations or to write code to plot gyroscope data. Having an intuitive, flexible tool in Foxglove Studio has really empowered him and the rest of the team.

I also love that Studio is open source and allows us to write custom panels specific to our project. There are so many excellent JavaScript visualization libraries out there, like [D3.js](https://d3js.org/) – having an extension API lets me leverage any of these tools to accomplish my task. I recently hosted a workshop at WHOI’s new innovation accelerator for Autonomous Vehicles and Sensor Technologies (AVAST), where we took an example polar plot and turned it into a Studio extension showing live ROS data in under an hour.

**What do you have planned next?**

I’m really excited about supporting the Scibotics Lab at WHOI with developing a [long-range autonomous underwater vehicle (LRAUV) for under-ice oil spill detection and mapping](https://www.whoi.edu/oceanus/feature/lrauv-arctic-oil-spill-response/), in collaboration with the Monterey Bay Aquarium Research Institute (MBARI) and the Arctic Domain Awareness Center. The goal is to be able to quickly deploy the LRAUV at the site of an environmental disaster to survey and help mitigate its impact.

![LRAUV](/images/blog/spotlight-ryan-govostes-software-engineer-at-WHOI/lrauv.jpg)

_The LRAUV can be deployed from a helicopter to provide first responders with crucial situational awareness to manage an environmental crisis. (© Woods Hole Oceanographic Institution, S. P. Whelan)_

The vehicle is a remarkable piece of engineering that has tremendous potential in enabling new scientific missions. It combines elements of traditional AUV propulsion with the greater endurance of gliders. We’ve started integrating ROS on the vehicle, so we can share software capabilities across the Institution’s heterogeneous fleet of autonomous vehicles.

**What advice would you share to someone who's interested in getting into robotics, either as a hobbyist or a professional contributor like yourself?**

I admittedly took a pretty unusual path into robotics, and believe there is a lot of value in being multidisciplinary and drawing inspiration from other fields, even if they seem superficially unrelated.

One of my favorite examples of this comes from an incredible team at WHOI called the WARPLab that is using topic modeling techniques to inform exploration. Topic modeling is typically associated with natural language processing – it’s often used to categorize bodies of text by subject matter without explicitly knowing the subjects ahead of time.

This team is [applying this idea to camera images](http://warp.whoi.edu/iros2018/), so their AUV can differentiate patches of seagrass from, say, a sandy region it saw a moment ago. As such, it can recognize when it has encountered something novel and decide to investigate.

This is all to say that you may be surprised how often what you already know translates to a new field. It’s very possible you have already developed skills that could be usefully applied to robotics.

_Hero image © Woods Hole Oceanographic Institution, R. Govostes_



Earlier this year, we announced support for [playing back ROS 2 bags](/blog/announcing-ros2-bag-support) in Foxglove Studio. We were thrilled to see the community's enthusiasm for broader ROS compatibility, so in the weeks since that release, we've been working hard to expand Studio's ROS 2 support.

Today, we're excited to announce that [v0.18](https://github.com/foxglove/studio/releases/tag/v0.18.0) of Studio allows you to **connect to a live ROS 2 stack** with two new data source options – `ROS 2` (beta) and `ROS 2 Rosbridge`. Whether you're running ROS on your local machine or a physical robot, you can now connect directly to your ROS 2 nodes to visualize their data in real-time.

# Share your feedback

While we're approaching feature parity with ROS 1, our long-term vision for ROS 2 compatibility is still evolving. Beyond the preliminary support in this release, we're still working with the larger ROS community to bridge the gap between what we can achieve now and the upstream work that remains (like message definitions [at runtime](https://github.com/ros2/ros2/issues/1159) and [in bag files](https://github.com/ros2/rosbag2/issues/782)).

With that said, we're excited for you to test this feature out in the field, and to give us feedback on how we can improve and expand on it. Feel free to share your thoughts in [Slack](/join-slack), file bug reports and feature requests on [GitHub](https://github.com/foxglove/studio), or message us directly on [Twitter](https://twitter.com/foxglovedev).

If you experience any issues with native ROS 2 connections, you can always fall back to our more battle-tested `ROS 2 Rosbridge` connection option. Check out [our docs](/docs/app-concepts/data-sources) for more details on how to connect to all our supported data sources.



For today's spotlight, **we invited Aditya Kamath, a robotics systems engineer, to talk to us about his latest project – a robot platform to help the next generation of roboticists learn ROS**. As someone who’s spent years tinkering with prototyping tools, computers, and electronics, Aditya had an obvious solution for countering his lockdown boredom – building (multiple!) robots from scratch. We discuss the exciting milestones he’s reached, and his future plans to help aspiring roboticists follow in his footsteps.

## Our team first stumbled upon your work when we saw you [using Foxglove Studio](https://twitter.com/kamathsblog/status/1403717948582379525) to visualize your robots. Tell us a little bit about yourself, and how you first became interested in robotics!

**I am Aditya Kamath, an Embedded / Robotics Systems Engineer based in Eindhoven, the Netherlands.** I currently work as an Embedded Systems Consultant, designing embedded software for complex machines and mechatronic components. I keep my passion for robotics alive by tinkering around with robotic builds, microcontrollers / single-board computers, and rapid prototyping tools like 3D printing.

![Aditya Kamath](/images/blog/spotlight-aditya-kamath-robotics-systems-engineer-on-akros/profile.jpeg)

I earned my Bachelor’s in Electronics and Communication Engineering from MIT Manipal in India (2014). Then, I received a Master of Science (MSc) in Systems and Control and a Professional Doctorate in Engineering (PDEng) in Mechatronic Systems Design from TU Eindhoven (2019). **Despite studying many different things, the common denominator in my education and extracurricular experiences has always been the field of autonomous robotics.**

## What exciting robotics projects are you working on now?

With all my spare time during the COVID-19 lockdown last year, I decided that I wanted to build some robots to experiment with ROS. I also saw this as an opportunity to learn more about 3D SLAM and depth and point cloud processing. **The first several months, I focused on building a few Jetson Nano-based ROS robots using some rapid prototyping tools.** For one, I used the [NVidia JetRacer](https://github.com/NVIDIA-AI-IOT/jetracer), an autonomous AI racecar with Ackermann steering, with custom add-ons. For another, I used the [NVidia JetBot](https://github.com/NVIDIA-AI-IOT/jetbot), an open source robot with differential drive geared towards educational purposes, again with custom add-ons.

Then at the start of this year, I was able to get a LIDAR and an [OAK—D camera](https://store.opencv.ai/products/oak-d). With these exciting new supplies, **I decided to use everything I had learned the previous months to design and implement my own ROS 1 robot platform from scratch**. I named the project AKROS, a working name that combines my initials and ROS – for until I come up with something more original.

![robot with LIDAR and camera](/images/blog/spotlight-aditya-kamath-robotics-systems-engineer-on-akros/with-lidar-and-camera.png)
_The robot with the new LIDAR and OAK—D camera attached._

All the software for AKROS ([ROS Noetic](http://wiki.ros.org/noetic)) runs on a [Raspberry Pi 4](https://www.raspberrypi.org/products/raspberry-pi-4-model-b/). It has a mobile base with omni wheels and four-wheel drive, with each wheel powered by a motor that computes the robot's odometry using encoders. The mounted [RPLIDAR A2](https://www.slamtec.com/en/Lidar/A2) provides a 2D laser scan, while the OAK—D camera provides RGB images, stereo depth images, and spatial object detection outputs using the on-board [DepthAI](https://docs.luxonis.com/en/latest/) module. The [Intel Realsense T265 tracking camera](https://www.intelrealsense.com/tracking-camera-t265/) provides simultaneous localization and mapping.

## What development tools do you use for iterating on AKROS?

I do all my development using [JupyterLab](https://jupyter.org/) – the server runs on the Raspberry Pi, and I open my workspace from any browser on the network. However, this method limits you to the terminal, so for tools like [RViz](http://wiki.ros.org/rviz), I have to install and configure ROS on my Windows laptop. **After that, I still have to go through three to four additional steps to launch RViz, which unfortunately cannot run directly on Windows.** On top of all this, it's easy to forget setting the necessary environment variables to connect to the remote ROS master. The whole process is very annoying and manual.

**Once I started playing around with Foxglove Studio, I thought it was a really convenient way to visualize ROS topics and messages from a remote desktop.** I love that it works on Windows, because that means I don’t have to go through the process of launching ROS from Windows, setting environment variables, and finally launching RViz. Foxglove takes care of all this for me!

<a href="https://www.youtube.com/watch?v=J-kTdkJawAM" target="_blank">
    <img alt="Using Foxglove Studio" src="/images/blog/spotlight-aditya-kamath-robotics-systems-engineer-on-akros/foxglove-demo.png"/>
</a>

_A demo of AKROS's SLAM and object detection capabilities, visualized in Foxglove Studio._

**Using Studio on Windows is as easy as using Rviz on a Ubuntu laptop.** Now, I can run my ROS launch files from the JupterLab terminal in my browser, then open up Foxglove Studio to start visualizing everything I want – no more complex setup process.

## We're so glad to see that Studio has been able to streamline your development process. What do you have planned next on your roadmap?

While this robot fulfills my personal learning goals, **I want to take AKROS and develop it into a product that could be used to teach robotics or to perform research in applications like control systems, computer vision, and AI.** In fact, I am currently in talks with university students and professors to understand their requirements for a ROS robot platform. Eventually, I want to design a polished product for these end-users, instead of a hacky prototype for just myself.

At this point of time, I am still in the development process. **I have two main short-term goals – to set up the ROS navigation stack** with the RPLIDAR’s laser scans and Intel Realsense tracking camera’s odometry, **and to set up 3D mapping** with the OAK-D camera. It's been an iterative process – I recently had to [completely redesign my navigation module](https://adityakamath.github.io/2021-08-01-navigation-module-design-update/) to accommodate my tracking camera.

![unmounted and mounted navigation module](/images/blog/spotlight-aditya-kamath-robotics-systems-engineer-on-akros/unmounted-and-mounted-navigation.png)
_Aditya spent quite some time iterating on his navigation module - adding components, fine-tuning its assembly, redesigning the frame - before mounting it on the rest of his robot._

## Do you have any advice for someone who's interested in getting into robotics, either as a hobbyist or startup founder?

**The best experience you can get is by building your own robot –  it’s also much more fun than doing an online course or working with a simulation.** There are lots of (free and open source) resources out there that will teach you the fundamentals step by step, and cheap robot kits are abundant online. By building your own physical robot, you learn a lot more about robotics than just how to program.

<hr/>

To keep up with Aditya's latest work, check out his project updates on his [Twitter](https://twitter.com/kamathsblog), [Instagram](https://www.instagram.com/kamathsblog/), and [blog](https://adityakamath.github.io/).

If you're a university professor or student interested in collaborating with Aditya for his AKROS project, you can get in touch with him directly on [Twitter](https://twitter.com/kamathsblog).

_This interview has been edited and condensed for clarity._



**We sat down with Steve Crowe, the editorial director of [The Robot Report](https://www.therobotreport.com/), to pick his brain on the state of robotics.** As someone who’s covered the industry for about a decade, Steve has seen a lot of ideas (and companies) come and go. We discuss the major industry trends he’s reported on over the last decade, and where he sees the field going in the next ten years.

## As the editorial director at The Robot Report, you write about the business and applications of robotics. Can you tell us a little bit about what sparked your initial foray into the world of robotics?

I didn't go to school for robotics or have any interest in the industry before my career path steered me in that direction. I’m a trained journalist, and the publishing company I was working at happened to own a robotics brand, [Robotics Business Review](https://www.roboticsbusinessreview.com/). After several years of building out another brand, I was asked to steer that ship to accelerate its online growth. Then in early 2018, my colleague Dan Kara reached out to me as he was growing the robotics editorial team at WTWH Media – that’s how I got started working on [The Robot Report](https://www.therobotreport.com/).

It’s now been about a decade since I started covering robotics – it’s a ton of fun. I'm located in Boston, which is certainly in the top three robotics clusters in the world, along with California and Austin. It's a great community, a pretty small community of folks.

## What does a typical day for you look like?

**If I have to summarize what I do, it's educating the robotics community on how to create better, more intelligent systems.** Whatever the medium is, whatever the format of the content is, we're trying to build a relationship with stakeholders in the robotics community and gain their trust. We show them that they can come to us to learn about the innovations driving the industry forward. They can come to our events to network with each other, or to learn about the products that will help them create the next generation of commercial robotics systems.

As far as what my typical day looks like, it depends. This week we announced [the winners for our annual RBR50 Robotics Innovation Awards](https://www.roboticsbusinessreview.com/2021-rbr50-robotics-innovation-award-honorees/). I also moderated a webinar with Rich Mahoney, the former director of [SRI International](https://www.sri.com/) and current VP of Research at [Intuitive](https://www.intuitive.com/), the world’s leading robotic assisted surgical company.

This is on top of producing engaging content across our 4 websites – we’ve built a diverse portfolio and grown the group pretty significantly. Our flagship brand [The Robot Report](https://www.therobotreport.com/) covers engineering news, emerging technologies, and investments and acquisitions. [Collaborative Robotics Trends](https://www.cobottrends.com/) teaches companies about the benefits of collaborative robotic arms. [Robotics Business Review](https://www.roboticsbusinessreview.com/), the original robotics brand I’d worked on, provides breaking news and informed analysis on the robotics and automation sectors. And finally, [Mobile Robot Guide](https://mobilerobotguide.com/) educates companies about the benefits of autonomous mobile robots (AMRs), automated guided vehicles (AGVs), and other mobile robots in logistics and supply chain environments.

## How have you seen the industry evolve in the last several years? Are there any particularly interesting industry trends that you’ve seen emerge?

**For so long, robotics referred to the industrial robot arms at manufacturing plants** – particularly automotive plants. As enabling technologies came down in price and industries were educated about the benefits of adopting these systems, **robotics has expanded out of those plants and research labs into a lot of different areas**.

One industry that’s attracting more players is food services. [Miso Robotics](https://misorobotics.com/), for example, is automating kitchen operations within fast food restaurants. With COVID and other airborne and foodborne illnesses, you’re seeing more companies trying to make the food preparation process safer.

<img src="/images/blog/spotlight-steve-crowe-editorial-director-of-the-robot-report/flippy.jpeg" alt="Miso Robotics' Flippy robot flips burgers" />

_Miso Robotics' Flippy is an AI-powered kitchen assistant boosting performance and safety in restaurants._

**Construction and agriculture are two markets that have often been talked about as being ripe for automation, and now it seems to be happening.** Several companies have been developing picking robots to help farms with the harvesting process. They’ve built pretty sophisticated AI that uses vision algorithms to understand when different types of produce are ready to be picked, and what exact mechanism is needed to harvest them.

**E-commerce is another area where robotic systems are currently gaining a lot of traction.** Mobile robots that transport items throughout a facility are one of the hottest markets right now. It was already growing at a pretty good rate before 2020, but the pandemic introduced more people to online shopping. The only way for many e-commerce players to keep pace with the demand was to automate parts of their operation. Amazon’s own AMRs, for example, have been vital in helping them keep up with the demand.

## What do you see as the industry’s biggest technical challenge?

**I'm pretty excited to see how the legged robotics market develops** – [Boston Dynamics](https://www.bostondynamics.com/) and [ANYbotics](https://www.anybotics.com/) are interesting ones to watch. There's been a lot of skepticism about the benefits of legged robots – they’ve been under research and development for decades, but nobody ever really thought that they were commercially viable or that there was a compelling use case for them. But within the last year, several quadruped robots have been commercialized, the most well known one being Boston Dynamic’s [Spot](https://www.bostondynamics.com/spot). Humans have teleoperated this system to inspect sites around the Chernobyl nuclear power plant disaster for radiation. It can even be set up to perform autonomous missions to keep humans out of harm's way – a great use case for robotics.

<img src="/images/blog/spotlight-steve-crowe-editorial-director-of-the-robot-report/spot.jpeg" alt="Boston Dynamics' Spot robot out in the field" />

_Boston Dynamics' Spot allows users to automate routine inspection tasks and capture data safely and accurately._

As for bipedal robots, [Agility Robotics](https://www.agilityrobotics.com/) has a fascinating one named Digit. They've studied animals – in particular, ostriches – to inform their locomotion design and planning algorithms. With Digit, they're targeting not last mile delivery, but something like last hundred feet delivery. They've done some crazy tests with partners like Ford where these robots get out of the car, deliver a package to a door, and then reload themselves into the car.

<img src="/images/blog/spotlight-steve-crowe-editorial-director-of-the-robot-report/digit.jpeg" alt="Agility Robotics' Digit robot unloads a package to deliver" />

_Agility Robotics has partnered with Ford to test doorstep deliveries with its bipedal robot Digit._

**These robots have very sophisticated locomotion capabilities** – they can climb stairs and navigate over all sorts of terrain – **giving them a major advantage over wheeled robots**. The construction, oil and gas, mining industries are all trying to figure out how to put these platforms to use. We’ll have to wait and see whether they prove themselves useful outside the lab.

## What are the major business hurdles that robotics startups face today?

**A lot of this stuff is hardware-centric – it just takes so much capital to develop these systems that don't have a tremendous amount of customers**. For example, [Locus Robotics](https://locusrobotics.com/) is an AMR company that’s been around for about 7 years and recently raised $150 million in Series E funding. They’ve said that their next step is to IPO, but here’s the thing – they have roughly 40 actual customers. It takes an incredible amount of time and money to develop these things, to hire the incredibly smart and hardworking people to iterate on these systems. Building robots and gaining any traction is incredibly challenging.

<img src="/images/blog/spotlight-steve-crowe-editorial-director-of-the-robot-report/locusbots.jpeg" alt="Locus Robotics' AMRs navigate a warehouse" />

_Locus Robotics offer automated warehouse robots that increase productivity and order accuracy._

**Interoperability is another hurdle that's talked about a lot now.** Mobile robots developed by different companies often operate in roughly the same way, but they all have their own proprietary communication systems and operating systems. So **there's an emerging problem with adopting heterogeneous fleets of robots from different manufacturers**, since these robots must operate together in the same environment, but can’t communicate with each other on their own. If you have robots from 4 different vendors that you want to reprogram in any way, there’s a good chance you have to learn 4 different operating systems. It makes the whole operation much more challenging for the end user.

Thankfully, **there's a lot of work being done around setting a standardized communication protocol**. For example, [READY Robotics](https://www.ready-robotics.com/) is creating a universal operating system so that users only have to learn one approach, regardless of how many different robotics vendors they choose. [Open Robotics](https://www.openrobotics.org/), the group behind the [Robot Operating System](https://www.ros.org/), has been collaborating with a hospital network in Singapore to develop a fleet management system that allows their healthcare robots communicate with each other. These robots were eventually able to recognize each other’s movements as they navigated hallway traffic and called elevators to travel between floors.

<img src="/images/blog/spotlight-steve-crowe-editorial-director-of-the-robot-report/romi-h.png" alt="A robot navigates a hospital with both humans and other robots" />

_Open Robotics has been collaborating with Singapore’s Centre for Healthcare Assistive and Robotics Technology (CHART) to develop an open source infrastructure framework that simplifies cross-fleet collaboration._

## Have there been any sectors of the industry that haven’t been able to live up to the hype?

**There hasn’t been much progress in the consumer market.** iRobot was very successful with its robot vacuums, but there really hasn't been much progress with other robots for your home. iRobot had been talking about a robot lawnmower for over a decade, and they finally teased it last year. But they eventually shelved it, [citing fallout from the COVID-19 pandemic](https://techcrunch.com/2020/04/28/irobots-terra-lawnmower-will-not-launch-in-2020-as-company-reprioritizes/). Social robots have often been hyped as the next in-home consumer robot, but it's been a disaster of an industry with a few high-profile failures.

One consumer robot company that does seem to be having some success is [Embodied](https://embodied.com/). Their Moxie robot helps young kids build social-emotional skills, by recognizing what a particular child is struggling with and then serving up content that might be beneficial to that child. I think they’ve been successful, because they weren't trying to be an all consumer companion robot – they honed in on one specific application.

<img src="/images/blog/spotlight-steve-crowe-editorial-director-of-the-robot-report/moxie.jpeg" alt="Embodied's Moxie robot helps children ages 5-10 build social-emotional skills" />

_Moxie uses play-based activites created by child development experts to promote children's mental well-being._

## What are some of your favorite robotics projects out there in the wild today?

**I love self-driving cars.** I love following [Cruise](http://getcruise.com/), [Waymo](https://waymo.com/), and the other leading autonomous vehicle developers. It's just a fascinating concept and a super challenging problem from an engineering standpoint. There are so many people who are killed in careless car accidents, so **if these systems could be scaled and deployed commercially all over the world, that would be tremendously valuable to society**.

There are also so many applications for an autonomous vehicle outside of a robo taxi. I've seen use cases where autonomous luggage cart vehicles operate on fixed repeatable routes – at a certain terminal, multiple times a day. There’s opportunity there.

<img src="/images/blog/spotlight-steve-crowe-editorial-director-of-the-robot-report/ba-luggage-carts.jpeg" alt="British Airways started trials of autonomous luggage carts in 2019" />

_British Airways started trials of driverless luggage carts at Heathrow Airport in 2019 to speed up loading carts._

And I’ve said it before, but **the realm of legged robotics is super exciting as well**. I'm sure you’ve seen the viral videos of Boston Dynamics’ two-legged robot doing [gymnastics routines](https://www.youtube.com/watch?v=YdnJI9T-yXI) and going for a jog. But we have to see how that progresses. That's what gets me fired up – seeing these challenges that are still left for us to solve.

## What advice do you have for someone who's interested in entering the robotics field, whether as an entrepreneur, engineer, or investor?

**If you’re starting a robotics startup – don't create a solution that's looking for a problem**. Don't just go build a supply chain robot because you know that’s a really hot market right now. Find a problem that has yet to be solved or automated, and figure out how to do that. So many companies we've reported on have gone out of business, because there was never a great use case for what they built. They took a variety of enabling technologies that had improved and come down in price over the years, and fused them together. But the problem it was solving was never really there. **They were so concerned about building the coolest product with the coolest tech that they forgot about their end user.**

**The startups that are still around and getting customers and funding have that one thing in common – they really understand the problem that their user is trying to solve.** An example is [6 River Systems](https://6river.com/), which was acquired by Shopify for almost half a billion dollars. Its executives may have had some robotics knowledge, but their expertise really lay in the logistics business. They had great firsthand knowledge of the challenges that supply chain operators face on an everyday basis. They could think clearly about the tools and system that they could develop to help these operators overcome those challenges.

<img src="/images/blog/spotlight-steve-crowe-editorial-director-of-the-robot-report/chuck.jpeg" alt="6 River System's Chuck promises faster and safer fulfillment solutions" />

_6 River System's Chuck uses machine learning and artificial intelligence to fulfill orders more efficiently._

Also, talk to as many people as you can. **Find people who have the domain knowledge you want to learn, and pick their brains** – that's really the key. It’s a pretty small community, and most people that I've met in my 10 years in the industry are super passionate about what they do. They’d love to talk to you about what they're working on, or about the business and technological challenges they face on a regular basis. **They want nothing but to move this industry forward – to get more robots out in the world helping people overcome daily challenges.**

A lot of the general public is afraid of these systems, as they start to appear out there in the real world. Sci-fi over the years has probably done a major disservice to the industry. But so many folks in the industry want to reverse that misconception and educate people about how these systems actually work and how they can help us tackle daily challenges.

<hr/>

To keep up with Steve’s latest work, you can follow him on [Twitter](https://twitter.com/stevecrowe) or read his pieces on [The Robot Report](https://www.therobotreport.com/).

_This interview has been edited and condensed for clarity._



Today, we're excited to announce that **Foxglove Studio now supports loading ROS 2 bags for playback and visualization**. With [v0.13.0](https://github.com/foxglove/studio/releases), a new data source option (`ROS 2 Bag Folder (local)`) will be available in the sidebar’s _Connection_ tab. With this option, you can load [rosbag2](https://github.com/ros2/rosbag2) files for playback.

# Current limitations & future plans

While we’re excited for ROS 2 to reach true feature parity with ROS 1 support in Studio, it’s important to note several important differences in the features as they stand today.

For one, since ROS 2 bags currently do not contain the [message definitions](http://docs.ros.org/en/rolling/Concepts/About-ROS-Interfaces.html?highlight=idl#message-description-specification) that were used to generate the bag files, **Studio is currently unable to load custom message definitions**. However, it will still support standard ROS message definitions (e.g. [std_msgs/ColorRGBA](http://docs.ros.org/en/api/std_msgs/html/msg/ColorRGBA.html), [geometry_msgs/Pose](http://docs.ros.org/en/api/geometry_msgs/html/msg/Pose.html), etc.) as it does today. With that said, **the Foxglove team is [working closely with ROS developers](https://github.com/ros2/rosbag2/issues/782) to ensure that future ROS 2 bags will include the message definitions used to generate them**. This will allow third-party tools like Studio to decode and inspect ROS 2 data.

**Studio also does not yet support preloading ROS 2 data**. After initializing a data source connection, Studio typically preloads data to pre-populate panels like the [Plot](/docs/panels/plot) panel. Adding this same support for ROS 2 sources is high on our list of priorities – [join our Slack community](/join-slack) or [sign up for our newsletter]() to be the first to know when this functionality lands.

![preloading data in plot panel](/images/blog/announcing-ros2-bag-support/preloading.png)
_Preloading ROS 1 data into the Plot panel before starting playback._

**We're also working on allowing users to connect to live ROS 2 data sources**. In the same way that users can currently connect to a live ROS 1 stack in Studio via a native connection or a Websocket bridge, we want our users to be able to listen live to their ROS 2 nodes.

# Share your feedback

Foxglove Studio support for ROS 2 is still a work in progress, so **let us know what improvements or new features you'd like to see**!

We also know not all roboticists use ROS as their robotics platform of choice, so **we’d love to hear your thoughts on how we should expand our supported data sources in the future**.

To share your feedback, join our [Slack channel](/join-slack), get involved in our [GitHub community](https://github.com/foxglove/studio), or message us on [Twitter](https://twitter.com/foxglovedev).

Check out [our docs](/docs/app-concepts/data-sources) for details on all our currently supported data sources.



Whether you just got [started with ROS](/blog/running-your-first-ros-node) or have a [robotic duck](/blog/building-and-visualizing-your-first-robot) roaming around your house, **giving your robot more sensor inputs can be a great next step to take in your robotics journey**, and adding a camera to your robot can be a simple and powerful way to do just that.

**Teaching a robot to understand what it's seeing through a camera might sound complicated, but it doesn't have to be!** As machine learning has become popular, it's also become much more accessible to non-experts, and there are now plenty of off-the-shelf toolkits that make it simple to analyze data and detect patterns.

When analyzing your sensor data, it can be helpful to see your outputs visually. A common approach for camera images is to produce an image that has been annotated with relevant markers. However, there are several limitations to drawing directly on the image – markers can cover pixels in the original image, appear blurry on small images, and be costly to store.

You can tackle these issues by using **image markers**, a kind of vector graphics that Studio can draw on top of the image. Studio even enables you to easily turn them on or off in your layout's [_Image_](/docs/panels/image) panel.

In this post, we'll set up a camera, run a face detection algorithm on the images, and use image markers in Studio to visualize the results. (If you're not yet familiar with ROS _nodes_ and _topics_, try reading [The Building Blocks of ROS](/blog/the-building-blocks-of-ros) first — we'll be using them in this tutorial.)

# Setting up a camera node

_This post will assume you have a working ROS 1 installation on your computer. I'm using ROS Noetic on Ubuntu 20.04, running in a virtual machine. If you need to set up ROS, follow these [installation instructions](https://wiki.ros.org/Installation/Ubuntu)._

The first thing we need is a camera connected to our robot. For the purposes of this demo, I'm using my laptop's webcam, but you might be using another camera that plugs into a USB port. Let's install a ROS package called [usb_cam](https://wiki.ros.org/usb_cam) to interface with the camera and produce ROS [`sensor_msgs/Image`](https://docs.ros.org/en/melodic/api/sensor_msgs/html/msg/Image.html) messages:

```sh
$ sudo apt install ros-noetic-usb-cam
```

`usb_cam` comes with a [launch file](https://wiki.ros.org/roslaunch) that will start up the camera driver and an image viewer, so we can see that our camera is working:

```sh
$ roslaunch usb_cam usb_cam-test.launch
```

![image_view window displaying images from the camera](/images/blog/annotate-your-robots-camera-images-with-image-markers/camera-test.png)

(If this doesn't work for you, you might need to modify the `usb_cam` parameters to select the correct camera device, pixel format, and other settings. You can create your own copy of the [`usb_cam-test.launch`](https://github.com/ros-drivers/usb_cam/blob/develop/launch/usb_cam-test.launch) file and adjust the parameters there.)

# Looking deeper with Foxglove Studio

Behind the scenes, this has launched a ROS node that publishes `sensor_msgs/Image` messages on a new topic. To get a better idea of how this works, let's hook up [Foxglove Studio](/download) and peek at the ROS graph. First we'll [connect to a ROS data source](/docs/app-concepts/data-sources). Since I'm using Ubuntu inside a virtual machine, I need to type an IP address where I can reach the Ubuntu guest:

![connecting to ROS in Foxglove Studio](/images/blog/annotate-your-robots-camera-images-with-image-markers/ros-connection.png)

Next we'll create a new [layout](/docs/app-concepts/layouts), add an [_Image_](/docs/panels/image) panel, and display the `/usb_cam/image_raw` topic:

![viewing camera images in an Image panel](/images/blog/annotate-your-robots-camera-images-with-image-markers/image-panel.png)

Let's also open up a [_Topic Graph_](/docs/panels/topic-graph) panel to visualize the flow of data in the system. Here, blue rectangles represent nodes, and pink diamonds represent topics. You can see how raw images, camera parameters, and log messages flow between the `/usb_cam` driver node, the `/image_view` window, and of course Studio itself.

![visualizing the flow of data with a Topic Graph panel](/images/blog/annotate-your-robots-camera-images-with-image-markers/topic-graph-panel.png)

# Creating an image processor node in Python

Now that we've set up a camera and understood how data flows between nodes in the graph, we can begin processing images. We'll use [rospy](https://wiki.ros.org/rospy) to write a simple ROS node in Python.

First, let's set up a node to receive camera images from the driver in a new `my_node.py` file:

```py
import rospy
from sensor_msgs.msg import Image

def image_callback(msg: Image):
   rospy.loginfo("Got an image!")

def main():
   rospy.init_node("my_node")
   rospy.Subscriber("/usb_cam/image_raw", Image, image_callback, queue_size=1)
   rospy.spin()

if __name__ == "__main__":
    main()
```

Make sure `roslaunch usb_cam usb_cam-test.launch` is still running, before running your script in a new terminal window:

```sh
$ python3 my_node.py
```

By adding a [_Rosout_](/docs/panels/rosout) panel to our current layout, we can view the output from `rospy.loginfo` directly in Studio. We can also confirm that `/my_node` has appeared in the graph, and is receiving data from `/usb_cam/image_raw`.

![viewing log messages in a Rosout panel](/images/blog/annotate-your-robots-camera-images-with-image-markers/node-step1.png)

The next step is detecting faces! We'll use a library called [Dlib](http://dlib.net), which provides a face detector out of the box:

```sh
$ pip3 install dlib
```

We can call `dlib.get_frontal_face_detector()` to create a face detector. The face detector only accepts grayscale images as input, so when we receive an image in our `image_callback`, we'll first use [cv_bridge](https://wiki.ros.org/cv_bridge) to convert it to an OpenCV image, then convert that to grayscale using `cvtColor`.

```py
import cv2
import dlib
from cv_bridge import CvBridge

cv_bridge = CvBridge()
face_detector = dlib.get_frontal_face_detector()
```

Finally, we'll run the face detector, which returns a list of rectangles where faces were found. The callback logs a message indicating how many faces were detected — moving your face around on camera should affect the node's output in the _Rosout_ panel.

```py
def image_callback(msg: Image):
   # Convert the ROS Image to a grayscale OpenCV image
   cv_img = cv_bridge.imgmsg_to_cv2(msg)
   grayscale_img = cv2.cvtColor(cv_img, cv2.COLOR_BGR2GRAY)

   # Run the face detector on the grayscale image
   rects = face_detector(grayscale_img, 0)
   rospy.loginfo(f"Detected {len(rects)} faces")
```

![log messages showing face detections](/images/blog/annotate-your-robots-camera-images-with-image-markers/faces-detected.png)

Now that we've determined whether a camera image contains faces, let's use image markers to see where these faces were detected.

# Adding image markers

ROS provides a message called [ImageMarker](https://docs.ros.org/en/melodic/api/visualization_msgs/html/msg/ImageMarker.html), which describes a **visual annotation on an image**. Since we might have a variable number of faces in the image, we'll need an array of markers. This isn't provided by default with ROS, but Foxglove has created our own [ImageMarkerArray](https://docs.ros.org/en/noetic/api/foxglove_msgs/html/msg/ImageMarkerArray.html) data type. You can install it by running:

```sh
$ sudo apt install ros-noetic-foxglove-msgs
```

Now let's update our code to publish an image marker for each face in the image! We're going to **publish a new topic called `/usb_cam/face_markers`**. (We use the same `/usb_cam` namespace prefix as the camera driver to indicate that the face markers use the same coordinate space as the camera driver's images.)

In order to do this, we need to reorganize our code a little bit, moving the subscriber and publisher into a [class](https://docs.python.org/3/tutorial/classes.html) so they can share state via `self`.

```py
from std_msgs.msg import ColorRGBA
from foxglove_msgs.msg import ImageMarkerArray
from visualization_msgs.msg import ImageMarker
from geometry_msgs.msg import Point

class Node:
   pub_markers: rospy.Publisher

    def __init__(self):
       rospy.init_node("my_node")
       rospy.Subscriber("/usb_cam/image_raw", Image, self.image_callback, queue_size=1)

       # On initialization, set up a Publisher for ImageMarkerArrays
       self.pub_markers = rospy.Publisher(
           "/usb_cam/face_markers", ImageMarkerArray, queue_size=1
       )
       rospy.spin()
```

Then in our `image_callback`, we'll publish an `ImageMarkerArray` every time we receive an image from the camera:

```py
markers = ImageMarkerArray()
for rect in rects:
    markers.markers.append(
        # Display each face as a cyan rectangle
        ImageMarker(
            header=msg.header,
            type=ImageMarker.POLYGON,
            outline_color=ColorRGBA(0, 255, 255, 1),
            points=[
                Point(rect.left(), rect.top(), 0),
                Point(rect.right(), rect.top(), 0),
                Point(rect.right(), rect.bottom(), 0),
                Point(rect.left(), rect.bottom(), 0),
            ],
        )
    )
self.pub_markers.publish(markers)
```

Finally, we'll add `Node()` to our `def main()`.

Back in Studio, the _Image_ panel now shows our new topic in the dropdown menu under "markers".

![dropdown menu to enable image markers](/images/blog/annotate-your-robots-camera-images-with-image-markers/select-image-markers.png)

When I turn it on, a box appears around my face — the detector is working! The `/usb_cam/face_markers` topic also appears in the graph.

![camera image overlaid with a box around the detected face](/images/blog/annotate-your-robots-camera-images-with-image-markers/face-box.png)

# Predicting facial features

Let's take it one step further. Not only does Dlib provide a face _detector_ — it also provides a _facial landmark predictor_, which understands the location of several common points on a face. To use it, we'll need a trained model file, which you can download from [this link here](http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2) (99MB decompressed). Information about the model is provided in the `imutils` package, so let's install that too.

```sh
$ wget http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2
$ bunzip2 shape_predictor_68_face_landmarks.dat.bz2
$ pip3 install imutils
```

We can initialize the predictor with `dlib.shape_predictor()`, passing it the path to the data file we just downloaded and extracted.

Once we've detected a face is present in a rectangular region, we'll run the predictor on that region to get a series of 68 **points**. Then we'll interpret those points using `imutils.face_utils.FACIAL_LANDMARKS_68_IDXS`, which groups them into 8 facial **features** (`mouth`, `inner_mouth`, `right_eyebrow`, `left_eyebrow`, `right_eye`, `left_eye`, `nose`, and `jaw`).

```py
from imutils.face_utils import FACIAL_LANDMARKS_68_IDXS
```

We'll upgrade our image markers by drawing a line of a unique color for each facial feature.

```py
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")
COLORS = [
   ColorRGBA(0, 0, 0, 1),
   ColorRGBA(0, 0, 255, 1),
   ColorRGBA(0, 255, 0, 1),
   ColorRGBA(0, 255, 255, 1),
   ColorRGBA(255, 0, 0, 1),
   ColorRGBA(255, 0, 255, 1),
   ColorRGBA(255, 255, 0, 1),
   ColorRGBA(255, 255, 255, 1),
]
```

Then, in our `image_callback`:

```py
for rect in rects:
    # Run the predictor, which returns a list of 68 facial landmarks as (x,y) points
    points = predictor(grayscale_img, rect).parts()

    # Draw a line around each face region
    for region_idx, (name, (start_idx, end_idx)) in enumerate(
        FACIAL_LANDMARKS_68_IDXS.items()
    ):
        # Connect the points for each region in a loop, except for the jaw
        type = ImageMarker.LINE_STRIP if name == "jaw" else ImageMarker.POLYGON
        markers.markers.append(
            ImageMarker(
                header=msg.header,
                type=type,
                points=[Point(p.x, p.y, 0) for p in points[start_idx:end_idx]],
                outline_color=COLORS[region_idx % len(COLORS)],
            )
        )
```

And that's all for this tutorial — thanks for following along! Now that your robot can analyze images, and you can visualize the results with image markers, I hope it will be easier for you to add amazing new functionality to your robot.

![face with lines drawn around eyebrows, eyes, nose, mouth, and jaw](/images/blog/annotate-your-robots-camera-images-with-image-markers/face-features.png)

Full sample code for this post is available [on GitHub](https://gist.github.com/jtbandes/938a6d39308f145ebc59b4b1c5c8a11d).

For more information, see the following reference materials:

- [`usb_cam` documentation](https://wiki.ros.org/usb_cam)
- [Dlib documentation](http://dlib.net)
- [Detect eyes, nose, lips, and jaw with dlib, OpenCV, and Python](https://www.pyimagesearch.com/2017/04/10/detect-eyes-nose-lips-jaw-dlib-opencv-python/) by Adrian Rosebrock

If you have questions or just want to chat about robotics visualization, join us in our [Slack community](/join-slack).



The typical cycle for robotics development usually starts with a deployed robot collecting data on a real-world excursion – whether that world is [a warehouse](https://www.youtube.com/watch?v=RFNrPXSGRIM), [a public road](https://www.youtube.com/watch?v=Pa7xU7PW-oM), or [the ocean's "twilight zone"](https://www.youtube.com/watch?v=5hjZtBvsmVY).

Typically, this data is offloaded into storage at the end of the excursion, and then analyzed to determine the improvements the robot's software requires. The necessary changes are made, the new and improved software is deployed onto the robot, and that robot is sent back out into the world to collect more data.

And so the cycle continues.

In the robotics community, there is no shortage of tools that can help you inspect, analyze, and visualize your data. From native Robot Operating System ([ROS](https://www.ros.org/)) tools like [rqt](http://wiki.ros.org/rqt) and [rviz](http://wiki.ros.org/rviz), to newer generations of tools like [PlotJuggler](https://github.com/facontidavide/PlotJuggler) and [Foxglove Studio](https://foxglove.dev/), there are a multitude of ways you can inspect the inner workings of your robot's brain.

**But how do you get to a point where you have robotics data to visualize in the first place?** What technical decisions do you need to make, and what steps do you need to take, to collect your data? How do you weigh all the possible strategies for this vital step?

Though there isn't an industry-wide standard or one right answer, **there are basic trade-offs to consider when deciding how to collect your robotics data**. The decisions you make will depend on your project's specific goals and constraints. By having a list of options to weigh, you can make more informed choices about the best data strategy for your project.

# Data collection approaches

Before you can store or analyze any data, you need to collect it. There are several ways you can do this:

- **Local disk recording**
- **Live telemetry**
- **Hybrid approach**

All these methods have their pros and cons – **the frequency with which your robot needs to send real-time data, the volume of data it records on a typical excursion, and its offload bandwidth will all inform which method you should use**.

Your guiding principle should be finding the strategy that will most consistently return uncompromised data from the dynamic real-world environments that your robot navigates. Your data will guide future iterations of your robot, so the method that allows you to collect and learn from your data most reliably is often a good place to start.

## Local disk recording

Robots are often deployed to areas that are inconvenient or dangerous for humans to go, and consequently cannot rely on a consistent internet connection to safely offload data. **These robots with inconsistent or limited in-the-field connectivity can write their data to local storage to ensure that it can be collected and retrieved properly**.

![robot in hazardous environment](/images/blog/recording-your-robotics-data/local-disk.jpeg)

**Your robot's disk space** will also affect how you collect data. With limited space, your robot must be more discerning about what data to record and what data to throw away. Running more business logic on the robot itself will conserve disk space – in this scenario, only the lighter-weight derived insights would be recorded to disk. While this “stretches out” disk space by writing a "lower resolution" version of your data, processing raw data live on your device will impact your robot's real-time performance.

Finally, **the importance of real-time feedback** will determine how you record your data. If your data does not need to be referenced until the robot is back from the field, purely local disk recording is ideal. However, if you need to monitor your robots in real-time, local disk recording would not be able to address this requirement.

## Live telemetry

**If it is crucial for you to know at any given time where your robot is, your robot must be able to offload data to a storage solution – like the cloud or your laptop – as it actively collecting it**.

![autonomous cars](/images/blog/recording-your-robotics-data/live-telemetry.jpeg)

Unlike local disk recording, which is blissfully independent of an internet connection, **real-time offload requires a consistent and reliable connection to your robot – either wirelessly or via a physical cable**. Since network connections can be unreliable, your robot can use in-memory buffering to avoid losing vital data whenever connectivity drops, transmitting these stored messages once a stable connection returns.

Due to bandwidth constraints, live telemetry is usually reserved for a small subset of your data. However, **real-time offload of full-resolution data can be particularly useful in controlled scenarios**, when your internet connection is reliable or when the scope of your task is within certain limits – for example, when you are conducting a sanity check on your robot or calibrating its sensors at homebase before sending it off into the field.

## Hybrid approach

Both local disk recording and live telemetry can be present at once on a given robot – some data can be analyzed asynchronously after the robot has returned, and other data can be immediately transmitted for real-time monitoring.

**Given that these approaches are useful in different scenarios, you can mix and match local disk recording and live telemetry for subsets of your robotics data**. Ultimately, the environment that your robot navigates, the type of data it wants to collect, and its goals while out in the field will all determine how you record your data.

**A hybrid approach is especially useful when you want both real-time insight into your fleet and high resolution data to diagnose potential failures**. With live telemetry, you can monitor your fleet's location and current status. In the event of a failure, you can upload the full fidelity data recorded on the robot's local hard disk to review it for diagnosis. In short, the hybrid approach navigates the trade-off between how much local disk space you have (local disk recording), and how much data you can reliably offload via your available bandwidth (live telemetry).

# Recording formats

Knowing how to record your robotics data is only half the battle. Another equally important question to address is the format in which to record this data.

There are several main criteria to consider when weighing data format options. Namely, you will want to choose a format that makes your data:

- **Conducive to analysis**
- **Trivial to recover in case of hardware or software failure**
- **Possible to record with low overhead**
- **Easy to offload**
- **Self-describing**

**Different data formats facilitate different types of analysis**. Choose a format while keeping in mind the type of analysis you may want to perform – whether it's visually playing back a recorded session, or using SQL to query petabytes of recordings.

**Your data should be stored in a format that makes it easy to recover**. For example, in the event of an unexpected shutdown or accident, the data your robot has already recorded should be preserved and remain uncompromised.

**Your robot's recording process should be IO and CPU efficient**. The volume of data you write and its compressibility will all affect performance and offloading.

Finally, **your data should contain self-describing schemas for easier deserialization and compression**. This enables you to decode your data without having to reference separate interface definition files, which may change frequently.

In short, choose the data format that allows your robot to record and offload its data for future analysis – all within its performance and bandwidth constraints. Let's cover your options, based on your robotics framework.

## ROS 1 robots

ROS is currently the industry standard for robotics data recording, mainly because it allows you to get up and running quickly, and easily integrate features that would be very time and labor-intensive to build yourself – i.e. modules for computer vision, SLAM, etc. This allows you to leverage the expertise of your peers, and makes collaborative robotics development extremely easy.

ROS allows you to write the [messages](http://wiki.ros.org/Messages) published by your [nodes](http://wiki.ros.org/Nodes) throughout the course of a robot's excursion to a bag file. [Bag files](http://wiki.ros.org/Bags) store ROS messages in a format that can then be played back, visualized, and analyzed – along with message definitions for all these messages. Many tools like [`rqt_bag`](http://wiki.ros.org/rqt_bag) and [`rosbag`](http://wiki.ros.org/rosbag) have been written to help you record, play back, and otherwise interact with these bag files. Writing your data to bag files optimizes recording and playback performance, because you avoid deserializing and reserializing your messages.

## ROS 2 robots

Whereas ROS 1 is a bundle of data formats, transport protocols, and reference implementations (C++ and Python), ROS 2 is a more generic C API with pluggable implementations. Even though ROS 2 has a default network transport protocol ([DDS](https://www.dds-foundation.org/what-is-dds-3/)) and a default bag storage implementation ([`sqlite3`](https://docs.python.org/3/library/sqlite3.html)), ROS 2 functions less as specific implementations and more as an abstract middleware interface for serialization, transport, and discovery. As such, [rosbag2](https://github.com/ros2/rosbag2) allows recording and replaying of different data formats via a flexible interface, improving communication over different networks.

While ROS 2 seems like a promising new direction for ROS, there are a few in-progress issues that could pose a problem to recording and playback. Most notably, [ROS 2 bags](https://github.com/ros2/rosbag2) currently do not store message definitions alongside their recorded data. This omission makes it more difficult for external tools like [Foxglove Studio](https://foxglove.dev) to inspect and analyze this data. With that said, storing message definitions in ROS 2 bags is an [open issue](https://github.com/ros2/rosbag2/issues/782) that the `rosbag2` team is actively tracking.

## Non-ROS robots

Outside of the ROS ecosystem, there are no clear-cut standards for data serialization formats.

Some teams use an open source data serialization format like [Apache Avro](https://avro.apache.org/), which optimizes for space and performance by storing data in compact binary format. Avro files are easy to process and read, because they store their data type schemas in JSON format. While JSON alone would get unwieldy for repeated fields across records, AVRO files use a direct mapping that makes it efficient for higher volume usage.

Other teams record a raw stream of [protobuf](https://github.com/protocolbuffers/protobuf) messages to a file. Protobuf, Google's mechanism for serializing structured data, transmits data as a compressed binary to conserve space, bandwidth, and CPU usage. Because there isn't a storage protocol, you will have to do some upfront work to get started. You can use [length-delimited streams](https://seb-nyberg.medium.com/length-delimited-protobuf-streams-a39ebc4a4565) to distinguish between subsequent messages, and design your own container format to provide message definitions for your data stream.

It is not uncommon for non-ROS teams to convert their data in custom recording formats into bag files, so they can load their data with ROS analysis tools.

# Summary

There are many tradeoffs to consider before your robot even starts collecting data. What we covered is not a one-size-fits-all approach, nor is it an exhaustive list of all the ways you can optimize data collection. Knowing your robot's working environment, your constraints, and your data's usage is paramount to assessing how you should tackle these questions.



While our team’s North Star is to provide a suite of general-purpose tools for visualizing and debugging robotics data, **we know many users have domain-specific needs that our out-of-the-box offering does not address**.

At Foxglove, we believe in focusing on the tools that most of our users find useful, but we want the best of both worlds. We want to provide features applicable to most roboticists, but also **empower you to develop bespoke tools customized to your specific needs**, while leveraging Studio’s existing data and layout management features.

**That’s why we’re very excited to announce the alpha release of Foxglove Studio [extensions](https://foxglove.dev/docs/extensions/getting-started)**. We developed an [SDK](https://github.com/foxglove/fox) to help you easily build and install your own custom [panels](/docs/app-concepts/panels) in Studio. Instead of creating your own tool or shoehorning yet another app into your workflow to accomplish some task, you can rely on Studio to be a one-stop shop for investigating your robotics data. **With extensions, there is no limit to how uniquely tailored your Studio environment can be to your project**.

**We’ve also opened up the first version of our [extensions marketplace](https://github.com/foxglove/studio-extension-marketplace)**, where you can publish your Studio extension or search for and install other users’ contributions. With this marketplace, we hope to take a page from the open source community’s handbook and help the robotics world become that much smaller and more collaborative.

# Stay tuned

Building flexible and extensible software is a central tenet of our development philosophy. Our users should be able to adapt Studio to their use cases, without being limited by our roadmap priorities or blocked by our finite development resources. With the new Studio extensions API and marketplace, we hope to facilitate more collaboration and empower you to do more. **But we’re not stopping just yet!**

**We already have a few ideas in the works for improving our extensions library**, such as allowing custom panels to publish messages and supporting more “genres” of extensions including [custom data formats](https://github.com/foxglove/studio/discussions/1268) and message decoders.

# Share your feedback

As we brainstorm new ways for Studio to accompany you to the bleeding edge of robotics, **we’d love to hear your feedback to inform future development of this extensions feature**. Our extension API remains experimental in this alpha release, so feel free to let us know what improvements or new features you'd like to see!

Join the conversation on our [Slack](https://foxglove.dev/slack) or [Twitter](https://twitter.com/foxglovedev). You can also check out our [GitHub](https://github.com/foxglove/studio) to get involved in [discussions](https://github.com/foxglove/studio/discussions) or to file your own [feature requests](https://github.com/foxglove/studio/issues).

Read [our docs](https://foxglove.dev/docs/extensions/getting-started) for a step-by-step tutorial on how to get started on your first Studio extension. Happy hacking!



# Overview

Now that we've learned about the [building blocks that make up a ROS system](/blog/the-building-blocks-of-ros), let's apply our knowledge to see these components in action!

In this post, we'll cover **how to install ROS on your computer, run your first ROS node, and interact with your data using Foxglove Studio**. Whether you want to inspect your active nodes, chart values of interest, or publish ROS messages back to your stack, we'll cover a few of the many ways that Studio can enhance your data exploration experience and accelerate your robotics development.

# Installing ROS

To get started with developing robotics software with ROS, you’ll first need to install it on your machine – whether your platform of choice is Ubuntu, macOS, or Windows.

## Ubuntu

ROS officially supports Ubuntu, so it's easiest to get up and running on this platform – we recommend using Ubuntu 20.04 and following [these instructions on their website](http://wiki.ros.org/Installation/Ubuntu) to finish installation.

## macOS or Windows

To install and run ROS on macOS or Windows, you must have access to a Linux system.

You can set one up by installing [VirtualBox](https://www.virtualbox.org/), a free and open source virtualization software package that allows you to run Linux on macOS or Windows. [Parallels](https://www.parallels.com/) is another virtualization option, albeit paid, for macOS. Again, we recommend using Ubuntu 20.04 and following the same instructions for [installing ROS](http://wiki.ros.org/Installation/Ubuntu) to get started.

# Running your first ROS node

Let's test that ROS was installed correctly by **running our first ROS node**. For this tutorial, we'll be launching [`turtlesim`](http://wiki.ros.org/turtlesim), a ready-to-launch ROS tutorial conveniently packaged as a node.

Open two terminal windows. In the first, start your ROS master:

```bash
$ roscore
```

In the second, start `turtlesim`:

```bash
$ rosrun turtlesim turtlesim_node
```

If everything worked as expected, you should see logs similar to the following in your `turtlesim` window:

```bash
[ INFO] [1622151455.538755408]: Starting turtlesim with node name /turtlesim
[ INFO] [1622151455.540372304]: Spawning turtle [turtle1] at x=[5.544445], y=[5.544445], theta=[0.000000]
```

You should also see a window pop up, showing a tiny turtle peacefully floating in the middle of a vast ocean:

![turtlesim](/images/blog/running-your-first-ros-node/turtlesim.png)

Now that we have our simulation running, **let's interact with our turtle by controlling its movements in this ocean world**. Open another terminal to run the teleop program, conveniently packaged as another node in the same `turtlesim` package:

```bash
$ rosrun turtlesim turtle_teleop_key
```

Once running, the `turtle_teleop_key` node binds to your keyboard's arrow keys to determine what ROS messages to publish back to the `/turtle1/cmd_vel` topic, which in turn controls where our turtle swims. You should see logs similar to the following in your terminal:

```bash
Reading from keyboard
---------------------------
Use arrow keys to move the turtle.
```

If we're focused on this terminal window and pressing arrow keys, we'll see our turtle following our directions, leaving behind a trail indicating where it has been.

![turtlesim teleop](/images/blog/running-your-first-ros-node/turtlesim-teleop.png)

# Using Foxglove Studio to interact with turtlesim

Now that we've been able to interact with our simulated turtle in a few ways with the `turtlesim` package, **let's use Foxglove Studio to look behind the scenes at how this all works**.

[Download](/download) and open Studio – if this is your first time opening the app, you’ll be greeted with a Welcome layout and introductory message.

When you're ready, use the sidebar's _Layouts_ tab to load a "New" empty layout.

![new layout](/images/blog/running-your-first-ros-node/new-layout.png)

To explore our `turtlesim` data , **we must first connect Studio to our ROS system**. You can use a native ROS connection within your Ubuntu environment (whether that's inside a virtual machine or not) – simply enter your ROS master's IP and port (e.g. "https://localhost:11311") to connect to your ROS system.

![connection tab](/images/docs/app-concepts/data-sources/tab.png)

## View the simulation's ROS graph

To start off our data exploration, let's try to **understand the connections between the running processes within our ROS system**. Conveniently enough, Studio has a panel for exactly this.

Use the sidebar's _Add Panel_ tab to drag and drop a [_Topic Graph_](/docs/panels/topic-graph) panel into your layout. This panel will display an interactive graph visualization of the current node and topic topology.

![add panel](/images/docs/app-concepts/panels/tab.png)

If your connection was successful, your panel should come alive with a graph of your active nodes, the topics they are publishing and subscribing to, and the services attached to them:

![turtlesim topic graph](/images/blog/running-your-first-ros-node/topic-graph.png)

Click and drag overlapping items in the graph to reposition them for easier viewing. We can see from this graph that our `turtlesim` node subscribes to the `/turtle1/cmd_vel` topic and publishes the topics `/turtle1/pose` and `/turtle1/color_sensor`.

## Plot the turtle's position

Next, let's use Studio to **investigate how the values in our published messages change over time**.

From our [ROS computation graph](http://wiki.ros.org/ROS/Concepts#ROS_Computation_Graph_Level), we learned that our turtle publishes its position on the topic `/turtle1/pose`. Let's examine this topic using the [_Raw Messages_](/docs/panels/raw-messages) and [_Plot_](/docs/panels/plot) panels.

Use the sidebar's _Add Panel_ tab to drag and drop both panels into your layout. In the _Raw Messages_ panel, add `/turtle1/pose` to see the structure of the topic's messages:

![raw messages](/images/blog/running-your-first-ros-node/raw-messages.png)

In the _Plot_ panel, add `/turtle1/pose.x` and `/turtle1/pose.y` as the custom values to plot on your x-axis and y-axis, respectively. These topic paths use [message path syntax](/docs/app-concepts/message-path-syntax) to drill down into the `/turtle1/pose` topic's `x` and `y` field values.

You should start to see the corresponding values populating your plot:

![turtlesim plot](/images/blog/running-your-first-ros-node/plot.png)

Go back to the `turtle_teleop_key` terminal and navigate our turtle around. As expected, you'll see the plot's values change as our turtle’s position changes.

## Navigate the turtle

Finally, let's use Studio to **publish messages back to our live ROS stack** to test and debug our turtle simulation.

Earlier, we saw that the `turtlesim` node subscribes to the `/turtle1/cmd_vel` topic, which contains our turtle's position. Knowing this, we should be able to publish navigation commands as ROS messages on this same topic, directly from within Studio, to control our turtle's position.

Add the aptly named [_Publish_](/docs/panels/publish) panel to your layout, for a convenient interface that allows you to draft and publish ROS messages on a given topic. Specify `/turtle1/cmd_vel` as your topic to publish on, and select [`geometry_msgs/Twist`](https://docs.ros.org/en/melodic/api/geometry_msgs/html/msg/Twist.html) as your datatype. This will automatically populate the input field with a JSON message template that adheres to the `geometry_msgs/Twist` schema.

Edit any of the message's field values, and click "Publish":

![publish](/images/blog/running-your-first-ros-node/publish.png)

Notice that manually publishing our edited message to `/turtle1/cmd_vel` causes our turtle to move in the simulation! This in turn also affects the trajectory of our _Plot_ panel's charted line.

# Continue exploring with Foxglove Studio

While `turtlesim` may be a basic example of a ROS simulation, its foundational concepts mirror those present in more complex robotics systems in production – both real and simulated.

Simulation fills a crucial role in robotics development by enabling fast iteration without putting expensive hardware or human safety at risk. Many robotics companies simulate hundreds or thousands of times as much data as they generate in real life.

We hope this tutorial helped you **review some foundational ROS concepts, see the role simulation plays in robotics development, and learn how Foxglove Studio can shed light on the inner workings of your robot** (or in this case, simulated turtle). Since Studio integrates a variety of traditional robotics tools into one user-friendly app, we hope it helps you spend less time spinning up different tools and more time making your robot go.

For other ways to explore your robotics data, check out the rest of Studio's panels (e.g. [_Table_](/docs/panels/table), [_Data Source Info_](/docs/panels/data-source-info), [_Node Playground_](/docs/panels/node-playground)), and stay tuned for our yet-unreleased extensions library.



> **Note**: The concepts and CLI commands covered in this post are applicable across ROS 1 and 2. With that said, newer versions of ROS 1 and ROS 2 have integrated several packages (e.g. `rqt_console`, `rqt_graph`, `rqt_plot`, etc.) as [plugins](http://ros.org/wiki/rqt_common_plugins) into a parent `rqt` package. For the purposes of this post (i.e. covering different features in discrete steps), we've opted to use the older, but still supported, version of these CLI commands (e.g. `rosrun rqt_graph rqt_graph` vs. `rqt`). Check out the [`rqt` docs](http://wiki.ros.org/rqt) for more information.

If you’ve kept up-to-date on tech’s latest robotics projects or tinkered around with your own robot kit, you’ve probably come across mentions of ROS.

**At its core, the Robot Operating System (ROS) is an open source software suite built to simplify robotics development.** It provides a collection of tools that runs everything from low-level device control, hardware abstractions, and package management. Due to its distributed design, users can pick and choose what parts of ROS to integrate into their own projects.

ROS was born in 2007, out of roboticists’ desire to more easily share resources and build upon each other’s work. These experts experienced firsthand how difficult it was building software for an increasingly diverse array of robots across a variety of platforms, and realized that no one person was equipped to tackle the field’s challenges alone. They hoped **a deliberately modular framework like ROS would foster more collaborative robotics development and encourage code reuse among the community’s brightest minds**.

Nearly 15 years after its conception, the ROS ecosystem now consists of tens of thousands of users worldwide, with applications ranging from [COVID-19 disinfection robots](https://clearpathrobotics.com/blog/2021/04/clearpaths-boxer-drives-autonomous-disinfection-robot-in-fight-against-covid-19/) to [humanoid robotic hands](https://www.shadowrobot.com/). Its members have contributed cutting-edge libraries like map generation systems, navigation tools, and computer vision utilities, ensuring ROS’s continued relevance in the robotics world.

# Basic concepts

**ROS is a distributed framework of self-contained processes, or _nodes_**, that discover and communicate with each other at runtime. Any given robot can have multiple nodes, each of which owns a subset of the robot’s tasks – one node may keep track of the robot’s velocity, another its steering angle, and still another its camera feed. **All nodes are registered with the _ROS master_**, a service registry that helps nodes locate each other.

**Nodes communicate with each other by publishing or subscribing to _messages_ on different channels of information, called _topics_.** In this way, we can separate concerns into their respective nodes, while still empowering the robot to make complex decisions to execute a task. **Nodes can be grouped into _packages_**, which can then be easily shared and distributed among ROS users.

![ROS nodes](/images/blog/the-building-blocks-of-ros/ros-nodes.png)

**Messages for a topic adhere to a fixed schema, called a _message type_.** For example, one topic can contain messages with just position coordinates ([`geometry_msgs/Point`](http://docs.ros.org/en/noetic/api/geometry_msgs/html/msg/Pose.html)), another lidar point cloud data ([`sensor_msgs/PointCloud`](http://docs.ros.org/en/melodic/api/sensor_msgs/html/msg/PointCloud.html)), and still another text logs ([`std_msgs/String`](https://docs.ros.org/en/melodic/api/std_msgs/html/msg/String.html)).

A message type is defined via multiple fields, each with a specified type. For example, the [`geometry_msgs/Pose`](https://docs.ros.org/en/melodic/api/geometry_msgs/html/msg/Pose.html) message type consists of a `position` and `orientation` field, each of which have their own message type definitions:

![message types](/images/blog/the-building-blocks-of-ros/message-types.png)

In summary, if nodes are like walkie-talkies that can communicate with each other, then topics are the channels that they broadcast audio on, and messages are the words that travel across these channels. In the same way that certain walkie-talkie channels are reserved for specific types of communication, each ROS topic is also reserved for messages that adhere to a specific type.

# Useful commands

To get started using ROS, you will need to install it on your machine – ROS officially supports only [Ubuntu](http://wiki.ros.org/Installation), but you can get set up on Windows ([Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/about)) and MacOS ([Docker for Mac](https://docs.docker.com/docker-for-mac/)) as well.

**Before running any nodes, you must run [`roscore`](http://wiki.ros.org/roscore)** to spin up the following components:

- **ROS master** – Service registry that helps nodes locate each other.
- **`rosout`** – Logger for nodes' debugging output.
- **Parameter server** – Server that allows nodes to store and retrieve parameters at runtime.

![roscore](/images/docs/app-concepts/data-sources/ros/roscore.png)

Now that you have your core ROS system up, you can **use [`rosrun`](http://wiki.ros.org/rosrun) to find ROS packages that contain nodes you want to run**.

For example, `rosrun turtlesim turtlesim_node` will find the [`turtlesim`](http://wiki.ros.org/turtlesim) ROS package and run its `turtlesim_node` to bring up a simulation of swimming turtles on your desktop. Running `rosrun rqt_graph rqt_graph` will run [that package’s](http://wiki.ros.org/rqt_graph) `rqt_graph` node to render a graph of your nodes connected by their publications and subscriptions.

Once you have a few nodes from different packages running, **use [`rosnode`](http://wiki.ros.org/rosnode) to display information about your currently active nodes**.

- `rosnode list` – Lists current nodes.
- `rosnode info /your_node_name` – Provides more information about a given node, including publications and subscriptions.

Once you have several nodes running, **use [`rostopic`](http://wiki.ros.org/rostopic) to fetch information for any published topic**.

- `rostopic echo /your_topic` – Displays messages published to a given topic.
- `rostopic list` – Prints information about active topics.
- `rostopic type /your_topic` – Returns the message type for a topic.
- `rostopic pub /your_topic package_name/MessageType [args]` – Publishes data to a given topic with a specified message type.

Finally, **use [`rosmsg`](http://wiki.ros.org/rosmsg) to inspect messages and their types** in more detail.

- `rosmsg list` – Lists all message types.
- `rosmsg show package_name/MessageType` – Displays the fields for a given message type.

# Useful packages

Given how extensive the ROS package environment is, it is easy to get overwhelmed by the selection. Here are a few popular packages that you can get started with:

- [`rqt_console`](http://wiki.ros.org/rqt_console) – Provides a GUI for displaying and filtering ROS messages.
- [`rqt_graph`](http://wiki.ros.org/rqt_graph) – Visualizes the ROS computation graph – i.e. nodes and the topics they publish and subscribe to.
- [`rqt_image_view`](http://wiki.ros.org/rqt_image_view) – Displays camera images using [`image_transport`](http://wiki.ros.org/image_transport).
- [`rqt_plot`](http://wiki.ros.org/rqt_plot) – Plots numeric values on a 2D chart.

# How Foxglove Studio fits in

While the ROS ecosystem offers an impressive array of features and services, it can be overwhelming to find, install, and run them all at once. ROS is also tightly bound to specific versions of Ubuntu, which introduces significant overhead for users interested in quickly viewing their data.

**Studio has integrated many of ROS's visualization and debugging tools into one cross-platform desktop application**, so that instead of memorizing package names and CLI commands, you can simply drag and drop a panel from our menu into your layout to start seeing results. For example, we've incorporated features similar to [`rqt_console`](http://wiki.ros.org/rqt_console), [`rqt_graph`](http://wiki.ros.org/rqt_graph), [`rqt_image_view`](http://wiki.ros.org/rqt_image_view), and [`rqt_plot`](http://wiki.ros.org/rqt_plot), into our [_Raw Messages_](/docs/panels/raw-messages), [_Topic Graph_](/docs/panels/topic-graph), [_Image_](/docs/panels/image), and [_Plot_](/docs/panels/plot) panels, respectively.

Studio further streamlines visualization and debugging workflows by allowing users to load `.bag` files directly into the app – you no longer need to run `roscore` and [`rosbag`](http://wiki.ros.org/rosbag) in multiple terminals every time you want to inspect your data.

![Studio panels vs ROS tools](/images/blog/the-building-blocks-of-ros/studio-panels-vs-ros-tools.png)

We’re currently working to support user contributions to an extensions library, which will allow you to write your own custom plugins, build project-specific panels, and ultimately customize every aspect of your Studio development workflow.

To learn more about how Foxglove Studio can accelerate your robotics development, feel free to check out [our docs](https://foxglove.dev/docs) or reach out to us directly in [our Slack community](https://foxglove.dev/slack).



**If you're looking to get into robotics, there's no substitute for having a physical robot.** It's the only way to get a feel for how sensors and actuators work together and to observe noisy real-world data coming in from sensors. Understanding how everything wires together is a fundamental part of the experience.

Fortunately, all-in-one robotics platforms are becoming increasingly accessible. If you want to focus on machine learning model training, Amazon's [DeepRacer](https://aws.amazon.com/deepracer/) provides a ready-to-race autonomous RC car. A wide array of [robotic arm](https://www.robotshop.com/en/robotic-arms.html) kits are also available to purchase at newly reasonable prices. **But for the novice roboticist looking for a complete robotics development experience, my current favorite is the [Duckiebot](https://www.duckietown.org/).**

# What is Duckiebot?

![Duckiebot](/images/blog/building-and-visualizing-your-first-robot/duckiebot.jpg)

**Duckiebot is unique, because it provides a deep dive into the real-world challenges and trade-offs that a roboticist would encounter, while still giving you the breadth of exposure to the entire end-to-end system.** The Duckiebot kit consists of an embedded computer, a collection of sensors, and a 3D printed frame where you assemble everything by hand. The only tool required is a screwdriver (included in the kit), and you can build the robot in an afternoon.

To give your robot a place to drive, the Duckietown driving track kit comes with foam tiles and masking tape for arranging your tracks and personalizing your miniature town. Best of all, EdX launched a free online companion course ([Self-Driving Cars with Duckietown](https://www.edx.org/course/self-driving-cars-with-duckietown)) this year, taking you from a basic working knowledge of software development to a broad understanding of the disciplines involved in building robots.

![Duckiebot tracks](/images/blog/building-and-visualizing-your-first-robot/duckiebot-tracks.png)

**One of the most fundamental challenges in robotics is system comprehension.** Understanding what the robot is sensing (_perception_), what decisions it is making based on the perceived environment (_planning_), and how those decisions translate to actuator commands (_controls_) will be a core part of your development experience. For the Duckiebot, we have a few different tools at our disposal to start grappling with this comprehension challenge.

After you've assembled the hardware, flashed the SD card, and pressed the power button, you are rewarded with the first set of feedback: LEDs and a small display with system health metrics. The Duckiebot kit also provides a web interface with more health metrics and system status.

![Duckiebot web UI](/images/blog/building-and-visualizing-your-first-robot/duckiebot-ui.png)

**Getting to the next level of depth requires connecting directly to the robot's software.** Like many robotics platforms, Duckiebot builds on top of the Robot Operating System ([ROS](https://www.ros.org/)) framework.

Though I won't go into a complete overview of ROS in this post, it's helpful to know that ROS-based robots are organized into modules called _nodes_. Nodes share information by publishing and subscribing to _topics_. Topics can be thought of as a stream of messages sharing a common data type, such as GPS coordinates, camera images, or steering angle commands.

To get a deeper look into what my Duckiebot is seeing or thinking, we need a tool that helps us inspect this underlying ROS data...

# Visualizing ROS robots with Foxglove Studio

This is where [Foxglove Studio](/) comes in. Our team's long experience in robotics found that the existing tooling is often too rigid or limited to solve many common problems. Robotics tooling should be as accessible and easy to use as Unity3D is for game development, or as Photoshop is for photo editing.

**Foxglove Studio uses a concept of _panels_, where different types of panels visualize information in different ways.** These panels include camera feeds, time series plots, an extensible 3D view, a map, a connectivity graph, and more. **You can arrange these panels into several dashboard views, or _layouts_, to investigate what your robot is doing.**

**Studio works on any platform** (in my case, Windows), despite most ROS tools only supporting Linux. It also works out-of-the-box with ROS robots, so we can connect to Duckiebot (or any robot) by opening a new [ROS connection](/docs/app-concepts/data-sources) to `<your-robot-host>.local` - for me, this is `darkwing.local`.

![Connecting with ROS](/images/blog/building-and-visualizing-your-first-robot/ros.png)

First, to inspect the published sensor data, let's add an [Image panel](/docs/panels/image) to our layout to get a camera feed.

![Image panel](/images/blog/building-and-visualizing-your-first-robot/image-panel.jpg)

Our duck is alive!

To get a high-level understanding of what software is running on the robot, you can add a [Topic Graph panel](/docs/panels/topic-graph), which shows ROS nodes and topics and how they are connected to each other.

![Topic Graph panel](/images/blog/building-and-visualizing-your-first-robot/topicgraph-full.png)

A lot is going on in the Duckiebot!

To start with a simplified overview, let's turn off the topics and services using the floating toolbar on the right.

![Topic Graph panel, simplified](/images/blog/building-and-visualizing-your-first-robot/topicgraph.png)

This view allows us to focus on what ROS nodes are currently running and how they connect to each other. Not all nodes are actively publishing data at the moment, since this is a brand new Duckiebot that hasn’t been fully developed into a self-driving robot yet.

# Continue exploring

From here, you can add more panels and experiment with configuration options to see what other data you can visualize. As a first step, try using the [Plot panel](/docs/panels/plot) to create a time series plot of your Duckiebot’s time-of-flight sensor.

For a head start on experimentation, download this <a href="/downloads/foxglovestudio-duckiebot.json" download>Foxglove Studio layout</a>, which has been preconfigured with the following panels for easy Duckiebot analysis:

- [Data Source Info](/docs/panels/data-source-info)
- [Diagnostics – Summary](/docs/panels/diagnostics)
- [Image](/docs/panels/image)
- [Parameters](/docs/panels/parameters)
- [Plot](/docs/panels/plot)
- [Rosout](/docs/panels/rosout)
- [Topic Graph](/docs/panels/topic-graph)

You can then import this layout file via the _Layouts_ tab to load it into your Studio app.

![Import Duckiebot layout](/images/blog/building-and-visualizing-your-first-robot/import-layout.png)

**Congratulations on building and visualizing your first robot!**

If you have any questions or ideas about robotics visualization, we'd love to chat with you in our [Slack community](http://foxglove.dev/slack).



At [Foxglove](https://foxglove.dev/download), our team is intimately familiar with the challenges that come with robotics development. Existing tools – like data inspection applications, visualization programs, or debugging utilities – are often difficult to extend, clunky to use, and unsuitable for team collaboration, making for a painfully slow development workflow. **We knew there was a better way to develop and debug our robots.**

**Our vision for [Foxglove Studio](https://foxglove.dev/docs) is to ease these workflow pains and to accelerate robotics development for everyone.** During our time working at [Cruise](http://getcruise.com/), we experienced firsthand the problems that a fully integrated [data visualization app](https://medium.com/cruise/webviz-fb5f77ebe52b) could solve for a self-driving car company. We started Foxglove when we recognized the opportunity to build on top of this work and bring the same benefits to the wider robotics community.

Whether you’re building an autonomous car, a warehouse robot, or a [self-driving rubber duck](https://www.duckietown.org/), we hope Foxglove Studio can help you **spend less time fighting your tools, and more time focusing on what your robots are doing**.

# Why Foxglove Studio?

With Studio, our goal is to empower the robotics community to more easily make sense of their complex and noisy real-world data. We’ve taken the dizzying array of tools that robotics development usually requires, and integrated them into a single seamless developer experience.

## Cross-platform

**Foxglove Studio is conveniently packaged as a desktop app for Linux, Windows, and macOS.** Making Studio a desktop application has unlocked some unique benefits. Because Studio is no longer limited by a sandboxed browser environment, it can connect directly to your ROS 1 stack via the [native ROS connection](https://foxglove.dev/docs/app-concepts/data-sources). It also remains unconstrained by the performance restrictions that come with running huge amounts of robotics data through a browser.

Most ROS tools are only supported on Linux, but **Foxglove Studio works cross-platform** – even if your ROS stack is running on a different operating system, Foxglove Studio can communicate with your robot out-of-the-box.

## Modular & extensible

Every view in Foxglove Studio is a customizable layout, made up of configurable data exploration tools called panels. Our panels can do everything from rendering interactive charts and 3D scenes, to displaying camera images and diagnostic logs. **Using these modular panels, you can effectively build a custom robot control center from scratch to support your every possible workflow.**

Here are some of our most commonly used panels:

- **3D panel** – Display visualization markers (e.g. point clouds, bounding boxes, classification labels) in a 3D scene
- **Image panel** – View images from your robot’s cameras
- **Plot panel** – Plot values from incoming data in interactive charts
- **Raw Messages** panel – Drill down into your incoming data

![common panels](/images/blog/introducing-foxglove-studio/common-panels.png)

Most recently, we’ve added the following panels to tackle common robotics tasks:

- **Map panel** – Display [`sensor_msgs/NavSatFix`](https://docs.ros.org/en/api/sensor_msgs/html/msg/NavSatFix.html) messages as points on a map.
- **Parameters panel** – Read and set rosparams from your connected ROS source.
- **Topic Graph panel** – Render a graph showing the current ROS node and topic topology.
- **URDF Viewer panel** – Use a [Unified Robot Description Format](http://wiki.ros.org/urdf) (URDF) file to visualize your robot model.

![new panels](/images/blog/introducing-foxglove-studio/new-panels.png)

Studio also provides a flexible [Node Playground panel](https://foxglove.dev/docs/panels/node-playground), where users can write custom data transformations in a code editor sandbox – to test out experimental outputs, or to create visualizations on the fly.

## Open source

**We’re partnering with our most important collaborators – our users – to build the future of Foxglove Studio.**

Even with our best efforts to release and maintain tools that help every roboticist, we know a one-size-fit-all approach doesn’t cover every possible use case or meet every specific need. We don’t want Studio to limit your workflow options, or otherwise constrain how you can leverage our tools in the future.

That is why we’ll be implementing a **public API for roboticists to build their own panels** that encapsulate the project-specific functionality they need. This will go hand-in-hand with our extensions library, also in the works: in the same way Visual Studio Code has a rich ecosystem of plugins, we want Studio users to be able to browse a **library of user-contributed extensions build to customize every aspect of the in-app experience**. These changes will empower roboticists with any niche development needs to contribute their own custom tools that plug and play within the Studio ecosystem.

We’re also looking to integrate with new data formats beyond ROS 1. Please [reach out](https://foxglove.dev/slack) with any data protocols or serialization formats you would like Studio to support.

# Stay in touch

We hope this post helped introduce you to the world of Foxglove Studio. At Foxglove, our North Star is to **streamline how you explore and extract insights from your robotics data**. We hope Studio helps lower the barrier to entry for understanding your robot – on your journey from prototype to polished product.

We’re thrilled you’ve stumbled upon [our community](https://foxglove.dev/community), and we hope you get involved! Let us know what you think is worth building next. Our codebase is written in TypeScript for [easy contributing](https://github.com/foxglove/studio) – feel free to open a pull request, or share your feature requests via our [Slack concierge](https://foxglove.dev/slack).

**To see Studio in action, [download the app](https://foxglove.dev/download) and [start tinkering](https://foxglove.dev/docs/getting-started)!**

### _Special thanks_

_We know Foxglove Studio could not exist without the hard work of our colleagues at Cruise, especially those building [Webviz](http://webviz.io/). We stand on the shoulders of giants, and we hope our own open source contributions help the robotics community rush towards the bleeding edge of our field._


Blog