Implementing a macOS Search Plug-In for Robotics Data

Jacob Bandes-Storch ·

18 min read

Published December 7, 2022

Foxglove Studio provides users with a flexible interface of highly customizable panels, allowing them to load and visualize a variety of robotics data sources. But as a hybrid web and desktop app, Studio can offer an enhanced user experience by integrating more deeply with the operating system.

In this post, I’ll describe the development process behind an integration we recently added to the Studio desktop app on macOS: the Spotlight Importer for MCAP files.

Why Spotlight?

The Foxglove Studio desktop app already offers a few basic integrations:

• Deep links open directly in the app
• Double-clicking .bag and .mcap files, .urdf(.xacro) models, and .foxe extensions will open them in the app
• The app responds dynamically to your system’s color scheme setting

On macOS, we took this a step further by integrating with Quick Look. Foxglove Studio’s Quick Look Preview Extension offers previews of .bag and .mcap files. Finder displays these previews in several places, like the “Get Info” and “Quick Look” windows, allowing you to peek into your data files – even when Foxglove Studio isn’t running:

But roboticists often work with many of these files on a regular basis — so many that it can be hard to sift through them and find what they’re looking for.

Spotlight allows macOS users to search all the files on their computer. It can search by file name, type, and contents, but has only a basic understanding of most files by default. To enable more detailed indexing and searching, Apple created the Spotlight Importer plug-in system, which allows third-party apps to enhance Spotlight’s metadata index with more information about their custom files.

Since robots often produce large amounts of data that’s time-consuming to search, many users and teams will store their data in a data center, or use a cloud service like the Foxglove Data Platform. But working with files on your local computer can be an indispensable part of robotics development workflows, and a Spotlight importer would add metadata search to the arsenal of tools available to a roboticist.

In this post, let's dive deep into the process of developing a Spotlight importer for MCAP files, using MCAP's native Swift API. Foxglove introduced MCAP to address the need for a common format for data storage and transfer in the robotics industry.

Let’s start with a few strategies to make iteration and debugging more efficient.

Developing difficult-to-test software

Developing a custom Spotlight Importer comes with some challenges. Since macOS automatically runs Spotlight indexing in the background, it can be hard to see how code changes affect the metadata Spotlight receives. The following strategies helped me effectively iterate and debug the importer during development.

Start with something that works

It might sound obvious, but the surest way to write new code that works is to start from existing code that already works, and make small modifications. This doesn’t apply just to code — it’s equally easy to accidentally break configuration files, like the Info.plist file in a plug-in bundle. Make sure to clean up anything left over from your starter project before you ship.

You can often find working examples either in the form of official sample code, starter project templates, or other projects on the internet. When developing this Spotlight importer, I used and modified Apple’s CoreRecipes sample app as well as the PlaygroundMDImporter project on GitHub.

Practice your iteration cycle

One of the biggest impediments to debugging is a slow or unreliable iteration cycle. Whether your code is working or not, it’s important to be able to test changes quickly, and to be confident that the tests are providing up-to-date results.

Testing can be particularly difficult with plug-in systems, which are executed by a host process instead of your own process. Spotlight is an especially unpredictable plug-in host, since it’s composed of a number of different commands and background processes that macOS runs at arbitrary times beyond your control. While it may require a lot of effort up front, figuring out how to coerce the system into giving you reliable test results will boost your future development speed and productivity.

I eventually settled on the following steps to test new builds of the importer plug-in more quickly:

• To ensure Spotlight can load the plug-in, copy it to ~/Library/Spotlight. (In production, the importer is included in the Foxglove Studio app bundle, but using this directory makes for quicker local testing.)
• To ensure a cached version of the importer is not being used, kill any Spotlight-related processes. Doing this after each rebuild may not always be necessary, but it helps provide confidence in test results. There are several candidate processes, so I killed all of them each time with sudo killall mds mdworker mdworker_shared corespotlightd mdbulkimport. Using /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister -kill may also help.
• Check whether the plug-in has been loaded by running mdimport -L. Sometimes the output doesn’t update immediately when adding or deleting the plug-in from the ~/Library/Spotlight folder, but it can still provide some helpful signal.
• To run the importer on a file, use mdimport -d2 -t path/to/file.mcap. If the command’s output includes “...with no plugIn”, the importer was not correctly loaded. Imported attributes are displayed as part of the output.

Log, log, log

In a plug-in system where you don’t control the host process, using a full-featured debugger may be out of reach. In that case, you can fall back to the age-old strategy of debug logging. In recent years, the macOS logging system has been revamped, and it takes a little practice to use it.

Here are the tips that worked for me:

• Create a Logger instance with custom “subsystem” and “category” strings using let log = Logger(subsystem: "com.example.my-subsystem", category: "SomeCategory").
• To log the state of your code, use log.debug("Example message \(someData)").
• To enable debug-level logs for your subsystem, run sudo log config --subsystem com.example.my-subsystem --mode level:debug.
• To view live log output, before running the code, start up a terminal window with log stream --level debug --predicate 'subsystem = "com.example.my-subsystem"'.

Apple also provides their own notes on Troubleshooting Spotlight Importers.

How Spotlight indexes your files

Let’s take a look at how Spotlight Importers work, and what it takes to create one using Swift!

The Spotlight system is composed of a few different parts:

• Background process – runs automatically, scanning your filesystem for files to index
• Importer plug-ins – extract metadata from different types of files
• Database – stores metadata and indexes for quick search and retrieval
• API – allows searching the database for files and metadata, to power experiences like Finder’s search functionality and the “Get Info” window

Some importer plug-ins are built into macOS, but others can be bundled inside of third-party applications. Our goal is to create an importer plug-in that enables Spotlight to index MCAP files.

Core concepts of macOS plug-ins

A macOS plug-in, such as a custom Spotlight importer, takes the form of a bundle. Bundles encapsulate executable code and resources into a logical unit. The most common type of bundle you’ve encountered on macOS is a .app, but bundles can have many different extensions, such as .pkg for a software installer, .appex for an app extension, .saver for a screen saver, or even just .bundle. Spotlight importers use the extension .mdimporter (short for “metadata importer”) — fundamentally, Spotlight is all about searching your files’ metadata.

Under the hood, each bundle is actually a directory that follows a certain structure, including an Info.plist file. Finder displays the bundle as a single file, but you can see inside by right-clicking the file and selecting “Show Package Contents”.

A Spotlight Importer is actually a specific type of bundle called a CFPlugIn. A CFPlugIn includes code that can be dynamically loaded and invoked by another process (the plug-in “host”). The CFPlugIn architecture is more than 20 years old, and — surprisingly for an Apple technology — is based on Microsoft’s Component Object Model (COM).

Note: Why are we using a 20-year-old API whose documentation was last updated in 2005? Believe me, I’d rather not. In fact, Apple has been transitioning to a newer, simpler plug-in architecture called App Extensions, and it’s even possible to create a Spotlight importer in the form of an App Extension by subclassing CSImportExtension. Unfortunately, CSImportExtensions just don’t work on the Mac today. This will probably be fixed at some point, but as of macOS 13.0, they aren’t usable.

To understand how Spotlight discovers and loads these plug-ins, let’s look at the CFPlugIn architecture in more depth.

CFPlugIn basics: types, factories, and interfaces — oh my!

The IUnknown interface, inherited from COM, is a standard set of functions and memory layout that allows a host process to load and communicate with a plug-in. A host may define other interfaces for specialized behaviors, but each interface must at minimum include the base functionality from IUnknown.

CFPlugIn adds a layer above interfaces called “types” and “factories”: a type represents a group of interfaces, and a factory is a function that creates an object for a particular type. Each unique type, factory, and interface is identified by a UUID.

For example, Spotlight defines a plug-in type for importers (kMDImporterTypeID, identified by the UUID 8B08C4BF-415B-11D8-B3F9-0003936726FC). To be a Spotlight importer, a plug-in must provide a factory for this type. When Spotlight wants to gather metadata about a file on your Mac, it finds all plug-ins supporting this type, and invokes their factory functions, which are expected to return an object matching the IUnknown interface.

Spotlight also defines a few interfaces under this type — for example, kMDImporterURLInterfaceID (13F60F02-3622-4F35-9891-EC10E6CD08F8), which corresponds to the MDImporterURLInterfaceStruct structure. Spotlight uses the IUnknown’s QueryInterface function to query for the 13F6… interface. If that succeeds, it knows the resulting object provides the functions defined in MDImporterURLInterfaceStruct, so it can then call ImporterImportURLData to extract file metadata.

Implementing interfaces via memory layout

Since COM allows plug-ins to be implemented in any language, a plug-in exposes its functionality to the host by providing a structure (i.e. a block of memory) with a predetermined layout. Specifically, the structure’s first item must be a pointer to a virtual function table, or vtable, which is a list of pointers to functions.

Since every interface is required to inherit from IUnknown, the first entries in the vtable must match the IUnknown interface — one field reserved for padding, and then three pointers to the following functions:

• QueryInterface – Used by the host to determine if a plug-in actually supports a specialized interface, such as the Spotlight importer interface. If it does, the plug-in returns a pointer to a vtable for that specialized interface (which must also conform to IUnknown), or if it doesn’t, it can simply return a null pointer.
• AddRef and Release – Manage the interface’s reference count. Reference counting is a form of memory management: a plug-in allocates a block of memory to hold the vtable, and the plug-in host uses the AddRef and Release functions to increment and decrement the reference count, informing the plug-in of when it’s finished using the interface so the memory can be freed again.

Any specialized functions for the interface, such as the ImporterImportData function for Spotlight, come after the above:

Now that we’ve covered the fundamentals of plug-ins, let’s see what it takes to implement one in Swift!

Implementing a CFPlugIn with Swift

Back when the CFPlugIn architecture was designed, the programming languages of choice for Mac software were C and Objective-C. Swift is a relatively young language, but more and more of Apple’s new APIs, frameworks, and educational materials are being designed with Swift in mind. It has seen wide adoption in the Apple developer community, and is overtaking Objective-C in popularity, so Swift feels like a natural choice for modern macOS software.

However, we should expect some friction when using Swift with older APIs like CFPlugIn. In particular, managing memory using reference counting requires extensive use of APIs marked “unsafe”. (It’s not that the C-based equivalents are any safer, it’s just that Swift forces us to admit it!)

To make our Swift code feel natural and idiomatic, let’s implement the IUnknown base functionality and the Spotlight importer interface in a Swift class.

First, let's scaffold an ImporterPlugin class that will encapsulate our plug-in’s data (the reference count) and logic (QueryInterface, AddRef, and Release methods):

final class ImporterPlugin {
  var refCount = 1

  func queryInterface(uuid: UUID) -> /* TODO */ {
    // Check whether the requested interface is one we support
    if uuid == kMDImporterURLInterfaceID || uuid == IUnknownUUID {
      /* TODO */
    }
    return nil
  }

  func addRef() {
    precondition(refCount > 0)
    refCount += 1
  }

  func release() {
    precondition(refCount > 0)
    refCount -= 1
    if refCount == 0 {
      /* TODO */
    }
  }
}

To finish the queryInterface and release functions, we need to tackle the issues of memory layout and memory management in Swift.

Controlling memory layout with tuples

For the host to load our CFPlugIn successfully, we need to create a factory that returns an interface struct with a specific memory layout. However, Swift generally does not give programmers control over the exact layout of custom types (enums, structs, and classes), so our ImporterPlugin class can’t be provided directly to the host.

There are a couple of exceptions that we can use to work around this limitation. First, tuples use a C-style layout, so we can use a tuple to create a struct that correctly begins with a vtable pointer. Second, types imported from C headers use the same layout in both Swift and C, so we can use the pre-defined vtable type for the Spotlight importer interface we’d like to implement (MDImporterURLInterfaceStruct).

Let’s define a Wrapper type to represent the block of memory that our factory will provide to the plug-in host. The wrapper holds a pointer to the vtable and a pointer to an instance of the ImporterPlugin class. The class instance will manage the interface’s reference count, and holds a reference to its factory and wrapper.

final class ImporterPlugin {
  typealias VTable = MDImporterURLInterfaceStruct
  typealias Wrapper = (vtablePtr: UnsafeMutablePointer<VTable>, instance: UnsafeMutableRawPointer)

  let wrapperPtr: UnsafeMutablePointer<Wrapper>
  var refCount = 1
  let factoryUUID: CFUUID
  …

Managing object lifetimes with UnsafePointer and Unmanaged

Since the plug-in host ultimately controls the interface’s lifetime, we can’t rely completely on Swift’s Automatic Reference Counting (ARC) to ensure the class instance is deallocated at the right time — we need a way to create it on demand, and only release it when the plug-in’s reference count has gone down to zero.

Instead of ARC, let’s use Swift’s manual memory management APIs. UnsafeMutablePointer<T> and UnsafeMutableRawPointer represent plain pointers that we can pass to and from the CFPlugIn APIs, and provide functions for manually allocating and deallocating memory. Unmanaged<T> is used to take a class instance created in Swift and pass it along as an unsafe pointer, while extending its lifetime so that ARC doesn’t cause it to be deallocated prematurely.

Let’s encapsulate the creation of the class, vtable, and wrapper struct in a static allocate method, which our factory can use to create and return the interface. This function uses UnsafeMutablePointer.allocate(capacity:) to create the wrapper struct and vtable, and Unmanaged.passRetained to inform ARC that we need to extend the instance’s lifetime and will manually release it later.

// An initializer is required in order to set up an instance of the class.
// It's marked private because this is an implementation detail — the `allocate`
// function below is the only public API we provide for creating an instance.
private init(wrapperPtr: UnsafeMutablePointer<Wrapper>, factoryUUID: CFUUID) {
  self.wrapperPtr = wrapperPtr
  self.factoryUUID = factoryUUID
  // Inform the CFPlugIn system about our newly created plug-in instance
  CFPlugInAddInstanceForFactory(factoryUUID)
}

deinit {
  // Inform the CFPlugIn system that our instance has been deallocated
  CFPlugInRemoveInstanceForFactory(factoryUUID)
}

static func allocate(factoryUUID: CFUUID) -> Self {
  // First, allocate memory for the wrapper block and vtable.
  let wrapperPtr = UnsafeMutablePointer<Wrapper>.allocate(capacity: 1)
  let vtablePtr = UnsafeMutablePointer<VTable>.allocate(capacity: 1)

  // Create an instance of the class, and convert it to an unmanaged reference
  // with +1 retain count.
  let instance = Self(wrapperPtr: wrapperPtr, factoryUUID: factoryUUID)
  let unmanaged = Unmanaged.passRetained(instance)

  // Set up the vtable's function pointers.
  vtablePtr.initialize(to: VTable(
    _reserved: nil,
    QueryInterface: /* TODO */,
    AddRef: /* TODO */,
    Release: /* TODO */,
    ImporterImportURLData: /* TODO */
  ))

  // Place the vtable pointer and the class instance in the wrapper.
  wrapperPtr.initialize(to: (vtablePtr: vtablePtr, instance: unmanaged.toOpaque()))
  return instance
}

Now, we can come back to complete the queryInterface and release functions.

If queryInterface is called with one of the supported interfaces (the Spotlight importer interface or the base IUnknown interface), it can return the wrapper pointer, since the wrapper struct is compatible with those interfaces. It also increments the reference count, since QueryInterface is expected to return an object that’s already been retained, and the host will release it later if necessary.

func queryInterface(uuid: UUID) -> UnsafeMutablePointer<Wrapper>? {
  // Check whether the requested interface is one we support
  if uuid == kMDImporterURLInterfaceID || uuid == IUnknownUUID {
    addRef()
    return wrapperPtr
  }
  return nil
}

Note: In theory, an object might return different vtables for different queried interfaces, possibly even allocating them dynamically, but in practice, we only need one specialized interface plus the base IUnknown functions, so we can just use a single wrapper, and the interface can return a pointer to itself.

release decrements the reference count stored within the instance. When it reaches zero, the plug-in host no longer needs to use the interface, and the memory holding the instance, vtable, and wrapper can be freed. To do so, we perform the opposite of all the operations we did in the allocate function: first use the Unmanaged.release method to release the class instance, and then deinitialize and deallocate the vtable and wrapper.

func release() {
  precondition(refCount > 0)
  refCount -= 1
  if refCount == 0 {
    Unmanaged<ImporterPlugin>.fromOpaque(wrapperPtr.pointee.instance).release()
    wrapperPtr.pointee.vtablePtr.deinitialize(count: 1)
    wrapperPtr.pointee.vtablePtr.deallocate()
    wrapperPtr.deinitialize(count: 1)
    wrapperPtr.deallocate()
  }
}

Creating a vtable with closures

Now that we’ve handled memory layout and allocation, let’s fill in the vtable.

The plug-in system expects the vtable to hold a function pointer for each interface method. Swift allows us to pass a closure expression as a C function pointer, but a closure used this way is not allowed to capture any variables from the local scope, so to perform logic specific to this plug-in, we need an additional context parameter.

The COM interface is designed to handle this, and it does so by passing an extra first argument to each interface function: a pointer to the interface object (the wrapper) itself. The closure can then act as a trampoline, forwarding the function call to a method on the wrapped class instance.

This “self” pointer is imported to Swift as an UnsafeMutableRawPointer. We’ll need to use Unmanaged.takeUnretainedValue() to get a reference that plays nicely with ARC, without releasing the underlying object. This is a bit unwieldy to work with, so let’s create a helper function:

static func fromWrapper(_ plugin: UnsafeMutableRawPointer?) -> Self? {
  if let wrapper = plugin?.assumingMemoryBound(to: Wrapper.self) {
    // Create a Swift-managed reference to the instance without consuming its +1 retain count.
    // We’ll perform the final release explicitly when the reference count drops to zero.
    return Unmanaged<Self>.fromOpaque(wrapper.pointee.instance).takeUnretainedValue()
  }
  return nil
}

Now we can initialize the vtable with closures that invoke the methods we wrote earlier:

vtablePtr.initialize(to: VTable(
  _reserved: nil,
  QueryInterface: { wrapper, iid, outInterface in
    // `iid` represents the requested interface’s UUID.
    // `outInterface` parameter is an output parameter which we can mutate to provide a pointer to the requested interface.
    // Success or failure is indicated by returning the status codes S_OK or E_NOINTERFACE.
    if let instance = ImporterPlugin.fromWrapper(wrapper) {
      if let interface = instance.queryInterface(uuid: UUID(iid)) {
        outInterface?.pointee = UnsafeMutableRawPointer(interface)
        return S_OK
      }
    }
    outInterface?.pointee = nil
    return HRESULT(bitPattern: 0x8000_0004) // E_NOINTERFACE <https://github.com/apple/swift/issues/61851>
  },
  AddRef: { wrapper in
    if let instance = ImporterPlugin.fromWrapper(wrapper) {
      instance.addRef()
    }
    return 0 // optional
  },
  Release: { wrapper in
    if let instance = ImporterPlugin.fromWrapper(wrapper) {
      instance.release()
    }
    return 0 // optional
  },
  ​​ImporterImportURLData: /* TODO */
))

Importing metadata for Spotlight — finally!

Now that our plug-in is set up, let’s implement the actual Spotlight interface with the ImporterImportURLData function!

Unfortunately, due to a bug in Spotlight, the first wrapper parameter to the ImporterImportURLData function actually holds an invalid pointer. This means we’re unable to access the ImporterPlugin class instance, and therefore unable to call an instance method. Instead, we’ll just have to implement the metadata importing in a static or global function:

​​ImporterImportURLData: { _, mutableAttributes, contentTypeUTI, url in
  // Note: in practice, the first argument `wrapper` has the wrong value passed to it, so we can't use it here
  guard let contentTypeUTI = contentTypeUTI as String?,
        let url = url as URL?,
        let mutableAttributes = mutableAttributes as NSMutableDictionary?
  else {
    return false
  }

  // Bridge between CFDictionary, NSDictionary, and Swift's Dictionary to improve ergonomics for the core importing logic
  var attributes: [AnyHashable: Any] = mutableAttributes as NSDictionary as Dictionary
  let result = importMCAPAttributes(&attributes, forFileAt: url, contentTypeUTI: contentTypeUTI)
  mutableAttributes.removeAllObjects()
  mutableAttributes.addEntries(from: attributes)
  return DarwinBoolean(result)
}

The core logic can then be implemented as follows:

func importMCAPAttributes(_ attributes: inout [AnyHashable: Any], forFileAt url: URL, contentTypeUTI: String) -> Bool {
  // Use the MCAP Swift APIs to read the file, and populate `attributes`.
}

Providing a plug-in factory

This is where I must admit that my plug-in implementation is not entirely written in Swift. The plug-in bundle must provide a factory function that the plug-in host can call, but Swift does not provide an official way to expose a global function to C or Objective-C code.

A simple workaround is to expose an @objc class, and use a short Objective-C file that exposes the desired function. We’ll define a PluginFactory class in Swift that calls the allocate function we defined previously:

@objc public final class PluginFactory: NSObject {
  @objc public static func createPlugin(ofType type: CFUUID, factoryUUID: CFUUID) -> UnsafeMutableRawPointer? {
    if UUID(type) == kMDImporterTypeID {
      return UnsafeMutableRawPointer(ImporterPlugin.allocate(factoryUUID: factoryUUID).wrapperPtr)
    }
    return nil
  }
}

Then, let’s add a short main.m file that defines a UUID for our factory, implements the factory function, and invokes the Swift code:

#import <CoreFoundation/CoreFoundation.h>

#import "MCAPSpotlightImporter-Swift.h"

// You should generate a new UUID for your factory, e.g. using `uuidgen`.
#define PLUGIN_FACTORY_ID "675EC679-0E71-4A8C-8A00-7F01C7412BA5"

void *MCAPImporterPluginFactory(CFAllocatorRef allocator, CFUUIDRef typeID) {
  return [PluginFactory createPluginOfType:typeID factoryUUID:CFUUIDCreateFromString(NULL, CFSTR(PLUGIN_FACTORY_ID))];
}

Packaging the plug-in with Info.plist and schema.xml

To ensure Spotlight can load the plug-in code, this factory UUID and the MCAPImporterPluginFactory function name must match the entries in the plug-in’s Info.plist file under the CFPlugInFactories key. Additionally, the CFPlugInTypes key must be set, as it associates our plug-in type (8B08…, a.k.a. kMDImporterTypeID) with our factory for that type. The CFPlugIn infrastructure will read Info.plist to determine which factories are available for Spotlight’s plug-in type, and which function to call to invoke each factory.

Finally, a Spotlight importer should have a schema.xml file describing the metadata attributes the importer can extract from files, as well as a localized schema.strings file containing human-readable names and descriptions for the attributes. This helps macOS determine what to show in Spotlight, Finder, and other places search results are shown.

Read more about the schema file in the Spotlight Importer Programming Guide.

The MCAP importer defines attributes named “Topics” and “Schemas”, corresponding to concepts in the MCAP file format.

Stay tuned

The Spotlight importer for MCAP files is included in the free Foxglove Studio desktop app for macOS. You can view the full code discussed in this post in the foxglove/MCAPSpotlightImporter repo on GitHub.

At Foxglove, we’re committed to empowering roboticists and their teams to understand and organize their data. Join our public Slack community to ask any questions, or to reach out to us directly with your feedback.

Read more:

tutorial

visualization

data management

MCAP

Announcing FlatBuffers Support in Foxglove

Analyze your FlatBuffers data with Foxglove Studio and Data Platform

Sam NosenzoSam Nosenzo

Esther WeonEsther Weon

December 12, 2022·6 min read

tutorial

ROS

visualization

Announcing Message Converter Extensions in Foxglove Studio

Use message converters to visualize your custom messages

Esther WeonEsther Weon

Jacob Bandes-StorchJacob Bandes-Storch

December 6, 2022·5 min read