Skip to main content

Adding Metrics

The Metrics tab is your go-to space for tracking all the key performance indicators (KPIs) and model performance metrics related to your AI project. Turo makes it easy to bring in data using two methods:

  1. Turo APIs – Built on top of the open-source OpenTelemetry APIs, this method is perfect for pushing data from any system or workload into Turo.
  2. MLflow Integration – Seamlessly ingest model performance metrics directly from MLflow.

Let’s dive into how each method works!

Using Turo APIs​

This method allows you to push data points to Turo via an API. Turo expects a time-series data, with no specific cadance. Here’s how to get started:

  1. Navigate to the “Metrics” tab and click on “Connections” in the top-right corner.

Connect model

  1. Click “Add a Connection”
  2. Choose who will add the metric. For this flow, select “Myself” (Choosing "Someone else" will open a form to send a request to a colleague to push a metric)
  3. Add a name for your connection.
  4. Select “OpenTelemetry Endpoint”
  5. Decide where the metric will appear on the metrics tab:
    1. Choose “Business (Key Performance Indicator)” if you’re pushing a business KPI.
    2. Choose “Model Performance” if you’re pushing a model performance metric.
  6. Click “Continue” where you can see a summary.
  7. Finish the wizard.

You’ll see an API endpoint and a key—keep these handy! You’ll need them to push data to Turo.

Opentelemetry success

Now you can use the API endpoint to push data Turo, either using HTTP requests or using the openTelemetery SDK

A Few Things to Note:

  • You can generate a new API key anytime by opening the Connections panel, expanding the connection, and clicking “Generate New API Key.”
  • Data points cannot be deleted once pushed. If you need assistance, reach out to the Turo team.

Connecting MLflow​

If your model is hosted on MLflow, you can easily register it with Turo. Here’s how:

  1. Make sure you have the following information:
    1. MLflow Model URL – The URL where your MLflow model is hosted.
    2. MLflow HTTP Authentication Credentials – The credentials required to access your MLflow instance.
  2. Follow the same steps as “Using Turo APIs,” but select “MLflow” instead of “OpenTelemetry Endpoint.”
  3. Enter the MLflow details from Step 1 into the form and submit.

MlFlow form

Once registered, you’ll see a confirmation message letting you know the model has been successfully added.

Note: Turo automatically picks up models tagged with the “production” tag in the provided Model URL and displays all associated metrics by default.

Metrics Features​

Turo’s Metrics tab provides 3 main features to help you stay on top of your project’s KPIs. Here’s what you can do:

Set Alerts​

Admins and Project Editors can set alerts for each metric to get notified if new data points go out of bounds. This allows you to act quickly and keep your project on track.

To add an alert:

  1. Navigate to the metric of your choice.
  2. Click on the “Bell” icon.

bell icon

Note: Alerts are user-specific, meaning only the user who sets an alert will see it or receive notifications when it’s triggered.

You can create as many alerts as you need! There are two types of alerts:

  • Min/Max Alert: Triggered when a new value falls below the minimum or exceeds the maximum threshold. You can specify either or both thresholds for a single alert.
  • Percentage Change Alert: Triggered when a new value changes by more or less than a defined percentage compared to the last registered value.

Important: The percentage change alert uses the last registered value for the metric at the time of creating the alert as the reference point.

Reorder Metrics​

Turo lets you reorder them with a simple drag-and-drop feature to prioritise which metrics to see first

  1. Click the drag button next to a metric.

drag button

  1. Move it to your desired position.

Note:

  1. Metrics can only be reordered within their respective sections (e.g., Business KPIs or Model Performance).
  2. The order is project-specific, meaning that the same order will appear to all users

Add Metadata to Metrics​

Give your metrics more context by adding metadata. Here’s how:

  1. Click on the “Metadata Panel” button for the metric you want to update.

metrics metadata button

  1. Add the following details:
  • Desired Direction: Specify whether the metric should increase (e.g., number of users) or decrease (e.g., number of complaints). This helps Turo provide a clearer picture of your project’s performance.
  • Description: Add a brief explanation of the metric, including where it’s being ingested from and any other critical details.