How do you measure AI ROI in healthcare?

If you read our earlier field note, The keys to successful AI implementation in healthcare, you’ll remember that one of the strongest leading indicators of AI ROI is adoption, more than the model itself. Before you can measure financial impact, clinical outcomes, or operational improvements, you first need to know whether people are using the system.

This article is the technical follow-up. Rather than looking at implementation strategy, it focuses on the measurement framework underneath it. What should you collect? Where should it come from? And how do you connect AI usage with the operational and financial metrics that ultimately determine whether a project has been successful?

When organizations evaluate an AI project, the conversation often jumps straight to return on investment.

That makes sense. Every project is expected to improve efficiency, reduce costs, or generate measurable value. The problem is that those outcomes are difficult to measure if you don’t first understand whether the system is being used.

Before looking at financial outcomes, you need to answer some basic questions:

Without that information, it’s difficult to say much about ROI.

The most valuable usage data is collected by the application itself.

If an AI system is expected to become part of a clinical or operational workflow, it should record how people interact with it from day one.

Simply knowing that someone logged in once rarely tells you very much.

The product should be designed to capture this information from the beginning, not added as an afterthought once the rollout is complete.

Usage only becomes meaningful when you know who could have been using the system.

That depends on the application.

For an ambient documentation tool, it might be every clinician using the EMR.

For an AI-assisted billing system, it might be every billing specialist.

For an AI system analysing healthcare call centre conversations, it could be every licensed user of the call centre platform.

Once you know the size of that population, you can compare it with the number of active users and see how adoption changes over time.

Collecting usage data is only the first step.

To understand whether adoption is creating value, those metrics need to be combined with the rest of the organization’s operational data.

That usually means moving application analytics into a data warehouse or data lake alongside information from clinical systems, billing platforms, financial reporting, banking systems, or call centre software.

Platforms such as Databricks and Snowflake make it possible to bring those datasets together into a single reporting environment.

Once AI usage data sits alongside operational and financial data, reporting can compare adoption against outcomes such as billing performance, workflow efficiency, or other operational metrics.

Once the data is in one place, you can start comparing AI usage with operational metrics.

Reporting should be considered before an AI pilot begins, not after it has already been deployed.

The application needs to capture the right information from day one because those interactions usually cannot be recreated later.

Building dashboards is relatively straightforward. Making sure the product collects the right data is the part that requires planning.

Every AI implementation should have a way to answer a simple question: is the system being used, and is that usage leading to measurable improvements elsewhere in the organization?

That starts inside the product itself, with analytics that capture how people use the system. It continues by bringing that information into a reporting environment alongside operational and financial data.

If that measurement framework is designed from the beginning, adoption, workflow improvements, and ROI can all be evaluated using the same reporting environment.