User Roles and Responsibilities

There are different types of users/roles that can be created in MedPerf:

Benchmark Committee

Include regulatory bodies, groups of experts (e.g., clinicians, patient representative groups), and data or Model Owners wishing to drive the evaluation of their model or data. Benchmark Committee do not have admin privileges on the system, but they have elevated permissions regarding the benchmark they define, such as deciding which model and Data Providers will participate.

Data Providers

Include hospitals, medical practices, research organizations, and healthcare insurance providers that own medical data, register medical data, and execute benchmark requests.

Model Owners

Include ML researchers and software vendors that own a trained medical ML model and want to evaluate its performance.

Platform Providers

Organizations like MLCommons that operate a platform that enables a benchmark committee to run benchmarks by connecting Data Providers with Model Owners.