The NEF DSI/AI project addresses local validation of AI-based clinical decision support (CDS)/decision support interventions (DSI) in provider settings.

Overview

Development stage

In development or acquisitionSource: a) Pre-deployment – The use case is in a development or acquisition status.

High-impact designation

Not high-impactSource: c) Not high-impact

Impact justification

Not reported

Start date

Not reported

Withheld from public reporting

NoSource: a) No

Mission

Topic area

Health and Medical

Operational functions

Clinical and medical Derived by keyword rules; see methodology

Problem the AI is intended to solve

Assess quality of AI-based clinical decision support (CDS) tools

Expected benefits

[LAVA, or "Local AI Evaluator", does not itself use AI.] LAVA would assist clinicians in assessing the accuracy, and therefore usefulness, of AI diagnosis tools. AI diagnosis tools are developed to apply to a national population, rather than to smaller, local populations, such as those served by small providers with one or few physical locations. These smaller, local patient populations may have different demographics than that on which the AI-based tool was trained, so the LAVA tool can help illuminate these differences and how the AI tool may apply to the local population. This can help providers learn how to best use their AI diagnosis tools.

System outputs

Outputs are not generated by AI, but rather use open source information to assess outputs from other AI-based tools. This tool's outputs are metrics that measure, for example, accuracy and precision of external AI predictions of disease onsets in local patient populations.

Technology

AI classification

Other

System name(s)

Not reported

Custom-developed code

Not reported

Public source code

Not reported

Sourcing

How it was built

Not reported

Vendor (as reported)

Not reported

Vendors (standardized)

None on the standard list

A vendor is the supplier named by the agency. It does not identify the underlying model or AI technology.

Data and privacy

Involves PII

Not reported

Privacy Impact Assessment

Not reported

Authorization to Operate

Not reported

Demographic features

Not reported

Training and evaluation data

Not reported

Federal Data Catalog

Not reported

Governance

0 of 8 minimum-practice questions answered. A blank answer means the agency reported nothing; it does not mean the practice is absent.

Potential impacts and how they were identified

Not reported