SCORE (Structured Clinical Output Rating Engine)

Overview

Development stage

In development or acquisitionSource: a) Pre-deployment – The use case is in a development or acquisition status.

High-impact designation

Not high-impactSource: c) Not high-impact

Impact justification

Not reported

Start date

Not reported

Withheld from public reporting

Not reported

Mission

Topic area

Health and Medical

Operational functions

Translation and transcription, Clinical and medical Derived by keyword rules; see methodology

Problem the AI is intended to solve

SCORE (Structured Clinical Output Rating Engine) uses o3-mini LLM to grade the quality of clinical notes that have either been created by a human or AI and provides feedback to the user. When a transcript of the patient encounter is available, SCORE employs embedding-based semantic analysis to enhance patient safety and documentation accuracy. Embeddings convert text into numerical vectors that capture meaning, enabling the system to measure how semantically similar—or different—statements are between the clinical note and the source transcript. This capability powers two critical safety detectors: Hallucination Detection identifies factual-sounding claims in the note that lack support in the transcript. This is essential for AI-generated documentation, where models may produce plausible but ungrounded statements about medications, allergies, diagnoses, or procedures. Claims with weak transcript support are flagged with risk levels (high/medium/low) based on their potential clinical impact. Contradiction Detection finds instances where the note and transcript discuss the same clinical topic but present conflicting information—such as differing medication dosages, mismatched vital signs, or inverted symptom reports (e.g., "denies chest pain" versus "reports chest pain").

Expected benefits

Improve clinical documentation quality through standardized evaluation and scoring and create efficiency for documentation audit staff.

System outputs

Output: Quality Score & Actionable Guidance Returns a hybrid quality score (1.0–5.0) and letter grade computed from three weighted components: PDQI-9 Analysis (70%) – AI-driven evaluation of 9 clinical documentation dimensions (up-to-date, accurate, thorough, useful, organized, concise, consistent, complete, actionable) Heuristic Metrics (20%) – Rule-based assessment of length, structure, and redundancy Factuality Verification (10%) – Claim validation against source transcript (when available) Each dimension includes narrative feedback, supporting evidence excerpts from the note, and specific improvement suggestions to guide documentation enhancement.

Technology

AI classification

Generative AISource: Generative AI: AI that generates new or synthetic content (e.g., images, videos, audio, text, code).

System name(s)

Not reported

Custom-developed code

Not reported

Public source code

Not reported

Sourcing

How it was built

Not reported

Vendor (as reported)

Not reported

Vendors (standardized)

None on the standard list

A vendor is the supplier named by the agency. It does not identify the underlying model or AI technology.

Data and privacy

Involves PII

Not reported

Privacy Impact Assessment

Not reported

Authorization to Operate

Not reported

Demographic features

Not reported

Training and evaluation data

Not reported

Federal Data Catalog

Not reported

Governance

0 of 8 minimum-practice questions answered. A blank answer means the agency reported nothing; it does not mean the practice is absent.

Potential impacts and how they were identified

Not reported