Using LLM to optimize National Health Interview Survey (NHIS) case note information

Overview

Development stage

PilotSource: b) Pilot – The use case has been deployed in a limited test or pilot capacity.

High-impact designation

Not high-impactSource: c) Not high-impact

Impact justification

Not reported

Start date

2025-03 (month precision)Source: Mar-25

Withheld from public reporting

NoSource: a) No

Mission

Topic area

Health and Medical

Operational functions

No function tag matched Derived by keyword rules; see methodology

Problem the AI is intended to solve

The National Health Interview Survey (NHIS) employs Census field representatives (FRs) who use open text fields, referred to as ‘case notes,’ to document their interactions with households during screening and interview processes. These case notes serve as a valuable resource, offering insights into the nature of these interactions and aiding in the identification of ‘base cases’—instances that may reveal significant data quality issues. Currently, the review of case notes is performed manually on a case-by-case basis, which limits opportunities for optimization. The objective of this initiative is to explore how large language models (LLMs) can enhance the efficiency and effectiveness of the case notes review process.

Expected benefits

Utilizing large language models (LLMs) for case note reviews provides several advantages, including substantial time and cost savings, improved data quality post data collection, and the creation of more effective training programs. These enhancements not only optimize operational efficiency but also support the goals of public health organization by ensuring that high-quality data is readily available for informed decision-making.

System outputs

Identifying additional problematic cases not referred by Census; examining all cases from some FR whose case was referred to confirm whether similar issues exist in other cases the FR worked on; identifying themes in the case notes like certain letters/respondent materials that are in use, problematic interview strategies, or respondent confusion with questions.

Technology

AI classification

Natural language processingSource: Natural Language Processing: AI that processes, interprets, and shares information in human language.

System name(s)

EDAV (Enterprise Data Analytics and Visualization Platform)

Custom-developed code

No

Public source code

Not reported

Sourcing

How it was built

Developed in-houseSource: b) Developed in-house

Vendor (as reported)

Not reported

Vendors (standardized)

None on the standard list

A vendor is the supplier named by the agency. It does not identify the underlying model or AI technology.

Data and privacy

Involves PII

No

Privacy Impact Assessment

Not reported

Authorization to Operate

Yes

Demographic features

k) None of the above

Training and evaluation data

Not reported

Federal Data Catalog

Not reported

Governance

0 of 8 minimum-practice questions answered. A blank answer means the agency reported nothing; it does not mean the practice is absent.

Potential impacts and how they were identified

Not reported