HHS · no agency ID · record R0610

Qualitative Analysis

Department of Health and Human Services · HHS/ACF

Overview

Development stage: DeployedSource: c) Deployed – The use case is being actively authorized or utilized to support the functions or mission of an agency.
High-impact designation: Not high-impactSource: c) Not high-impact
Impact justification: Not reported
Start date: 2023-03 (month precision)Source: Mar-23
Withheld from public reporting: NoSource: a) No

Mission

Topic area: Other
Operational functions: No function tag matched Derived by keyword rules; see methodology

Problem the AI is intended to solve

How can thematic coding and trend analysis across qualitative data be done more efficiently? ACF staff often conduct surveys and interviews, which generate qualitative data that needs to be analyzed for themes and trends. The standard approach involves multiple human passes of labeling the data for analysis, which is very time-intensive.

Expected benefits

Faster initial labeling of qualitative data that human reviewers are then able to correct and iterate from

System outputs

ACF employees have several tools available to them to support qualitative analysis. Typically the tools are asked to assist with one of the following scenarios: - Take a user-provided list of topics and text passages to initially categorize passages by topic(s) - Suggest potential categories for organizing text passages - Identify thematic trends across a corpus of narrative data - Conduct sentiment analysis

Technology

AI classification: Natural language processingSource: Natural Language Processing: AI that processes, interprets, and shares information in human language.
System name(s): NVivo, Qualtrics, Credal, Ask Sage (decommissioned)
Custom-developed code: No
Public source code: Not reported

Sourcing

How it was built: Purchased from vendorSource: a) Purchased from a vendor
Vendor (as reported): Lumivero, Qualtrics, Credal, Ask Sage
Vendors (standardized): CredalAsk SageQualtrics

A vendor is the supplier named by the agency. It does not identify the underlying model or AI technology.

Data and privacy

Involves PII: Yes
Privacy Impact Assessment: www.hhs.gov ↗
Authorization to Operate: Yes
Demographic features: k) None of the above
Training and evaluation data: RAG implementation using commercially-available LLMs and user-provided narrative data
Federal Data Catalog: Not reported

Governance

0 of 8 minimum-practice questions answered. A blank answer means the agency reported nothing; it does not mean the practice is absent.

Pre-deployment testingNo answer reported

AI impact assessmentNo answer reported

Independent reviewNo answer reported

Ongoing monitoringNo answer reported

Operator trainingNo answer reported

Fail-safeNo answer reported

Appeal processNo answer reported

User and public consultationNo answer reported

Potential impacts and how they were identified

Not reported