HHS · no agency ID · record R1022

Enhancing the RCDC with Generative AI

Department of Health and Human Services · HHS/NIH

Overview

Development stage: DeployedSource: c) Deployed – The use case is being actively authorized or utilized to support the functions or mission of an agency.
High-impact designation: Not high-impactSource: c) Not high-impact
Impact justification: Not reported
Start date: 2024-05 (month precision)Source: May-24
Withheld from public reporting: NoSource: a) No

Mission

Topic area: Science
Operational functions: Summarization, Translation and transcription Derived by keyword rules; see methodology

Problem the AI is intended to solve

The goal is to assess whether generative AI can enhance the RCDC process by minimizing time-consuming, resource-intensive routine and manual tasks and improving overall efficiency.

Expected benefits

Gen AI is expected to save time, reduce manual workload, and improve productivity by automating resource-intensive processes. These efficiencies support the agency's mission by enabling better resource allocation and enhanced categorization of NIH research.

System outputs

The Azure cloud platform provides secure access to OpenAI’s ChatGPT model, which is connected to search indexes pre-loaded with relevant datasets. These datasets include internal agency-provided data, such as meeting transcripts, and publicly available data from NIH RePORTER. ChatGPT responds to prompts to execute various tasks such as summarizing meeting transcripts and notes, and the scientific content within curated sets of grant applications. ChatGPT is also utilized to recommend an appropriate RCDC category for a grant application and provide explanations for its recommendations. Additionally, ChatGPT is prompted to predict semantic types for thesaurus concepts, identify hierarchical relationships between concepts, cluster similar concepts, and suggest synonyms for specified terms.

Technology

AI classification: Generative AISource: Generative AI: AI that generates new or synthetic content (e.g., images, videos, audio, text, code).
System name(s): Not reported
Custom-developed code: No
Public source code: Not reported

Sourcing

How it was built: Purchased from vendorSource: a) Purchased from a vendor
Vendor (as reported): Microsoft
Vendors (standardized): Microsoft

A vendor is the supplier named by the agency. It does not identify the underlying model or AI technology.

Data and privacy

Involves PII: No
Privacy Impact Assessment: Not reported
Authorization to Operate: No
Demographic features: k) None of the above
Training and evaluation data: Text data from meeting transcripts and notes, and publicly available grant data from NIH RePORTER database. The data is indexed using Azure Cognitive Search AI and securely stored in an Azure cloud storage container. A Retrieval-Augmented Generation (RAG) chatbot is employed to retrieve the indexed data, and prompts are developed to effectively query the data and generate responses using ChatGPT. LLMs respond in a manner that can cite specific language in the data sources, allowing subject matter experts to validate LLM-generated outputs.
Federal Data Catalog: Not reported

Governance

0 of 8 minimum-practice questions answered. A blank answer means the agency reported nothing; it does not mean the practice is absent.

Pre-deployment testingNo answer reported

AI impact assessmentNo answer reported

Independent reviewNo answer reported

Ongoing monitoringNo answer reported

Operator trainingNo answer reported

Fail-safeNo answer reported

Appeal processNo answer reported

User and public consultationNo answer reported

Potential impacts and how they were identified

Not reported