TREAS · REQ-2024-01873 · record R2779

Large Language Model (LLM) Coding for Proof of Concept

Department of the Treasury · Bureau of the Fiscal Service (BFS)

Overview

Development stage: PilotSource: b) Pilot – The use case has been deployed in a limited test or pilot capacity.
High-impact designation: Not high-impactSource: c) Not high-impact
Impact justification: The tool, once deployed, is only for productivity and does not impact any of the defined high-impact areas
Start date: 2024-06-10 (day precision)Source: 6/10/2024
Withheld from public reporting: NoSource: a) No

Mission

Topic area: Information Technology
Operational functions: Search and knowledge retrieval Derived by keyword rules; see methodology

Problem the AI is intended to solve

Searching for agency information using a knowledge retrieval system. Contextual Research (Exploring policies, regulations, trends, and strategic topics).

Expected benefits

Test standing up an LLM and train it using documentation such as PDF files and web pages to be knowledgeable of internal systems. Subsequently use the LLM to power a chat bot that will be used by internal customers. Test to see if the chatbot can provide fast and consistent answers to customer queries.

System outputs

FAQ handling and Case Deflection

Technology

AI classification: Generative AISource: Generative AI: AI that generates new or synthetic content (e.g., images, videos, audio, text, code).
System name(s): Not reported
Custom-developed code: Yes
Public source code: Not reported

Sourcing

How it was built: Developed in-houseSource: b) Developed in-house
Vendor (as reported): Not reported
Vendors (standardized): None on the standard list

A vendor is the supplier named by the agency. It does not identify the underlying model or AI technology.

Data and privacy

Involves PII: No
Privacy Impact Assessment: Not reported
Authorization to Operate: No
Demographic features: k) None of the above
Training and evaluation data: Internal documentation such as PDF files and web pages to be knowledgeable of internal systems
Federal Data Catalog: Not reported

Governance

0 of 8 minimum-practice questions answered. A blank answer means the agency reported nothing; it does not mean the practice is absent.

Pre-deployment testingNo answer reported

AI impact assessmentNo answer reported

Independent reviewNo answer reported

Ongoing monitoringNo answer reported

Operator trainingNo answer reported

Fail-safeNo answer reported

Appeal processNo answer reported

User and public consultationNo answer reported

Potential impacts and how they were identified

Not reported