Local LLMs Evaluation

Overview

Development stage

In development or acquisitionSource: a) Pre-deployment – The use case is in a development or acquisition status.

High-impact designation

Not high-impactSource: c) Not high-impact

Impact justification

The tool, once deployed, is only for productivity and does not impact any of the defined high-impact areas

Start date

2025-08-05 (day precision)Source: 8/5/2025

Withheld from public reporting

NoSource: a) No

Mission

Topic area

Information Technology

Operational functions

No function tag matched Derived by keyword rules; see methodology

Problem the AI is intended to solve

AI Enablers (Enterprise solutions that enable AI to integrate into IT services infrastructures)

Expected benefits

An open-source tool that allows you to run large language models (LLMs) locally on your own computer. Can be used to run and experiment with various LLM's locally and gain greater control over data .

System outputs

Local LLM evaluation

Technology

AI classification

Generative AISource: Generative AI: AI that generates new or synthetic content (e.g., images, videos, audio, text, code).

System name(s)

Not reported

Custom-developed code

Yes

Public source code

Not reported

Sourcing

How it was built

Contract and in-houseSource: c) Developed with both contracting and in-house resources

Vendor (as reported)

Not reported

Vendors (standardized)

None on the standard list

A vendor is the supplier named by the agency. It does not identify the underlying model or AI technology.

Data and privacy

Involves PII

No

Privacy Impact Assessment

Not reported

Authorization to Operate

No

Demographic features

k) None of the above

Training and evaluation data

N/A. Evaluating tools only at this stage

Federal Data Catalog

Not reported

Governance

0 of 8 minimum-practice questions answered. A blank answer means the agency reported nothing; it does not mean the practice is absent.

Potential impacts and how they were identified

Not reported