Tax Disclosure Text Clustering

Overview

Development stage

DeployedSource: c) Deployed – The use case is being actively authorized or utilized to support the functions or mission of an agency.

High-impact designation

Not high-impactSource: c) Not high-impact

Impact justification

The output is not presumed to be high-impact and is not used as the principal basis for significant decisions/actions

Start date

2024-09-30 (day precision)Source: 9/30/2024

Withheld from public reporting

NoSource: a) No

Mission

Topic area

Administrative Functions

Operational functions

No function tag matched Derived by keyword rules; see methodology

Problem the AI is intended to solve

This project takes written data from uncertain tax disclosures, cleans up errors in the text, incorporates additional attachment information not present in the Compliance Data Warehouse (CDW), and groups the disclosures into semantically similar topics. This creates a single source to find uncertain tax disclosure declarations and additional information to facilitate review. The process follows three steps: 1) extraction of text from Form 8275-R, Regulation Disclosure Statement, and Schedule UTP, Uncertain Tax Position Statement, from CDW and supplemental attachments, 2) embedding of text into numeric vectors using the E5-large model, and 3) performing hierarchical clustering on the text vectors to group the disclosures.

Expected benefits

The primary goal of this use case is to gain insights useful for improving future examinations and outcomes.

System outputs

Written descriptions provided by taxpayers disclosing items or positions not otherwise adequately disclosed on a tax return to avoid certain penalties are grouped into semantically similar topics to create clusters. The model produces a table of cleaned tax disclosures and their assigned cluster (numerical label).

Technology

AI classification

Natural language processingSource: Natural Language Processing: AI that processes, interprets, and shares information in human language.

System name(s)

Not reported

Custom-developed code

Yes

Public source code

Not reported

Sourcing

How it was built

Contract and in-houseSource: c) Developed with both contracting and in-house resources

Vendor (as reported)

Not reported

Vendors (standardized)

None on the standard list

A vendor is the supplier named by the agency. It does not identify the underlying model or AI technology.

Data and privacy

Involves PII

No

Privacy Impact Assessment

Not reported

Authorization to Operate

Yes

Demographic features

k) None of the above

Training and evaluation data

Hierarchical Clustering determined by cutting the dendrogram at a user specified height, which in this case represents the minimum cosine distance at which clusters are determined to be distinct. This consisted of evaluation by the subject matter experts (SMEs). One critical aspect to the evaluation involved setting a minimum similarity threshold for the clustering process. We also conducted spot checks on a random sample of groups to ensure consistency.

Federal Data Catalog

Not reported

Governance

0 of 8 minimum-practice questions answered. A blank answer means the agency reported nothing; it does not mean the practice is absent.

Potential impacts and how they were identified

Not reported