Living Literature Review: Semi-automated Literature Screening

Overview

Development stage

DeployedSource: c) Deployed – The use case is being actively authorized or utilized to support the functions or mission of an agency.

High-impact designation

Not high-impactSource: c) Not high-impact

Impact justification

The output of this AI use case does not serve as a principal basis for decisions or actions that have a legal, material, binding, or significant effect on rights or safety.

Start date

2023-07-05 (day precision)Source: 07/05/2023 00:00:00

Withheld from public reporting

NoSource: a) No

Mission

Topic area

Science

Operational functions

No function tag matched Derived by keyword rules; see methodology

Problem the AI is intended to solve

The Agency conducts reviews of scientific literature in many contexts, including periodic revision of NAAQS as mandated by the Clean Air Act, other standards, but also one-time assessments of published evidence. Given the scope of some of those reviews and the very large of peer-reviewed articles to consider, a system was required to rank and prioritize them for faster screening, based on past reviews and ongoing expert criteria.

Expected benefits

Faster, more responsive evidence reviews.

System outputs

The AI system queues up references for expert review with the references most likely to be relevant first, and continuously improves the queue as the experts go through it.

Technology

AI classification

Classical / predictive MLSource: Classical/Predictive Machine Learning: Models trained on data to make predictions or classifications based on identified patterns or relatio…

System name(s)

Not reported

Custom-developed code

Yes

Public source code

Not reported

Sourcing

How it was built

Developed in-houseSource: b) Developed in-house

Vendor (as reported)

Not reported

Vendors (standardized)

None on the standard list

A vendor is the supplier named by the agency. It does not identify the underlying model or AI technology.

Data and privacy

Involves PII

No

Privacy Impact Assessment

Not reported

Authorization to Operate

No

Demographic features

k) None of the above

Training and evaluation data

AI is simple machine learning that trains anew for each literature screening project, using user decisions to select relevant articles and deselect irrelevant ones. Users can also provide a warmup set of past relevant articles on a per-project basis.

Federal Data Catalog

Not reported

Governance

0 of 8 minimum-practice questions answered. A blank answer means the agency reported nothing; it does not mean the practice is absent.

Potential impacts and how they were identified

Not reported