Expert Annotation for Datasets Critical to AI systems Evaluation, Validation, and Safety Verification

The Department of Commerce is seeking to establish multi-award Basic Ordering Agreements to support critical research needs for the National Institute of Standards and Technology. The primary objective is the development of expert annotated datasets, benchmarks, and measurement instruments designed to enhance the ability of the AI Safety Institute to rigorously evaluate the functionality and capabilities of frontier AI models. These efforts are intended to improve the validation and safety verification of advanced AI systems, facilitating their potential adoption within the government and enabling the tracking of global AI competition and developments. The work will be performed in Gaithersburg under NAICS code 541690, with coordination managed by the Department of Commerce.