Request for Information (RFI) Automated Scoring Project for the National Assessment of Educational Progress (NAEP)

The National Center for Education Statistics (NCES) in the U.S. Department of Education (Department) is conducting market research to identify state-of-the art capabilities and the level of interest in an anticipated National Assessment of Educational Progress (NAEP) Automated Scoring Challenge. The government is interested in the existing capabilities, accuracy metrics, the underlying validity evidence of assigned scores, and costs of using automated scoring with NAEP assessment items for Reading and Mathematics. This notice is NOT a request of proposals but is an invitation to potential offerors to express interest and provide information regarding their scoring platform. Information submitted in response to this Request for Information (RFI) may be used to assist the Department in finalizing its acquisition strategy for a forthcoming Automated Scoring Challenge (Challenge) and for the future of NAEP and its potential uses of automated scoring. Participation in the Virtual Vendor Fair, while strongly encouraged, is NOT a pre-requisite for a prospective vendor to participate in the Challenge. Interested vendors may also submit an optional capability statement and any questions they may have. Disclaimer: The Department will not pay for the provision of any information nor will it compensate any respondents for providing this information. This notice in no way obligates the Government to award any contract to any interested party. Feedback will not be provided to any interested party who submits information to the Department in response to this notice. Automated scoring (AS) is the ability of computer technology to evaluate and score written prose in an assessment context. Accurate AS platforms frequently use natural language processing and machine learning algorithms to produce accurate and interpretable scores. (further defined in the - Background section in the attachment). This technology is one of the most widely deployed uses of machine learning in education, with results demonstrated to be on par with human inter-rater reliability for many items. Automated scoring technology platforms deploy these algorithms using secure, efficient, and repeatable processes and are integrated into assessment scoring systems. Please refer to the attachment for complete instructions and information regarding the Virtual Vendor Fair, and the submission of capability statements. This is NOT a request for proposals. UPDATE! The Virtual Vendor Fair has been rescheduled to July 8, 2021 at 12:30 p.m. ET. Please see the new attachment for updated instructions and the Microsoft TEAMS invitation. UPDATE!! Thank you to all those who could attend the Vendor Fair. The slide deck has been uploaded to this posting. We will continue to accept questions and/or suggestions for the automated scoring challenge until Wednesday, July 14, 2021, at 2:00 p.m. ET. Depending on the amount of questions received, the Department will post responses as an update to this posting shortly thereafter. All parties are encouraged to continue to monitor this posting for updates, including information regarding the scoring challenge. UPDATE with Scoring Challenge Information: ED.gov National Assessment of Educational Progress (NAEP) Automated Scoring Challenge Accurately & Transparently Predicting Scores of Test Items TOTAL CASH PRIZES OFFERED: $30,000 SUBMISSION START: 09/16/2021 08:00 AM ET SUBMISSION END: 11/28/2021 05:00 PM ET For more information about the challenge, see: https://www.challenge.gov/challenge/naep-automated-scoring-challenge/