Evaluation of the National Assessment of Educational Progress (NAEP) Achievement Levels

THIS IS NOT A SOLICITATION OR REQUEST FOR PROPOSALS. NO PROPOSALS ARE SOUGHT. The following is a Request for Information (RFI). This synopsis is for information and planning purposes and does NOT constitute a Request for Proposal (RFP). It is issued under the authority of FAR 15.201("Exchanges of information before receipt of proposals"). Background The National Assessment of Educational Progress (NAEP), also known as the Nation's Report Card, is the largest nationally representative and continuing assessment of what America's students know and can do in various subject areas at grades 4, 8, and 12. Congressionally authorized since 1969, NAEP is a project of the National Center for Education Statistics (NCES) within the Institute of Education Sciences (IES) of the U.S. Department of Education. The NCES Commissioner is responsible for carrying out the NAEP project. Under the provisions of P.L. 107-279, the Secretary of Education is required to provide for continuing review of the NAEP assessment. The legislation identifies the issues to be addressed in the reviews, one of which includes the requirement to evaluate whether the NAEP achievement levels, are - reasonable, valid, reliable and informative to the public. P.L. 107-279 (2002), Sec. 303(e)(2)(C) states that NAEP achievement levels shall be used on a trial basis until the Commissioner for Education Statistics determines, as a result of an evaluation under Sec. 303(f), that such levels are - reasonable, valid, and informative to the public. Pursuant to the legislative authorities described above, the Institute of Education Sciences (IES) has been asked by the Secretary of Education to conduct an independent and objective evaluation of the NAEP achievement levels. This contract seeks to obtain the services and expertise of an organization with experience evaluating large scale assessments to conduct an evaluation of the achievement levels used to report NAEP results. This effort is intended to provide the Commissioner of Education Statistics with information necessary to inform his decision about whether the current trial status of the NAEP achievement levels can be removed or whether they should remain in trial status. NAEP policy and achievement levels are established by the National Assessment Governing Board (NAGB). NAGB members are appointed by the Secretary of Education, and by law, are independent of the Secretary and the Department. As part of its responsibilities, Congress authorized NAGB - to develop appropriate achievement levels for reporting NAEP results. NAGB established three achievement levels'Basic, Proficient, and Advanced'that are the primary means of reporting the performance of students on The Nation's Report Card. The achievement levels represent informed judgments of "how good is good enough" and help explain results of The Nation's Report Card beyond scale scores. The achievement levels are set independently for each subject and grade and began being used in the early 1990s. Purpose The purpose of this notice is to request information from interested parties, including potential offerors, about the feasibility of contract arrangements to provide an independent and objective evaluation of whether the Reading and Mathematics NAEP achievement levels for grades 4, 8 and 12 are - reasonable, valid and informative to the public. The evaluation should reflect and be consistent with current professional standards (as defined in the Standards for Educational and Psychological Testing; AERA, APA and NCME, 1999), large-scale assessment research and practices. The evaluation should be evidenced-based, relying solely on objective data, not on opinion. The evaluation should use existing data, impartial information, research study results from earlier evaluations, the technical reports of the NAEP achievement level-setting studies, and other sources as appropriate. New data gathering and studies may be proposed, but should be kept to a minimum and should be limited in size and scope. The intention is to carry out a fresh, objective, unbiased review of the body of evidence pertaining to the NAEP reading and mathematics achievement levels that has accumulated over two decades. The evaluation findings and evidence should be summarized along with a presentation of the strengths and weaknesses of the evidence in relation to the evaluation criteria (reasonable, valid and informative to the public). The ultimate objective is to present the evidence in a manner by which the Commissioner of Education Statistics can make a determination about whether to lift or maintain the trial designation of the NAEP achievement levels in reading and mathematics. ED will consider responses from sources that are able to demonstrate the following capacities and capabilities: 1. Conduct expert, independent evaluations related to K-12th grade education; 2. Make valid conclusions that meet the purpose of proposed evaluations and present results to inform policy-making decisions; 3. Meet applicable professional standards required of the research field; and 4. Have prior experience related to standard setting methodologies. If ED decides to move forward with a full and open competition for this requirement, it is anticipated that a firm-fixed-priced contract will be awarded to the successful offeror. At this time, this Request for Information Notice is issued to assist the agency in performing market research to determine whether there are qualified and capable sources to provide the aforementioned services and to receive suggestions and comments from the industry. Interested sources are requested to submit a response to Thomas.Smith@ed.gov by June 6, 2013, 12:00 PM EST. Late responses will not be considered. Additionally, responses should include recent (within the past three years or work that is on-going) and relevant experience (work similar in type and scope) to include contract numbers, project titles, dollar amounts, points of contact, and labor rates. For points of contact, we request telephone numbers where the responder performed the relevant work. Marketing brochures and/or generic company literature will not be considered. Not addressing all the requested information may result in the Government determining the responder is not capable of performing the scope of work required. No additional information regarding any potential future RFP or Government requirements will be provided during this market research. Requested Information Interested vendors are requested to submit a maximum 5-page statement of their knowledge, experience, and capabilities to perform the following: - Conducting independent evaluations of government-funded programs; - Conducting standard-setting evaluations; - Experience with standardized national assessments measuring student academic achievement; - Establishing panels comprised of subject matter experts; - Convening two to five work group meetings per year; - Facilitating discussions with the goal to encourage committee members to reach consensus; and - Make valid conclusions that meet the purpose of proposed evaluations and present results to inform policy-making decisions. The draft Statement of Objectives (SOO) is included with this notice. ED encourages comments and feedback from capable and interested vendors to the draft SOO, including comments and feedback to improve the scope of work, proposed methodology, and recommend possible contract arrangements for consideration. Please limit feedback and comments to 5 pages. Instructions for Responding to this Notice Replies will be separated from, and have no bearing on, subsequent evaluation of proposals submitted in response to any resulting formal RFPs. The use of information received in response to this notice may be used by ED for acquisition planning and solicitation preparation activities. Eligibility in participating in a future acquisition does not depend upon a response to this notice. ED will not critique the responses to this notice and the notice should not be used by Offerors to market their products/services. ED does not intend to pay for the information solicited and will not recognize any costs associated with responding to this RFI. Proprietary information is neither sought nor desired by ED. If such information is submitted, it must clearly be marked "proprietary" on every sheet containing such information, and the proprietary information must be segregated to the maximum extent practicable from other portions of the response (e.g., use an attachment or exhibit). ED has developed a list of questions to facilitate discussion and influence the final version of the RFP. ED intends to use the answers to refine the RFP and enhance selection criteria. 1. Company info: Name, Cage Code, and DUNS 2. Do you believe there is a more appropriate NACIS than 541720? If so, which one, and why do you believe it is more appropriate? a. What is the size of your company under 541720? b. What is the size of your company under any alternate NAICS that you may have provided under question 2? 3. What do you see as the risks to performance, cost, and schedule associated with the effort? 4. What pricing arrangements do you recommend? a. If a firm-fixed-price contract is used what types of information would be required to price this effort? 5. Does this SOO provide enough information to prepare a contractor performance work statement? If not, please provide recommendations.