Workforce Data Quality and Public Use Data

The Division of Strategic Planning and Performance (DSPP) is charged with the responsibility for providing executive leadership for the Employment and Training Administration (ETA) in the development of the strategic plan, measurement of performance and results, and increased accountability for the workforce investment system to customers, partners, and stakeholders. DSPP facilitates federal reviews of performance and oversees a system for measuring results. In addition, DSPP ensures the integrity of workforce system performance data and enhances the public awareness of performance accountability through various means, including website management, public forums, and agency publications. ETA's "Performance and Results" website provides important information on how performance is measured, reported, and evaluated (http://www.doleta.gov/performance). I. SCOPE OF WORK The contractor will provide analysis of the performance data and direct-to-grantee technical support with regard to performance reporting for the grantees of the following programs and their predecessors: - Workforce Innovation and Opportunity Act (WIOA) title I Adult, Dislocated Worker, and Youth - WIOA title III Wagner-Peyser Employment Services - National Dislocated Worker Grants (DWG) - Indian and Native American Programs (INAP) - National Farmworker Jobs Program (NFJP) - Jobs for Veteran State Grants (JVSG) This work is to be performed under the three main tasks as described in section III. II. DESCRIPTION OF TASKS, SKILLS, AND DELIVERABLES This is a labor hour type task order against the awardees' GSA Schedule. The Contractor must demonstrate understanding of and have experience with analyzing Federal Workforce Programs and providing technical assistance to the related to Federal Agencies, states, and other grantees. III.1 DESCRIPTION OF TASKS III.1.1 SUPPORT FOR WORKFORCE PERFORMANCE REPORTING AND DATA ANALYSIS The contractor shall provide direct-to-grantee technical assistance on reporting procedures, performance measurement, and data analysis for WIOA funded programs and their predecessors. This task includes a data quality review of Participant Individual Record Layout (PIRL) data, providing cleansed data files to ETA, preparing descriptive and analytic reports, and conducting ad hoc analyses of data collected for the WIOA Adult, Dislocated Worker, and Youth, Wagner-Peyser Employment Service, JVSG, and DWG. This task will include the following activities: - Provide direct-to-grantee technical assistance on workforce system reporting and performance. - Respond directly to grantees' requests for technical assistance, regarding the submission of performance data, received by telephone and email. These responses must meet the following criteria: - Responses must be accurate and in accordance with relevant statutes, regulations, guidance, and technical assistance materials. - Responses must be timely, such that acknowledgment of the request is sent within 2 business days and an initial resolution is sent within 5 business days of the time the request was received. ? During the critical time periods identified as 15 calendar days prior to the Quarterly and Annual reporting deadlines, acknowledgment of the request must be sent within 1 business day, and an initial resolution with 3 business days of the time the request was received. - Provide ETA with documentation of technology-related issues within the reporting system that arise from grantee requests for assistance. - Provide ETA with a monthly summary of the grantee requests and the responses, for the purpose of identifying common issues that could be resolved with system improvements, additional guidance, or general technical assistance. - Prepare technical assistance materials as requested by ETA. Technical assistance materials may include guides and narrated presentations that can be distributed by e-mail, online or on CD. - Provide technical assistance to ETA and states related to performance reporting, data validation, methodology and data quality improvement. - Assist in implementing and formulating data validation tools, resources, procedures, policies, and protocols. - Conduct analyses of data validation results as requested by ETA (ongoing); Such analyses may include preparing tables of results, or writing short papers describing results. - Assist ETA with any data validation/integrity pilot approaches. - Conduct quarterly data quality reviews of program performance data to identify potential anomalies or illogical data that should be corrected in the national-level data files or corrected by states. - ETA will provide the contractor with data files containing individual level information on individuals served by programs described above, as submitted and certified by grantees to ETA. - Conduct a variety of analyses to identify areas of concern in the performance data. - Inform ETA by email of any significant issues with data submitted by states within two weeks of receiving the file so that ETA can request revisions from states, if necessary. - Compare performance outcomes based on individual record performance data with those in annual and quarterly aggregate reports obtained from ETA. - Notify ETA offices by e-mail of any potential problems with ETA's approach to data collection and provide appropriate technical assistance to resolve data problems. - Provide a summary of the findings from the quarterly data quality reviews and annual and quarterly report comparisons that meets the following requirements: - Must be visual, highly consumable, and descriptive. - Must provide a means of tracking grantees' progress over time and documenting responses from grantees and any corrective actions taken. - Prepare and send final Full and Public versions of the clean performance data incorporating quality corrections and confidentiality restrictions pertinent to ETA. - Provide ETA with support in the analysis of WIOA data. - Prepare Descriptive Reports from the individual record performance data. - Prepare National Summary Tables. - Prepare a National Data Book, which provides detailed information on the programs, including information about who is served, what services are provided, and the outcomes attained by participants. The Data Book is based on the individual records and contain information reported by grantees to ETA. The National Data Book will be provided to ETA in electronic form only. - Prepare state versions of the Data Book. State Data Books will be provided to ETA and states in electronic form only. - Conduct ad hoc analyses of performance data as requested by ETA (ongoing). Ad hoc analyses may include creating research and/or special request datasets. Preparing tables of results, and writing short papers describing results when requested by ETA. III.1.2 SUPPORT FOR INDIAN AND NATIVE AMERICAN PROGRAMS PERFORMANCE MEASUREMENT AND REPORTING The contractor shall provide support for Indian and Native American (INA) performance measurement and reporting including the following: - Statistical and Analytic Support - Develop worksheets and other supporting materials that INA grantees use in developing their performance goals. These worksheets will make use of data on INA grantees' prior years' performance and service levels, as well their service area characteristics as measured from a variety of Census, BLS, and other data sources. - Process individual participant level data, each quarter, which shall include identifying which grantees are delinquent in reporting, testing the data for quality and completeness, and running basic statistics describing key program features. - Process quarterly reports, which shall include identifying grantees that are delinquent in reporting, checking for consistency with the record layout as specified by ETA, and running basic tabulations describing key program features. - Use quarterly individual participant level data and program performance report data at the end of each program year, and program performance report data for prior years to profile the INA program's trends over time, including with respect to aggregate service levels, participant characteristics, and performance. - Work with ETA to develop, and update as needed, a statistical regression model used to calculate grantee performance goals in accordance with statutory requirements. - Annually, assess each grantee's performance relative to its performance targets and develop summary performance statistics. - Conduct ad hoc analyses as requested by ETA, which may include preparing tabulations of data and writing short papers describing results. - Technical Assistance and Other Support - Prepare for and deliver training session on performance and reporting issues for INA grantees at national and regional conferences. - Participate in Work Group meetings relating to performance and reporting issues for the INA program, as requested. - Provide additional support on an ad hoc basis. Such support might consist of meeting with ETA or providing assistance to grantees related to performance and reporting. III.1.3 SUPPORT FOR NATIONAL FARMWORKERS JOB PROGRAM REPORTING AND PERFORMANCE MEASUREMENT The contractor shall continue to provide support for National Farmworker Jobs Program (NFJP) performance measurement and reporting, including: - Statistical and Analytic Support - Review quarterly PIRL data for NFJP grantees. Provide feedback to grantees and to staff on data quality issues, and provide quarterly summaries of grantees' participant characteristics, services, and outcomes. Acknowledgment of the request must be sent within two business days and an initial resolution must be sent within 5 business days of the time it was received. - Assess NFJP grantees' quarterly performance relative to established performance goals. Prepare summary sheets for staff, identifying which grantees met or exceeded their performance targets and which failed to do so. Develop worksheets and other supporting materials that NFJP grantees use to assess their performance relative to established benchmarks. - Develop annually a document that provides detailed information on participant characteristics, services, and outcomes nationally and by individual grantee. - Conduct ad hoc analyses of PIRL data as requested by ETA (ongoing). Ad hoc analyses may include preparing tables of results and writing short papers describing results when requested by ETA. - Develop national and grantee Data Books on an annual basis to summarize program activities for each program year. Include detailed information on participant characteristics, services, and outcomes nationally and by individual grantee. - Develop, and update as needed, a statistical adjustment model to be used to determine grantee performance levels. Provide other statistical and analytic support to ETA for the process of setting performance levels and negotiating performance levels with grantees. - Work with ETA to develop, and update as needed, a statistical regression model used to calculate grantee performance goals. Provide other statistical and analytic support to ETA for the process of setting performance goals and negotiating goals with grantees. - Technical Assistance and Other Support - Attend National and regional conferences of farmworker grantees and provide presentations on reporting and performance issues within NFJP as requested by ETA. - Participate in quarterly feedback discussions for NFJP employment and training grantees. - Provide ETA with a monthly summary of the grantee requests and the responses, for the purpose of identifying common issues that could be resolved with additional guidance or general technical assistance. - Provide targeted technical assistance to individual NFJP grantees' requests or coalitions of grantees to improve their data collection, reporting and/or calculated performance. Acknowledgment of the request must be sent within 48 hours and an initial response must be sent within 5 business days of the time it was received. - Develop technical assistance materials on performance and reporting, such as tutorials or training resources, as requested by ETA - Meet with National NFJP staff, as requested, to discuss PIRL reporting and/or performance issues. - Assist federal staff, upon request, in addressing policy inquiries from ETA leadership in a timely manner. - Maintain availability to answer questions posed by ETA and process time sensitive ETA data-related requests. III.2 LABOR CATEGORIES The contractor shall provide the necessary personnel to accomplish the tasks in accordance with the statement of work. The labor categories are as follows: - Project Manager I (Workforce Innovation and Opportunity and Act) - Project Manager II (Indian and Native American Program) - Project Manager III (National Farmworker Jobs Program) These three categories are identified as Key Personnel. The contractor shall provide a Project Manager I who has a minimum of five years of experience analyzing workforce investment programs and providing technical support related to the performance of these programs. The Project Manager I must have at minimum a Bachelor's degree in economics, political science, public administration, statistics, or similar. The Project Manager I for this project must also have extensive work experience analyzing Federal workforce programs and providing technical assistance related to workforce performance to federal agencies, states, and other grantees. The Project Manager must have customer experience where states and grantees have been the customers of technical assistance. The Project Manager must also have experience with implementing and developing measures to ensure data validity, integrity and accuracy. The experience and qualifications of both Project Manager II and Project Manager III proposed for the project shall have a minimum of three years' experienced each in analyzing workforce investment programs and providing technical support related to the performance of these programs. Both Project Manager II and Project Manager III must have at a minimum a Bachelor's degree in economics, political science, public administration, statistics, or similar. The Project Managers for this project must also have extensive work experience analyzing Federal workforce programs and providing technical assistance related to workforce performance to federal Agencies and states and other grantees. In addition to the Project Managers, it is expected that the contractor will have additional personnel that are able to perform the required tasks, these positions include: 1. Programmer: Responsible for writing program code for designated projects and ensuring that all program requirements are met. Writes program code for projects. Develops interactive courseware from script storyboards using various authoring systems. Capable of performing interim and final testing of developed programs. 2. Policy Analyst: Provides subject matter experts to review policies and programs, make recommendations for improvement, draft technical documents, and offer expert opinions in matters pertaining to workforce programs. These subject matter experts must be required to have expertise in a wide range of workforce programs and technical related experience. 3. Statistician: Analyzes and interprets data from various sources. Compiles reports, charts, and tables based on established statistical methods. Familiar with a variety of the workforce analysis concepts, practices, and procedures. Relies on extensive experience and judgment to plan and accomplish goals. Performs a variety of complicated tasks. 4. Workforce Data Analyst: Knowledge of Workforce (job training and employment) research and workforce programs development and evaluation for the purpose of furthering the development and utilization of the Nation's workforce resources, and skill to research and develop labor market information relating to occupational analysis, skills assessment and policy implications. 5. Admin/Clerical Staff: Responsible for providing administrative support to all levels of the organization by performing duties such as client contact, preparing correspondence, reports, proposals, organizing meetings, making travel arrangements, and providing marketing support. The experience and qualifications of additional personnel that the Project Managers may call upon to provide additional assistance are as follows: A resume and letters of intent shall be provided for the professional personnel, including employees, contractors, or a contingency hire (defined as person not currently employed but who have executed a binding letter for commitment for employment with the offeror, if the offeror receives award under this solicitation); Letters of intent must be dated and include signatures from the individual and the offeror/contractor. The letter must state that the individual will be available for the number of hours stated in the proposal. In addition, the offeror must acknowledge the adequacy of the time commitment of all personnel assigned to the project, according to each task and sub-task (the number of hours per year that the individual will devote to each aspect of the project). III.3 DELIVERABLES Task Deliverable Description Due Date III.1 Direct responses to grantees' requests for technical assistance regarding the submission of performance data received by telephone and email. Acknowledgment of the request must be sent within 2 business days and an initial resolution must be sent within 5 business days of the time the request was received As Requested Reports by state related to data validation, methodology and data quality improvement Quarterly Documentation of technology issues within the reporting system that arise from grantee requests for assistance. As Necessary Monthly summary of the grantee requests and responses, for the purpose of identifying common issues that could be resolved with additional guidance or general technical assistance Monthly Quarterly summary of the data quality issues by state Quarterly Full and Public versions of the clean performance data incorporating quality corrections and confidentiality restrictions pertinent to ETA Quarterly National Summary Tables Quarterly National Data Book providing detailed information on the programs, including information about who is served, what services are provided, and the outcomes attained by participants Annually State Data Books Annually Ad hoc analyses of performance data, including but not limited to research, the creation of special datasets, tables of results, and short papers describing results As Requested III.2 Statistical and analytic support Technical assistance and other support As Requested III.1.3 Feedback on quarterly PIRL data. Acknowledgment of the request must be sent within 48 hours and an initial response must be sent within 5 business days of the time it was received. Quarterly Analyses of PIRL data. Ongoing Responses to federal staff's and NFJP grantees' requests for technical assistance. Acknowledgment of the request must be sent within 48 hours and an initial response must be sent within 5 business days of the time it was received. As requested Monthly summary of the grantee requests and responses, for the purpose of identifying common issues that could be resolved with additional guidance or general technical assistance. Monthly Summary sheets for staff, identifying which grantees met or exceeded their performance targets and which failed to do so. Technical assistance materials on performance and reporting, such as tutorials or training resources. As requested Statistical adjustment model used to negotiate grantee performance levels. Annually Presentations on reporting and performance issues. As requested Worksheets and other supporting materials that NFJP grantees use to assess their performance relative to established benchmarks. As requested Develop national and grantee Data Books on an annual basis to summarize program activities for each program year. Include detailed information on participant characteristics, services, and outcomes nationally and by individual grantee. Annually Meet with National NFJP staff. As requested III. PERIOD OF PERFORMANCE To be determined. All interested parties are invited to submit a capabilities statement, to Jeffrey Brown, by February 11, 2019 by 2 p.m. EST. Electronic copies of capability statements are preferred and may be emailed to ETA.Lockbox@dol.gov with Workforce Data Quality and Public Use Data in the Subject Line. The response must be specific to each of the tasks listed above to demonstrate that the respondent is capable. The submittals shall not exceed 20 pages, TOTAL. A determination by the Government to proceed with the acquisition as a set-aside is within the discretion of the Government. Please be advised that telephone inquiries Will Not be accepted. PLEASE NOTE THAT THIS IS NOT A REQUEST FOR PROPOSAL.