- SPECIAL NOTICE/REQUEST FOR INFORMATION - Robust Artificial Intelligence Test Event (RAITE) 2025

N0016425SNB44 - SPECIAL NOTICE/REQUEST FOR INFORMATION - Robust Artificial Intelligence Test Event (RAITE) 2025 - FSC/PSC AC33 - NAICS 541715 ISSUE DATE: 5 MAR 2025 CLOSING DATE: 4 APR 2025 12:00PM ET THIS IS A REQUEST FOR SUBMISSIONS ONLY. THIS REQUEST IS FOR PLANNING PURPOSES AND DOES NOT CONSTITUTE AN INVITATION FOR BIDS, A REQUEST FOR PROPOSALS, A SOLICITATION, A REQUEST FOR QUOTES, OR AN INDICATION THE GOVERNMENT WILL CONTRACT FOR THE ITEMS CONTAINED IN THIS NOTICE. THE GOVERNMENT WILL NOT REIMBURSE RESPONDENTS FOR ANY COSTS ASSOCIATED WITH SUBMISSION OF THE INFORMATION BEING REQUESTED OR REIMBURSE EXPENSES INCURRED TO INTERESTED PARTIES FOR RESPONSES OR PARTICIPATION. This Special Notice is being issued by Naval Surface Warfare Center, Crane Division (NSWC Crane), in support of the Chief Digital and Artificial Intelligence Office (CDAO) and other entities within the Department of Defense (DoD). This Special Notice is intended to be a Request for Information/Submissions with relevant details outlined below for participation consideration in RAITE 25. RAITE is structured as a red/blue team test event focusing on Artificial Intelligence (AI) and autonomy technologies on a cyber-physical range. The mission of RAITE is to provide academia, industry, and government a unique environment to holistically test AI capabilities. The collaborative, real-time, representative test pushes AI capabilities in terms of attacks and defenses to address critical technology gaps. Given the extraordinary reliance of the DoD on commercial vendors (small and large) and private sector solution providers, NSWC Crane will require a combined effort of industry, academia, and government experts to bring innovative initiatives into the experimentation event to achieve definable specific outcomes. The Government is not responsible and will not reimburse any costs associated to this announcement. The Government will not reimburse costs for responses to this announcement nor participation in the RAITE event. Background for 2025 RAITE started in 2023 and has evolved each year. The 2025 event is continuing to support scenarios in the areas of multimodal data (computer vision, radar, e.g.), cyber-enabled AI networks (such as critical infrastructure), UxV collaboration and environment interaction. In addition, RAITE 2025 will add LLM (Large Language Model) Test & Evaluation as a dedicated scenario. Classification RAITE 2025 will be held at the unclassified level. All submissions shall be UNCLASSIFIED. Data Publication: Universities under contract with the Department of Defense must follow contractual obligations when publishing results. U.S. government entities and contractors should refer to relevant Security Classification Guides (SCGs) or reach out to Naval Surface Warfare Center, Crane Division if questions arise. Submission Guidelines: If your organization is interested in participating in RAITE 2025, please send the following details to RAITE Teams at RAITE@us.navy.mil . Organization details: Organization name Submitters name, phone number, and email City, State of organization or where submitter is based If any team member is not a US citizen Scenario and team of interest (e.g., Scenario 2 Blue Team): There is no maximum number of scenarios or teams an organization can support. For each capability interested in bringing to RAITE: Name or identifier of the system Short description (3-5 sentences) Technical Readiness Level Whether the capability does or can run real-time (a general estimate, no post-processing during event for red and blue teams) Verify technology is unclassified or can be run in an unclassified environment Submissions may be either a Word document or a PowerPoint (not both): Word Documents or PDFs Not to exceed 5 pages Organization details listed on first page PowerPoints Not to exceed 5 slides Organization details listed on first slide Files should not exceed 10MB Execution Schedule Tentative Planning Schedule Request for Information (RFIs) March 2025 Initial Planning Conference (IPC) March-April 2025 (virtual) Team Meetings and Planning April - Oct 2025 Mid-Planning Conference (MPC) August 2025 Range walk August 2025 White-box teams deliver tech TBD RAITE 2025 TBD; Tentatively late Oct 2025, (New Mexico/virtual), 5 testing days, a total of 9 days including setup/tear down Path Forward from RFI Submission: Within 45 days of the RFI closing date, submitters will be notified if they have advanced to the next stages of RAITE 25, if more information is needed from them before acceptance, or if they have not been accepted to participate in RAITE 25 this year. Approved respondents will receive details on next steps and event participation. Submitters who need to provide more information, will be contacted to provide specific information needs. Then submitters, if accepted, will receive details on next steps and event participation. Scenarios Scenario 1: Attack Aware Sensor Suite BLUE TEAMS: A computer vision and radar system will run a multimodal tracking algorithm. RED TEAMS: Affect respective blue team through cyber and/or physical means. All attacks must be reversable. Expect black to gray box access to blue teams. PURPLE TEAMS: Collect or analyze data from red/blue teams. Passive teams. Goals: The [Primary task] of the scenario is combining a stationary computer vision and radar system to effectively find/track UAV(s) and/or UGV(s), while moving around an outdoor space that has partially obstructed views. [Secondary task]: For blue teams, know specifically when you have been attacked, which sensor, and alert a human for intervention. Scenario 2: Multi-Agent Autonomous Search BLUE TEAMS: One or more Unmanned Aerial Vehicles (UAV(s)) and Unmanned Ground Vehicles (UGV(s)), employ a sensor suite capable of searching for a physical target and navigating the autonomous system near the item. If two or more autonomous systems are used, the search task should be collaborative between systems. RED TEAMS: Affect respective blue team through cyber and/or physical means. All attacks must be reversable. Expect black box access to blue teams. PURPLE TEAMS: Collect or analyze data from red/blue teams. Passive teams. Goals: [Primary] A UAV(s) and UGV(s) must collaboratively search a space, find a physical target, and move with a certain distance of the specified item. [Secondary] UGVs interact/ - disarm the object found in the primary task. UAVs - save the object by moving it a certain distance from the original point. Scenario 3: Data Poisoning Default scenario is white box, but black/gray box is an option, depending on RFI submissions. BLUE TEAMS: Provide a white box computer vision or LLM model to common folder provided by the US Navy. Red teams will access the files for manipulations. Red teams will attack the model through any data poisoning technique and upload the attacked model to the shared folder. Blue teams will them download the attacked model to run during RAITE. RED TEAMS: Attack blue team models in any way consistent with data poisoning, data manipulation, or similar. Attacks can be overt or covert. Attacks can permanently affect the model. PURPLE TEAMS: Collect or analyze data from red/blue teams. Passive teams Goal: [Primary] Attack a computer vision or LLM model to poison/alter/etc. algorithm effectiveness. Benign and poisoned datasets are evaluated during RAITE by the blue teams. Scenario 4: LLM Attacks LLM T&E is a new scenario for 2025. We encourage broad submissions for this topic. A potential red/blue team outline is below but may change based on community interest. BLUE TEAMS: Run a local LLM where the operator has a set of prescribed prompts or actions. The LLM can be purely language-based or multi-modal. RED TEAMS: Affect respective blue team through cyber means. All attacks must be reversable (for hardware, not software). Decrease LLM capability or LLM operator confidence in algorithm. PURPLE TEAMS: Collect or analyze data from red/blue teams. Passive teams. Scenario 5: Real-time Attacks on Computer Vision (CV) BLUE TEAMS: Run a real-time CV algorithm on a provided security camera system to detect and track humans moving throughout a building. RED TEAMS: Affect respective blue team through cyber and/or physical means. All attacks must be reversable. Expect a gradient of white box to black box access to blue teams. PURPLE TEAMS: Collect or analyze data from red/blue teams. Passive teams. Goal: [Primary] This scenario combines open source and custom blue team CV models where red teams use physical and digital attacks to disrupt the blue team's ability to perform correctly. Scenario 6: The Big Picture The intent of this scenario is to demonstrate a series of physical/digital attacks on multiple AI and non-AI targets. The likely implementation, as used during 2024, will be attacking a critical infrastructure target. The critical infrastructure consists of various Supervisory Control and Data Acquisitions (SCADAs), monitored by AI-enabled security cameras, and a monitored cyber system. BLUE TEAMS: Defends critical infrastructure through any single system or across all systems (cyber, cameras, SCADAs). RED TEAMS: Affect respective blue team through cyber and/or physical means. All attacks must be reversable. Prioritize covert attacks but not a necessity. PURPLE TEAMS: Collect or analyze data from red/blue teams. Passive teams. The primary task of the scenario is to cripple the range's critical infrastructure by avoiding detection in the security cameras and LAN (e.g. red team can - see the SCADA but no attack has taken place - a.k.a. sleeper cell). The secondary task is to deliver an attack on the SCADAs that disrupts the SCADA hardware in a covert manner. All inquiries (technical and administrative) or clarifications of any kind must be submitted via email to the POCs listed below with the appropriate classification markings. All contracting related responses should be unclassified and for general access by the Government, other evaluators, and other respondents. All material provided (responses, discussions, questions/answers, and other data) that is not related to competition sensitive or marked contractor proprietary material may be provided to other respondents. Any material provided may be used in development of future requirement documents. Any proprietary information submitted must be clearly and separately identified and marked and will be appropriately protected. The Government will NOT be responsible for any proprietary information not clearly marked. Responders may include information that is not within the scope of this request for submissions, but must be relevant to the overall goals. In addition, if responses include technical requirements that have been identified as CUI in accordance with NIST SP 800-171, the documents shall be identified with the appropriate category markings. See https://www.archives.gov/cui/registry/category-marking-list for guidance. This Special Notice is issued solely for information and planning purposes and does not constitute a program or contractual solicitation. NSWC Crane is under no obligation to acknowledge receipt of submissions or of the information received, or to provide feedback to respondents with respect to any information submitted under this request for submissions. In accordance with FAR 15.201(e), responses to this notice are not offers and cannot be accepted by the Government to form a binding contract. Responders are solely responsible for all expenses associated with responding to this request for submissions. The Government intends to use the information received in response to this request for submissions for planning purposes. The Government will not award a contract directly on the basis of this request for submissions or to otherwise pay for the information solicited. Participants reviewing submissions to this request for submissions may include: The US Government; Federally Funded R&D Centers; Systems Engineering and Integration (SE&I) government support contractors; all are bound by appropriate non-disclosure requirement and bound by all proprietary laws. Regarding this request for submissions, no contact is allowed with these organizations. Respondents must submit program/idea/concept white paper(s) and associated briefing(s) for review and consideration to NSWC Crane no later than 12:00PM ET 4 April 2025. No additional forms or other materials are needed. Any changes or updates to this announcement can be found at the Government Point of Entry SAM.gov. Individual notifications will not be sent to respondents. It is the respondents responsibility to monitor and respond accordingly to Government announcements.