Testing, Validating, and Protecting Army Data Sets for Use in Artificial Intelligence (AI) and Machine Learning (ML) Applications

DEPARTMENT OF DEFENSE Department of the Army Army Science Board Request for Information: - Testing, Validating, and Protecting Army Data Sets for Use in Artificial Intelligence (AI) and Machine Learning (ML) Applications AGENCY: Department of the Army, DoD. ACTION: Request for information (RFI) in support of an Army Science Board (ASB) study titled - Testing, Validating, and Protecting Army Data Sets for Use in Artificial Intelligence (AI) and Machine Learning (ML) Applications Summary: Pursuant to the Federal Advisory Committee Act of 1972 (5 U.S.C., Appendix, as amended), the Sunshine in Government Act of 1976 (U.S.C. 552b, as amended) and 41 Code of the Federal Regulations (CFR 102-3.140 through 160) the Department of the Army requests industry information on products, science and technology (S&T) research, operational concepts, and mission support innovations to support the Army in evaluating its current state of AI data set protection and testing and in developing recommendations for future consideration. No funds are available for any information submission and submitting information does not bind the Army for any future contracts/grants resulting from this request for information: (a) The Government does not intend to award a contract on the basis of this Request for Information or to otherwise pay for the information solicited except as an allowable cost under other contracts as provided in subsection 31.205-18, Bid and proposal costs, of the Federal Acquisition Regulation. (b) Although - proposal and - offeror are used in this Request for Information, your response will be treated as information only. It shall not be used as a proposal. (c) This Request for Information is issued for the purpose of market research. NARRATIVE: The Army is dedicated to advancing and incorporating cutting-edge technology to enhance military capabilities and to streamline operations in battlefield applications. A crucial aspect of this integration will be the deployment of AI/ML technologies. In support of that effort, the ASB is conducting a study titled, - Testing, Validating, and Protecting Army Data Sets for Use in Artificial Intelligence (AI) and Machine Learning (ML) Applications. To complete its analyses and explore as many viable sources of data as possible, the ASB is soliciting information from organizations external to the Army, including industry (traditional defense contractors as well as and non-traditional and/or small businesses), Government laboratories, Federally Funded Research and Development Contractors (FFRDCs), and academia. Based on information submitted in response to this request, the Board may conduct additional market research. Organizations are invited to submit information on products or technologies to support on the protection, testing and validation of Army data sets for use in AI/ML. Lines of inquiry fall into two broad areas: First, the study team seeks details on methodologies and techniques that should be used for data set protection and security, to include the following: 1. Data Encryption: The utilization of cryptographic algorithms and advanced security measures to protect sensitive data, specifically focusing on attack vectors that would be unique to AI data sets that require special considerations 2. Data Privacy: The incorporation techniques such as data anonymization, pseudonymization, or synthetic data to preserve privacy while retaining analytical value. 3. Security Auditing and Testing: Strategies for inspection, analysis and evaluation of data set security, and identification of potential vulnerabilities (may differ from other data protection requirements given that the AI data sets would be used for training Army battlefield systems). 4. Integrity and non-repudiation: Methods to ensure that alterations of data sets as integrated have not been poisoned to detrimentally alter the behaviors of battlefield systems away from their intended mission capabilities. In addition, methods to build user trust by demonstrating trustworthiness, reliability, effectiveness, etc. 5. Restoration of capabilities: Following data set compromise, what methods, practices, or remediations should be employed to restore capabilities expeditiously? Second, the Board seeks information on testing AI-enhanced systems in battlefield applications (autonomous operations, intelligence, surveillance, and reconnaissance (ISR) and other military missions) to include the following: 1. Testing Methodologies and Environments: Approaches to real-world and simulated testing, including robustness against adversarial AI technologies and assessment of system performance under various realistic scenarios. 2. Evaluation: Quantifiable measures and frameworks for assessing the effectiveness, accuracy, and reliability of AI-enhanced systems in military context against representative threats (that are also assumed to employ AI/ML) in operationally realistic scenarios (for example, pitting Army units, against an opposing force with intent to win, in joint experimentation or training exercises). 3. Validation and Verification: How V&V apply to AI data sets. Given the specific characteristics of AI training data, how does V&V apply, if at all? 4. Integration and Interoperability: Strategies for seamless integration of AI/ML capabilities within military units, with existing military systems, platforms, and communication networks. The Army's AI enhanced systems will be sourced from a variety of companies, over potentially significant timespans, having been trained in various ways. What policies or practices need be incorporated by the Army to ensure unit level performance? 5. Effectiveness / Confidence: Demonstrate to users and stakeholders that the AI system is trustworthy, reliable and effective, increasing user confidence and adoption. Submission Instructions and Format: To respond to this Request for Information, interested parties should submit their packages electronically by 5:00 p.m. Eastern Daylight Time on May 12, 2023. Responses should be no more than 10-pages in length. ADDRESSES: Electronic responses will ONLY be accepted and should be submitted to the: Army Science Board, ATTN: Designated Federal Officer / Alternate Designated Federal Officer at: heather.j.gerard.civ@army.mil, vinson.l.bullard.civ@army.mil, and Tom Coryell walter.t.coryell2.civ@army.mil In response to this Request for Information, kindly provide your comprehensive assessment addressing the above-mentioned points, inclusive of any additional insights and suggestions that you believe would be pertinent to the Army's integration of AI/ML technologies specific to their protection, testing, and validation. Any responses received that are marked - Proprietary or include any usage restrictions will NOT BE REVIEWED by ASB. Submissions are to be electronic ONLY to the POC listed above. Any responses received that are marked - Proprietary or include any usage restrictions will NOT BE REVIEWED by ASB. All Proposers should review the NATIONAL INDUSTRIAL SECURITY PROGRAM OPERATING MANUAL, (NISPOM), dated February 28, 2006, as it provides the baseline standards for the protection of classified information and prescribes the requirements concerning Contractor Development information under paragraph 4-105. Defense Security Service (DSS) Site for the NISPOM is: http://www.dss.mil/isp/fac_clear/download_nispom.html. Only unclassified white papers must be emailed to the POC listed (see ADDRESSES and FOR FURTHER INFORMATION CONTACT). A listing of respondents and whether or not their submission was utilized will be made available for public inspection upon request. Open deliberation by the full committee is anticipated on or about July 23, 2023 in Irvine, CA. FOR FURTHER INFORMATION CONTACT: Mr. Vince Bullard at (571) 215-1408 or via vinson.l.bullard.civ@army.mil or Tom Coryell at walter.t.coryell2.civ@army.mil