Nuclear Explosion Monitoring (NEM) FY20 R&D

29 Aug 19, NOTE: Responses are due on 17 Oct 19 MST @ 1500 hours. See paragraphs 4 and 5, below. REQUEST FOR INFORMATION (RFI) AIR FORCE RESEARCH LABORATORY (AFRL) SPACE VEHICLES (RV) DIRECTORATE Nuclear Explosion Monitoring (NEM) Research Scientific Solutions 1. CONTRACTING OFFICE ADDRESS AND POINTS OF CONTACT (POC) Department of the Air Force, Air Force Materiel Command (AFMC), Air Force Research Laboratory Detachment 8 - Space Vehicles Directorate, 3550 Aberdeen Ave, SE, Kirtland AFB, NM 87117-5776 CONTRACTING POINTS OF CONTACT (POC) Mr. Eric Price, Contract Specialist AFRL Det 8/RVKB Phone Number: 505-846-5011 Email: eric.price.5@us.af.mil Ms. Anne C. Peel, Contracting Officer AFRL Det 8/RVKB Phone Number: 505-846-1075 Email: anne.peel@us.af.mil TECHNICAL POINTS OF CONTACT Dr. Raymond J. Willemann, Senior Scientist AFRL/RVBN Phone: 505-853-4333 Email:raymond.willemann@us.af.mil Dr. Frederick R. Schult, Senior Scientist AFRL/RVBN Phone: 505-846-6101 Email: frederick.schult@us.af.mil The NAICS Code for this RFI is 541715. 2. DESCRIPTION OF INTEREST 2.1 The Government seeks to form a consortium to carry out fundamental and advanced research that will accelerate development of decisive mission capabilities in battlespace awareness through adoption of machine intelligence in processing seismic and other geophysical signals to detect, identify, and track rapidly evolving and emerging threats from underground nuclear tests. Machine intelligence is defined broadly as technologies that transform a monitoring system from one that that is based primarily on decisions by human analysts - albeit aided by processing - to a system relying heavily on trust in automated products. Machine intelligence includes machine learning, such as training of deep neural networks, but the scope extends beyond that to use of Bayesian inference, template matching, and other approaches to discovering useful information in big data. 2.2 Applied Research The topics of applied research (i.e. technical readiness level 2, https://www.army.mil/e2/c/downloads/404585.pdf) that the consortium would address in machine intelligence for processing geophysical signals would include the following four categories of processing,with a caveat that the promise of machine intelligence extends to the possibility of producing lists of well- located and reliably characterized seismic sources directly from seismic data streams. That is, it may be possible to dispense with intermediate products - signal detections, phase identifications, and measurements of signal features such as onset times, amplitudes, and slowness vectors - as constructs for human analysts that are no longer needed. Nevertheless, there are numerous challenges to the envisioned transformative improvement in automated processing. Thus, the fundamental research to support the transition may include developing explicit or implicit capabilities to complete the steps described here. 2.2a. Detect signals of interest among clutter and noise. Topics of consortium research might include signal detection. The goals of signal detection include picking onset times and durations of signals of interest even with amplitudes smaller than human analysts normally discern. Simultaneously, it is necessary to limit the number of "false detections" (i.e., incorrect decisions that segments of a data stream are signals of interest) to a small fraction of the true detections. Widely-used detection methods are based on ratios of energy in different time windows after frequency filtering, summation of signals from array elements using delays computed from plane wave models or computed empirically [Gibbons et al., Geophys. J. Int., 208, 101-117 (2017)]. Alternative detection methods have been developed, including comparison of continuous data streams with past signals of interest, which in some cases have been libraries that are manually updated only occasionally and in other cases are automatically updated frequently in order to continue reliable and efficient operation during sequences of aftershocks from a large earthquake or during an extended period of eruption or other volcanic activity [e.g., Harris, Lawrence Livermore National Laboratory Report UCRL-TR-222758 (2006)]. A narrow objective might be to improve on existing detection methods to improve the threshold for detecting signals of interest while also reducing the number of false detections. A broader objective to progress towards an integrated system might include developing confidence metrics that would be useful in the subsequent steps of operations: signal categorization, signal association, and source characterization. Confidence metrics might include likelihood that the detection is real or false, uncertainty of onset time, and uncertainty of signal duration. 2.2b. Measure signal features and categorize signals. Topics of consortium research might include signal categorization. Typically, signals are categorized primarily to facilitate the subsequent steps of associating signals detected at sparsely distributed stations and estimating seismic source characteristics. Widely-used signal categorization methods are based on various signal measurements, such as ratio of vertical amplitude to horizontal amplitude, ratio of amplitude in different frequency bands, ground particle motion, and apparent velocity across an array. If those subsequent steps could be executed efficiently and accurately without prior signal categorization, then an integrated system might omit this step. Nevertheless, an integrated system might be designed to include categorization of detected signals in one or more of several different ways, including: - Propagation mode. E.g., P (primary compressional), S (secondary shear), Rayleigh wave, or Love wave. - Propagation distance. E.g., local (2km < D < 200 km), regional (200 km < D < 2000 km), or teleseismic (2000 km < D) - Event type. E.g., similarity to signals from past earthquakes (either tectonic, volcanic, or anthropogenic) or mining events (either collapses, ripple-fired, or single-shots). An incremental machine intelligence approach might improve existing capability by discovering diagnostic combinations of measurements that had not been tested previously. In a more ambitious approach, artificial neural networks might be trained to recognize waveforms of signals known to be in various categories without a priori specifying which measurements to use. 2.2c. Discover seismic sources by confirming or rejecting association hypotheses. Topics of consortium research might include signal association. Recognizing that signals of interest at two or more seismic sensor locations originate from the same event (i.e. the same earthquake or mining event) is challenging. Seismologists have used computers to execute the iterative association process for decades [e.g., Stewart, Bull. Seismol. Soc. Am., 67, 433-452 (1977)] but to this day the results include many false associations and missed events, and therefore must be reviewed and corrected by human analysts. Approaches used today to associate signals from networks covering broad geographic areas include the widely-known computer programs Global Association [Lebras et al., DOE F08606-90-D-0005 Final Report (1994)] and NetVISA [Arora et al., Bull. Seismol. Soc. Am., 103, 709-729 (2013)], as well as the recently- developed programs GLASS [Yeck et al., Bull. Seismol. Soc. Am., 109, 1469-1478 (2019)] and ALEUMA [de Groot-Hedlin and Hedlin, Bull. Seismol. Soc. Am., 108, (2018)]. The existing computer programs typically produce results that are marginally accurate enough for efficient human review and correction, but only after a burdensome trial-and-error effort to tune the processing parameters for the unique objectives of any given processing center and distinctive features of any given network of sensors, which is especially problematic if data from potentially useful alternative sensor sites are unexpectedly available. Initial efforts to train algorithms to associate waveforms from the same event recorded at widely separated sensors have been very encouraging, but have addressed smaller scale problems [e.g. McBrearty et al., Seismol. Res. Lett., 90, 503-509 (2019); Dickey et al., arXiv:1904.07936 (2019)]. Nevertheless, many serious challenges remain to extend these results to global networks and to simultaneously associating local, regional, and teleseismic events while monitoring to the lowest thresholds possible. An incremental approach towards using machine intelligence in association might comprise automated tuning of an existing association program, perhaps with modifications of the existing program to use a broader set of signal measurements than any of them do currently. A superior automated association step might integrate re-measurement of signal features using array beams or radial/transverse components computed from provisional initial event hypotheses. In a more ambitious approach, provisional associations might be assessed by a directly evaluating the compatibility of waveforms at different sensor sites with each other or with an event hypothesis, perhaps ideally including features to dynamically adapt to new conditions such as addition or loss of sensor sites from a network or natural events such as earthquake sequences related to a volcanic eruption or a large main shock. 2.2d. Estimate source characteristics, including source depth, energy release, and focal mechanism. Topics of consortium research might include source characterization. Attempts to automatically estimate source characteristics and categorize events extend back decades [e.g. Dysart and Pulli, Bull. Seismol. Soc. Am., 80, 1910-1933 (1990)], but have not yet achieved sufficient reliability to be useful in operations. Source types often are conditionally identified by empirical criteria, such as ratios of different types of magnitudes [Napoli et al., Bull. Seismol. Soc. Am., 105, 2235-2241 (2010)], network averages of cross- spectral ratios [Walter et al., Bull. Seismol. Soc. Am., 85, 1050-1067 (1995)], and ratios of signal amplitude to signal duration [Koper et al., Geophys. Res. Lett., 43, 10,638-10,645 (2016)]. Rapidity and confidence level of discrimination of possible underground nuclear explosions from the far more numerous earthquakes is critical for operating efficiently and focusing expert effort where it is most needed. Machine intelligence might incrementally advance automated source type identification by training neural networks to recognize diagnostic combinations of signal measurements. Machine intelligence could be used to integrate more diverse information than human analysts typically include in prompt evaluation. The promise of machine intelligence might be best fulfilled by automating at least some steps that human experts typically undertake for selected events that are suspected to be nuclear tests. For example, human experts may compare the seismic waveforms associated with a suspected test to waveforms from similar nearby earthquakes, mining events, and nuclear test in the past, or to numerical simulations of signals expected from an event of a certain type with certain characteristics, such as depth, yield, and location relative to significant topographic features. A machine intelligence approach may also dramatically advance automated source type identification by training an algorithm to directly classify events, or indirectly improve classification by accurately projecting data into source and propagation spaces [e.g. .Dickey et al. (2019) cited above]. 2.3. Advanced Research Beyond these several aspects of applied research, proof of concept (TRL 3) and laboratory environment validation (TRL 4) research would comprise linking various capabilities and demonstrating that together they generate more accurate, reliable, and timely high-fidelity monitoring products than are achieved today. Advanced research might be based at least partly on funding from direct customers of AFRL or other interested agencies. Nevertheless, the core role of the consortium would include activities to support advanced research, such as: - Developing a strategic plan that outlines the advanced research needed to develop and validate a complete system; - Creating computational infrastructure to facilitate integration of products from fundamental machine intelligence NEM research into the complete system and test performance compared to a baseline system. 2.4. Data Used in Research Raw seismic data are available from numerous organizations, including the NSF-supported IRIS Data Management Center, other centers that distribute data from members of the International Federation of Digital Seismic Networks, and the Comprehensive Nuclear Test Ban Treaty Organization (CTBTO). Products from processing and analysis of seismic data are available from numerous organizations, including the USGS National Earthquake Information Center, national earthquake monitoring agencies of states of the United States and of other countries, and the CTBTO. The CTBTO data and analysis products are particularly relevant to the objectives of the Air Force in forming the consortium. It will be the responsibility of the consortium to make arrangements directly with such organizations for access to data and data products and establish the utility of their selected data as surrogates for data used in operational nuclear explosion monitoring. The consortium will be responsible decisions about use of the data, such as whether all consortium projects make use of a shared set of tags on the data and shared data subsets for development and validation, respectively. The consortium will be responsible for quality control, such as ensuring that the data tags are accurate and that the subsets are representative of data typically used in operational nuclear explosion monitoring. 2.5. Consortium Specifics 2.5a. Request for Information Potential offerors are invited to submit white papers with consortium plans and rough order of magnitude cost estimates. White papers should describe the structure of the consortium, demonstrate the expertise in the fields involved, and identify likely areas of fruitful research. While preparing white papers, potential offerors may discuss ideas with NEM staff members and with other organizations with whom they might form a consortium. The goals of soliciting white papers include: - Encourage potential consortium leaders to formulate a role for intellectual leadership, rather than merely being a funding pass-through organization. - Emphasize the importance of substantive roles for consortium partners that make substantive contributions to developing and implementing a strategic plan. - Enable proposals for a consortium that is broad enough to include expertise in the research community that is already transforming seismic data processing. 2.5b. Request for Proposal The Air Force could possibly utilize the STAR Advanced Research BAA solicitation # FA9453-19-S-0002 under topic area two for mission effects and capabilities based on the responses of this Request for Information. 2.5c. Award: Type, Ceiling, and Performance Period The Air Force is considering an award as an Other Transaction (OT) for Research & Development. Other funding vehicles, such as a Procurement or Cooperative Agreement, will be considered and white papers should include indications of preferred vehicles. The objective is a vehicle for sustained efforts comprising a broad range of research and development that will transformatively improve nuclear explosion monitoring, other Air Force missions, and missions of customers of other AFRL/RV programs. The Air Force may award one or more vehicles that will not exceed ceiling of $20,000,000 (for all awards) and a performance period up to 5 years. 2.5d. Funding After allowing for incremental costs on existing contracts and initial costs of anticipated new stand-alone awards, NEM expects to have at least $700,000 from its own funding during each year of the award. NEM would seek contributions from its Air Force customer and other agencies with relevant interests so that total core funding would be $1,000,000 annually. The consortium leader would be expected to use less than half of the core funding for coordinating activities described in the proposal. During the first year, the Air Force expects the majority of funding to be used for fundamental research projects undertaken by the consortium members and described in the proposal. During subsequent years, the Air Force expects the majority of core funding to be used for research projects that advance the strategic plan, selected by the consortium internally using procedures that avoid a priori favoring the consortium leader or any particular consortium member. Contingent on available funding, the Air Force may provide additional funding during certain years, for research projects selected by the consortium using the same procedures and that accelerate progress towards the objectives of the strategic plan. 2.5e. Special Projects Additional AFRL/RV funding during some years might be for special research projects by subsets of the consortium members. AFRL/RV would provide a statement of objectives to the consortium leader and all of the consortium members about interest in a special research project. Through processes agreed upon within the consortium, some set of consortium members would participate in preparing a proposal that the consortium leader would submit to AFRL/RV. AFRL/RV would participate with the consortium to perform a technical evaluation of the proposal and, if it is acceptable, the funding would be provided to the consortium leader, which would distribute it within the consortium in accordance with the negotiated terms of the award. 2.5f. Other Funding To pursue further research projects related to the objectives of the Air Force or to enhance the consortium coordinating activities related to the objectives of the Air Force, and in consultation with AFRL/RV, the consortium could seek and accept further funding from other Federal government agencies, from U.S. state government agencies, from U.S.-based and internationally recognized corporations and foundations, or from intergovernmental agencies. 2.5g. Leadership The leader of the consortium would be the organization that submits the proposal. The consortium leader would be responsible for submitting vouchers to the Government and for preparing and submitting quarterly, annual, and final reports on consortium activities, to include consortium-funded research projects by other members of the consortium. Further consortium coordinating activities would be expected to foster wide expertise and a collaborative approach to delivering transformative capabilities in nuclear explosion monitoring. The proposal is expected to be specify coordinating activities that the consortium leader and members deem well suited to these objectives, which might include some or all of these examples: - Develop a strategic plan to identify essential avenues of research, carry out research into various avenues, and integrate research products into an effective system. - Hold meetings of researchers from consortium members and from selected other organizations to review research accomplishments and, as necessary, update or revise the strategic plan. - Establish a mechanism to share intellectual resources, such as tagged datasets. Some of the resources might be restricted to researchers affiliated with institutions that are members of the consortium, while other resources might be openly available. - Organize on-line activities, such as webinars, to develop shared expertise. - Convene short courses or training institutes for early-career researchers affiliated with consortium members and from selected other organizations to develop a collaborative community of next-generation researchers. - Organize challenges, perhaps with cash prizes, in which groups compete to create and configure an algorithm to process signals during a future time period from a global seismic network (e.g. the CTBTO International Monitoring System) that best matches human analysis (e.g. the CTBTO International Data Center). 2.5h. Membership The members of the consortium would be the organizations named as members in the proposal. Members could leave the consortium by notifying the leader and AFRL/RV. The consortium could add new members by procedures agreed upon within the consortium, but consultation with AFRL/RV would be expected. 3. REQUIRED ADDITIONAL INFORMATION TO BE SUBMITTED: a. Company Name and Mailing Address b. CAGE Code and DUNS Number c. Point(s) of Contact d. POC Telephone Number and e-mail Address e. Business Size Status (i.e. 8(a), HUBZone, woman-owned, Service Disabled Veteran-Owned, Small business, Large Business, Historically Black College/University) under NAICS 541715; the associated small business size standard is 1,000 employees f. Relevant experience in the past five years in the technical areas of interest g. Availability of an adequate number of qualified personnel to support the effort h. If an Other Transaction is desired a rough contract is requested with terms and conditions * Note: If a solicitation is released, it will be synopsized on the Federal Business Opportunities (FedBizOpps) website (www.fbo.gov). It is the responsibility of potential offerors to monitor the website for additional information pertaining to this notice or a requirement that may relate to this notice. 4. FORMAT & SUBMISSION Interested businesses are requested to submit a white paper no later than 17 October 2019; 3:00 PM MST. Please provide sufficient detail to indicate technical approach, consortium methodology, contract type preference, rough order of magnitude (ROM), previous experience, and supporting information that directly relates to the areas of interest identified in this request for information. Submittals should consist of an original plus 2 copies. In addition to the paper submittals, the Government desires an electronic version (CD-ROM) in searchable .pdf or Microsoft Word compatible format. Electronic submissions are NOT permitted. Please MAIL packages to the following address: AFRL/RVKB; (General Mail Room) ATTN: Mr. Eric Price or Ms. Anne Peel 3550 Aberdeen Ave SE; Bldg 570, Suite: 1160; Kirtland AFB, NM 87117-5776 Responses must be unclassified and any proprietary information provided must be portion marked accordingly. To protect such data, each line or paragraph on the pages containing such data must be specifically identified and marked with a legend similar to the following: "The following contains proprietary information that (name of responder) requests not be released to persons outside of the Government, except for the purposes of review." Submitted responses should be provided on standard letter size 8-1/2 by 11 inch paper, limited to a maximum of fifteen (15), single-sided, single-spaced pages. The font for text should be Times New Roman 12-point or larger. The Responder may use oversized pages (including "foldouts") where appropriate to contain graphic presentations. Oversized pages do not count as extra pages within the page limitations. Existing commercial documentation and product literature can also be submitted in addition to the 15 pages above and is not subject to a page limitation. All information received shall be safeguarded from unauthorized disclosure. Responses must be unclassified and any proprietary information provided must be marked accordingly. This is not a Request for Proposal (RFP) and is not to be construed as a commitment by the Government to issue a solicitation or ultimately award a contract. Any costs incurred as a result of this request for information shall be borne by the respondent and will not be charged to the Government for reimbursement. If your interest is only in subcontracting possibilities, please clearly indicate this on the cover page. Foreign firms are advised that they will NOT be allowed to participate under ITAR rules; please contact the Contracting Officer for conditions and restrictions. All submissions become Government property and will not be returned. 5. REQUESTS FOR CLARIFICATION A Responder may request clarification in writing to the Contracting Office for any requirement that is unclear by sending an e-mail to the Contract Specialist as identified above. Any requests for clarification must be received no later than 3 October 2019; 3:00 PM MST. Note: clarifications may be posted to benefit all interested Responders, do not include proprietary information in the clarification request. Interested parties are encouraged to periodically check the website during the response period for clarifications. 6. DISCLAIMER This is a Request for Information (RFI) ONLY as defined in FAR 15.201(e) to obtain information about capabilities and market information related to the technology of interest for planning purposes. This RFI is NOT a request for competitive proposals; therefore, responses to this notice are not considered offers and cannot be accepted by the Government to form a binding contract. Companies that respond will not be paid for the information submitted, nor is the Government required to provide feedback to RFI responders.