Visual AI for Space Robotics Obstacle Detection

Amendment 004 - An attachment has been added to extend the closing date for this challenge to November 3, 2022 at 14:00 EDTAmendment 003 - An attachment has been added to extend the closing date for this challenge to October 27, 2022 at 14:00 ESTAmendment 002 is raised to add details of the security requirement, update the link for the Contract Security Manual and to add the Security Requirement Checklist as attached document, Annex C*Please note this challenge has Reliability Security Clearance requirements.Before award of a contract, the following conditions must be met:(a) the Bidder must hold a valid organization security clearance as indicated(b) the Bidder's proposed individuals requiring access to classified or protected information, assets or sensitive work sites must meet the security requirements(c) the Bidder must provide the name of all individuals who will require access to classified or protected information, assets or sensitive work sites;See the Contract Security Manual for more information -  Contract Security Manual – Security requirements for contracting with the Government of Canada - Security screening - National security - National Security and Defence – Canada.ca (tpsgc-pwgsc.gc.ca)AMENDMENT 001 - An attachment has been added. The document contains questions and answers related to the Challenge.This Challenge Notice is issued under the Innovative Solutions Canada Program (ISC) Call for Proposals 003 (EN578-20ISC3). For general ISC information, Bidders can visit the ISC website.Please refer to the Solicitation Documents which contain the process for submitting a proposal.Steps to apply:Step 1: read this challengeStep 2: read the Call for ProposalsStep 3: propose your solution hereChallenge title: Visual AI for Space Robotics Obstacle DetectionChallenge sponsor: Canadian Space Agency (CSA)Funding Mechanism: ContractMAXIMUM CONTRACT VALUE:Multiple contracts could result from this Challenge.Phase 1:The maximum funding available for any Phase 1 contract resulting from this Challenge is : $150,000.00 CAD excluding applicable taxes, shipping, travel and living expenses, as required.The maximum duration for any Phase 1 contract resulting from this Challenge is up to 6 months (excluding submission of the final report).Estimated number of Phase 1 contracts: 2Phase 2:Note: Only eligible businesses that have successfully completed Phase 1 will be invited to submit a proposal for Phase 2.The maximum funding available for any Phase 2 contract resulting from this Challenge is : $1,000,000.00 CAD excluding applicable taxes, shipping, travel and living expenses, as required.The maximum duration for any Phase 2 contract resulting from this Challenge is up to 24 months (excluding submission of the final report).Estimated number of Phase 2 contracts: 1This disclosure is made in good faith and does not commit Canada to award any contract for the total approximate funding. Final decisions on the number of Phase 1 and Phase 2 awards will be made by Canada on the basis of factors such as evaluation results, departmental priorities and availability of funds. Canada reserves the right to make partial awards and to negotiate project scope changes.Note: Selected companies are eligible to receive one contract per phase per challenge.TravelFor Phase 1, a successful bidder may need to travel to the Canadian Space Agency (Longueuil, QC) for the Final Review Meeting.Kick-off meetingTeleconference/videoconferenceProgress review meeting(s)Teleconference/videoconferenceFinal review meetingLongueuil, Quebec or teleconference/videoconferenceAll other communication can take place by telephone, or videoconference.Challenge Statement SummaryThe Canadian Space Agency (CSA) is seeking solutions that will improve the autonomy of future space robotic systems through the use of a vision system based on Artificial Intelligence (AI) techniques to detect in real-time potentially hazardous obstacles and verify clearance margins in dynamic and uncertain environments.Challenge StatementCollision-free autonomous path execution in a dynamic and uncertain environment is an important issue which CSA seeks to resolve in order to increase robotic system autonomy for future space missions. Robotic systems operate on space infrastructure or in other environments where collisions can potentially be catastrophic. Future space systems will need to have greater autonomy in order to be less reliant on communications with the ground and to alleviate crew workload; consequently they will need to rely on multiple layers of intelligent software for supervision and safety assurance. Cameras both built into the robot and mounted to the infrastructure provide a relatively low mass, power, and volume solution for sensing the robotic workspace. Canadarm2 for example is equipped with on-board cameras in its end-effectors and along its booms in order to allow human operators to guide and monitor its operations. Future missions such as the cislunar Gateway call for autonomous robotic operations during periods when video cannot be downlinked to the ground for oversight. This challenge seeks to explore the feasibility of an AI-based vision system to assist or offload the need for human ground controllers to monitor clearances. For CSA, an efficient robot supervisory system translates into improved safety and operational efficiency of the mission. CSA believes that a computer vision system based on machine learning will be able to detect obstacles and monitor proximity to structure and obstacles while handling payloads. CSA can provide Mobile Servicing System and International Space Station video and telemetry data during phase 1 and 2.Desired outcomes and considerationsEssential (mandatory) OutcomesThe solution must:Operate in real-time, in a space environment, under all solar lighting conditions.Use 2D camera data (still images and video streams) from existing cameras built into a space manipulator or mounted to its infrastructure and not assume any additional sensor functionality.Have sufficient resolution and accuracy to detect the presence of slender objects (~2 cm wide) such as cable harnesses and EVA tethers.Have a false positive rate of 2 % or lower and a false negative rate of 1% or lower.Have to assume that the cameras could be mounted on the manipulator's booms or end-effector (e.g. may not be stationary during operation).Work with a data bandwidth (video stream from the camera to processor) limited to 512 kbps and an update rate of at least 2 Hz.Provide a 3D virtual world display of the manipulator and any obstacles detected.Provide obstacle detection along the entire length of a space robotic arm and about any attached or manipulated payloads.Be able to operate on a typical space platform such as a 2-4 core, ARM-based FPGA System on a chip, running VxWorks or Linux, a few GB of RAM, a solid-state drive, a power budget of approximately 10 Watts, Ethernet connectivity capable of transfer rates of at least 100Mbps.Additional OutcomesThe solution should:Have robust performance with respect to distorted and blurred images streamed from the onboard cameras due to the oscillations caused by movements of the robotic arm.Incur minimum budget increase to power, mass, volume, and computation resources (CPU and memory) allocated to a space robot's avionics concept.Investigate the feasibility of adding path-to-flight computing hardware to the on-board avionics if the computation power of the space robot’s avionics is found to be insufficient.Be able to detect features inside of a given clearance volume or a keep out zone, for example a cylinder with depth of 2.4 m and diameter 2 m.Investigate multiple cameras to cover the entire volume of the keep out zone.Use a vision algorithm requiring a minimum number of independent camera views (expected to be part of study).Be applicable to a 7 degree of freedom, offset joint robot similar to the Special Purpose Dexterous Manipulator on the International Space Station.Background and ContextIt is a critical safety consideration for a space servicing robotic manipulator to avoid unintended contact during operations. For example, collision prevention in Canadarm2 begins at the planning stage. Robotic trajectories are designed using a kinematic simulator that uses high-fidelity graphical models of the external International Space Station (ISS) structure, the Space Station Remote Manipulator System (SSRMS), and the Special Purpose Dexterous Manipulator (SPDM). Robotics mission designers use this tool to define a series of poses and trajectories for the manipulator. A graphical planning tool can be used to verify that the trajectory meets the needs of the mission while avoiding singularities, self-collisions, and joint limits, and respecting clearances to structure and any defined keep-out zones (e.g., a volume around a radiating antenna). While these planned paths are a good starting point, they cannot be executed blindly for the following reasons:Disturbances in the station geometry due to pressure effects at docking ports and thermal expansion/contraction cannot be accounted for in the Computer-Aided Design (CAD) model.Differences may exist between the as-designed model and the as-built modules.Space vehicles have appendages such as solar arrays and antennas which can deploy or articulate to maintain their required pointing. If their articulation states are unknown, the robot must avoid their entire possible swept volume.The exterior configuration of the vehicle may be physically altered by Extra-Vehicular Activity (EVA) crew members, left-behind debris, or installation of logistics and science payloads which were not represented in the model at planning time. Faults in the attachment of external components may occur, such as cable harnesses coming loose or floating vehicle tether, and these may interfere with the operational workspace of the robotic system. EVA crew members may not be aware of or misjudge upcoming manipulator motion, so may accidentally be in the planned path of the end-effector or booms.Again in the example of Canadarm2, at execution time, when the manipulator’s end effector approaches proximity to the station, a human operator must observe all motions to ensure that clearance is maintained between the station structure, arm, and payload. The SSRMS elbow pan-tilt-zoom cameras, the station infrastructure cameras, or both can provide the necessary video views for this task. For SPDM operations, ideal camera views are often not available. It is important to note that the clearance monitoring hinges on the human’s perceptual system to filter out noise, infer depth, and accommodate the harsh and variable on-orbit lighting conditions. For future space missions, such as the cislunar Gateway station, the majority of robotic operations are expected to be performed autonomously with no crew onboard and no communication link to the ground. As such, the hazard control function that was previously provided by the operator watching the downlinked video must be performed by some aspect of the onboard system. The AI-based vision system is one possible solution for providing this hazard control. It may be possible for the robot to build or correct its world model based on some high-density exteroceptive sensors (e.g., stereo camera or lidar). This approach would require accurate and reliable 3D reconstruction, registration, and segmentation, and these problems have not been solved to the point where they can be fully trusted with the safety of a crewed space vehicle without human supervision. As such, a reactive behavior-based system that uses real-time sensing to gracefully override commands which would result in collisions seems to be a more appropriate solution. The solution being sought takes the converse approach by using existing or easily accommodated sensors (i.e., cameras), and a more advanced decision-making algorithm. AI-based approaches such as machine learning, pattern recognition, or spatiotemporal reasoning can be used to detect obstacles and monitor proximity to structure. The operation is then halted (or potentially adjusted autonomously) if any unexpected object is found to be in proximity to the arm. An example use case of the vision system is as follows:A dexterous manipulator is autonomously maneuvering to insert a payload into a berthing location.Mission control, and in particular the robotics personnel who planned the operation, are unaware that a slender antenna from an adjacent site has come loose, and is now floating and partially blocking the approach corridor.As the payload is being moved into position, the vision system estimates that there is insufficient clearance between the payload and the antenna.The autonomous agent halts the motion of the arm, and signals to the ground that the operation cannot continue as planned.ENQUIRIESAll enquiries must be submitted in writing to contact@example.gov no later than ten calendar days before the Challenge Notice closing date. Enquiries received after that time may not be answered.How to Prepare a Bid