Cyber Space Odyssey (CSO) is a novel serious game supporting computer networking education by engaging students in a race to successfully perform various cybersecurity tasks in order to collect clues and solve a puzzle in virtual near-Earth 3D space. Each team interacts with the game server through a dedicated client presenting a multimodal interface, using a game controller for navigation and various desktop computer networking tools of the trade for cybersecurity tasks on the game's physical network. Specifically, teams connect to wireless access points, use packet monitors to intercept network traffic, decrypt and reverse engineer that traffic, craft well-formed and meaningful responses, and transmit those responses. Successful completion of these physical network actions to solve a sequence of increasingly complex problems is necessary to progress through the virtual, story-driven adventure. Use of the networking tools reinforces networking theory and offers hands-on practical training requisite for today's cyberoperators. This paper presents the learning outcomes targeted by a classroom intervention based on CSO, the design and implementation of the game, a pedagogical overview of the overall intervention, and four years of quantitative and qualitative data assessing its effectiveness.

Once realized, autonomous aerial refueling will revolutionize unmanned aviation by removing current range and endurance limitations. Previous attempts at establishing vision-based solutions have come close but rely heavily on near perfect extrinsic camera calibrations that often change midflight. In this paper, we propose dual object detection, a technique that overcomes such requirement by transforming aerial refueling imagery directly into receiver aircraft reference frame probe-to-drogue vectors regardless of camera position and orientation. These vectors are precisely what autonomous agents need to successfully maneuver the tanker and receiver aircraft in synchronous flight during refueling operations. Our method follows a common 4-stage process of capturing an image, finding 2D points in the image, matching those points to 3D object features, and analytically solving for the object pose. However, we extend this pipeline by simultaneously performing these operations across two objects instead of one using machine learning and add a fifth stage that transforms the two pose estimates into a relative vector. Furthermore, we propose a novel supervised learning method using bounding box corrections such that our trained artificial neural networks can accurately predict 2D image points corresponding to known 3D object points. Simulation results show that this method is reliable, accurate (within 3 cm at contact), and fast (45.5 fps).

Achieving precise 6 degrees of freedom (6D) pose estimation of rigid objects from color images is a critical challenge with wide-ranging applications in robotics and close-contact aircraft operations. This study investigates key techniques in the application of YOLOv5 object detection convolutional neural network (CNN) for 6D pose localization of aircraft using only color imagery. Traditional object detection labeling methods suffer from inaccuracies due to perspective geometry and being limited to visible key points. This research demonstrates that with precise labeling, a CNN can predict object features with near-pixel accuracy, effectively learning the distinct appearance of the object due to perspective distortion with a pinhole camera. Additionally, we highlight the crucial role of knowledge about occluded features. Training the CNN with such knowledge slightly reduces pixel precision, but enables the prediction of 3 times more features, including those that are not initially visible, resulting in an overall better performing 6D system. Notably, we reveal that the data augmentation technique ofscalecan interfere with pixel precision when used during training. These findings are crucial for the entire system, which leverages the Solve Perspective-N-Point (Solve-PnP) algorithm, achieving 6D pose accuracy within 1 $$^\circ$$∘ and 7 cm at distances ranging from 7.5 to 35 m from the camera. Moreover, this solution operates in real-time, achieving sub-10ms processing times on a desktop PC.

In-flight refueling of unmanned aerial vehicles (UAVs) is critical to the United States Air Force (USAF). However, the large communication latency between a ground-based operator and his/her remote UAV makes docking with a refueling tanker unsafe. This latency may be mitigated by leveraging a tanker-centric stereo vision system. The vision system observes and computes an approaching receiver’s relative position and orientation offering a low-latency, high frequency docking solution. Unfortunately, the boom – an articulated refueling arm responsible for physically pumping fuel into the receiver – occludes large portions of the receiver especially as the receiver approaches and docks with the tanker. The vision system must be able to compensate for the boom’s occlusion of the receiver aircraft. We present a novel algorithm for mitigating the negative effects of boom occlusion in stereo-based aerial environments. Our algorithm dynamically compensates for occluded receiver geometry by transforming the occluded areas into shadow volumes. These shadow volumes are then used to cull hidden geometry that is traditionally consumed, in error, by the vision processing and point registration pipeline. Our algorithm improves computer-vision pose estimates by 44% over a naïve approach without shadow volume culling.

This paper presents a comprehensive approach to enhancing autonomous docking maneuvers through machine visual perception and sim-to-real transfer learning. By leveraging relative vectoring techniques, we aim to replicate the human ability to execute precise docking operations. Our study focuses on autonomous aerial refueling as a use case, demonstrating significant advancements in relative navigation and object detection. We introduce a novel method for aligning digital twins using fiducial targets and motion capture data, which facilitates accurate pose estimation from real-world imagery. Additionally, we develop cost-efficient annotation automation techniques for generating high-quality You Only Look Once training data. Experimental results indicate that our transfer learning methodologies enable accurate and reliable relative vectoring in real-world conditions, achieving error margins of less than 3 cm at contact (when vehicles are approximately 4 m from the camera) and maintaining performance at over 56 fps. The research findings underscore the potential of augmented reality and scene augmentation in improving model generalization and performance, bridging the gap between simulation and real-world applications. This work lays the groundwork for deploying autonomous docking systems in complex and dynamic environments, minimizing human intervention and enhancing operational efficiency.