作者: HA0568

The Street View Running System aims to address the various obstacles that hinder individuals from exercising regularly by bringing the outdoor running experience indoors. Designed to enhance the enjoyment of indoor running, this system displays Google Maps Street View images directly to users while they run on treadmills or indoors. The system comprises three main components:

1. Kinect System: Developed by Microsoft, the Kinect system detects the user’s running movements.
2. Real-time Google Maps Integration: Based on the user’s running speed, the system displays and moves the Street View images along the selected running route on Google Maps.
3. Database of Famous Running Routes: The system incorporates a database of renowned running routes worldwide.

Users can engage in indoor running within the effective range detected by the Kinect system, eliminating the need for a treadmill. As users perform running motions recognized by Kinect, the system calls the Google Maps API to display the selected running route’s Street View images. With this system’s assistance, users can conveniently engage in running exercises indoors while immersing themselves in the realistic street views from various locations worldwide, enhancing the experience of their indoor runs.

The depth CNN face recognition system consists of four main steps: face detection, localization, feature extraction, and matching. Face detection and localization are based on existing methods but have been modified to achieve faster computation speed and higher accuracy. For feature extraction, deep learning techniques are employed. We propose using Extreme Feature Mapping as the non-linear function in the CNN and introduce Residual Networks (ResNet) to develop a deep neural network with a total parameter count of just over eight million. With a training dataset comprising over a million images of about twenty thousand faces, we can extract 256-dimensional face feature vectors. In testing with the LFW face database, the system achieves a recognition accuracy of up to 98% from face detection, localization, feature extraction, to recognition.

 

For the analysis of flying areas, this subcategory employs SegNet to segment images, distinguishing between objects of interest and the background. Subsequently, it utilizes the method of Deeper Depth Prediction with Fully Convolutional Residual Networks to predict depth and applies it to aerial photographs. Finally, it combines deep learning with YOLOv2 to detect buildings from any angle, determining whether there are structures within the flight area. This method is then used for the analysis of flying areas.

For constructing environment 3D models and dynamic textures, we utilize an image-based approach to calculate sparse point clouds from 2D images, which are then used to reconstruct the 3D model of the real scene. After reconstructing the model, in order to reduce the computational load for graphics processing and improve the performance of the UAV simulator, we must simplify the 3D model. Here, we employ the model simplification algorithm in Meshlab for this purpose.

The establishment of the unmanned aerial vehicle (UAV) simulator is based on Microsoft’s developed AirSim simulator. Through this simulator, we can freely control UAVs in a virtual world without limitations on venue, weather, or time. Additionally, there is no need to worry about battery life issues or safety concerns such as crashes due to improper operation or loss of control. Furthermore, we integrate the remote controller with VR display, allowing users to see the UAV in the virtual world via VR goggles and control it through the FrSky remote controller.

In the field of feature matching research, utilizing deep learning in this field to overcome variations lighting condition across different images has long been a challenging task. In this study, we propose III-Net, a novel training model that addresses feature matching using our proposed double-triplet loss with softmax loss. This approach effectively enhances the overall performance and stability of feature matching. Unlike previous deep learning architectures, III-Net specifically handles scenarios with significant variations in lighting conditions during feature matching. As a result of this additional processing, our approach demonstrates superior performance in real-life scenarios compared to all previous research outcomes.

The intelligent obstacle detection system utilizes the camera on the unmanned aerial vehicle (UAV) to directly detect obstacles in the scene, employing two frameworks:

1. Framework One: Integrates obstacle detection based on deep learning and depth prediction methods into a single unified framework, achieving real-time obstacle detection and depth prediction.
2. Framework Two: Utilizes deep learning to combine obstacle detection methods with scene depth prediction methods, establishing a neural network model for real-time obstacle detection and scene depth prediction.

The obstacle avoidance control using deep learning employs the deep deterministic policy gradient (DDPG) as the deep reinforcement learning algorithm. This algorithm involves two neural networks, including the decision network and the evaluation network, interacting with the environment. The system is utilized the potential field algorithm based on the information of surrounding obstacles and the target position to construct a virtual potential field. Then, the algorithm calculates the acceleration vector at that location as the direction of movement and speed judgment for the unmanned vehicle. By incorporating a small amount of expert operation information as guidance, the algorithm is encouraged to develop better strategies while mimicking the expert. This approach is known as “from demonstration,” imitating and refining strategies more effectively from demonstrated policies.

The Simulation Environment Automatic Flight (Imitation Learning) Training system references the Pilot Net model architecture published by Nvidia in 2016. It develops a supervised learning Convolutional Neural Network (CNN) model. After feeding a photo into the system, it responds an action that the UAV should take at that moment, mimicking the judgments made by human experts when observing the scene.

The Unmanned Aerial Vehicle (UAV) Intelligent Sensory Feedback System utilizes sensory systems other than human vision to establish a multi-sensory intelligent connection between the UAV sensing system (including flight attitude, flight environment, etc.) and the operator. The first step is focusing on tactile and auditory senses to conducts feed-back experiments on information concretization. The feedback information includes two categories: flight information (such as ascent/descent, acceleration/deceleration, wind resistance, orientation, obstacle detection) and flight environment (airspeed, temperature). The sensory types include vibration, pressure, and sound.