XR for Helicopter Pilots

Olga Resnik
12 minutes ago
4 min read

I want to talk more about complex use cases for professional, para-military and military market. Essentially this market has been using and developing XR technologies for decades, and this used to be a niche XR market.

Now gaining more and wider spread through different fields of application, still XR products are well established mostly in defense and military applications, where the use cases are very clear and the value XR brings is undisputable.

One of such application is XR for Helicopter Pilots: landing a helicopter in low visibility conditions.

Use Case: Helicopter landing in low visibility conditions

In critical defense fields, situational awareness in extreme environments is key. Navigating and flying the helicopter in low visibility conditions, in low-light or night environments, in presence of such effects as brown-out in a desert condition, or white-out in a snow storm, identifying hazards, locating people and safely performing find-and rescue and other critical missions require the most advanced systems, high-end engineering and expertise feature integration.

Here you absolutely need a custom tailor-made solution without any compromises.

How can XR help safely land a helicopter in low visibility or complex terrain conditions?

Let’s dive in the XR Feature Analysis.

XR for Helicopter Pilots Feature Analysis

🟩 Key Features

Display (Feature#1): Visual instructions and data are essential in a complex low-visibility environment for improving scene visibility.
AR Overlay (Feature#5): The situational awareness is super critical in this application, so it must be elevated with additional AR overlayed data, assuring perfect alignment of the digital data on the real-world objects and landmarks.
Data storage (Feature#6): Video recording for real-time processing and post-mission debriefing and learning.
Super Vision (Feature#8): Multi-spectral imaging (infrared, thermal) is the core enabler in this task: enhancing vision in low-visibility environments sometimes induced by the helicopter itself or the external conditions.
3DoF (Feature#9): Use 3Dof tracking to keep the projected data including object markers steady during head movements, or

🟨 Trade-Offs & Optional Enhancements

Some decisions involve trade-offs and design choices that set up the optimal basic functionality point. Each trade-off decision has its implications, and these are not easy decisions to make.

3D Display (Feature#2): A 3D display can enhance depth and distance to objects visual perception. This optional feature is most commonly realized and selected as a base feature since most helicopter pilots XR systems use binocular vision. Thus, adding 3D display comes without changing the hardware, only implementing the software 3D images construction using convergence between the 2 eyes.
Smart Vision (Feature#7): AI-powered video processing for objects recognition to identify obstacles and people, create virtual terrain profile based on real-time data can elevate the accuracy of the mission critical data.
6DoF (Feature#10): 6DoF full spatial tracking for complete orientation and spatial awareness in low visibility environment is a potential upgrade.
User Input (Feature#11): Voice control and pilot fatigue monitor and detection via eye tracking can provide added value and increase pilot safety during the mission.
System Output (Feature#12): Alternative or additional system-user communication such as audio cues or haptic feedback, can improve situational awareness and reduce cognitive mission load.
Multi-User Interaction (Feature#13): Adds real-time team collaboration to share insights and info between different teams on the spot in real time. This feature can help coordinate the teams and share the data between the multiple users and a commanding station. This can further improve the mission efficiency and upgrade the functionality.

Use-case based System Development

This use case is another good example of a field of application where XR’s true potential shines—solving high-complexity challenges where precision, safety, and mission success are at stake.

Moreover, this use case is well established and the XR products in this and other use cases in the military and para-military fields exist for many years. Moreover, the approach of use-case based XR development is a standard in this field. Many attempts to switch the approach and create platforms that may cover many potential use cases has yet been unsuccessful – mainly because this approach proposes non-optimal solution for any specific use case.

After mapping all the required features, it’s evident that a dedicated design and system development is required and there are no shortcuts here. This mapping in the beginning of design stage helps to set system architecture and development strategy for this unique product.

XR Platform Design Partner

Here JOYA Team brings our highest value as experts in XR development and design, making us the top partner for companies that aim to design tailor-made products especially in cases like this one.

TWe can explain each trade-off decision and its impact on the system costs, risks and development timeline. We can help you select the right technologies, optimize the architecture and design, fully understanding the impact of every decision along the development source.

We also help you focus the design efforts, bring our vast experience to your services and help you minimize the design budget and timeline.

Find out more about our experience in the AR Design field.

Contact us to learn more and we will develop your XR platform together.