Increasing safety through situational awareness enhancement tools

When faced with multiple courses of action during an airborne operation, a set protocol helps narrow down the selection process quickly and easily. The reason that airborne decision making (or ADM) is different from other operational considerations is that the consequences of ‘getting it wrong’ are potentially catastrophic. At the very least, they can lead to mission failure and wasted time and money. At worst, especially in situations such as search and rescue, it could cost someone their life.

Airborne decision-making generally takes a four-step approach:

1. Defining the problem
2. Considering the options
3. Implementation
4. Post-mission analysis.

This streamlines the operational process and helps operators make a considered decision quickly and with the maximum amount of informational analysis. Following this protocol makes operators more likely to make the correct decision.

A systematic approach to decision making ensures the safety of the aircraft and the team onboard. It also enables coordinated responses on the ground and at central control points. In the case of airborne surveillance, this ensures that operations are carried out effectively, and that data transfer occurs seamlessly to maximize efficiency.

FlySight’s solutions have been selected by several players for the operator cognitive enhancement by introducing special features such as 2D/3D Moving Map, Augmented Reality and video processing techniques.

The first aspect of airborne decision making is to define the problem. For example, in a surveillance operation within an urban area, this could be identifying a target vehicle in a built-up area with multiple potential targets. The potential for ‘data overload’ could result in the surveillance operation being inefficient and, ultimately, ineffective.

Another element of problem definition is how it could impact the operation of the flight itself, including identifying potential hazards to safe flight. Again, in an urban environment, this could include areas where elevated hazards such as power lines could impact the operation’s success and the aircraft’s safety.

Considering the options for pre-flight working plans

At its most basic level, the 5P plan is most commonly used to prepare for an operation and will allow the crew and ground support teams to consider the problems and potential solutions before take-off. The 5Ps – Plan, Plane, Pilot, Passengers and Programming – work synergistically to create a pre-flight working plan that considers potential problems such as terrain considerations and excessive data gathering.

Operators can fine-tune their plan at this juncture, especially using the final P: Programming. Using software such as OPENSIGHT’s innovative Automatic Target Recognition (ATR) takes much of the hard work away from operators. It provides an ATR system that enables instant automatic real-time detection, classification and identification. Utilizing Deep Learning protocol, it can be fully integrated into existing operations and exploits the benefits of AI processing for exceptionally fast and accurate data acquisition.

Additional Programming tools such as Augmented Reality can give operators vital real-time information in 3D overlays, defining potentially hazardous areas and positioning the aircraft in its exact geospatial position. This assists pilots in planning not only the most effective and efficient route but the safest too.

Operations systems such as OPENSIGHT have a significant advantage because they are easily integrated into existing ground-level or onboard set-ups. They are also intuitive, allowing operators to quickly familiarize themselves with the processes and to adapt and course-correct mid-operation if necessary. The implementation of a progressive and augmented reality system such as OPENSIGHT provides a multitude of options for operations that are quickly and easily implemented.

A system incorporating ATR can eliminate one final yet crucial factor – operator fatigue. By analyzing the data and removing any extraneous data automatically, it allows the operators to focus more on the crucial mission-critical elements, reducing the chances of operator error through fatigue or informational overload.

Post-mission analysis allows revision of operational parameters

The final aspect of ADM is reviewing the outcome or post-mission analysis. Because systems such as OPENSIGHT use Deep Learning techniques, the mission can be revised, and future operations parameters can be redefined using the data gathered in real-time. This makes post-mission reviews more beneficial to future operations. The 5P system can be streamlined, including pre-mission checks and hazard identification protocols. Risk evaluation can be factored into future decision-making processing.

With the use of AR software, airborne decision making can be refined to a much higher degree, ensuring the safety of crews, the safety of the aircraft, and a much more efficient use of airborne units in all types of operations.

Mobile phone detection, location and communication systems in airborne SAR scenarios makes a real difference to outcomes. Nearly everyone these days carries a mobile phone. This has been a real boon to Search and Rescue teams, as it is now much easier to locate individuals in distress, even in remote areas. Traditionally, poor coverage meant that in some more isolated locations, it was impossible to use a mobile phone as an actual phone. This limited the mobile phone’s effectiveness in Search and Rescue operations.

However, using new systems that can locate a phone via geolocation technology effectively turns a mobile phone into a beacon for SAR teams. This is what ARTEMIS has achieved, and thanks to a collaboration with FlySight, it could be a revolution in SAR operations. This technology could save lives.

What is ARTEMIS?

ARTEMIS is a mobile phone detection, location and communication system specifically designed by Smith Myers for Search and Rescue operations. Because traditional methods of location can be slow, ARTEMIS has been fine tuned to give accurate geolocation information based on the signal picked up from a mobile phone, even in a technical ‘dead zone’ such as halfway up a mountain. ARTEMIS geolocates a handset quickly and accurately using not one but two geolocation techniques.

With a range of up to 19 nautical miles, ARTEMIS has a much wider reach than traditional geolocation systems. It enables operators to send texts and make calls in no-service areas, potentially allowing them to contact a target before reaching them to assure them that the target has been located and that help is on the way.

The system has been tested and is certified as RTCA DO-160-G, meaning it can operate even in the most challenging and rigorous operational environments. The system can be adapted to both airborne SAR units and ground teams, coordinating the data to provide shareable information in real-time. The hardware is compact, requiring only two small antennae.

Crucially, the software integration has been condensed into a single API document that covers all mission systems and sensor integration. Basically, ARTEMIS is a plug-and-go system that can be applied to any SAR scenario.

Using the highly advanced OPENSIGHT system, FlySight and Smith Myers have joined forces to deliver ARTEMIS to SAR teams around the globe. By integrating ARTEMIS and OPENSIGHT, operators can get a much clearer operational picture of any scenario. By combining ARTEMIS with OPENSIGHT technology and plug-ins such as Fog Suppression Retinex, SAR teams can get a much more detailed overview of the terrain from an airborne SAR unit.

Using ARTEMIS, they can then home in on the target’s mobile phone signal and geospatial position. At the same time, OPENSIGHT’s multi-layer AI and enhancements make it easier to direct ground units towards the target via the shortest and most easily accessible route.

The additional data provided by mass-mapping and geofencing ensures that crews work as efficiently as possible, shortening the time it takes to find the target and minimizing crew fatigue which could impact their ability to locate a person in distress.

Using their advanced Augmented Reality software, the information gathered from ARTEMIS can be overlaid and accurately rendered into a 3D map. This gives ground-based SAR teams a more in-depth understanding of the terrain, allowing them to choose the safest and/or fastest route to the target. So not only does the ARTEMIS/ OPENSIGHT combination provide a faster rescue for the target, but it ensures the safety of SAR teams, both on the ground and in the air.

Whether it’s a mountain rescue or a coastal search, any system that allows SAR teams to locate the exact position of their target quickly has a massive impact on a successful outcome. ARTEMIS and OPENSIGHT could revolutionize modern SAR tactics and, consequently, save countless lives in the future.

Data collection is key to airborne special missions operators; how do the high-quality digital imaging options that FlySight offers ensure that data is accurate and accessible to users in the air and on the ground?

Searching through massive amounts of metadata can not only be costly and time-consuming, but it can jeopardize the success of an ongoing operation. Information management systems need to be broad-ranging but tunable so that specific data can be quickly processed, identified and analyzed.

What is OPENSIGHT?

OPENSIGHT is a multi-platform providing processing, exploitation and dissemination of data. By integrating a geo-exploitation toolbox with augmented reality capability and the option to create multi-layer processing in real-time, OPENSIGHT gives the ability to gather, analyze and dissimilate data quickly and easily.

OPENSIGHT complies with all current NATO Standardization Agreement (STANAG) guidelines and is based on over a decade’s worth of experience in software interfacing for airborne information processing.

How does OPENSIGHT work?

OPENSIGHT uses modular plug-ins that can be customized to the user’s requirements. These plug-ins work symbiotically with similar applications, allowing the user to create a library of plug-ins that can be accessed using a touchscreen. The Software Development Kit enables users to create plug-ins specifically designed for applications. These can include geospatial awareness that provides more accurate data and gives the mission context. This is particularly useful in operations such as border control, where data analysis may indicate a ‘weak spot’ in a border defense system.

Real information is overlaid onto a synthetic environment, allowing mission managers to create scenarios and accurate forecasts. Using our border example again, they may be able to analyze the flow of traffic across a particular crossing point at various times of the day and night.

By analyzing the information gathered during the mission and putting it into context using geospatial positioning, along with real-time and mission-archived data, mission controllers may be able to predict more accurately when the crossing point is under the greatest strain or, at its worst, most vulnerable.

Situational awareness systems

The most important thing about any data analysis tool is its adaptability. With OPENSIGHT Software Development Kits, users can develop solutions for every aspect of the process, from planning and operations initiation to analysis, reports and even forecasting.

The system also allows system integrators to manage information in a synthetic environment for Geospatial Situational Awareness, including NATO systems and existing geodatabases, giving users pinpoint accuracy.

This ability to process data from a variety of resources also allows the user to layer information onto a 3D map, which will give the user the exact location and elevation of their airborne platforms. Combining this with augmented reality makes this process practical in both urban and rural settings, adding a new dimension to acquiring data points and data collation.

This is particularly useful in urban areas where the issue of data ‘noise’ can be at its worst. The algorithmic analysis of the data with pre-determined parameters ensures that the user avoids the ‘too much data’ conundrum.

By making the most of the technology available, operations managers can narrow down and define exactly what information they need to make mission-critical decisions quickly and easily. These decisions can then be passed upstream to the operators, and both air and ground crews to action.

The result is a far more efficient system with the flexibility to adjust the defined algorithmic parameters to suit each individual mission target.

Acquisition to examination – disseminating information from the source

It is vital that the information that the unit gathers while in operations is not only accurate and data rich but is also easy to disseminate back in the control room.

Real-time data can be assessed, analyzed, and acted upon faster and more effectively. The data can also flow both up- and downstream, providing all teams with crucial information when it’s needed most.

OPENSIGHT Analysis Console

To process the sheer amount of data that can be acquired on a mission and to extrapolate the essential points from that data while filtering out the ‘noise’ that has no relevance, OPENSIGHT Analysis Console has been developed to enhance and streamline the process.

The system can handle the extensive amount of STANAG 4609 data produced by an integrated collection system. By programming specific criteria into the search parameters, the console can quickly and accurately identify what is relevant and what isn’t.

However, nothing is better than an experienced ‘pair of eyes’ on any situation. OPENSIGHT Analysis Console allows operations managers and analysis experts to supervise real-time operations and liaise with pilots and airborne operators to adjust the mission based on real-time analysis.

As a decision-support system, OPENSIGHT Analysis Console demonstrates how information sharing can be achieved quickly and in real-time operations – essential for the effectiveness of any operation, especially those involving airborne units. It also provides a cohesive and wide-ranging integrated system that works with STANAG existing data-collection systems, upgrading the data acquired with augmented data, georeferencing and full motion video.

This level of sophistication ensures that operations can be planned, executed and debriefed efficiently, regardless of the mission parameters or the number of agencies involved.