Flight simulation has been a supplementary part of live flight training since Edwin Link delivered his first simulator in 1927. With aeromedical and air rescue providers around the world seeing increased operational tempo, more complex missions and shrinking budgets, it is no wonder that defence, law enforcement and paramilitary rescue agencies have steadily moved to increase simulation as a means to enhance and optimise training, safety and operational readiness.

Industry challenges

Operators have to support training systems for longer periods of time as budgets become more stretched, so it is important to provide simulation products that will protect customer investment and sustain training requirements throughout the lifetime of a programme. This means offering a product that is mature and stable, with a complete set of features that utilises advanced image generation rendering techniques, while also supporting the latest sensors used in modern military, aeromedical and SAR platforms. Also, as training life cycles become longer, it is important to provide solutions that will allow for technology insertion to take advantage of hardware / software updates while maintaining the customers initial investment – this means that simulators must be scalable and easily upgradeable.

Dr James Frey of Plexsys Interface Products is a retired US Navy helicopter pilot and a globally recognised expert in the training and simulation industry. He spoke to AirMed&Rescue about the tasks facing the simulation technology sector. “The challenge is that we, in the simulation innovation side of the industry must prove that simulation can replicate the necessary tasks and cognitive stressors needed for effective training. That is, true cognitive stimulation through quality simulation – the kind where the student is physically and mentally immersed in the scenario – this is the new baseline to meet if we are to get this right for students and convince them to go to the schoolhouse instead of the runway for the majority of their training,” said Frey. “Increasingly, both government and commercial clients are realising training simulation is as much about the software as it is the hardware. While procurements need to maintain focus on vehicle or cockpit realism in fit, form, and function, we are reaching a tipping point where the industry is able to provide realism for training with means other than just realistic control sticks. Hybrid cockpit solutions with augmented reality or mixed reality technology surrounded by higher-resolution visual solutions are finally getting the attention they deserve. These technologies have the potential to transform the fleet of what we already have into a fleet of what we actually need for modern, high-value, meaningful operational training.”

Solutions

Innovation has led to the push for immersive technologies such as live, virtual, constructive (LVC) – or real mixed with synthetic training – as an evolution to traditional simulator technology. Another adjunct to simulation is the evolution of gaming technology, which was initially designed with the goal of entertainment, and is now perfectly acceptable for operational training and rehearsal. By leveraging the latest simulation and appropriately scoped visualisation technologies, students or crews can exploit the flexibility and speed of games to maximise user experiences in challenging and realistic environments.

While leading Operation Blended Warrior at the 2017 Interservice / Industry Training, Simulation and Education Conference (I/ITSEC), Dr Frey was able to link live players from three continents for the first time ever

While leading Operation Blended Warrior at the 2017 Interservice / Industry Training, Simulation and Education Conference (I/ITSEC), Dr Frey was able to link live players from three continents for the first time ever. The Navy and US Marine Corps team, led by CDR Gay (US Navy) and Dr Frey, used multiple industry partners, including live military simulators from Sweden, France and Brazil, and Serious Game hardware and software, across the showroom floor from companies like Bohemia, CAE, Lockheed Martin, Plexsys, and Rockwell Collins. It was the first time this scope had been attempted in a live demonstration which linked players from across the globe into the showroom floor to demonstrate the Viking 2018 exercise research and development efforts for NATO.

“Games for training is not a replacement for live behavioural learning. There will always be a need to get one’s boots dirty, but if done well, this truly is where training becomes learning,” said Frey.

Also speaking about game-based training was Adam Breed, Prepar3D engineering project manager at Lockheed Martin: “Overall, it is proven that game-based simulation can really enhance training. There’s greater learner engagement and it increases the speed of learning. Additionally, self-based learning has proven effective in controlled experiments and it leads to a more effective use of academic time.”

The art of simulation

The art of creating an immersive simulated environment is achieved through scientifically designed components that consist of advanced image generator systems and high-density databases which coalesce to form what appears to users as a live-action picture.

Andrew Fernie, Senior Technical Fellow at CAE, shared his views on image generators for simulators: 

“Users continue to look for better image quality and more realism. Each person evaluating a visual system makes a comparison, consciously or unconsciously, against what they see with their own eyes in the real world, and no matter how far the industry has come, we still have work to do to satisfy that test of making the synthetic environment more like the real world. Improvements in resolution, density of the content, and the fidelity of special effects will contribute to meeting expectations, and will require work on all aspects of the visual system: database content, image generator, and display system. At the same time, however, we are expected to reduce the life cycle costs associated with the visual system.”

Users continue to look for better image quality and more realism

Trainees have traditionally interacted with visuals via projection onto screens, however the growing use of head (or helmet) mounted displays (HMD) is an effective way to display visuals which trainees can immerse themselves in. The key strengths of HMD technology in training is its lower cost, smaller footprint, and transportability.

In most current cases, the use of HMD in simulation is focused on entry-level training and individual skills training; it is also well-suited for special purpose trainers where there is no need for a physical cockpit or instrument panel.

“Many of our military pilot clients require training in skill sets involving realism in visualisation and task saturation. Situational awareness, close air-support, hazard avoidance, gauge monitoring, delicate stick skills in operations like refueling or formation flight do not require switches and knobs; they need flight controls and good proprioceptive feedback (sound, vibrations, visualisation). This does not require expensive full cockpits,” Dr Frey told AirMed&Rescue. “We often find ourselves trying to help customers understand they can do more with less, get more training and learning for less cost, and maintain safe currency with training systems which put pilots and aircrew into the scene rather than into a cockpit.”

While it appears to be a hard argument to make, HMD technologies can put trainees into an environment where their eyes are seeing the correct reactions from the aircraft and surrounding environment. Dr Frey explained during a speech at I/ITSEC that if one couples this with simplified but accurate flight controls and pedals, they have designed a low-cost, high-use learning system that ‘could fold and fit onto a single pallet, and be deployed worldwide’. Some contend that more advanced applications will continue to rely on complex displays using projection, while others see HMDs employing blended or mixed reality as the next evolution in simulation training.

Taking the next step

The commercial market is making advances in  VR, AR and MR, however the big unknown is whether the performance will get to the point to where they can displace traditional out-the-window projection systems.

In MR, user(s) are in computer-generated environments, interacting with inserted real and / or computer-generated entities, data and people that are registered to the computer-generated environment. One of Lockheed Martin’s prototype solutions in this area is Deploy3D, a deployable projector-less 360-degree, mixed reality visual system that realistically simulates an out-the-window display and an instrument control panel of vehicles. This is a cost-effective, immersive and lightweight platform agnostic training solution.

Deploy3D is derived from the use of green screen technology with geometry-based masking providing a 360-degree field of view to the user. This is fully adaptable from a laptop training device with HMD, to a mission rehearsal trainer. It provides on-demand training, reducing training time needed in large dome simulators with a lower life cycle and sustainment cost.

With Coalescence, we took COTS helmet mounted display technology and we coupled that with high-end computational solutions that we developed; we solved some of the key problems the industry was struggling with, and some latency issues

Rockwell Collins has fielded a proprietary technology that allows one to train in blended or mixed environments. Using commercial-off-the-shelf-technology (COTS), the company has developed its Coalescence Mixed Reality system, which merges the trainee’s real-world view with a synthetic environment. Coalescence enables one to train on a variety of tasks, including mobile flight trainers. The system lets the trainee see and interact with real equipment with his/her hands while mixed with the synthetic environment. In layman’s terms, this is like Google Glass, where a virtual environment is layered over a real-world view and Augmented Reality is the result. Through the use of an AR HMD, the trainee can see real-world elements – including his/her hands – in an immersive semi-virtual environment and the Rockwell Collins’ software tracks and responds appropriately with pop-up menus for interactive visual feedback.

“With Coalescence, we took COTS helmet mounted display technology and we coupled that with high-end computational solutions that we developed; we solved some of the key problems the industry was struggling with, and some latency issues,”

said Nick Scarnato, Director of Strategic Development at Rockwell Collins. “With our solution, we have the ability to mix the virtual world with real human hands so you can have tactile feel and have the ability to manipulate objects that are real in your scene, and everything else could be virtual. That allows us to immerse the student as close to the real world as you can get.”

One negative aspect to HMDs is they cannot be worn for long periods of time without causing head and neck fatigue, sometimes nausea, and close-light induced eye strain. These aspects are being continually improved and it’s not unreasonable to believe HMDs will become as innocuous as a pair of eyeglasses in the future.

Delivering simulation for training

Platform acquisition requirements rarely define of the type of simulators needed for training. In fact, it is more common that types of missions much better define requirements, with simulators typically forming one aspect of a comprehensive training system. When thinking of simulators, many think of large dome Level D full flight simulators, but there are in fact many simulators including part task trainers, integrated procedures trainers, and systems simulators that replicate the myriad of sensors common to modern aircraft, from electro-optical and infrared sensors, to synthetic aperture radars.

There is a global trend of outsourcing training services due to the costs associated with delivering a robust training solution. Aeromedical operators like Helijet in British Columbia send their pilots to Florida to train on S-76 simulators operated by FlightSafety International.

Another major player in training and simulation is CAE, a global training systems integrator which is capable of offering governments, defence, paramilitary and original equipment manufacturers a comprehensive range of innovative training solutions designed to cost-effectively meet specific training and readiness requirements.

In 1997, the UK awarded CAE a private finance initiative contract to establish the Medium Support Helicopter Aircrew Training Facility at Royal Air Force Base Benson in the UK. This is one of the first programmes of its kind where a military customer pays ‘by the hour’ for a turnkey training service.

In 2018, CAE was selected by the Qatar Emiri Air Force to provide a comprehensive NH-90 training solution, including a training centre facility, suite of simulators and training devices, and training support services. Following delivery of the training centre facility and NH-90 training devices in 2021, CAE will commence providing training support services, including classroom and simulator instructors. CAE will also design and develop a Tactical Control Centre to be used for managing networked mission training exercises.

CAE believes this type of training services delivery approach will be more commonly used because it frees up active duty personnel for operations, leverages capital investment made by industry, and often enables customers to acquire the required training services more quickly and efficiently. 

In Canada, KF Aerospace leads the Allied Wings consortia which trains Royal Canadian Air Force (RCAF) pilots through the Contractor Flying Training and Support (CFTS) programme. That programme recently took delivery of Frasca International’s Bell 206 Level 7 Flight Training Device which is fitted with a six degree-of-freedom electric motion platform and has a FlightSafety International VITAL 1100 image generator that is supported by a 220 x 60-degree direct projection display. The device also has Frasca’s SimAssist automated instructor operating system that allows instructors to create pre-programmed training sorties. In addition to the new Bell 206 FTD, other equipment includes desktop and FTDs for the Grob 120A, Bell 412 and Beech C-90.

“Conducting a portion of the basic, and highest risk, activities of this training in a safe and controlled environment will enable rapid repetition of sequences free from the random and often adverse effects of weather,” said Peter Fedak, CFTS Site Manager for KF Aerospace. “Over 50 per cent of this program is conducted in the synthetic environment utilising one of the two advanced flight simulation devices at CFTS – one Transport Canada certified Level D Night Vision capable full flight simulator and one Level 7 flight training device (FTD) with visuals. The students will graduate with their RCAF Pilot Wings from this platform having conducted all the sequences necessary to prepare them for their operational flying training on any of the RCAF’s modern rotary wing fleets.”

Also speaking to AirMed&Rescue was Lockheed Martin, which is supporting US Air Force’s Pilot Training Next/Learning Next initiative focused on providing a personalised learning environment for every student through immersive technology for effective, fast and deep learning. The Pilot Training Next initiative leverages Lockheed Martin’s Prepar3D as the core simulation platform powering its training devices.

Prepar3D is hardware agnostic, and its nonproprietary nature provides an environment where users can leverage what they want

Prepar3D delivers a full spectrum learning environment and can deliver procedural training to full mission rehearsal. Using Prepar3D’s Software Development Kit, customers can access, modify and supplement the simulation platform to create advance training solutions.

Prepar3D is hardware agnostic, and its nonproprietary nature provides an environment where users can leverage what they want, including investments from all previous efforts, and integrate capabilities directly. Training packages can be deployed across any hardware configuration from tablets to desktops, virtual reality and domes.

“Prepar3D’s open architecture is what facilitates its thriving ecosystem, which is composed of hundreds of independent companies and individuals developing supplement content and capabilities to make Prepar3D the most powerful training platform possible,” said Breed. “With this product, customers are not locked into a single provider. Using Prepar3D’s Software Development Kit, developers and users can create, integrate and include enhanced databases and terrain to supplement the base environment. Prepar3D’s enhanced tools such as SimDirector and SimOperator are used by instructors to create and control multiplayer scenarios.”

Conclusion

The reality is that training budgets are usually the last to receive attention, while the need for training never diminishes. Level D Full Flight Simulators will always have a place in aviation training, however newer modalities of delivering simulation are taking root such that customers now understand that using a broader range of simulation tools to optimise the training pipeline will yield benefits, particularly as aircraft and instructors are in short supply and comprise the two largest costs of a programme. It is therefore reasonable to assume that simulation that can deliver quality training without negative transfer will likely see an increase in implementation.