Virtual reality (VR), assisted reality (AR), augmented reality (also AR, somewhat confusingly), mixed reality (MR) — even virtual factories? There’s a lot of acronyms and a lot of hype about these technologies within aviation, especially as the industry continues to focus on digitalisation and new tech. We sat down with experts from Collins Aerospace, part of RTX (previously Raytheon Technologies) for the latest in our ab initio primer guides.
VR, the two types of AR, and MR are “immersive technologies that offer various levels of virtual experiences,” Collins Aerospace senior technical fellow of advanced visualisations Ryan Wheeler tells us. “They rely on varying levels of computer-generated graphics and visual elements to enhance or create virtual content.”
This might be displayed on a computer, tablet or smartphone, or even via goggles, a headset, heads-up display, or smart glasses. By and large, virtual reality sees the user immersed into a non-physical created realm that they interact with solely using computer controllers, whether that is a joystick, game pad, keyboard, mouse or other kind of wearable like smart gloves. Assisted (sometimes called augmented) and mixed reality, meanwhile, combine the real world with this virtual realm.
Virtual reality
VR is perhaps most familiar of all the technologies.
“At Collins, VR technology has been utilised several ways, including in design and engineering where, for instance, VR systems have been used to present customers with aircraft interior solutions, layout configurations and product design and selection options,” Wheeler explains. “VR has also been used in varying areas of Collins and RTX for employee training — providing everything from a basic immersive experience of a product’s assembly and disassembly processes to a more complete immersive experience, including up to certification-level training.”
Team collaboration and connection is also a key use case, including using VR and 360-degree cameras to allow dispersed teams to tour a manufacturing space without disturbing the factory floor. Here, the visiting teams can view the floor using a range of technologies, from a simple computer showing static images (think Google Maps’ Street View, but inside the factory) all the way up to live audio-video being transmitted to a visitor wearing an immersive headset who can turn their head to watch what’s going on live. This is much cheaper and faster than (say) flying head-office execs out to a production site.
Assisted reality
“Assisted reality,” Wheeler continues, “is a technology that overlays digital information onto the real-world environment and involves the use of devices such as smartphones, tablets or smart glasses. Assisted reality performs best in situations where remote parties need to see a person’s perspective, but where object tracking isn’t needed. The main use case for assisted reality in A&D manufacturing is ‘see-what-I-see’ remote assist — with use-cases including remote customer support, remote supplier support for downed machines, internal remote support for individuals working from home or at another location, regulatory & customer audits, and product inspections.”
There’s a strong MRO use case for assisted reality, and indeed some MRO houses used assisted reality — via tools all the way from a simple FaceTime-style one-to-one video call through to group meetings and the use of more complicated two-way goggles.
Augmented and mixed reality
“Augmented and mixed reality (AR/MR) combines virtual elements with the real world in a way that allows interaction between the two. The two are like VR except that AR and MR show, and are responsive to, the real-world environment. AR and MR share many similarities, but what differentiates them for most people is the degree of interplay between physical and virtual objects,” Wheeler explains. “AR and MR fit best where physical items are available and present, making their primary use case in manufacturing the real-time instruction toward assembly and MRO operations. Many AR and MR use cases can be performed with a head-mounted display or with a handheld device like a tablet.”
Examples here include augmented reality displays where a small camera aimed at a workbench with a wiring harness shows a live image on a nearby screen — which then has virtual instructions overlaid, step-by-step, to guide the worker through a complicated series of tasks. The overlay is targeted via a simple marker, sort of like a mini 3D barcode.
Mixed reality, in essence, is a more advanced version of this, which recognises the shape of the wiring harness assembly in 3D, without the need for the AR marker. Here, Wheeler says, “the operator would just look at the assembly, the headset would recognise and register the assembly spatially — a virtual model of the cable harness would render itself into the physical assembly, and anywhere the cables aligned behind physical items, those sections of the model wouldn’t render to simulate occlusion.”
The virtual factory
Going beyond a simple 3D static or live tour is the virtual factory. Szymon Mazepa, Collins’ senior principal engineer of factory automation, explains that “the virtual factory is a digital representation of an entire manufacturing site – including layout, equipment and processes — providing advanced decision support capability.”
This goes past simpler data input like images, video, machine models and products to add real-time data from a variety of sources: sensors, process monitoring systems, inspection results, and more. In many ways, the virtual factory starts to approach a sort of digital twinning for the factory.
Once the virtual factory is established, it unlocks other Industry 4.0 technologies, processes and methodologies, including the Internet of things, big data, advanced analytics, artificial intelligence, machine learning, robotics, cobotics, automation and more.
“The integration of sensors allows for real-time data collection and monitoring of equipment, production, and supply chain processes,” Mazepa explains. “Data can be utilised for predictive maintenance and process optimisation. Fata analytics enable the identification of patterns to further optimise processes and support data-driven decision making. Machine learning algorithms can be trained and used to predict the quality of the parts as they are being produced, reveal root causes of defects, and optimise production processes. The virtual factory is a testing platform for automation and robotic solutions before implementation minimalising risks and enabling optimisation.”
Adding all these technologies together doesn’t just sum up to make the factory of the future incrementally better: rather, each one multiplies the effects of the other to improve it exponentially. Fundamentally, that technology multiplying effect is one of its key benefits.
Published 15 August 2023
