Remote Assist App — Scalable Solution for Virtual Troubleshooting

For many of us, technology is a staple commodity. It permeates a growing number of households, and factories increasingly rely on highly specialized machinery. And while technology has catapulted the evolution of our civilization, it’s prone to breaking.

But technicians who can fix problems may not always be available, and on-site service is often costly. Pandemic-related border closures and travel restrictions also limit the availability of specialists. In this context, remote assist solutions emerge as reliable support across industries.

The State of Technology Adoption

In the last 30 years, the adoption of specific technologies in the US alone has skyrocketed.

In the business world, technology has spurred production and manufacturing on a palpable scale. According to the World Economic Forum, OECD producers that adopted technology “have grown at a rate of 3.5%, compared with an anaemic 0.5% for the laggards.”

Even small manufacturing facilities have seen an increase in the diffusion of technological development, which led to a more productive workforce and output increases.

Developing countries have also observed an increase in technological innovation, albeit at a lower scale.

The global technological advancement and the ubiquitous presence of technologies in many sectors call for engineers and technicians who can not only drive that innovation but also maintain it.

Yet there’s an alarming shortage of skilled engineers and technicians who can carry out innovative infrastructure projects. On the other hand, less qualified personnel can complete tasks with varying levels of complexity given professional assistance.

Remote assistance might therefore become an affordable and scalable solution that bridges the gap between the need for support and the lack of on-site skill.

Augmented Reality Devices for Remote Assistance

Remote assist is most often associated with augmented reality headsets. While headsets usually offer convenience (both hands free), the price tag might be steep for some.

But the high price of AR headsets is only one of the many barriers to mainstream adoption of these devices. That said, the global number of smartphone users in 2021 is 3.8 billion, which makes this technology available to almost half of the world. Out of that number, there’s an estimated 2 billion devices with AR capability (1.25 billion running iOS and 851 million with Android).

With so many AR-enabled smartphones, access to mobile remote assist is significantly greater.

How Remote Assist App Helps the Technology-Reliant World

Remote Assist App brings skilled technicians directly to the users in need of technical support. Remote Assist App provides visual and audio content via an intuitive interface housed in a smartphone.

‍

How Remote Assist App Works

The app has two modes: consultant and user.

User

When a user connects, they choose a problem from a list of available topics.

The user then waits for a connection with a technician.

Consultant

After logging in, the technician sees a list with active user sessions awaiting help for a given topic.

Once connected with the user, the technician has access to the user’s rear camera.

The technician communicates with the user:

• Through voice
• By placing 3D objects on objects in the user’s environment
• By marking objects in the user’s environment using a yellow marker

Remote Assist App: Technology Stack

Remote Assist has two parts:

• The cross-platform element build in Flutter
• The native element that runs video connections through WebRTC and delivers AR features
  

Flutter

Remote Assist App was built in Flutter because the framework makes it possible to share the codebase between multiple platforms. As a result, Remote Assist App can be extended to iOS, the web, and dedicated desktop apps without rewriting the code.

No backend

The application doesn’t need a backend to support video connections — WebRTC works in a peer-to-peer mode. This lets Remote Assist run multiple connections. Firebase is responsible for connecting users with technicians.

AR element

We chose Android to check how the solution would work on a theoretically weaker AR platform. ARKit — the AR framework for iOS — provides an overall better experience than Android’s AR platform.

Potential Extensions

The current version of Remote Assist App is a prototype in which we wanted to validate assumptions. In the future, the solution can be extended and adapted to specific business cases.

Possible extensions of Remote Assist App:

• Flutter lets us share the application’s code between a mobile app and website (and other platforms, e.g, desktop apps). With a single codebase, it’s a relatively low investment to add a web panel for technicians.
• Remote Assist App can be expanded to include an automatic system that facilitates the connection between technicians and users.
• In cases where both hands are needed to perform tasks, Remote Assist App can be used in tandem with Google XR.

Sample Use Cases for Remote Assist App

Thanks to the ubiquity of smartphones, Remote Assist App can be used for virtual troubleshooting in many industries and sectors.

Here are the major use cases for Remote Assist App:

• Call centers and technical support — The solution can be implemented in businesses that solve issues with hardware, e.g., broken TV decoder or WiFi router.
• Training — Remote Assist helps expand training opportunities by allowing trainers and teachers to hold courses independently of the students’ location.
• Machinery repairs — Whenever repairs of highly specialized machinery require the technicians to be on-site (e.g., go to a different country), Remote Assist App can be used as an affordable tool that connects skilled engineers with available on-site workers.
• Car service — With the growing technical advancement of cars, mechanics may lack the necessary skills to conduct repairs and need guidance from manufacturer’s engineers.
• Miscellaneous tasks — For example, tapping a keg, asking a partner where to find something.

Scalable Virtual Troubleshooting with Remote Assist App

Remote Assist App helps increase the efficiency of virtual troubleshooting by giving technicians greater context. In other use cases, Remote Assist App eliminates the need for on-site professionals to solve problems. In the technology-dependent world, Remote Assist App can be a viable solution to cut servicing costs, decrease training overhead, and promote innovation inclusivity.

Remote Assist App is our internal R&D project created in n/Labs.