Mobile
Web
AR

Remote Assist App

mobile phone viewing an object in augmented reality

Overview

Remote Assist App is a solution that connects users in need of help with consultants who show them how to fix appliances and other objects in their environment.

main technologies
ARCore
ARKit
WebRTC
Flutter
Metal
Ruby on Rails
React
Year
2021
Service
Mobile app development
Design
Augmented reality
Technologies
ARCore
ARKit
WebRTC
Flutter
Metal
Ruby on Rails
React

Augmented Reality Devices for Remote Assistance

The global number of smartphone users in 2021 was 3.8 billion. Out of that number, there’s an estimated 2 billion devices with AR capability (1.25 billion running iOS and 851 million with Android), which makes smartphones an excellent and highly affordable vehicle for augmented reality. 

With so many AR-enabled smartphones, access to mobile remote assist becomes significantly greater, giving everyday users a convenient tool that lets them easily connect with specialists (or their friends) to get advice. 

Remote Assist App brings together two people: one in need of help, the other willing to provide it. When a person in need of help gets connected with an assistant, the assistant immediately sees the user’s environment and can point at specific elements in that environment.

Help centers could use Remote Assist App to instruct customers how to troubleshoot problems with a router or other appliance. This could potentially decrease the time to solve the problem and eliminate many friction points that occur when the assistant can’t see the device in question.

or download it from
App store app buttonGoogle Play app button

Building Remote Assist App

We used a multi-technology stack to build Remote Assist App. The solution contains many parts that work together and help in code reuse between platforms.

The mobile component (for iOS and Android):

  • Partially written natively and in Flutter
  • The iOS version is also powered by WebRTC, ARKit, ARCore, and Metal

The frontend component:

  • For the consultant's user panel and dashboard we used React.js

The backend:

  • Ruby on Rails (Thin) that transfers WebRTC packets that enable the connection. THe backend is scalable and can be adjusted to large traffic requests.

or download it from
App store app buttonGoogle Play app button

Remote Assist App use cases

1

Remote training

Remote Assist App helps teachers provide better learning opportunities for students by holding lessons everywhere, without the need for physical contact. By using the app, teachers simply walk the students through the subject in question, pinpointing elements that need more explanation.

2

Machinery repairs

Remote Assist App fits into contexts where the help of skilled engineers is needed on-site, but they are hundreds of miles away. With the assist app, engingeers can instruct on-site workers and guide them through the fixing process of highly specialized machinery.

3

Car service

Cars increasingly rely on electronics and computer-operated maintenance. Car shops may often lack the specialized workforce to perform repairs according to the manufacturer's requirements. Remote Assist App is an easy way through which mechanics can concult with engineers and service a car.

4

Call centers and technical support

Call center operators have a carefully designed guidelines on how to troubleshoot customer problems. However, they lack a visual on the problem that could let them pinpoint the exact problem faster and improve issue resolution times. Remote Assist App lets customers show the problematic hardware and apply the consultants advice faster.

5

6

The Remote Assist App has two operating modes:

User mode — the user has an issue and needs the help of a consultant. The user launches the app and calls the consultant. Before the consultation, the user generates a code that's shared with the consultant. Thanks to that users can safely use the app without signing up for an account. When the consultation starts, the user shares their back camera view. In that setup, the consultant sees the source of the problem clearly and can guide the user how to fix it.

Consultant mode — the consultant inputs the code received from the user, which enables the consultant to see the camera view shared by the user. The consultant can mark specific objects in the user's environment and draw custom shapes that can further help the user perform a task.

a user flow of remote assist app

or download it from
App store app buttonGoogle Play app button

Validate your tech idea in our boutique R&D department

Contact us
next project

PONS XR Interpreter

Mobile
AR

PONS XR Interpreter is an augmented reality proof of concept for communication and real-time translation. Built for Magic Leap, XR Interpreter lets professionals work together across distances and languages.

View project
Remote Assist app view

Validate your tech idea in our boutique R&D department

Contact us
See the product in your space
View in AR
Want to view the product to see how AR works?
Show product in AR
Visualize it better in AR
See in your space
On a laptop? Scan the QR code below to see how it works.