Built in the Altom test lab, AltTap is a robot designed to help with and mitigate some of the critical challenges of test automation. It is especially useful in scenarios involving the running of tests on devices of varying screen sizes and resolutions. AltTap’s special power is that of performing automated tests on any type of touchscreen device.

How does it do that exactly? It simulates a human tester by the using of a stylus to perform click actions on a touchscreen. The process is based on an image-recognition algorithm.

The components of AltTap

AltTap consists of the following elements:

1. a work platform upon which the device is placed
2. an external camera used for taking screenshots of the device screen
3. an end effector which is moved into position using two NEMA 17 stepper motors
4. a stylus driven by a servomotor that has a feedback system which allows it to adapt to the device screen thickness
5. a microcontroller which helps manage all the components and ensures communication with the PC from which the tests are actually run
6. a power supply

Some of these components are 3D printed.

It is important to note that AltTap works together with a PC, which:

• provides the support for creating tests.
• processes the images.
• sends instructions to microcontrollers.
• checks if a test has passed or failed.

How does it work?

Tests are set up on a PC AltTap is connected to. AltTap will play the role of the tester by performing various click actions on the screen of the device. It will locate elements by using an image recognition algorithm.

Thus, as a prerequisite, the user needs to create a database with the images of the elements to be interacted with during the tests.

The OpenCV image algorithm used utilizes feature detection to create a match between the searched image and the screenshot of the device. If the match is successful, the algorithm will return the coordinates of the searched element. These coordinates are expressed in pixels. The PC then converts these coordinates to millimeters and sends them to the microcontroller. Once identified, elements are interacted with according to instructions. 

If you prefer to think in diagrams and arrows, here is a visualization of the process:

Why work with AltTap?

We have put it to the test and are happy to declare that AltTap is a valuable tester’s helper. Here is a list of things we appreciate about working with AltTap:

• you create the script once and run it multiple times on multiple devices with different resolutions.
• AltTap never gets tired or bored: it can repeat a test for as many times as you wish, taking identical steps each time – and it can do this for a looong time!
• it records everything, taking up an important part of the documentation activity.
• it can test on screens small and large.
• it can interact with device features that are not accessible programmatically: e.g. on iOS,  it can enable/disable the wi-fi connection or bluetooth.
• it can also be used when you cannot interact with an application using software testing tools or frameworks: e.g. testing embedded software on industrial touch panels; or, testing an application that does not expose elements – the element cannot be called and you use a different method such as image recognition.   

The limitations of AltTap

Just as its enthusiastic builders, AltTap has its limitations. We are constantly looking for ways to improve and help AltTap grow, but so far:

• it cannot perform multi-touch gestures such as pinch, zoom, tilt, rotate.

Made with enthusiasm and ingenuity, open about its limitations, AltTap is a reliable and competent helper. It aims to bring value and improve test automation experiences for both testers and project stakeholders. Continuously supporting AltTap’s development, we are committed to seeing it achieve that goal. This is AltTap for now – we are terribly excited about what AltTap will be next!