Product Guide Studio Tester4.0 DeviceFarm

Click on the [Manage Test Script] button from the “Project Detail” screen
From the “Manage Test Script” Screen, click on the [Create with ATOMP Studio] button
Select a device to create the test script. Users can select either an Android or an iOS device

4. Users can select either [NATIVE_APP] or [WEB_VIEW] by clicking on the [NATIVE_APP] button

Step 4: Create a test script with ATOMP Studio

After selecting the device, Test script creation screen will be displayed. This screen shows the stream of the actual device. Please refer to “Part 2: Create a Test script with Studio” for details on how to create a test script.

Step 5: Run a test script with Tester 4.0

After being created, a test script will be saved and displayed in the “Manage test script” screen in Tester 4.0. Users can then use the script for test execution. For details on how to execute a test script, please refer to “Part 3: Execute a Script with Tester 4.0”.

II. Launch App using Chrome or Safari

How to launch Safari on iOS: On the script creation screen, click on the “Device” item from the right menu => Select the “App” item => Select the “Launch app” button => Popup “Launch app” will be displayed => Enter an id for the “Bundle Id” field and press the [Execute Action] button

How to launch Chrome on Android: On the script creation screen, click on the “Device” item from the right menu => Select the “Android Activity” item => Select the “Start Activity” button => Popup “Start Activity” will be displayed => To start an app, input a package name and an activity name for the following fields: “App package”, “App activity” and press the [Execute Action] button.

B. Start using ATOMP with PC Web

Step 1: Add a new project (Tester 4.0)

Log in to: https://tester40.atomp.io/
1. When the Dashboard is shown, click on “Add Project”
1. Pick a name and set a type for the new project (Type: PC Web)
1. Click on the [Create New Project] button to finish creating a new project

Step 2: Install the Studio Web extension on your browser

The ATOMP Studio Web extension has not been installed on your browser
1. Click on the [Manage Test Script] button from the “Project Detail” screen
2. From the “Manage Test Script” Screen, click on the [Create with ATOMP Studio] button
3. Inside the popup “ATOMP STUDIO WEB”, click on the [Download Studio Extension] button to download the extension file namely “studio-web-extension”
4. Download the Studio Web extension and extract the compressed zip file to your computer
5. Install the Studio Web extension via Chrome.
  - Go to Chrome > Manage Extensions.
  - Turn on the “Developer Mode”.
  - Click on the “Load unpacked” button and select the extracted folder of Studio Web extension.

Update Studio Web extension to the latest version
- After clicking on the [Create With ATOMP Studio] button from the Manage Test Script screen, the popup “Update Version” will be displayed
- User can download the latest version by clicking on the [DOWNLOAD LATEST VERSION] button

Step 3: Create a test script with ATOMP Studio Web

After opening the Studio Web extension, the Studio Web primary screen will be displayed. This screen shows all the commands/steps that will be executed inside your desired web app. Please refer to “Part 2: Create a Test script with Studio” for more details on how to create a test script.

Step 4: Run a test script with Tester 4.0

After being created, a test script will be saved and displayed inside the “Manage test script” screen in Tester 4.0 where users can execute this script on. For details on how to execute a test script on Tester 4.0, please refer to “Part 3: Execute a Script with Tester 4.0”.

Part 2: Create a Test script with Studio

A. Create a test script with Mobile

I. Create a Precondition

Precondition can be created and used for multiple scripts with the same precondition. Please follow these steps:

Step 1: Click on the [Manage Test Precondition] button.

Step 2: Click on the [Add Precondition] button.

Step 3: Select a device to start creating a precondition.

Step 4: Select a precondition parent (if applicable).

Step 5: Click on the [Record script] button to switch to the recording mode to create steps for the precondition.

➩ After selecting the pre-condition => the [Preview script] button will be displayed. You can select the [Preview script] button to run the pre-condition. (Follow step 10, 11)

Step 6: Select an element from the streaming screen of the device to perform actions.

Step 7: Choose the appropriate actions to take => The steps taken are recorded below.

Step 8: Click on the [Stop] button => the [Preview script] button will be enabled.

Step 9: Click on the [Preview script] button to run and view all created steps.

Step 10: Click on the [Run all] button to run all preconditions.

Step 11: Click on the [Run script] button to run the recorded steps one at a time => After running the script, the device’s streaming screen will show the live execution of whatever steps that were recorded. The step status will be displayed at the end of each step.

Step 12: Click on the [Back] button => the “Create script” screen will be displayed => Click on the [Save] button.

Step 13: Popup “Save as Pre-condition” will be displayed => Enter the information of the precondition that you want to save => Click on the [Save] button => This Precondition will be saved in the “Manage Precondition” screen.

II. Create a new test script

The script creation screen is displayed by step: Click on the [Manage Test Script] item => Click [Create with ATOMP Studio] => Select any device.

1. Upload a CSV file (if necessary)

Users can upload a CSV file containing two rows of data (key/value pairs) and use those pairs as variables inside ATOMP Studio.

From the “Predefined Actions” tab, click on “Expand” at “CSV External Input”, then click “Upload” to upload a CSV file.

A CSV file has the following format (Column A: Variable Name, Column B: Value)

After uploading the CSV file, users can use those variables in any textbox while creating test script by typing “$+variable name”

2. Switch to the [Recording] mode

To start creating a test script, you need to switch to the [Recording] mode by clicking on the button [Record script] or the Eye icon => below is a sample of the Recording mode.

Users can select a screen element (icon, button, text, etc…) to execute some actions.

3. Select Precondition(s)

Step 1: On the script creation screen, click on the [add new] button.

Step 2: Select one or more preconditions from the precondition list of the project

➩ When the script is running, these selected preconditions will run from top to bottom. List of selected preconditions will be displayed as below:

4. Create Test Steps

Step 1: Switch to the “Select Element” mode (by clicking on the icon )

Step 2: Select an element on the streaming screen

Step 3: Select an action you want to take with the element selected in step 2

If the selected element is an icon or a button, the available actions are (Tap, Double Tap, Long Press)
If the selected element is a Text Input box, click on the [Send Keys] button to input text to the element.

Step 4: If the action is [Send Keys] -> a popup will be shown for users to input characters.

➩ After executing 1 action, the tool automatically generates a test script for that action and displays as a step (refer to the picture below)

5. Create an Expectation

Step 1: Select an element that you need to check on the Streaming screen

Step 2: The PROPERTIES box will show the properties of the selected element –> choose a property to create an expectation that you want.

If you want to check the UI of the screen, select the icon camera inside the “Expectation” column to capture screen as reference for future comparison.

If you want to use AI Visual check, select icon to add AI Visual check expectation to compare visual images which help you increase accuracy and reduce effort for testing. AI Visual check will capture the screen to be a visual reference for later test and let you know if during regression test, the actual screen has difference with the captured screen.

You can drag mouse to specify which part is dynamic data and can be excluded in visual comparison. After that, you click on “OK” button to save Expectation or click on icon to delete selected data area.

2. You can also click the link reference image in Expectation to view again or edit chosen data by following part (1). Click on icon if you want to delete AI Visual check expectation.

User can view evidence at Tester40 [Test Result] page.

The system also supports comparing images of element when confirming exepectation. Please follow these steps:

Click on an element on the streaming screen à Click on [+] button besides text [Image] in Custom Properties.

Then [Compare image] pop up display. Popup has 2 options:
– Upload image: Allowing uploading an existing image or an image cutted from an xd file (JPG, JPEG, PNG format).
– Use curent image: Allowing to automatically capture the current screenshot, then based on the element position, automatically crop the image of that element.

Option 1: Upload image

Firstly, Users must click on tab Upload image then select image file and press OK. After selecting image, photo is uploaded to ATOM Storage. If there is an existing image, the new uploaded image will replace the old one. ==> After uploading, the expectation added <Element> displays the same as <filename of the uploaded image>

Option 2: Use current image. Click on button to use current element’s image as reference then click on [OK] button, image is used as expectation.

6. Feature IF – THEN – ELSE

Step 1: Add [IF] block

On the [PREDEFINED ACTIONS] tab, click on [Command] menu to expand it
Expand [Conditional] sub-menu
Click on [IF Statement] to add IF block to script

➩ After that, new block will appear on Recorded Script section

1) Hover on to open menu

Click on “Add condition” add a new condition to current [IF] block, it will open input form as above.
Click on “Add Sub-IF” to add a [Sub-IF] block to current block
Click on “Add ELSE IF” to add [ELSE IF] block after current [IF] block
Click on “Add ELSE” to add [ELSE] block after current [IF] block
Click on “Delete Block” to delete current block. If deleting a [IF] block will delete all [ELSE IF] and [ELSE] blocks that come with it. User must confirm before action to be executed.

2) Click on to stop working on current block

3) Click on to stop working on current block, to work on a block (pin – unpin block)

4) Delete a condition

5) Edit condition

6) Logical operator (uses when block has 2 more conditions)

7) Element to do comparison operation

8) Property of element (must be specified in order to save condition)

9) Operator

10) Value to compare (default is element’s property value)

11) Click to save condition

12) Click to cancel input condition

13) Block action (will be executed if condition is met)

Some cases when working with IF – THEN – ELSE function

When adding condition without select element or property to do comparison operation, a message will appear like bellow and condition will not be saved until user specify element & property.

When deleting all [IF] – [ELSE IF] block condition, a notification will appear with bellow content to inform user that block condition is empty

When working on [IF] – [ELSE IF] block which does not contain any condition, script will not be recorded and a notification with bellow content will appear

When trying to save script with an empty condition [IF] – [ELSE IF] block, script will not be saved and notify an error message like bellow

Preview Script with IF – THEN – ELSE function

All [IF], [ELSE IF], [ELSE] block which condition(s) does not meet will be ignore & show error message like above
When previewing script with empty condition [IF] – [ELSE IF] block, script will break at block which condition is empty!

7. Feature Select Step to take evidence on Tester4.0

Step 1: Tick on checkbox in the last column of test script which step you want to take evidence when running on Tester4.0

Step 2:After running sprint done, you click “Download evidence” button to get the evidence ticked in step 1.

User can view evidence at Tester40 [Test Result] page

8. Feature clone script

Step 1:

Clone one test step: Click on one test step which you want to clone => Right mouse and choose “Clone”.

Clone some test steps:

If the test steps are continuous: Click on the first or the last test step and mouse hover all test steps which you want to clone => Right mouse and choose “Clone”.

If the test steps are not continuous: Click on any test steps and hold Ctrl and click on other test steps => Right mouse and choose “Clone”

Step 2:

Choose one step to clone before or clone after

Click on one test step what you want to clone after or before that => Right mouse and choose “Clone before” or “Clone after”

Choose some step to clone before or clone after

If the test steps are continuous: Click on the first or the last test step and mouse hover all test steps which you want to clone => Right mouse and choose “Clone before” or “Clone after”.

If the test steps are not continuous: Click on any test steps and hold Ctrl and click on other test steps => Right mouse and choose “Clone before” or “Clone after”.

9. Preview Script

Step 1: Click the [Stop] button to finish recording the test script

Step 2: You can preview the created script from any available device by clicking on the [Preview script] button

From this screen, users can preview the script from another device by clicking on the [Select device] button => Select device screen will be displayed. (The device selected for creating the script will be selected as default for preview)

-> After selecting device, click the button to start previewing the script.

10. Save a script

Step 1: Click on the [Save script] button -> A dialog will be displayed

Step 2: Input the reference ID and description for the script

Step 3: Click on the [Save Test Script] button

➩

After a test script is saved, it can be found in the screen “Manage Test script” in Tester 4.0

11. Edit a script

Users can edit the created script by following these steps:

Step 1: From the “Manage Test Script” screen, select the “Edit” icon from the script you want to edit => Select device screen will be displayed => Perform a device selection just like creating a normal script

Step 2: After the device is selected => The script creation screen will be displayed, and the users can edit the script normally => Then the users can save the script by 2 methods:

Save to the existing script
Save as a new script

III. Mirror 1 action on multiple devices

Step 1: Click on the icon mirror -> The popup list of available devices will be displayed

Step 2: Select a device that will mirror the actions of the main device

After selecting a mirror device, the streaming of all mirror devices will be shown

User’s actions in the main device will also be performed in the mirror devices

Step 3: Capture the evidence on all devices by clicking on the icon capture screenshot from the tab bar

=> Screenshots in all devices will be shown (both main device and mirror devices)

Step 4: Click on the icon under each image => screenshot will be copied to the clipboard

Step 5: Click on the [Download] button => download evidence (screenshot) from all devices

IV. Ruler Measurement

This function is intended to increase productivity comparing the difference between UI and XD

To use this function, please follow the steps below:

Step 1: Click on the “Ruler” icon from the navigation bar to open the MEASUREMENT screen

Step 2: If you want to measure size and position please click [Ruler] tab, if you want to measure color code then click [Color] tab

– “Ruler” tab

To pin an element, click on the [Measure] button, then click on any element from the streaming screen, or from the Source list.

– “Color” tab

To pin an element, Click on the [Filter] button, then click on any element from the streaming screen, or from the Source list.

B. Create a test script with PC Web

I. Create a Precondition

Precondition can be created and used for multiple scripts with the same precondition. There are two ways to create a precondition: Create with ATOMP Studio and Import precondition.

Option 1: Create with ATOMP Studio

Step 1: Click on the [Manage Test Precondition] button.

Step 2: Click on the [Create with ATOMP Studio] button. The Studio Web primary screen will be displayed.

Step 3: Select a precondition parent (if applicable) by click on [+] button besides text [Precondition].

After popup Add Precondition display, user can tick precondition which you want to add then click on [Add Precondition] button. Parent Precondition will added to current precondition.

When you add parent precondition, you can click on that precondition to view test script and click on button to preview precondition.

Step 4: Click on the button [Record script] to start recording your browser’s interactions.

Step 5: Before you can start recording, you must specify a valid base URL for your project by clicking on the [START RECORDING] button to open your URL. Your test will start by navigating to this URL.

Step 6: Once the URL has been opened, after executing 1 action, the tool automatically generates a test script for that action and displays as a step (refer to the picture below)

Step 7: Click on the button to stop recording => the button will be enabled.

Step 8: Click on the button to run and view all created steps.

Step 9: Click on button to save precondition. Popup “Save Pre-condition” will be displayed => Enter the information of the precondition that you want to save => Click on the [Save] button => This Precondition will be saved in the “Manage Precondition” screen.

Option 2: Import precondition

If you have available precondition’s test script, users can import it into the system. Please follow these steps:

Step 1: Click on the [Add Precondition] button to open the [Add Precondition] pop up.

Step 2: Select the test script file you want to import (file format must be correct as defined by the system), then click the [Open] button.

I. Create a new test script

Click on the [Manage Test Script] button => Click [Create with ATOMP Studio]. The Studio Web primary screen will be displayed.

1. Start the [Recording] mode

Click on the button [Record script] to start recording your browser’s interactions. Below is a sample of the Recording mode.

Before you can start recording, you must specify a valid base URL for your project and click on the [START RECORDING] button to open your URL.

2. Select Precondtion(s)

Step 1: On the script creation screen, click on the [add new] button.

Step 2: Select one or more preconditions from the precondition list of the project then click on [Add precondition] button.

When the script is running, these selected preconditions will run from top to bottom. List of selected preconditions will be displayed as below:

3. Create Test Steps

Once the URL has been opened, after executing 1 action, the tool automatically generates a test script for that action and displays as a step (refer to the picture below)

Additionally, users can create a step manually by utilizing the “Command” input box.

Customize steps

Command has a series of commands that users can use to interact with the web app.
Target has some element types for users to choose from: (list target to find element)
+ xpath:attributes
+ css
+ xpath:position

Description: a step description
Delay(ms): a delay time before executing the current step
Capture: capture wep app screen during this current step’s execution
Value: input value for each command

User can select different element types for the target by clicking this icon

Here are some commands that are commonly used:

Click command
- The purpose of click command in is to click on any UI element in the application.
- While the row is selected, enter the Selenium IDE command ‘click’ to the Command box field and enter the XPath locating strategyinto the Target box field.

Assert text command
- The purpose of assert text command is to check the text on the UI element ( Checking the text on this button is a per our requirement)
- While the row is selected, enter the command ‘assert text’ to the Command box field, enter the id locating strategy into the Target box field:

In addition, the user can select the Mouse Over/Assert/Store/Verify/Wait For command by right clicking on the object that the user wants to check and as show bellow:
+ Mouse over : The purpose of the mouse over command is to perform mouse the hover action on the given UI element.
+ Assert: The purpose of the assert command is to check the value stored in a variable is according to the expected result. (Ex: assert text, assert not text,…)
+ Store: The purpose of the storecommand is to store any text into a variable for easy re-use. Follow (Ex: store xpath count, store text,…)
+ Verify: The purpose of the verifycommand is to check the value stored in a variable. The variable’s value will be converted to a string for comparison. (Ex: verify check, verify not text,…)
+ Wait for element visible: The purpose of the wait for element visible command is to wait for the element to get displayed on the page. (Ex: wait for element not visible, wait for element present,…)

store xpath count
- The purpose of the store xpath count command is to retrieve and store the count of the UI elements located using the given XPath Locating strategy into a variable in Selenium IDE
  - Gets the number of nodes that match the specified xpath

echo command
- The purpose of echocommand is to print the messages given in Selenium IDE. Useful for debugging

➩ The echo command usually comes with the store command (Ex: store attribute/ store xpath count/ store text….)

If and end command
- The purpose of the if command is to check whether the given condition is true or false. If the condition results in true, the Selenium IDE statements inside the if and end commands will be executed. If the condition results in false, the Selenium IDE statements inside the if and end commands won’t be executed.

Else if/ send keys command
- Else if is the command in Selenium IDE will be executed when the if condition results in false and we have another condition to be validated before going to the else block.
- Simulates keystroke events on the specified element, as though you typed the value key-by-key.

➩ All if and else/else/else if commands end with “end” command

4. Preview script

Step 1: Click the [Stop] button to finish recording the test script

Step 2: You can preview the created script from any available device by clicking on the icon

If you add precondition to test script, you can preview all precondition and test script by clicking on the icon .

5. Save scripts

Step 1: Click on the [Save script] button -> A dialog will be displayed

Step 2: Input the reference ID and description for the script

Step 3: Click on the [Save Test Script] button

➩ After a test script is saved, it can be found in the screen “Manage Test script” in Tester 4.0

6. Edit script

Users can edit the created script by following these steps:

Part 3: Execute Script with Tester 4.0

A. Execute script for Mobile

I. Execute a Script and check the test result

1. Run Now

Step 1: Click on the button [Run Test Script Now] -> The popup Run Test Script will be displayed

Step 2: Select Device (In the Run Test Script popup, select one or more Android devices)-> Click on the [Next] button

Step 3: Select Test Script (After selecting Android devices, the popup list of Android test scripts will be displayed –> select one, or more scripts that the users want to run –> Then click on the [Next] button)

Next, we can select iOS devices and iOS test scripts, similarly to step 2, 3 with Android

Step 4: Select test language(s)

Input a Job name and select test language(s)
Click on the button [Run Test Now]

Step 5: View running test script(s) (Show the correct number of devices and the name of each device selected for running from step 2 -> Click on the [View] button next to each device name to show the streaming screen of that device)

Step 6: After the test script execution is finished: The Test Report will be shown in the Project Detail screen. To view the detail of a Test report: click on each test report item in the list

2. Run a script On Schedule

Step 1: Click on the [Add Schedule Job] button

➩ The popup select devices & test scripts (similar to the “Run Now” feature) will be displayed

Step 2: Setting the time and date to run test scripts automatically:

Input the Job name
Select test language(s)
Select a time to run test scripts: including the time and the day of the week
Click on the [Add Schedule Test] button

➩ After finish adding a Schedule, at the scheduled time, the selected test scripts will be executed and the test result will be saved.

3. Check the test result

After the test script execution is finished, the Test Report will be shown in the Project Detail screen.

To view the detail of a Test report: click on each test report item in the list.

There are 2 types of results: Pass and Fail.

Pass: Script works, no script errors.
Fail: There was a script error or an input error, not getting results from Tester 4.0.

If user use AI to check the expectation in test script, it will have more result: Warning. The script works, there are no script errors, the AI returns the results (but not the Input Error), but the AI’s result is that something is wrong. Specifically: An element’s image comparison function returns FALSE. When the result is warning, the user needs to confirm the result.

AI when comparing 2 screen shots will give the same ratio of 2 images, the value runs from 0 to 100%.

In Tester 4.0 define a Threshold level so that if it is below this level, it is considered a problem. For example, if Threshold setting is 95% then:

A.I returns 2 images that are > 95% similar -> no problem.
A.I returns 2 images with the same 95% or less -> considered “problematic” à Warning.

At the moment, the Threshold setting is 95%.

III. Export – Import Test Script

1. Export Test Script

After successfully creating the test script, if you want to export the generated test script, you can follow these steps:

Step 1: From the “Manage Test script” screen, click on the “Export Test script” button to switch to the “Export Test Script” screen.

Step 2: Select the test script you want to export, then click the on the [Export X Test script] button (X is the number of test scripts you want to export)

1. Import Test Script

If you already have a test script file, you can import the file by following these steps:

Step 1: From the “Manage Test script” screen, click on the “Import Test script” button to open the “Import Test script” popup.

Step 2: Click on the [Browser] button, then select the test script file you want to import, then click the [OK] button

B. Execute script for PC Web

I. Execute a Script and check the test result

1. Run Now

Step 1: Click on the button [Run Test Script Now] -> The popup Run Test Script will be displayed

Step 2: Select environment (In the Run Test Script popup, select Platform, Browser, and Browser Version) -> Click Add button -> Click Next button

Step 3: Select Test Script -> Select one or more scripts that the users want to run -> Then click on [Next]

Step 3: Input a Job name -> Click on the button [Run Test Now]

Step 5: View running test script(s) (Show the correct number of browsers and the name of each browser selected for running from step 2)

Step 6: After the test script execution is finished: The Test Report will be shown in the Project Detail screen. To view the detail of a Test report: click on each test report item in the list

2. Run a script On Schedule

Run a script On Schedule same with “Run Now” feature (follow 1. Run Now)

II. Download-Add Test Script

1. Download test script

After successfully creating the test script, if you want to export the generated test script:

Step 1: Click on icon to download test script

Step 2: Select folder to save script

2. Add test script

If you already have a test script file, you can add the file by following these steps:

Step 1: From the “Manage Test script” screen, click on the “Add Test script” button to open the “Add Test script” popup.

Step 2: Select the test script file you want to import (file extension .atomp), then click the [Open] button

Part 4: Remote Control device with Device Farm

Log in to Device Farm: https://devicefarm.atomp.io/

I. Set up Executable files for the project

After users log in, in the Device List screen, users can see all the devices that are permitted to use.

Click on any device whose status is “Use” (which means that the device is available to use).

II. Interact with device

Users can click, swipe, tap and hold on the device screen to interact with device

III. Upload app on device

Step 1: Click on the [Dashboard] tab

Step 2: Drag and drop the file type IPA or APK to install applications into the current device

IV. Sound

Users can listen to the sound played from the device by clicking on the “Sound” icon.

V. Capture Screenshot

Users can interact with the device via the Streaming screen and can also capture the screenshot of the device

Step 1: Select the tab [Screenshots]

Step 2: Move to the screen needed to capture the screenshot

Step 3: Click on the button [Screenshot]

VI. Device information

Users can view the device information by selecting the “Info” item

Users can view the device’s files by selecting the “File Explorer” item

When the developers and testers want to collect the logs of the device they are using, they can go to “Logs”. And they can choose the type of logs they want to collect by selecting the “Logcat Level” item

Users can perform operations on the device such as opening the camera, turning the volume up or down and rebooting the device in the most convenient way by selecting the “Advanced” item

VII. Remote build and remote debug

1. Android

First, select available device and click Use to get instructions

If user is not on the same local network as the device, open terminal and run step 2 in the instructions. Example

If user connects to the device for the first time, go back to device farm web app and click Add Key allow access

Check device connection

Now user can use any IDE to build, debug just like a real device being plugged into user’s computer. Example build a calculator app with android studio:

Select device to Run:

Run project with remote device:

Return to device farm to see the app is installed

2. iOS (only work on MacOS)

First, select available device and click Use to get instructions

Install socat tool to connect to remote device:

Backup system file

*Restore after finishing work to ensure system work properly: sudo mv /usr/var/usbmuxx /usr/var/usbmuxd

Connect to remote device (using sudo)

Open Xcode to check device is connected. Wait a moment for Xcode to find the device

After the device is found, If it is the first time connecting to device, user may have to wait 3-10 minutes to prepare the device (LAN connection) or several hours (Internet connection)