In the last blog (Mobile Automation Test Framework Introduction and Design) we discussed some basic knowledge about automation framework and how to set up a proper automated testing framework. In this article, I will introduce how to select a suitable automation tool and how to design the framework library.

First, we need to define some important capabilities which we would like to see in an automation tool:

1. Cross-platform.
2. Can support native view and web view.
3. Easily extended, easily maintained, flexible and low-cost.
4. Easy to use and Community is active.

Then compare the most popular testing tools based on the above capabilities.

How to choose an automation tool for mobile testing

Decision Table:

Appium is obviously the best choice through the comparison in the above table, which is briefly summarized below:

• Appium is a cross-platform tool which supports multiple languages.
• Appium supports Web View and Native View.
• Appium is very extensible. We can integrate it with our favorite test data and report tool.
• Appium is based on the WebDriver API and supports both native and web view. It is very easy for tester engineers to develop test scripts.
• The Appium user community is also active. Any questions about this tool will be answered promptly in the community.

After selecting a suitable mobile automation testing tool, we need to consider how to set up the automation framework based on the selected tool.

Tools list for mobile automation framework

• Mobile automation framework is based on Appium.
• Git is used as a distributed revision control and source code management (SCM) system in this framework.
• Maven is used as the CM tool to configure project dependencies.
• GitLab provides continuous integration services for code development and scheduling test case execution.
• DataBase is used to manage the test data.
• Extent Report is used to manage the test report.

(We will discuss Data Base and Report Management in future blogs)

How to set up a mobile automation framework

How to design a Framework Library

This framework library is the core of the entire framework. We need to design the library following the architecture which we discussed in the last blog.

The major features in a Framework Library:

• Page Object: Encapsulation of all page elements info into the page object class, which consists of the Element Type Class definition, define page operation functions (business logic).

• TestBase: Base class, initial driver and test engine annotation.

• Element Type Definition: Define an element type, which includes element and other attributes belong to the element.
• TestData Management: Define the communication and procedure with Database.
• Report Management: Define the communication and procedure with Report Listener.

(Put all above features into System Util package)

• Method Encapsulation: this part contains the most useful user actions and verification actions.

(Method encapsulation classes are placed into Function Util package)

Up to this point, the principles of choosing a suitable automated testing framework has been well discussed. Details of designing the framework library to setup the mobile automated testing framework is presented.

My Mobile Automation Framework Blog Series:

Mobile Automation Test Framework Introduction and Design

Mobile Automation Test Tool Selection and Framework Library Design

Mobile Automation Test Data Management (coming soon)

Mobile Automation Test Report Management (coming soon)

Mobile devices are becoming more important than ever in our society. The demand for automated testing on mobile APPs is becoming significantly greater than before. This blog will discuss how to set up a mobile automated testing framework that is easy to create test scripts and maintain upon requirement change.

First, this blog will introduce a structure diagram of an automation framework so that you can have a basic understanding of every major module in the framework.

Mobile Automation Framework Simple Structure:

Propose an automation framework with five major modules as shown in the above diagram:

• The first module is the automation framework, upon which the test engineer will create the automated testing project.
• The second module is the Framework Library. This module represents the actual testing project library for the automated testing project.
• The Data and Report management module represent how test engineers manage test data and how to generate test reports.
• After that is the Test Scripts module, where the Testing Engineers create automated testing scripts for the testing project.
• Finally is the browser module or the web app itself. This represents the Mobile application under testing.

Next, we will discuss the advantages and disadvantages of several popular automated test framework types.

Popular Automation Test Framework Type Introduction:

1.The Data-Driven Testing Framework:

Description: Data-driven testing is a framework where test input and output values are read from data files, not from scripts.

• Benefits: Test Data can be managed separately.
• Disadvantages: The amount of work caused by any changes to the program under testing is the highest in all the automated testing frameworks. The maintenance cost is therefore very high.

2.The Test Script Modularity Framework:

Description: The test script modularity framework requires the creation of small, independent scripts that represent modules, sections, and functions of the application-under-test. These small scripts are then used in a hierarchical fashion to construct larger tests, realizing a particular test case.

• Benefits: When the element or business logic changed, maintenance is ONLY about updating related test scripts and test data.
• Disadvantages: All the maintenance work needs to be been finished by automated testing engineers.

3.The Test Library Architecture Framework

Description: The test library architecture divides the application-under-test into procedures and functions and puts them into a library. This framework requires the creation of the library that represents modules, sections, and functions of the application-under-test. These library files are then called directly from the test case script.

• Benefits: Test Scripts, Library, and Test Data can be updated by different engineers. Procedures and functions are divided from business logic.
• Disadvantages: More effort is required from automation engineers as they need to be in charge of frame library maintenance.

4.The Keyword-Driven or Table-Driven Testing Framework

Description: This framework requires the development of data tables and keywords. The automation scripts in the function level retrieve the test data from the developed data tables or keywords. Then the automation scripts in the test case level call the function-level scripts to perform the testing.

• Benefits: Automation engineers can focus on keywords development and maintenance. Detailed test scripts can be managed by other test engineers.
• Disadvantages: Higher requirements for automation engineers.

The best automation framework should combine all of the above frameworks’ benefits together.

Design Mobile Automation Framework according to above analysis results:

The Benefits of the custom automation framework diagram:

1. Input and output values are read from data files which are managed by the database. This conforms to the principle of a Data-Driven Testing Framework. The reason why we choose database as a management tool will be discussed in my future blogs.
2. The test suite contains several test scripts which are independent of each other. This conforms to the principle of the Test Script Modularity Framework.
3. Set up a framework library which provides procedures and functions instead of putting them into scripts. The test case scripts call those library files directly. Test data and reports are managed outside of the Library. This meets the principle Test Library Architecture Framework.
4. Repeatable user actions and major verification actions are encapsulated in a specific class. Test scripts can directly call those actions. This meets the principle of the Keyword-Driven or Table-Driven Testing Framework.

This is the first article of my blog series on the subject of the Automated Testing Framework for Mobile Testing, which introduced the major modules of an automation framework, analyzed several most popular frameworks and concluded the requirements for designing a proper testing automation framework.

My Mobile Automation Framework Blog Series:

Mobile Automation Test Framework Introduction and Design

Mobile Automation Test Tool Selection and Framework Library Design

Mobile Automation Test Data Management (coming soon)

Mobile Automation Test Report Management (coming soon)

Mobile devices are getting more important than ever in our society. A large amount of Android and IOS applications are developed to make our lives convenient in every possible way.

Most mobile apps are usually running on the devices with various models and different platforms. The automated testing for these apps therefore shall be conducted in diverse circumstances. Testers certainly like to see the automated testing scripts running across devices and platforms without making many changes. One of the solutions for this is from Appium, a well-known automated testing tool for mobile applications.

Appium: “cross-platform”

Appium is an open-source tool for automating native, mobile web, and hybrid applications on iOS and Android platforms. Appium is usually combined with Selenium for automated testing on mobile apps. Testing engineers with selenium experience can conveniently get familiar with this tool and develop automation scripts for mobile platform testing.

An important feature of this mobile testing tool is Cross-platform. Automation scripts are runnable on multiple platforms (iOS, Android, Windows) by using the same set of Appium API. Thus, the automation scripts become reusable between iOS, Android, and Windows test suites, which makes the scripting work more efficient.

How to configure different platform devices to support grid testing

The Appium tool contains two different grid models: Local Grid and Cloud Grid.  The major difference between these two models is whether the Appium Hub and the Node are set on the same machine.

However, we would recommend Cloud Grid as the model for grid testing because it can be applied to multiple testing environments. In this model, all the testing devices can be considered as a testing lab. Such labs can be remote, which means during a test, there is no need to have physical devices along with testers or developers.

Here is an example of setting up Cloud Grid for mobile testing.

The parameters of the hub and notes are listed below:

Hub: Windows (hub ip: 10.2.6.46, hub port: 4444)

Node1: Windows (node ip: 10.2.6.46, node port: 4723)

Node2: Mac (node ip: 10.2.6.138, node port: 4723)

(One Tip:  Appium Grid allows to set up multiple nodes in one machine with different udid, but one Mac machine can only control one iOS device as iOS automation needs Instruments)

The following are the commands to set up Appium Hub:

java -jar selenium-server-standalone-*.jar -role hub

The configuration of Appium Node is usually maintained with Jason configuration files as shown in the following diagram:

The above configuration is for Windows and Mac Nodes. After that, we need to set up the node by entering the following commands:

appium –nodeconfig (the configuration JSON file absolute path) –no-reset

Notes: the configuration JSON file absolute path is required here

We can also set up multiple nodes on one machine.  In this case, we need to specify listener ports for different devices.

The default port for Appium client and Server is 4723.

The default port for Appium server and Testing device is 4724.

However, we can specify the port number to other numbers to avoid conflicts. (e.g. 4725, 4726)

Once all the above steps are completed successfully, the mobile automation testing lab is set up for grid testing.