Complete Tutorial on Mobile apps testing- Learn how to perform mobile apps testing stey by step guide

Mobile application testing is a process by which application software developed for hand held mobile devices is tested for its functionality, usability and consistency.

In this tutorial we will completely guide you about mobile testing and how to perform mobile apps testing.

What is Mobile OS

A mobile operating system (mobile OS) is the operating system that controls a smart phone , PDA, or other mobile device.

Modern mobile operating systems combine the features of a personal computer operating system with touch screen, Bluetooth, WiFi, GPS mobile navigation, camera, video camera, speech recognition, voice recorder, music player, personal digital assistant (PDA), and other features.

Symbian
Foundation	Symbian Foundation
First Release	Symbian OS 6.0 and 6.1  - 1998
Blackberry OS
Foundation	RIM (Research in Motion)
First Release	Blackberry OS 1.0 – January 1999
Android
Foundation	Android Inc. – Google 2005
First Release	Android 1.0, September 2008
IOS
Foundation	Apple
First Release	iOS 1.0 – January 2007
Windows Phone OS
Foundation	Microsoft
First Release	Pocket PC 2000 – April 19, 2000 Windows Mobile  Ozone – June 23, 2003
webOS
Foundation	Palm  - hp 2010
First Release	Palm OS 1.0 – 1996 webOS – January 2000
Bada OS
Foundation	Samsung
First Release	Bada OS 1.0 – February 2010

Mobile Application Types

Testing of mobile applications depends upon

• Version/Build Numbers
• Handsets (Various Devices)
• Test locations (Mapping Apps, GPS etc)
• Testing Environment (Availability of Emulators / Device)
• Back end Infrastructure (Hardware Resources)
• Carrier (Network Support)
• Application (Hybrid, Domain, Consumer etc)
• Testing (Network, Hardware, UI, IA, Compatibility, Interrupt etc)
• Competencies
• (Diff OS Versions, Preferences, Crash Points, Low Connectivity, Low Memory, Sleep Mode Response, UI Response, Accelerometer Response, Battery Usage, Retrials Network Disconnect, Localization, Time Zone Change )

Important factors to be considered for the mobile application testing

• Understand Network & device requirements before testing
• Testing in uncontrolled real-world
• Selection of Automation Tool (If Req.)
• Support various resolutions, input (Touchpad, Keypad)
• Check end-to-end flow at least once for all platforms
• Conduct Performance, GUI & Compatibility using actual devices (Recommended)
• Recoverability (Crash / Transaction)
• Measure performance in realistic conditions (wireless traffic & user load)

Test Strategy for Mobile Apps Testing

 

• Phase 1 Test Planning. Test planning of the testing activates
• Phase 2 Environment Setup. Test team will plan and setup test environment
• Phase 3 Test Cases/Scripts Preparation. Preparation of test cases
• Stage 4 Test Cases /Script Executions. Execution of test Cases /scripts, management of test results and utilization of defect tracking process.
• Phase 5: Bug(s)/Status Reporting: Reporting of bugs, and Test status reporting to team.

 

Testing Types 

Installation Testing

 Purpose of this testing type is to test the installation and un installation of the app

• Can the app be found in the app store or play store? (Check after go-live)
• Does the app , shows all instructions to user while installing the application (for example access to network, notifications etc…)
• Can the app be installed on the device from the desired location?
• Can the new updated app installed without un uninstalling the last version
• Does the app removed successfully when the user un installed the app
• Is user able to continue the installation , if during installation any interruption occurred
• Does the application function as expected after re-installation?
• Does the app behavior is correct if phone memory is full and user tried to install the app

Connectivity Testing

• Does the app behave according to specification if connected to the internet through Wi-Fi?
• Does the app behave according to specification if connected to the internet through 4G?
• Does the app behave according to specification if connected to the internet through 3G?
• Does the app behave according to specification if connected to the internet through 2G?
• Does the app behave according to specification of the app is out of network reach?
• Does the app resume working when it gets back into network reach from outside reach of the network?
• Update transactions are processed correctly after re-establishing connection.
• Does the app behaves according to specification if the app switches between networks (Wi-Fi, 3G, 2G)
• Is downloading of the app prevented in case it’s bigger than the OS allows downloading when connected to cellular networks? 

Functional testing

The functional testing of Mobiles normally consists in the areas of testing user interactions as well as testing the transactions. The various factors which are relevant in functional testing are

Type of application based upon the business functionality usages (banking, gaming, social or business)

• Target audience type (consumer, enterprise, education)
• Distribution channel which is used to spread the application (e.g. Apple App Store, Google play, direct distribution)
• The most fundamental test scenarios in the functional testing can be considered as :
• To validate whether all the required mandatory fields are working as required.
• To validate that the mandatory fields are displayed in the screen in a distinctive way than the non-mandatory fields.
• To validate whether the application works as per as requirement whenever the application starts/stops.
• To validate whether the application goes into minimized mode whenever there is an incoming phone call. In order to validate the same we need to use a second phone, to call the device.
• To validate whether the phone is able to store, process and receive SMS whenever the app is running. In order to validate the same we need to use a second phone to send sms to the device which is being tested and where the application under test is currently running.
• To validate that the device is able to perform required multitasking requirements whenever it is necessary to do so.
• To validate that the application allows necessary social network options such as sharing, posting and navigation etc.
• To validate that the application supports any payment gateway transaction such as Visa, Mastercard, Paypal etc as required by the application.
• Verify whether the application has been launched successfully or not.
• Verify whether the splash screen is displayed for long enough to read the text or not.
• Verify that the application’s display is adapted to the screen size and all buttons and menus are easily clickable.
• Verify that soft keyboard automatically appears if the user’s main action is to enter some text.
• Check that each screen is appropriately displayed in each display mode (landscape, portrait).
• Verify that if the app contains user settings, does the application provide a form to change settings or not.
• Has the  fields validation for empty values been tested?
• Has the  fields validations for mandatory fields been tested?
• Has the   fields validations for mandatory fields been tested?
• Has the  form fields validations for field max input size been tested?
• Has the form fields validations for field min input size been tested?
• Has the all navigations of the applications been tested?
• Has the the GUI of the application been verified accordingly to the mockups?
• Has the spellings check been performed?
• Does the app behaviour is correct, if phone memory is full and user try to run the application
• For a multi-lingual app, verify whether the text content and button labels are properly translated or not.
• Verify if the application supports speech recognition feature, application can recognize the voice in the language of the application.
• Verify if the application uses sensors, application is correctly driven by these sensors.
• Verify if the application communicates with a webserver, if the network coverage fails, does the application automatically reinitiate the communication once the coverage is re-established or not?
• Verify if the application supports calendar and create events feature.
• Verify if the application support capture image.
• Verify if the application supports record sound.
• Verify if the application allows to control the volume, the user can alter the volume anytime.
• Verify if the password is displayed or not when its entered into the application.
• Verify when the device is suddenly powered off, the application does not corrupt data. Are the last saved data saved properly?
• Verify tap on the screen ten times at different positions, the application should work normally and not freeze.

Performance testing

his type of testing’s fundamental objective is to ensure that the application performs acceptably under certain performance requirements such as access by a huge number of users or the removal of a key infrastructure part like a database server.

The general test scenarios for performance testing in a Mobile application are:

• To determine whether the application performs as per the requirement under different load conditions.
• To determine whether the current network coverage is able to support the application at peak, average and minimum user levels.
• To determine whether the existing client-server configuration setup provides the required optimum performance level.
• To identify the various application and infrastructure bottlenecks which prevent the application to perform at the required acceptability levels.
• To validate whether the response time of the application is as per as the requirements.
• To evaluate product and/or hardware to determine if it can handle projected load volumes.
• To evaluate whether the battery life can support the application to perform under projected load volumes.
• To validate application performance when network is changed to WIFI from 2G/3G or vice versa.
• To validate each of the required the CPU cycle is optimization
• To validate that the battery consumption, memory leaks, resources like GPS, Camera performance is well within required guidelines.
• To validate the application longevity whenever the user load is rigorous.
• To validate the network performance while moving around with the device.
• To validate the application performance when only intermittent phases of connectivity is required.

Security testing

The fundamental objective of security testing is to ensure that the application’s data and networking security requirements are met as per guidelines.

The following are the most crucial areas for checking the security of Mobile applications.

• To validate that the application is able to withstand any brute force attack which is an automated process of trial and error used to guess a person’s username, password or credit-card number.
• To validate whether an application is not permitting an attacker to access sensitive content or functionality without proper authentication.
• To validate that the application has a strong password protection system and it does not permit an attacker to obtain, change or recover another user’s password.
• To validate that the application does not suffer from insufficient session expiration.
• To identify the dynamic dependencies and take measures to prevent any attacker for accessing these vulnerabilities.
• To prevent from SQL injection related attacks.
• To identify and recover from any unmanaged code scenarios.
• To ensure whether the certificates are validated, does the application implement Certificate Pinning or not.
• To protect the application and the network from the denial of service attacks.
• To analyze the data storage and data validation requirements.
• To enable the session management for preventing unauthorized users to access unsolicited information.
• To check if any cryptography code is broken and ensure that it is repaired.
• To validate whether the business logic implementation is secured and not vulnerable to any attack from outside.
• To analyze file system interactions, determine any vulnerability and correct these problems.
• To validate the protocol handlers for example trying to reconfigure the default landing page for the application using a malicious iframe.
• To protect against malicious client side injections.
• To protect against malicious runtime injections.
• To investigate file caching and prevent any malicious possibilities from the same.
• To prevent from insecure data storage in the keyboard cache of the applications.
• To investigate cookies and preventing any malicious deeds from the cookies.
• To provide regular audits for data protection analysis.
• Investigate custom created files and preventing any malicious deeds from the custom created files.
• To prevent from buffer overflows and memory corruption cases.
• To analyze different data streams and preventing any vulnerabilities from these.

Usability testing

The usability testing process of the Mobile application is performed to have a quick and easy step application with less functionality than a slow and difficult application with many features. The main objective is to ensure that we end up having an easy-to-use, intuitive and similar to industry-accepted interfaces which are widely used.

• To ensure that the buttons should have the required size and be suitable to big fingers.
• To ensure that the buttons are placed in the same section of the screen to avoid confusion to the end users.
• To ensure that the icons are natural and consistent with the application.
• To ensure that the buttons, which have the same function should also have the same color.
• To ensure that the validation for the tapping zoom-in and zoom-out facilities should be enabled.
• To ensure that the keyboard input can be minimized in an appropriate manner.
• To ensure that the application provides a method for going back or undoing an action, on touching the wrong item, within an acceptable duration.
• To ensure that the contextual menus are not overloaded because it has to be used quickly.
• To ensure that the text is kept simple and clear to be visible to the users.
• To ensure that the short sentences and paragraphs are readable to the end users.
• To ensure that the font size is big enough to be readable and not too big or too small.
• To validate the application prompts the user whenever the user starts downloading a large amount of data which may be not conducive for the application performance.
• To validate that the closing of the application is performed from different states and verify if it re-opens in the same state.
• To ensure that all strings are converted into appropriate languages whenever a language translation facility is available.
• To ensure that the application items are always synchronized according to the user actions.
• To ensure that the end user is provided with a user manual which helps the end user to understand and operate the application who may be not familiar with the application’s proceedings

Usability testing is normally performed by manual users since only human beings can understand the sensibility and comfort ability of the other users.

Compatibility testing

Compatibility testing on mobile devices is performed to ensure that since mobile devices have different size, resolution, screen, version and hardware so the application should be tested across all the devices to ensure that the application works as desired.

The following are the most prominent areas for compatibility testing.

1. To validate that the user Interface of the application is as per the screen size of the device, no text/control is partially invisible or inaccessible.
2. To ensure that the text is readable for all users for the application.
3. To ensure that the call/alarm functionality is enabled whenever the application is running. The application is minimized or suspended on the event of a call and then whenever the call stops the application is resumed.

Recoverability Testing

• Crash recovery and transaction interruptions
• Validation of the effective application recovery situation post unexpected interruption/crash scenarios.
• Verification of how the application handles a transaction during a power failure (i.e. Battery dies or a sudden manual shutdown of the device)
• The validation of the process where the connection is suspended, the system needs to re-establish for recovering the data directly affected by the suspended connection.

Interruptions Testing

• Does the app behave as designed/desired if there is an incoming call?
• Does the app behave as designed/desired if there is an incoming SMS?
• Does the app behave as designed/desired if the charger is connected?
• Does the app behave as designed/desired if the charger is disconnected?
• Does the app behave as designed/desired if the device goes to sleeping mode
• Does the app behave as designed/desired if the device resumes from sleeping mode
• Does the app behave as designed/desired if the device resumes from lock screen?
• Does the app behave as designed/desired if the device is shaken?
• Does the app behave as designed/desired if a local message is coming from another app (think of: calendar reminders, to-do task etc.).
• Does the app behave as designed/desired if a push message is coming from another app (think of: twitter mentions, whatsapp message, wordfeud invitation, etc).
• Does the app behave as designed/desired if the “Battery low” message is pushed
• Does the app behave as designed/desired if the sound on the device is turned off?
• Does the app behave as designed/desired if the device is in airplane mode?
• Verify that the app does behave as intended if the mobile device is shaken.
• Does the app write to the SD card?
• Does the app shows correct behaviour if the SD card is filled/full?
• Does the app shows correct behaviour if the SD card is removed?
• Does the app shows correct behavior if the SD card is changed?

Tips for reporting mobiles apps defects

• Never write ‘click’ in the defect report of a mobile application. Instead write ‘Tap/Double Tap’ as these are the correct terms.
• A mobile application has ‘screens’ and not ‘pages’. Never use the term ‘page’ in the defect report of a mobile application as only web applications have ‘pages’.
• The ‘keyboard’ that only contains numeric values is called ‘Keypad’. If this ‘Keypad’ is used for dialing numbers, it will be referred to as ‘Dial pad’. So always use the specific terms while reporting the defect.
• There are two screen views. i) Landscape, ii) Portrait. Used these terms instead of writing ‘horizontal screen’ and ‘vertical screen’ for Landscape and Portrait respectively.
• ‘Drag n Drop’ is a term that is used for web applications. For mobile application, ‘slide to rearrange’ must be used.
• ‘Slide’ or ‘Swap/Swipe’ must be used when moving down the application screen instead of ‘scroll’.

·         Particularly in iPhone, there is a three line button on top which is used to show/hide side menu. This button is called ‘hamburger’ button. While writing the defect report, it is mostly referred to as ‘menu icon’ or ‘side menu button’.

·         Use of the following terms will also make it easy for the developers to understand the defects properly.

• Tap: Opens or launches whatever you tap.
• Double Tap: Zooms in or out in stages.
• Pan: Moves through screens or menus at a controlled rate.
• Flick: Scrolls rapidly through menus or pages or moves sideways in hubs.
• Pinch: Zooms gradually out of a website, map or picture.
• Stretch: Zooms gradually in a website, map or picture.
• Rotate: Move a picture or other item(s) on the screen in a circular direction (clockwise or counter-clockwise).

Approach	Types of Testing	Manual Testing		Use Automation
		Use Multiple Devices	Use Emulators
	Functional Testing	Yes	Yes	Yes
Standard Testing	Installation Testing	Yes	No	No
	Interruption Testing	Yes	No	No
	Regression Testing	Yes	No	Yes
	Network Connectivity Testing	Yes	No	No
Special Type of Testing To Address Specific Challenges	Compatibility Testing	Yes	Yes	Yes
	GUI/Usability Testing	Yes	Yes	No
Types Of Testing More Relevant for enterprise Mobile Applications	Performance Testing	Yes	No	Yes
	Security Testing	Yes	No	Yes

Also visit the link to learn mobile apps terminologies :

Common terms used in mobile application development and testing