4 Benefits of Using the Espresso Test Automation Tool

If you’re an Android developer, you’re probably familiar with Google’s Espresso test automation framework. As an open-source tool, it’s very easy for developers to use and extend within their working environment (Android Studio IDE).

But before discussing the benefits of Espresso, let’s understand the motivations and pains developers and test automation engineers face today while trying to validate their Android application (APK) throughout the build/dev/test workflow.

  • Each build needs to be validated after code changes are made.
  • Dependencies on remote servers and other workstations for testing slow down the process.
  • Unit and functional tests need to be easy to execute from both an IDE and continuous integration perspective.
  • Apps need to be tested using the latest Android OS APIs that support new platform features and OS versions.
  • Testing needs to occur on both emulators and real devices.

In light of these challenges, it’s clear why the adoption of the Espresso automation framework is high. Even though Espresso is an instrumentation-based test framework, it has many benefits to both developers and test automation engineers. It uses Junit underneath the hood, so Espresso is easy to use within leading IDEs and provides useful testing annotations and assertions. It’s also fully integrated within the leading Google Android IDE – Android Studio.

Here are four main benefits of using Espresso:

1. Espresso workflow is simple to use

The way Espresso works is by allowing developers to build a test suite as a stand-alone APK that can be installed on the target devices alongside the application under test and be executed very quickly.

2. Fast and reliable feedback to developers

As developers are trying to accelerate deployment, Espresso gives them fast feedback on their code changes so they can move on to the next feature or defect fix; having a robust and fast test framework plays a key role.

Espresso does not require any server (like Selenium Remote WebDriver) to communicate with; instead it runs side-by-side with the app and delivers very fast (minutes) test results to the developer.

3. Less mobile testing flakiness

Because Espresso offers a synchronized method of execution, the stability of the test cycle is very high. There’s a built-in mechanism in Espresso that, prior to moving to the next steps in the test, validates that the Element or Object is actually displayed on the screen. This eliminates test execution from breaking when confronted with “objects not detected” and other errors.

4. Developing Espresso test automation isn’t hard

Developing Espresso test automation is quite easy. It is based on Java and Junit, which is a core skillset for any Android app developer. Because Espresso works seamlessly within the Android Studio IDE, there’s no setup or ramping up and no “excuses” – to actually shift quality in the in-cycle stage of the app SDLC.

In addition to the above, there is of course the large community powered by Google that pushes the Espresso test automation framework and allow easy and fast ramp up for newcomers.

Learn more using the Espresso Cheat sheet below:

Espresso Test Automation Framework

Perfecto is offering support for both Android Studio IDE as well as the ability to install and launch an Espresso test suite (APK) on real devices in the cloud across various locations and user conditions. For more information, please refer to the Perfecto Community and search for “Android Studio” or “Espresso.”

Joe Colantonio’s Test Talk: Mobile Testing Coverage Optimization

How does a company nowadays put together a comprehensive test strategy for delivering high-quality experiences for their applications on any device? I think this is the question I get asked most frequently and it is the biggest challenge in today’s market, how to tackle mobile testing and responsive web testing. The solution can be the difference between an app rated 1 star or an app rated 5 stars.

Play Podcast

I had a lot of fun talking to Joe Colantonio from Test Talks about how to create a successful app starting with my Digital Test Coverage Optimizer. Listen to the full talk to hear my ideas on moving from manual testing to automation, tracking the mobile market, the difference between testing in simulators and emulators versus real devices and more.

https://joecolantonio.com/testtalks/110-mobile-testing-coverage-optimization-eran-kinsbruner/

 

JC

#30DaysofTesting – Day 8 Reporting

As the #30daysoftesting challenges continues, i have decided today to put the famous iOS Native LinkedIn mobile app and perform some exploratory testing on it using my iPhone 6 Plus device running iOS 9.X

IMG_9531

Today’s challenge was about finding up to 5 different defects and reporting them back to the app vendor.

Here are my findings:

  • Searching through the contact list (my list of contacts overpasses 2500 members) is simply unusable since the A-Z side bar is non proportional with the page size, so basically trying to filter by letter (e.g. “K”) is very hard

IMG_9526

  • App crashed twice when entering long string of characters into the search bars either for searching contacts/groups or messages
  • Sharing a message from the app – DOES NOT WORK. You can only share from the app main screen an update but not a message.

IMG_9527

  • Redundant “Close Page” link in various EULA/Privacy web pages – accessible through setting screen (Privacy Policy, User Agreement, EULA)

IMG_9528

 

I’ve reported back to LinkedIn about these defects, below is their confirmation email – pending their response.

LIIssues

Happy #30Daysofesting To All Of You.

 

Eran

30 Days of Testing Challenge – I’m In!

For those who aren’t familiar with this month program led by Ministry of Testing, the full details are HERE.

I plan to stand up for this challenge (Tweeter handle to follow this program is: #30daysoftesting

For day 1, I’ve bought the following book (Testing in 30+ open source tools, 2nd Edition), and i plan on reading it (well – large portion of it, it’s more than 1200 pages) – the book highlight the main trend i am seeing over the past 12-18 months in the digital space (especially Mobile) where many new open-source test frameworks are being introduced to the market aiming to make both Dev and Testers lifes easier.

IMG_9439

With the amount of new open-source tools in the market agile teams can achieve the following:

  • Faster test development
  • Fine tune the test environment to meet complex product requirements
  • Develop and execute tests from the Dev/Test native environment (IDE)
  • Execute and receive actionable feedback faster

With the above benefits, it is clear that today’s agile teams have a lot to gain by embracing open-source compared to traditional proprietary testing solutions.

Looking forward to update on 2nd-30th day of this program

Eran

Selecting the Best Open-Source Test Automation Tool for You

There’s a shift to open-source mobile test automation tools happening today among developers and QA. And it’s not just happening in mobile testing. Many mature technology sectors are adopting lightweight, vendor-transparent tools to fulfill the need for speed and integration.

As with many free and open-source software markets however, a plethora of tools complicates the selection process. How do you know what to spend time learning, integrating and deploying in your own environment?

This post aims to help you choose which open-source test automation framework to use based on a number of critical considerations.

Criteria for Selecting an OSS Test Automation Framework

The rationale and advantages behind your choice of open-source test automation tools can be related to a few key benchmarks:

  1. Ease of script development and execution (supports agile processes and short iterations)
  2. Cross team collaboration capabilities (Both QA and Dev can easily use the same tools)
  3. Match app platform with test development language (ObjectiveC/Swift for iOS, Java for Android)
  4. No platform capabilities gap around testing (support for the latest OS features)
  5. Support for real devices as well as emulators/simulators
  6. Fully integrated tools within IDEs

Additionally, there are considerations that differ from mobile/web project to project:

What are the application use cases? What level of complexity needs to be tested?

  • Heavy UI elements?
  • Environment dependencies (Networks, GPS, Camera)?
  • What OS versions and API levels should be supported?

Does the app support multiple platforms ((Mobile (iOS/Android), Web))?

Based on these considerations, you are far more likely to see long-term success with your automated testing efforts than if you simply dive in to a given framework without understanding the implications.

SO what are the most popular OSS test automation frameworks?

When looking at today’s open-source mobile test automation landscape, there are five highly-adopted test frameworks:

  1. Selenium – The leading open-source test framework for web app test automation
  2. Appium – Open-source test automation framework for mobile native, web and hybrid apps
  3. Calabash – Behavior-driven development (BDD) test framework based on Ruby development language
  4. Espresso – Google open-source test automation framework within Android Studio
  5. XCUITest – Apple’s open-source test automation framework within XCode IDE

Each of these frameworks are being sponsored by a different community and have unique benefits to their target platforms and respective audiences. Though general-purpose frameworks cover a broad range of devices, they often lack late-breaking hardware support; conversely, frameworks that are device-specific often lack support for different scripting languages and approaches. It is therefore important to identify what is important to your team and project as part of the selection process, and avoid just selecting a framework based on technical requirements.

In the table below, we list the top test automation tools with their core capabilities and limitations:

Open source test automation tools

Why choose one framework over another?

You may already have portions of test automation frameworks and tooling integrated into your software delivery process. These decisions may not always have been made by a single person or team, but rather a collection of experiences and motivations over time. A few of these motivations are:

  • Teams trying to get fast quality feedback per each of their app builds and code commits
  • Teams testing UI and functionality of their app
  • Teams using behavior-driven testing tools to match their agile processes
  • Performing cross-platform testing of mobile and web
  • Complementing unit testing

The table above shows that not every tool or test framework can provide full coverage of your requirements, and may even come with a risk of limiting the quality of the mobile or web app under test do to their shortcomings. We have summarized each approach to help you decide which makes the most sense for you.

Appium

Appium is best suited to QA teams trying to test the functionality of native, mobile web and mobile hybrid apps across iOS and Android. This tool is less suitable for developers who wish to develop and perform unit testing since it uses a different scripting language than the app itself, (e.g. ObjectiveC). The generated Appium report is a bit limited from a debugging and fast feedback loop perspective, and does not include videos, network logs and key vitals information.

Selenium

The Selenium framework is the best choice for web test automation teams testing for RWD (responsive web design), or stand-alone web sites. It’s less suitable for developing unit testing, which makes this framework less appealing for developers. Core Selenium test reports are not highly informative and lack unique mobile related insights.

Calabash

Calabash is designed for organizations that work in BDD (behavior-driven development) workflows. The tool offers an easy path to develop the features in parallel with the tests for these features in an easy user-flow based language. Calabash is appealing for both dev and QA practitioners. The tool provides solid insights and reports to Dev and QA teams.

Espresso & XCTestUI

These two tools are are very similar in that they are both designed for target users. Espresso is for Android and XCTestUI is for iOS. Both tools are fully integrated into development IDEs such as Android Studio/Xcode, and offer very easy to develop techniques, including test recorders. These tools are fully maintained by Google and Apple, which assures that they always support the latest OS features (i.e. iOS Force Touch) so developers can stay ahead of the market and test accordingly. These tools support both unit testing types and functional UI testing. Both tools are app context only, which limits their abilities to test for user condition scenarios.

 

7 Mobile Test Automation Best Practices

Developing a mobile test automation scenario isn’t that complicated. Developers and testers use a variety of commercial test automation frameworks or open source tools such as Selenium and Appium to do automation. However, when trying to execute these tests on real devices or integrate them into an Agile or CI (continuous integration) workflow, things get a little complicated.

The major challenges around mobile test automation

The essence of developing test automation is to be able to use and re-use scripts many times, across platforms and environments. Test automation should be as maintainable as possible, especially as new platforms and product features are released. Many organizations that develop test automation for their mobile apps face the following challenges:

  1. Executing the tests against a variety of real mobile devices
  2. Executing these tests in parallel
  3. Leveraging existing test code (re-usability) for new tests
  4. Including real end-user environments/conditions (changing network conditions, low battery) in the tests
  5. Overcoming unexpected interruptions (incoming call, apps running in background)
  6. Running these tests unattended — over night, as part of a Jenkins CI job

These are just few of the challenges organizations confront when trying to progress from older SDLC processes and meet faster releases and enhanced Dev–>Build–>Deploy–>Test–>Deploy cycles.

7 practical test automation tips

Overcoming these challenges starts with few changes in the overall mobile app dev and test processes.

Consider these seven recommendations for building sustainable unattended automation.

Test automation

The key to mobile test automation is to start with a small number of test cases, automate them, and assure that they are robust enough and can be executed in parallel and unattended. Only then should you invest more and grow the test suite.

An important question to ask at the start is: What should I be automating? Organization often do not choose the right tests to automate, resulting in lost development time, weak ROI, and an over-reliance on manual testing.

To learn more about the 7 Ways to Overcome Test Automation Obstacles, please join us next week for a webinar hosted by myself, automation expert and author Daniel Knott, and Perfecto’s Director of Technology Uzi Eilon.

Responsive Web: The Importance of Getting Test Coverage Right

When building your test lab as part of a RWD site test plan, it is important to strategically define the right mobile devices and desktop browsers which will be your target for your manual and automated testing.

For mobile device testing you can leverage your own analytics together with market data to complement your coverage and be future ready, or leverage reports such the Digital Test Coverage Index Report.

For web testing you should also look into your web traffic analytics or based on your target markets understand which are the top desktop browsers and OS versions on which you should test against – alternatively, you can also use the digital test coverage index report referenced above.

Related Post: Set Your Digital Test Lab with Mobile and Web Calendars

Coverage is a cross organizational priority where both business, IT, Dev and QA ought to be consistently aligned. You can see a recommended web lab configuration for Q1 2016 below which is taken from the above mentioned Index – Note the inclusion of Beta browser versions in the recommended mix due to the nature silent updates of these versions deployment on end-user browsers.

WCReport
For ongoing RWD projects  – once defining the mobile and web test coverage using the above guidelines, the next steps are of course to try and achieve parallel side by side testing for high efficiency, as well as keep the lab up to date by revising the coverage once a quarter and assure that both the analytics as well as the market trends still matches your existing configuration.

As a best practice and recommendation, please review the below mobile device coverage model which is built out of the 3 layers of Essential, Enhanced and Extended where each of these layers includes a mix of device types such as legacy, new, market leaders and reference devices (like Nexus devices).

MobileCoverageLayers

To learn more, check out our new Responsive Web Testing Guide.

responsive web testing strategy