Mobile, Cross Browser Testing, DevOps and Continuous Testing Trends and Projections for 2018

As we about to wrap out 2017, It’s the right time to get ready to what’s expected next year in the mobile, cross-browser聽testing and DevOps landscape.

To categorize this post, I will divide the trends into the following buckets (there may be few more points, but I believe the below are the most significant ones)

  • DevOps and Test Automation on Steroids Will Become Key for Digital Winners
  • Artificial Intelligence (AI) and Machine Learning (ML)/ Tools alignment as part of Smarter Testing throughout the pipeline
  • IOT and Digital Transformation Moving to Prime Time

 

DevOps and Automation on Steroids

If in 2017, we’ve seen the tremendous adoption of more agile methods, ATDD, BDD and organizations leaving legacy tools behind in favor of faster and more reliable and agile-ready testing tools, such that can fit the entire continuous testing efforts whether they’re done by Dev, BA, Test or Ops.

In 2018, we will see the above growing to a higher scale, where more manual and legacy tools skills are transforming into more modern ones. The growth in continuous testing (CT), Continuous Integration (CI) and DevOps will also translate into much shorter release cadence as a bridge towards real Continuous聽Delivery (CD)

 

Related to the above, to be ready for the DevOps and CT trend, engineers need to become more deeply familiar with tools like Espresso, XCUITest, Earl Grey and Appium on the mobile front, and with the open-source web-based聽framework like the headless google project called Puppeteer, Protractor, and other web driver based framework.

In addition, optimizing the test automation suite to include more API and Non-Functional testing as the UX aspect becomes more and more important.

Shifting as many tests left and right is not a new trend, requirement or buzz – nothing change in my mind around the importance of this practice – the more you can automate and cover earlier, the easier it will be for the entire team to overcome issues, regressions and unexpected events that occur in the project life cycle.

AI, ML, and Smarter Test Automation

While many vendors are seeking for tools that can optimize their test automation suite, and shorten their overall execution time on the “right” platforms, the 2 terms of AI and ML (or Deep learning) are still unclear to many tool vendors, and are being used in varying perspectives that not always mean AI or ML 馃檪

The end goal of such solutions is very clear, and the problem it aims to solve is real –> long testing cycles on plenty of mobile devices, desktop browsers, IOT devices and more, generates a lot of data to analyze and as a聽result, it slows down the DevOps engine. Efficient mechanism and tools that can crawl through the entire test code, understand which tests are the most valuable ones, and which platforms are the most critical to test on due to either customer usage or history of issues etc. can clearly address such pain.

Another angle or goal of such tools is to continuously聽provide a more reliable and faster test code generation. Coding takes time, requires skills, and varies across platforms. Having a “working” ML/AI tool that can scan through the app under test and generate robust page object model, and functional test code that runs on all platforms, as well as “responds”聽to changes in the UI, can really speed up TTM for many organization and focus the teams on the important SDLC activities in opposed to forcing Dev and Test to spend precious time on test code maintenance.

IOT and The Digital Transformation

In 2017, Google, Apple, Amazon and other technology giants announced few innovations around digital engagements. To name a few, better digital payments, better digital TV, AR and VR development API and new secure authentication through Face ID. IOT this year, hasn’t shown a huge leap forward, however, what I did notice, was that for specific verticals like Healthcare, and Retail, IOT started serving a key role in their digital user engagements and digital strategy.

In 2018, I believe that the market will see an even more advanced wave in the overall digital landscape where Android and Apple TV, IOT devices, Smart Watches and other digital interfaces becoming more standard in the industry, requiring enterprises to re-think and re-build their entire test lab to fit these new devices.

Such trend will also force the test engineers to adapt to the new platforms and re-architect their test frameworks to support more of these screens either in 1 script of several.

Some insights on testing IOT specifically in the healthcare vertical were recently presented by my colleague Amir Rozenberg – recommend to review the slides below

https://www.slideshare.net/AmirRozenberg/starwest-2017-iot-testing/聽

 

Bottom Line

Do not immediately change whatever you do today, but validate whether what you have right now is future ready and can sustain what’s coming in the near future as mentioned above.

If DevOps is already in practice in your organization, fine – make sure you can scale DevOps, shorten release time, increase test and platform automation coverage, and optimize through smarter techniques your overall pipeline.

AI and ML buzz are really happening, however, the market needs to properly define what it means to introduce these into the SDLC, and what would success look like if they do consider leveraging such. From a landscape perspective, these tools are not yet mature and ready for prime time, so that leaves more time to properly get ready for them.

Happy New 2018 to My Followers.

Advertisements

The Essentials of iOS App Testing For iPhone X

48 hours ago, Apple revealed its new and futuristic聽iPhone X. Regardless of its design, and debatable price tag, this device also introduced a whole set of functionality, display, and engagement with the end-user.

iOS11聽is turning聽to be quite different from聽previous releases from both user adoption which is still low (~30%) and also from a quality perspective –聽4 patch releases in 1.5 months is a lot.

Most of the changes are already proving to cause issues for existing apps that work fine on iOS11.x and former iPhones like iPhone 8, 7 and others.

In this post, I’d聽highlight some pitfalls that testers as well as developers ought to be doing immediately if they haven’t done so already to make sure their apps are compliant with the latest Apple mobile portfolio.

The post will be divided into 2 areas:聽Mobile testing聽recommendations and聽App Development聽recommendations.

Mobile Testing for iPhone X/iOS 11

  • Test聽across聽all supported platforms聽as a general statement. iOS11 isn’t for every device, and apps are stuck on iOS10 that has different functionality than the iOS11. Test your apps across iOS9.3.5, iOS 10.3.3, and the latest iOS11.x
  • iPhone X comes from the factory with iOS11.0.1, requesting for an update to iOS1.1 – that means, this device will never get the intermediate iOS11.0.2/iOS11.0.3 – if customers haven’t yet updated to iOS1.1, you may want to have 1 device like iPhone 8/7 still on iOS11.0.3 so you have coverage for聽iOS11.Latest-1
  • Display and Screen Size for iPhone X聽specifically changed, and this device has a 5.8” screen size that is different for all other iPhones. Testing UI elements, Responsive apps layouts and other graphics on this new device is obviously a must (below is an example taken from聽CVS聽native app showing UI issues already found by me while playing with the device). This device is also full screen similar to the Samsung S8/Note 8 devices. A lot of tables, text field, and other UI elements need to be iOS11/iPhopne X ready by the developers.

Image title

  • New gestures and engagement flow聽impact usability as well as test automation scripts. In iPhone X, unlike previous iPhones, the user has no HOME Button to work with. That means that in order for him to launch the task manager聽 (see below) and switch or kill a background running app, he needs to follow a different flow. What that means is that at first, the app testing teams need to make sure that this new flow is covered in testing, and more important, if these flows are part of a test automation scenario, the code needs to be adapted to match the new flow.

Image title

In addition to the removal of the聽Home button聽that causes the new way of engaging with background apps, the way for the user to return to the Home screen has also changed. Getting back to the Home screen is a common step in every test automation, therefore these steps need to account for the changes, and聽replace the button聽press聽with聽a聽Swipe Up gesture.

  • Authentication and payment scenarios聽also changed with the elimination of the聽Touch ID聽option, that was replaced with the聽Face ID.聽While iPhone X introduced an innovative digital engagement with the Face recognition technology, the de-facto today to log in into apps, make payments and more, is still the Fingerprint authentication.聽Testing both methods聽is now a quality and dev requirement. From a scan that I ran through the leading apps in the market (see examples below), there is a聽clear unpreparedness聽for iPhone X. Most apps will either show on their UI the option to log in via Touch ID or if they support Face ID, they will allow users to use it, while still showing on the UI and in the app settings the unsupported option.

Image title

  • Testing mobile web and responsive web apps聽in both landscape and portrait mode with the unique iPhone X display is also a clear and immediate requirement. I also found issues mostly around text truncation and wrong leverage of the entire screen to display the web content.

Image title

 

In addition, trying to work with Hulu.com website proved to also be a challenge. Most menu content is being thrown to the bottom of the screen under the user control, making it simply inaccessible. Obviously, the site is not ready for iPhone X/Safari Browser.

Mobile Apps Development

  • Optimize existing iOS apps from both UI as well as authentication perspective. As spotted above, there are clear compatibility issues around the removal of the Touch ID option, that needs to be modified on the UI side of the apps when launched on iPhone X. In addition, scaling UI elements on the new screen whether for RWD apps or mobile apps needs refactoring as well. Apple is offering app developers a聽ui聽guidlines聽to help make the changes fast.Image title
  • Leverage advanced capabilities in iOS11聽that best suit the new chipset (AI11 Bionic) and the camera sensors, to introduce digital engagement capabilities around augmented reality (ARKit API’s) and others. Retail apps and games are surely the 1st most suitable segments to jump on these innovative capabilities and enrich their end users’ experiences.

Image title

Bottom Line

The new iPhone X might be paving the way together with the Android Note 8 for a new era of innovation that offers App developers new opportunities to better engage and increase business values. If quality will not be aligned with these innovative opportunities, as shown above, that transformation will be quite challenging, slow and frustrating for the end users’

It is highly recommended for iOS app vendors across verticals to get hands-on experience with the new platform, assess the gaps in quality and functionality, and make the required changes so they are not “left behind” when the innovative train moves on.

Trends in Cross Browser Testing and Web Development

Typically, i”ll write a lot on mobile app testing, tools, trends, coverage and such.

In this blog, I actually wanted to share some up to date trends as I see them in the web landscape.

The web market has shifted a lot over the past years alongside the mobile space. We see a clear use of specific development languages, development frameworks and of course specific test frameworks aimed to test Angular, jQuery, Bootstrap,.Net and other websites.

From a聽Dev Language perspective, the web FE developer is mostly using the following languages as part of his job:

Source:聽http://vintaytime.com/premium/top-programming-languages/

As a clear trend in web development, it shows that JavaScript is the leading language used by web developers. It’s actually not a huge surprise since if you move to the top聽frameworks used by these web developers, you will see quite a few that are based on JavaScript.

There are聽some trends seen recently by developers around shifting to聽nonAngularJS聽web development framework like聽Aurelia, React,聽and聽Vue.JS聽that are seeing a growing usage and adoption by developers due to considerations such as (larger list of Pro’s/Con’s are in source 1 below). With this trend in mind, and you’ll read in my references below, the new solutions are still not as complete as AngularJS is.

  • Shorter learning curve
  • Simple to use, clean
  • Flexibility
  • Lightweight聽compared to others (less than half the size of AngularJS e.g.)
  • Better performing
  • Easy to integrate with other front-end stack tools
  • Responsive server-side rendering (Vue.JS supports it, reduces time for users to see rendered content)
  • SEO Friendly
  • Good documentation and Community Support
  • Good debugging capabilities

Source 1:聽https://www.slant.co/topics/4306/~angular-js-alternatives

Source 2:聽https://w3techs.com/technologies/details/js-angularjs/all/all

Now, that we have seen the leading web development languages, and frameworks used these days, let’s drill down into what test automation engineers are adopting.

Selenium without a doubt is the leading and base for most frameworks, however, even in this space, we see new and innovative test frameworks such as聽Casper.JS,聽TestCafe,聽Buster.JS,聽Nightwatch.JS聽together with the traditional聽Webdriver.IO聽and of course聽Protractor.

If we examine the below visual (Source:聽NPM Trends), it’s a clear market dominance between Selenium and Protractor that underneath its implementation does uses Selenium WebDriver, and supports Jasmine and Mocha tools.

The advantage of tools like Protractor is that they support much easier web sites that were developed in various frameworks like AngularJS, Vue.JS etc. Such advantage allows test automation engineers to agnostically use them for multiple websites regardless to the frameworks they are built with.

It is not that easy, and pink as I described above, but it does give a good headstart when starting to build the test automation Foundation.

Thre are few other players in that space that are aimed at specific unit testing, and headless browser testing (Phantom.JS, Casper.JS, JSDom etc.).

As I blogged in the past, from a test automation strategy perspective, teams might find it beneficial and more complete to leverage a set of test frameworks rather than using only one. If the aim is to have non-UI headless browser testing together with Unit testing and also UI based testing, then a combination of tools like Protractor, Casper.JS, QUnit might be a valid approach.

I hope you find this post useful, and can “swim” in the hectic tools landscape. As always, it is important to match the tool to the product requirements, development methodology (BDD, Agile, Waterfall etc.), supported languages and more.

Recent Web Browser Quality Related Innovations

Yea, I know that my blog title is mobiletestingblog, but that’s not a mistake in the title 馃檪

There is no distinction anymore around which platform is used to consume content today, whether it’s a smartphone, tablet or a desktop browser when it comes to web聽apps.

If your company is developing a web app or responsive website, these sites ought to be tested thoroughly against all of the above platforms. The majority of web traffic BTW today is coming from mobile devices.

In general, it is good to know that from a desktop browser market share, there are less familiar players such as UC browser by Alli-baba and Samsung Internet browsers that hold a nice chunk of the market (globally) – so, avoiding them as 聽part of your test coverage matrix might not be a good strategy.

Source:聽http://gs.statcounter.com/browser-market-share

In general, the below would be the formula for web testing that I would recommend these days, however if from a web traffic analysis and supported geographies you have a requirement to target聽China, Europe, and others – then the above metrics should be added to the mix either in addition to the below, or as an adjustment.

With that in mind, I wanted to highlight in this post some recent web specific tools that are out there, free and can be extremely useful for both developers and testers.

In Google Chrome 59 (Beta is already available today!), Google is introducing new code coverage built-in tools that can allow both developers and testers to record the screen activities and report back in a nice dashboard how much of the site content (javascript, and more) was actually executed in an aim to optimize the website quality, performance and much more.

From a user perspective, they only need to enable the Code Coverage option from within the developer tools in Chrome, so it is added under the Sources menu option as seen in the below

Once that is done, simply start capturing the code coverage by clicking on the Record button to get an output like the below – simple, valuable, and unfortunately聽only available as free and built-in solution within this browser compared to FireFox/Safari and others 馃槮

I went and used this new tool on Geico.com responsive site and nearly completed the most common transaction of querying聽for a new car insurance. At the end of the recording, i received the below chart that as you’ll see – shows a usage of only ~60% of the site JavaScript code in this journey.

When drilling down deeper to a specific .JS source file, you can see a highlighted source with Green/Red where it is actually used and unused – this is what your web developers need to see and optimize wisely.

Let’s see a key feature that was recently introduced in FireFox also, and cab be useful for both Dev and testers.

2 weeks ago, Mozilla released FireFox 53聽that is their聽1st step in a new project called Quantum, that aims to enhance performance, stability and more.

Among the innovations in that release are compact themes, usability features like reading time for the page, new permission model (see below), faster performance and few other bug fixes for stability.

 

Detailed release notes on FF 53 can be found here:聽https://www.mozilla.org/en-US/firefox/53.0/releasenotes/.

In addition to the newly introduced features, and if you’re not aware of – FireFox offer quite useful developer tools including object inspector, performance monitor, debugger and network monitor that can also enhance your overall web Dev and Test activities (see example below)

Performance Monitoring Tools From Within FireFox Developer Tools

Network Monitoring Options From Within FireFox Developer Tools

Bottom Line

With Chrome and FireFox being the leading Desktop and Mobile browsers, it is very important for web teams to continuously monitor the early releases from Google and Mozilla, and as the 1st Beta or Dev branches are available to validate – Do It. This can not only reveal earlier regressions but might also as mentioned in this blog, offer you some new productivity tools that can increase the value to your overall Dev and Test activities.