Getting Your Test Tool Right is Key to Releasing Quality Software

choosing right software testing tools

Choosing and investing in a software testing tool or a test automation framework is like fitting into the right sports shoes – you need to run with them, walk with them, and if on an expedition climb with them. While you want the testing tool to work for you, you have multiple objectives to achieve with that tool/framework and finally get higher ROI.

Adrian Ludwig, Director of Android Security at Google, mentioned to a tech portal at a security conference that its recent launch Pixel and Pixel XL are as secure as iPhones. However, The Register recently reported that a group of Chinese white hat hackers could hack a Pixel in 60 seconds at the PwnFest hacking competition in Seoul.

With stakes so high, testing is getting redefined every second, reinforcing the need for an all-inclusive applicable testing tool for specific business requirements that would ensure quality deliverables.

What are the key activities of a tester that a testing tool can ease out? To name a few:

  • Building and sustaining a release /project cycle /component information
  • Maintaining test artefacts for every release/cycle
  • Create traceability index for test assets
  • Build support for Test execution, for example, test suite creation, test execution reports, etc.
  • Bug/Defect tracking and management

Modern day testing concepts and the new school of thought reinforces the application of testing way ahead in the development process. This has created a paradigm shift and has changed the relevance of testing in the overall SDLC process.

For instance, Shift-left approach, DevOps, Agile Testing are methodologies that encourage rigorous and constant testing right through development, with focus driven on quality deliverables.

Before you dive into the reasons/rationale behind choosing the right testing tool, here are some relevant reasons for implementing and investing in a testing or test automation tool.

  • Reduce errors due to repetition: Repetition is boring and can lead to errors if executed manually. If the tests are fed in constantly for regression tests and checked against coding standards, there are some prominent chances of errors.
  • Consistency and recurring tests: Some testing tasks need tremendous consistency and repeatability in exactly the same way. Manual efforts can lead to variations leading to errors. By implementing a tool, these tasks can be reproduced exactly and automatically to validate the consistency of the results.
  • External assessment: A testing tool can bring objectivity to the testing activity. A tool brings automation and cuts the subjective notions from the testing process. The assessment is done to identify defects.
  • Easy access to test reports: A testing tool facilitates well-designed and well-presented reports for assessing the test data and analysis. It provides information in the form of graphics, statistics, performance metrics, etc.

Testing tools are segmented according to the various testing activities that the tool is expected to execute. The market is flooded with Licensed as well as Open-source tools for diverse and increasingly challenging testing needs. The segmentation of testing tools can vary as per the changing requirements in the marketplace.

Experts have set some key predictions for Testing, namely, expansion of Mobile Application testing, rise of Agile and Continuous Delivery models, and increasing adoption of Open-Source testing tools. These intrinsic needs of products and services will keep on determining the adoption of testing and testing tools in the holistic software development process.

Types of testing tools

testing tools types

What are the questions you need to ask before choosing and finally investing in a testing tool? These questions/parameters will lead you nearer to your testing and product development goal – Quality!

Whether you are a start-up, a large conglomerate, a medium or large enterprise, these questions can further enable you to justify the costs and estimate the ROI from the tool to the management team. These parameters are applicable across any type of testing/test automation tool that you plan to invest in.

What are you expecting to achieve from the tool?

Get specific with your requirement and try to breakdown your testing goals. This will help you to further address your testing requirements and choose an appropriate tool.

Whom are you sourcing the tool from?

It is important to know the details of the vendor you are sourcing the tool from. Get to know about the company, the support that you will receive, and most importantly how stable is the company.

How established is the tool?

Before you invest time, money and efforts, know about the tool and get to know market feedback. Find out details such as how agile and adaptable it is, any expected upgrades in the near future, relevance for your business needs, and relevant Analyst feedback.

What is the quantum of training required?

First, decide between a Licensed or Open-source tool with proper required justification for your business/testing requirements. Understand the training process and requirements. Research if there is an active community supporting the tool and what is the estimated training cost.

This will help you understand the overall expected learning curve for your team and weigh it against the existing technical capabilities.

Is there a Proof of Concept (PoC)?

Free trials and demos is very much the trend amongst vendors and enterprises selling tools and related services. Additionally, it is important to figure out whether your tool vendor will provide you a proof of concept as well. It will give you a first-hand idea about the tool’s capabilities and understand whether it is in sync in with your business requirements.

Additionally, check for other related details, namely, warranty period, validity of free trials, number of free trials before purchase, etc.

What does the competition look like?

It is wise to not get carried away with only one offer in hand. Gauge the competition, know what other similar tool vendors are offering you, and experiment with various tools. This will help you justify the investment that you are planning to make in the chosen tool vendor.

If your business is not so technically aligned and your internal technical capabilities are questionable, it is suggested to on board an experienced testing partner. These partners may have partnerships with testing tool vendors to leverage for your project, or can confidently advise you on the right testing tool to choose for your Quality Assurance needs.

Over to you!

Gallop’s Advisory and Transformation Services collaborate with businesses to understand their QA focus areas – people, tools, and infrastructure across the delivery lifecycle. The Advisory Services enable clients to gauge their QA maturity and detect gaps in their practices.

Connect with us to embark on a QA transformation journey armed with the right testing tool, comprehensive testing framework, and a metrics driven QA approach. Build value-added engagements and ensure a high performance of your software/application with our Transformation services.

The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.

List of 100 Software Testing Tools To Meet Your Testing Objectives

top Software Testing tools
Online Black Friday threats, Security breaches, Online banking thefts, System downtime, and many such nightmares are giving sleepless nights to enterprises across the globe. Ensuring performance with loaded security and enhanced experience is a must-have for every competent player in the marketplace.

We bring to you a resourceful and loaded list of software testing tools that not only reduce the testing efforts, but also help you get your software/application faster to the market and sustain quality while at speed.

Check out this list that covers 100 open-source as well as licensed software testing tools. Most of the tools listed here offer free trial versions to give a chance to the users to check it out before final investment.

In this run on software testing tools, we have segmented the tools under 4 categories.

  1. Web Application testing tools
  2. Website Security testing tools
  3. Cross Browser testing tools
  4. Mobile Application testing tools

Please note that this list does not suggest any particular order or ranking of the tools. Gallop Solutions is not associated with any of the tool vendors listed in the article. We leverage our partnership with industry leading automation tool vendors to extend added benefits to our clients.

Web application testing tools

We have listed some key tools to execute Performance, Load and Stress Testing on the Web Applications. This ensures the application’s performance under diverse situations and challenging scenarios.

  1. Apache JMeter

apache-jmeter

It is an Open Source load testing tool, scripted in Java 6+ and can operate across platforms. Apache recently released JMeter ‘v2.11’ that supports all platforms and is used for load testing to analyze and gauge performance of the system/application.

Its strong GUI helps build Test Plan quickly and debug processes. It further helps analyze the overall performance under various loads on the server, and the network.

Official Website: Apache JMeter

2. NeoLoad

NeoLoad

This is a load and stress testing tool by Neotys built for Windows, Linux, and Solaris. It is available in English and French, and its latest version 4.2 helps measure, analyze, and improve the performance of the website.

While there are multiple users simultaneously accessing the website, the tool helps check the performance of the website under added load to ensure required user experience. It makes the testing process faster, efficient, and repetitive.

Official Website: NeoLoad

3. LoadRunner

LoadRunner

A load testing tool by HP that is built for Windows and Linux to test the web and other applications effectively. It is available in version 12.0 with multiple languages.

The tool helps evaluate the tool’s performance under heavy load, where there is a probability of multiple users accessing the website at the same time. Additionally, it handles numerous protocols for load testing – .NET Record/Replay, Database, DCOM, GUI Virtual Users, Java Record/Replay, Network, Oracle E-Business, Remote Access.

Official Website: LoadRunner

4. LoadUI

LoadUI

LoadUI latest version is available in version 2.7 that is written in Java, JaveFX, and Groovy language and supports diverse platforms. It mainly supports web services and SopaUI 5.0 (a functional testing tool).

The latest version has corrected issues that were present in the earlier version of LoadUI. The tool is currently available in Pro version with drag-and-drop interface that helps test the load in a real-time scenario.

Official Website: LoadUI

5. Loadster

Loadster

It is a commercial load testing tool implemented to test websites web applications/services (HTTP web services) that emulates the environment to test the application’s performance under heavy load, given by the tool. It supports Linux, Mac, and Windows.

It is a feature-loaded tool that works on web applications/services, cookies, sessions, custom header, etc. It is implemented to test the performance, resilience, stability and scalability of web applications. It simulates multiple users, networks with the site and collects statistics. This helps determine and eliminate performance bottlenecks to ultimately manage traffic and crisis situations.

Official Website: Loadster

6. Load impact

Load Impact

It is an online load testing tool used to test the performance of websites, web and mobile apps, and APIs under load and across various platforms. The tools enables to conduct tests online instead of having an offline platform in place.

While the testing process is fast, the user can also repeat the tests easily and within shorter time span.

Official Website: Load impact

7. Wapt

Wapt

It is a load and stress testing tool that works on Windows and provides an easy and cost-effective way to specifically test business applications websites, mobile websites, web portals, etc. It works on secure HTTPS websites, dynamic content and RIA applications under a data-driven mode.

Official Website: Wapt

  1. Qtest

Qtest

It is a web load testing tool by Quotium Technologies SA, analyzes the application and supports all Windows platforms. It provides easy to use interface and can be opted as an OnDemand hosted solution or an OnPremise application.

Official Website: Qtest

  1. httperf

httperf

Web server performance/benchmarking tool from HP Research Labs. Provides a flexible facility for generating various HTTP workloads and measuring server performance. Focus is not on implementing one particular benchmark but on providing a robust, high-performance, extensible tool. Available free as source code in C.

Official Website: httperf

  1. Telerik Test Studio

Telerik Test Studio

The test automation tool by Telerik tests web and desktop applications across all Windows OS. The tool tests the application’s functionality, performance, and load. Additionally, it offers a plugin for Visual Studio and a solo app to test the cross-browsing issue.

Test Studio supports various languages namely, HTML, AJAX, Silverlight, and ASP.NET and can run test across browsers.

Official Website: Telerik Test Studio

  1. AgileLoad

Agile Load

It is a load testing tool from Agile Load SA to test applications for the Web and Mobile. Its features include automatic recording of test scenarios, distributed load injectors, infrastructure monitoring, and creation of custom test reports for each user profile.

Additionally, it provides a script storyboard that provides graphical view of test scenarios that enables customization. The Load injectors can be installed on your network or on the Cloud.

Official Website: AgileLoad

  1. Tsung

Tsung

Free open-source multi-protocol distributed load testing tool supported by Process-One. Can be used to stress HTTP, WebDAV, SOAP, PostgreSQL, MySQL, LDAP and Jabber/XMPP servers. SSL is also supported. OS monitoring (CPU, memory and network traffic) using SNMP, Munin or Erlang agents on remote servers. XML configuration system; several sessions can be used to simulate different type of users. Dynamic sessions can be described in XML. User think-times and the arrival rate can be randomized using a probability distribution. HTML reports can be generated during the load to view response times measurement, server CPU, etc. Developed in Erlang.

Official Website: Tsung

  1. HttpRider

HttpRider

It is an Open-Source tool that is used for Load, Performance and Stress testing of Web applications. It is developed in .NET by Alexandru Botez. It creates http scenarios by reporting http requests using Fiddler and repeats them on the basis of the user’s configuration.

It can customize a chain of queries and parameters by attaching them to a customized data source with the purpose to emulate non-identical requests.

Official Website: HttpRider

  1. Sandstorm

Sandstorm

It is a load testing tool from Impetus Technologies Inc. that supports protocols for the Web, Mobile and Email. It facilitates test from multiple locations, across browsers and simulates cache with rich extensible framework for modification of test cases by using JavaScript and core Java APIs.

It provides User Interface for handling dynamic parameters and automated cookies. It also integrates resource monitoring for most of the popular Web applications and Database servers.

Official Website: Sandstorm 

  1. Multi-Mechanize

multi mechanize

It is an Open-Source framework by Corey Goldberg, implemented for testing web performance and load testing. It enables execution of simultaneous python scripts to generate load on a website/web application.

The results for which can be reported in a CSV format along with an HTML report containing statistics and graphs. If you have proficiency with Python, HTTP, then it is recommended to use this tool.

Official site: Multi-Mechanize

  1. Load2Test

load2test

It is a Performance/Load/Stress testing tool from Enteros Inc. that can simulate real production workload in the back-end for testing the application’s performance. It is also available in the Amazon EC2, RackSpace or PlatForm Labs cloud environments.

Its integrated performance management and root cause analysis system mechanically collect performance metrics across load test infrastructure and implement cross-component connections to detect performance bottlenecks.

Official site: Load2Test

  1. Xceptance LoadTest

xceptance loadtest

It is a load testing and regression tool by Xceptance Software Technologies Inc. for load testing performance of Web applications or any other applications. It holds recording capabilities with availability of XLT Cloud Service.

It can develop scripts with Script Developer IDE or code all tests in Java using WebDriver.

Official site: Xceptance LoadTest

  1. Blazemeter

blazemeter

It is a self-service, on-demand, cloud-based load testing tool that can simulate varied user scenarios for testing websites, web applications, mobile apps/services. It is Apache JMeter compatible – pre-configured JMeter environments with up to 144 CPU cores and 500 GB of memory. It enables you to set up tests across locations, access test results, evaluate reports, compare and evaluate against earlier reports on a unitary console.

The traffic can be generated using public cloud providers or install an on-premise load generator on your machine and test within the firewall on the internal network. It also provides free tools and resources for tips to optimize website and performance of the applications.

Official site: Blazemeter

  1. LoadStorm

loadstorm

It is a web-based load testing tool by CustomerCentrix, LLC and leverages Amazon Web services to scale up on demand for processing power and bandwidth. The load tests for web and mobile can be built using the tool to simulate multiple users with unique logins and purposes.

Official site: LoadStorm

  1. Neustar Web Performance

neustar-web-performance

It is a self-service, on-demand and pay-as-you-go service by Neustar that enables you to simulate massive volumes of real browsers that hit the website. It uses Selenium, cloud-based real/simulated browsers, AJAX and Flash support.

In its reports it incorporates screen shots of the browsers with errors.

Official site: Neustar Web Performance

  1. Pylot

pylot

It is an Open-source tool by Corey Goldberg that generates simultaneous loads, where you can define test cases in an XML file – specify requests – url, method, body/ payload, etc. It can be verified by matching content to regular expressions with HTTP status codes.

It helps monitor and execute test suites from GUI (wxPython), adjust load, agents, intervals, and test duration. It further displays real-time stats and reports errors.

Official site: Pylot

  1. AppLoader

apploader

It is a load testing app by NRG Global to test websites and other applications that can be accessible from Windows desktop and generates load from the end user’s perspective. It is protocol independent and supports a diverse range of enterprise class applications. It runs from Windows platforms.

Official site: AppLoader

  1. fwptt

fwptt

It is an Open-source tool by Bogdan Damian for load testing web applications and had capabilities to handle AJAX and can generate tests in C#.

Official site: fwptt

  1. JCrawler

jcrawler

It is an Open-source tool for stress testing web applications, comprising crawling and exploratory features. Here the user can provide JCrawler a set of URLs to start with and begin crawling from that point, so while going through any URLs it can generate load on the application.

The Load parameters (hits/sec) can be configured with central XML file and fire up as many number of threads as required to generate the load constantly. Additionally, the tool is platform independent.

Official site: JCrawler

  1. vPerformer

vperformer

It is a Performance and load testing tool from Verisium Inc. that can assess the performance and scalability of the web application. It users recorded scripts or customized scripts by using Javascript.

The target platform is Windows and the VCloudPerformer, which is a free cloud load testing tool can be used to evaluate the performance, scalability, and load stress capacity of web applications by implementing on-demand load servers on Amazon EC2 cloud.

Official site: vPerformer

  1. Curl-Loader

curl loader

It is an Open-source tool scripted in ‘C’ that simulates load and specific behaviour of thousands of HTTP/HTTPS and FTP/FTPS clients, with its own source IP-address. As against other tools, curl-loader uses real C-written client protocol stacks, namely, HTTP and FTP stacks of libcurl and TLS/SSL of OpenSSL.

With this tool you can log activities of each virtual client and the statistics collected comprise, resolution, connection establishment, requests generation, responses generation, errors recorded from the network, TLS/SSL and application (HTTP, FTP) level events and errors.

Official site: Curl-Loader

  1. The Grinder

the grinder

It is a Java-based load testing framework that is available for free under a BSD-style open-source license. It helps synchronize activities within a test script across processes, across machines by using graphical console application.

It comes with an evolved plug-in for testing HTTP services and a tool that allows HTTP scripts for recording automatically.

Official site: The Grinder

  1. Testing Master

testing master

It is a load testing tool by Novosoft and its capabilities include IP spoofing, multiple concurrent test cases and holds various testing features for sites with dynamic content and secures pages with HTTPS.

Official site: Testing Master

  1. Avalanche

avalanche

It is a load testing application from Spirent Communications used to design stress-test security, network, and Web application infrastructures by generating massive user and network traffic. It can emulate about two million synchronously connected users with unique IP addresses, emulates multiple Web browsers, with Web Services testing Supports

Official site: Avalanche

  1. Loader

loader

It is a stress testing tool that runs on WinXP and comes with free evaluation versions for two virtual users. The capture module provides a development environment that utilizes C# scripting and XML based data.

The Control module defines, schedules, and deploys tests, further defining the number of virtual users, etc. The Analysis module evaluates the results and brings reporting capabilities.

Official site: Loader

  1. Visual Studio Test Professional

visual studio test professional

It is a suite of testing tools for testing Web applications and services integrated within the Microsoft Visual Studio environment. The suite includes Lab Management capabilities and enables testers to script, execute, and mange tests within Visual Studio.

Official site: Visual Studio Test Professional

  1. OpenLoad

openload

It is a cost-effective and entirely web-based load testing tool by OpenDemand and does not require scripting languages and its web-based recorder can capture and translate user actions across the website/web application.

Additionally, it can generate up to 1000 simultaneous users with minimum hardware.

Official site: OpenLoad

  1. Siege

siege

It is an Open-source Stress/Regression testing tool that supports basic authentication, cookies, HTTP and HTTPS protocols. It enables testing a web server and can configure multiple simultaneous users.

With this tool you can stress test a single URL with a particular number of users or stress test multiple URLs concurrently. Developed by Jeffery Fulmer, the tool helps report total transactions, elapsed time, bytes transferred, response time, server response and so on.

Official site: Siege

  1. OpenSTA

opensta

The tool ‘Open System Testing Architecture’ is a free, Open-source web load/stress testing application that is licensed under the Gnu GPL. It utilizes a spread out software architecture based on CORBA.

Official site: OpenSTA

  1. PureLoad

pureload

It is a Java-based multi-platform performance testing and analysis tool by PureLoad Software that comprises ‘Comparer’ and ‘Recorder’ capabilities, enables dynamic data input, and edits/debugs the scenario and generates load for single or distributed sources.

Official site: PureLoad

  1. ApacheBench

36 Apachebench

It is a Perl API that is used for benchmarking and regression testing and can be used as a foundation for a complete benchmarking and regression testing suite for transaction-based perl sites. It can be used for stress-testing the server while verifying the correct HTTP responses.

Official site: ApacheBench

  1. Torture

torture

The Bare-bones Perl script by Lincoln Stein is used for testing the speed, dependability, responsiveness and testing stability of a particular Web server. It can send huge amounts of random data to a server and then evaluate the speed and response time.

Official site: Torture

  1. WebSpray

webspray

It is a cost-effective load testing tool from CAI Networks that enables link testing and can simulate up to 1000 clients from a single IP. Further also supports multiple IP addresses with or without aliases. It is built for Windows.

Official site: WebSpray

  1. WebPerformance

Load Testing by Web Performance

It is a load testing tool by Web Performance Inc. that focuses on offering ease to the users and supports all browsers and web servers. It further enables recording and allows the user to view exact bytes flowing between the browser and the server. It does not require scripting.

It can automatically tackle variations in session-specific items namely, cookies, usernames, passwords, IP addresses, and much more.

Official site: Load Testing by Web Performance

  1. Optima Quality Studio

optima quality studio

It is a collection of load testing, capture/playback, and similar tool from Technovations to enable performance testing of the websites. The modules include WebCorder, Load Director, Report Generator, Batch, Manager and many more.

It further enables authentication, SSL, cookies, redirects and records scripts that can be modified manually.

Official site: Optima Quality Studio

  1. QALoad

QAload

It is a tool by Microfocus for load/stress testing web, database, and character-based systems. It supports HTTP, SSL, SOAP, XML, Media streaming that operates across a variety of databases, middleware, and ERP solutions.

Official site: QALoad

  1. IBM Rational Performance Tester

ibm rational performance tester

It is a Performance testing tool by IBM/Rational and offers optional extensions to Seibel applications and SAP solutions. Additionally it supports Windows, Linux, offering high-level detailed views of the tests.

Official site: IBM Rational Performance Tester

  1. Radview’s WebLoad

radviews webload

It is a load testing tool by Radview Software, where test scripting is done via visual tool or Javascript with improved load testing objects and methods. It handles mobile load testing and comes with ability to record native or browser-based apps directly from the mobile handset. Additionally, it can integrate with selenium.

It can further run load generators in the cloud and does not require special licenses. It can generate tailored out-of-the-box reports/graphs with drill-down capabilities.

Official site: Radview’s WebLoad

  1. SilkPerformer

SilkPerformer

It is an enterprise-class load testing tool by Microfocus and can simulate thousands of users working with numerous protocols and across diverse computing environments. It facilitates behaviour predictions for the e-business environment prior to deployment, regardless of the size and complexity.

Official site: SilkPerformer

Web Functional/Regression Testing Tools

Functionality, behaviour, usability and security of websites/web applications are some rising concerns that are bothering enterprises across sectors. Appropriate testing tools help you eliminate bugs within shorter cycles and get your web application functional and market ready.

Most of the bugs occur due to functional issues within the website/web application. Below are some select web functional/regression testing tools for your quick reference.

  1. Selenium

selenium

Selenium is an acclaimed and popular automated testing suite amongst testers these lately. It has been designed to support and boost Test Automation for testing the functional aspects of the web applications across a diverse range of browsers and platforms.

The suite comprises 4 basic components namely, Selenium IDE, Selenium RC, WebDriver, Selenium Grid. The WebDriver directly communicates with the web browser and uses its native compatibility to automate.

Official website: Selenium

  1. TestCafe v14.1.3

TestCafe

It is a web testing framework that tests all web applications across Windows, MacOS, and Linux OS. It is popularly used for web application testing and generates clear and simple tests. It uses browsers that support HTML5 for recording and can run functional tests across Windows, Mac and Linux, and on remote computers.

For mobile devices it can run tests on iPhone, iPad, Android and Windows Phone. Additionally, it can simultaneously execute tests across multiple browsers and machines.

Official Website:  TestCafe v14.1.3

47. CasperJS v1.1-beta3

casperJS

It is an easy to use open source navigation tool for Windows, MacOS, and Linux OS, used for scripting & testing the PhantomJS and SlimerJS (Gecko) with the help of its useful and valuable functions, methods and syntactic sugar, written in Java script language. Basically, it works for forms, links, page screenshot, remote DOM, events sign-in process, etc. Also, downloads binary and other resources, writes functional test suites and save it in a JUnit XML format.

Official Website: CasperJS v1.1-beta3

48. Screenster

screenster

Screenster is an UI based functional and regression testing tool to test web applications and can work across all Windows OS that require all major browsers and Java6 or high. While operating it takes screenshots at every step and compares them to the baseline, enables verification and further provides complete access to Selenium API whenever required.

It is an UI based functional and regression testing tool for web applications, works on all Windows OS that requires all major browsers and Java6 or high. A very good screen validator uses to give 10x productivity without a single line of code. During operation, it takes screenshot on each and every step and compares them to baseline, permits verification of changes or lack of changes to UI, provides full access to Selenium API when needed

Official website: Screenster

49. SoapUI

soapui

It is a cross-platform free open-source functional testing tool written in Java language for service-oriented architectures (SOA) and representational state transfers (REST). It offers user-friendly interface with multiple features used for API testing that facilitates easy and speedy performance of functional/regression and load testing.

Its commercial version (SoapUI Pro) is particularly used to test functionality of the application. It supports all protocols and technologies connected to the particular application.

Official Website: SoapUI

50. Sahi

sahi

An open-source cross-platform web application testing tool that is scripted in Java and JavaScript, and released under an Open-Source Apache License 2.0. It is used to test multiple browser applications under web 2.0.

It identifies elements easily and works on applications that include dynamic ids and applications developed by using ExtJS, ZK, Dojo, YUI, etc.

Official Website: Sahi

51. Web Application Testing in .Net (Watin)

web-application-testing-in-net-watin

It is a cross-platform web application testing tool and its latest version is 2.1 that is developed in C# language. With influence of Watir, Watin was used to test a web application that was written in .Net in December 2005.

Since then, it has now become a user-friendly and stable framework with added functionality and supports all major HTML elements and dialogs. It works across Internet Explorer 6, 7, 8, 9 and FireFox 2 and 3.

Official Website: Watin

52. Web Application Testing in Ruby (Watir)

web-application-testing-in-ruby-watir

It is an open-source (BSD) cross-platform testing tool that is easy to use and offers flexible automation tool with Ruby libraries for web browser automation. Though it is loaded Ruby libraries, it also supports application scripted in any language and enables you to link with databases, export XML files, read files and spreadsheets, and synchronize code as reusable libraries.

It works across Internet Explorer, Firefox, Chrome, Opera and Safari browsers in the same ways.

Official Website: Watir

53. QTP (UFT)

CasperJS

It is a Windows based software testing tool by Hewlett Packard (HP) that is used to test the applications on the web or desktop for functional and regression testing. The UI is called Integrated Development environment (IDE) that comes with numerous functionalities that motivate testers to test more.

QTP uses VB Script language to run the script and supports Web, Java (Core and Advanced), .Net, Flex, Web Services, WPF, Delphi, Power Builder, Stingray 1, Terminal Emulator, SAP, Oracle, Siebel, PeopleSoft, Windows Mobile, VisualAge Smalltalk, Silverlight and mainframe terminal emulators.

The latest version available is UFT 12.0. UFT 12.0 that provides easier and minimum package to download the product. So, the installation gets easier. While QTP works only on Windows, UFT 12.0 supports Safari on a remote Mac, and all other popularly used browsers.

Official Website: QTP (UFT)

54. Ranorex

Ranorex

It is a Windows based GUI testing tool by Ranorex GmbH for testing desktop, web and mobile based applications that can handle testing for any dynamic GUI application.

It tests functionality of the application that involves both the tester and developer. This encourages reliable and distributed testing to detect bugs efficiently. It also supports technologies, namely, .NET, Winforms, WPF, Java, SAP, HTML5, Flash, Flex, Silverlight, iOS, Android, Windows Apps (Native/Hybrid), etc.

It can be used by bigger organizations and small testing teams too.

Official Website: Ranorex

Web Site Security Testing Tools

Financial services and the banking sector has been an all-time victim of security breaches, compromising massive sensitive user data. However, financial services are a must-have for everyone. So here we list some Security testing tools for building a robust application for the marketplace.

55. NTOSpider

ntospider

NTOSpider provides full security for your web applications/services, mobile, and RIA applications. It is a Windows based web security tool and importantly it scans your application completely in less time and ensures security to the system, cost-effectively.

Official website: NTOSpider

56. Brakeman v2.6.1

Brakeman 56-brakeman-v2-6-1

The tool is an Open-source scanner that checks Ruby on Rails apps for security vulnerabilities, available for Jenkins/Hudson and works on Rails 2.x, 3.x, and 4.x. It methodologically scans Rails application code to identify security issues at every stage.

Official Website: Brakeman

57. SiteDigger v3.0

SiteDigger

It examines Google’s cache, errors, configuration issues, proprietary information and major security nuggets on the websites. Additionally, it has a good user interface that provides real-time results with the ability to save signature selection and result set.

You are not required to have Google API License Key to access the tool, as it runs on across all Windows OS required Microsoft .NET Framework v3.5.

Official Website: SiteDigger

58. Netsparker

netsparker

It is a cross-platform web application security scanner that detects and reports website/web application vulnerabilities and issues with the security across any website/web app irrespective of the OS or technology it is built on.

Official Website: Netsparker

59. Network Mapper (NMap)

network Mapper

It is a cross-platform web security scanner scripted by Gordon Lyon founder of hosts and services on a computer network. It delivers required packets to the target machine and scrutinizes the response received.

Previously, it was only used for Linux, but it can be now used for Microsoft Windows, BSD variants – Mac OS X, AmigaOS, Solaris, HP-UX, , andSGI IRIX.

Official Website: NMap

60. Open Web Application Security Project (OWASP)

Open Web Application Security Project

It is a cross-platform online security project/ community that involves global organizations, educational institutions and individuals that focus on security for web applications by creating free articles, techniques, and tools.

Official Website: OWASP

  1. HP Fortify

HP Fortify

It is a security product suite by HP that embeds vulnerability detection, integrates static source code analysis, dynamic runtime analysis, and real-time monitoring to detect and prioritize critical security vulnerabilities.

Its Program Trace Analyzer (PTA) finds vulnerabilities that get visible only when the application is running. It integrates QA tests to find vulnerabilities when an application’s functional tests are going on.

Official site: HP Fortify

  1. HP WebInspect

HP WebInspect

It is an automated security assessment tool by HP for testing web applications and services. It helps identify known and unknown vulnerabilities and incorporates checks to validate effective web server configuration.

It requires Windows and MSIE.

Official site: HP WebInspect

  1. IBM Security AppScan

IBM Security AppScan

It is a tool suit by Rational/IBM that automates security testing for web and mobile applications. It enables defect analysis and recommends techniques to fix detected security loopholes.  Its assessment module can be used by auditors and compliance officers to conduct extensive audits and ensure compliance with security standards as well.

It enables static code analysis, offers support for JavaScript, HTML5, Cordova, Java and Objective-C.

Official site: IBM Security AppScan

Cross Browser Testing Tools

Considering seamless user experience is key for every brand and every enterprise, getting the right tool in place is indispensable for every tester. These are some tools for your reference.

64. Browsera

Browsera

It is an online cross-browser testing tool that operates on Windows and Mac OS X platform and detects cross-browser layout issues by comparing the output from each browser. When every test is completed, every JavaScript error is reported and collated. The entire website can be tested by using the crawling feature of the tool.

Official Website: Browsera

65. IE NetRenderer

IE NetRenderer

It is a free HTML tool for Apple iMac and Linux facilitates users to view the website displays in IE browsers 5.5 to 11. Besides, it also presents an extensive number of tasks in real time. These services can be executed at a fast paced.

Besides, the tool also helps capture various other tasks on the website across browsers and in real time.

Official Website: IE NetRenderer

66. IETester v0.5.2

IETester v0.5.2

It is a Windows web browser by Core Services that provides a user-friendly interface, enabling the users to get all functionalities rapidly and easily. It provides a configuration without customizations with focus on speed and efficiency.

With the Microsoft ribbon toolbar, the users can easily switch across Internet Explorer versions (5.5 to 9) with a single click. Apart from this it also facilitates compatibility tests.

Official Website: IETester

67. W3C Link Checker

W3C Link Checker

It is a link checking service provider by W3C that is used to links and anchors existing on a website page or across the entire website by using languages; HTML, XHTML or CSS.

Official website: W3C Link Checker

Mobile Web/App Testing Tools

68. Selendroid

selendroid

It is an Open-source test automation framework to test the UI of Android native, Hybrid apps and the mobile. The tests are scripted by using the Selenium 2 client API.

Additionally, it can be used on emulators and actual devices. It can also be integrated as node into the Selenium Grid to scale up and test simultaneously. Various gestures are supported by the Advanced User Interactions API. It can further simplify test case development.

Official site: Selendroid

  1. ios-driver

ios driver

It is an Open-source tool used to test iOS native, Hybrid, or web apps for the mobile by using the Selenium/Webdriver API. It can also be used as a Selenium grid node that runs tests synchronously within the same architecture as for the web.

Official site: ios-driver

  1. KIF (Keep It Functional)

Keep It Functional

It is an Open-source iOS integration test framework that automates by leveraging the iOS accessibility attributes. It builds and executes the tests by implementing a standard XCTest testing target.

In line with other iOS test tools, it uses undocumented Apple APIs, which is appropriate for testing and integrates with the user’s Xcode project. The tests for KIF are scripted in Objective C.

Official site: KIF

  1. Mobile-Friendly Test

Mobile Friendly Test

It is a free online test tool developed by Google that analyzes a URL and informs if the page is mobile-friendly/responsive. Additionally, it reports/defects detected and recommends solutions.

It also provides an extensive mobile responsive documentation on the site.

Official site: Mobile-Friendly Test

  1. Remote Test kit

Remote Test kit

The tool developed by NTT Resonant Inc. facilitates a real device test cloud that enables manual and automated testing across multiple real iOS and Android smartphones and tablets. It integrates with Xcode for iOS and Eclipse for Android.

Further to each test, the test data from all apps is erased the devices and are automatically re-initialized. Additionally, the tool supports Selenium Webdriver, Jenkins. Real-time Clipboard Synchronization facilitates copy pasting text that has been developed in advance on the user’s computer, or copying text from the iOS device to a local system.

Official site: Remote TestKit

  1. TestObject

TestObject

It is a device test cloud created by TestObject GmbH that has over 150 devices, mainly Android and some iOS. It provides access to real devices from the user’s browser for manual and automated testing.

It also supports Native, web and hybrid apps and offers additional facilities like screenshots, switching over system language to test, supporting testing frameworks that includes Robotium, Espresso and Appium; a REST API, Ant and Gradle plugins.

Official site: TestObject

  1. Frank

Frank

It is an Open-source framework developed by Thoughtworks that is used to write structured text tests by using Cucumber and execute them against the iOS application.

It necessitates installation of a Ruby and compiles/links the Frank servicer into the application that is being tested. It has a robust app inspector that can get detailed information from the application on the run. It executes tests on both the Simulator and Device.

Official site: Frank

  1. Android Testing Support Library

android-testing-support-library

It is a framework built by Google to test Android apps. It provides a set of Java APIs through the Android SDK Manager that helps build and run app test code, comprising JUnit 4 and functional UI tests.

Tests can executed from the Android Studio IDE or from the command line. It also comprises AndroidJUnitRunner: JUnit 4-compatible test runner for Android; Espresso: UI testing framework for white box functional UI testing within an app, and UI Automator: UI testing framework for black box functional UI testing across the devices or applications under test.

Official site: Android Testing Support Library

  1. Monkey

android-testing-support-library

It is a free command-line tool/program that can run on your emulator or device to generate pseudo-random series of user interactions/transactions, namely, clicks, touches, and system related gestures. Monkey can stress-test applications that are being developed randomly but repeatedly.

For instance, if the application crashes or doesn’t perform as expected/respond to system error, the tool will report the error occurred.

Official site: Monkey

  1. MonkeyRunner

android-testing-support-library

It is a free tool from Google that offers a Python API for scripting programs, controlling an Android device or an emulator not within the Android code. It can write a Python program that can install an Android application or test package, take screenshots of user interface, and more.

It is meant to test applications as well as devices for its functional and framework aspects. Importantly, you can apply one or more test suites across multiple devices or emulators, and it can be extended with Plug-ins.

Official site: MonkeyRunner

  1. Android Lint

android-testing-support-library

Lint, a tool from the Android Studio is a code scanning tool that can enable the user to easily detect and rectify issues with the structural quality of the code. This is done without running the application or scripting any tests.

It has a command-line interface that can easily integrate the tool within the user’s automated testing process. The tool checks the Android project source files to detect bugs and bring in enhancements for efficiency, usability, performance, and security.

Official site: Android Lint

  1. Calabash

calabash

It is a free open source framework from Xamarin Inc. that enables scripting and running automated Acceptance tests for mobile applications by using Cucumber and Ruby. It is a cross-platform tool that supports Android and iOS native apps, comprising libraries that facilitate test code to interact with Native and Hybrid apps.

Official site: Calabash

  1. Xamarin Test Cloud

xamarin-test-cloud

It provides a locally created, robust object-based scripting environment for simulating and automating actions that an actual user could take via a mobile app on iOS or Android device. Importantly, it provides a test device cloud, offering 1,800+ real devices for testing.

Here, the test scripts run concurrently across multiple devices at one time. Test Scripts can be created in Calabash (Ruby) and C# using Xamarin Studio or Visual Studio. The tool can integrate with any CI systems.

Official site: Xamarin Test Cloud

  1. TestMunk

testmunk

It is a mobile app test automation runner that is available on the web and enables execution of BDD-style test cases across virtually available emulators, user devices, or across a pool of TestMunk Android and iOS versions of devices. It supports Native and Hybrid apps.

It provides over 50 BDD-style TestSteps for both iOS and Android and can extend and script customized TestSteps and uses Ruby.  It can initiate and run tests locally on an emulator or a device at no cost.

Official site: TestMunk

  1. Crashlytics

crashlytics

It is a free set of tools developed by Twitter and can be implemented within iOS and Android apps. It enables the user to report detailed information on an application’s crash. It includes details about the devices that the application runs on, like whether it is happening only on the device or on a specific model in the series.

It provides ‘Answers kit’ that gives information on critical performance metrics of an app, growth and engagement indicators.

Official site: Crashlytics

  1. UberTesters

uber testers

It is a free set of tools for managing and reporting iOS/Android app and crowdsourced/beta testing. It needs the Ubertesters SDK, which integrates with multiple frameworks optimized for cross-platform development.

Additionally, it can integrate with many third party bug tracking systems or use the incorporated bug tracker. It capabilities also include in-app bug editing, marking, reporting and user feedback. It tracks all testing sessions in real-time sorted by devices or testers.

Official site: UberTesters

  1. TestFairy

testFairy

It is a set of tools developed by TestFairy to manage and report iOS/Android app crowdsourced/beta testing. It provides video that shows everything that takes place on the mobile device, comprising detailed internal metrics such as CPU, memory, GPS, logs, and crash reports. It also detects ANRs and alerts when the app shows abnormal results.

Official site: TestFairy

  1. HockeyApp

hockeyapp

It is a set of tools created by Microsoft to manage and report iOS/Android/WinPhone/OSX app crowdsourced/beta testing. It integrates the open source HockeyApp SDK for Android, iOS, Mac OS X, Win, Unity, Xamarin and then uploads the app’s build to HockeyApp.

It reports metrics presenting which devices were tested, by whom, the app was used for how long, the language tested, and more.

Official site: HockeyApp

  1. TestDroid

86-testdroid

It is a suite of Android application testing tools that can be implemented both for test automation and manual testing. It has been developed by Bitbar Inc. Testdroid Enterprise.

Testdroid Cloud is an online service that provides thousands of different real Android and iOS devices to run tests. The test runs begin with a device reboot, ensuring that the tests are executed on clean devices with no interference.

Official site: TestDroid

  1. eggPlant Mobile

eggplant-mobile

It is a test automation tool for the mobile that is image-based, low-overhead, for black-box testing. It is the combination of eggPlant Functional and eggOn, the mobile agent that enables to fully test mobile apps on iOS, Android, Windows Phone, BlackBerry, and any other mobiles devices.

Official site: eggPlant Mobile

  1. Maveryx

maveryx

It is a Test automation tool developed by Maveryx for Android and Java apps. It utilizes ‘GUI Objects Finder’ that recognizes GUI test objects with text while execution of tests. The object search engine supports exactly matching algorithms to identify test objects in the UI, even when there are partial or approximate matching, misspellings and synonyms or in case of change of objects from the time of test creation.

It supports JUnit assertion-based verification points, distributed testing, plugin for Eclipse, and even when customized extensions/plugins are created.

Official site: Maveryx

  1. Mobitaz

mobitaz

It is an Android/iOS test automation tool developed by MSys Technologies. Its features comprise Native app and Web test automation, end-to-end automation, etc. The Test cases can be easily created, without any requirement of source code.

Official site: Mobitaz

  1. Clang Static Analyzer

clang static analyzer

It is an Open-source source code analysis tool that detects bugs in C, C++, and Objective-C programs. It can be used either as a standalone tool or within Xcode. It is expected to run in tandem with a codebase Build.

It is implemented as a C++ library that can be leveraged by other tools and applications.

Official site: Clang Static Analyzer

  1. Roboelectric

roboelectric

It is an open source unit test framework that customizes Android SDK classes for the user to test Android apps within the JVM at the workstation within seconds.

Official site: Roboelectric

  1. TestArchitect Mobile Plus

TestArchitect Mobile Plus

It is a Test automation tool from LogiGear for native/hybrid apps on Android/iOS, and Web, Windows and Java applications. With this the tests can be executed across multiple local devices via USB and Wi-Fi, or on devices hosted by cloud-based mobile testing partners. It provides support for multi-touch gestures, access to the physical device buttons, and command-line execution. Image recognition feature enables testing for standard apps and games with fast, 3D, interactive graphics.

Additionally, it supports IE, Firefox, Chrome, Safari, WebView, HTML5, and customized actions can be created using C#, C++, JAVA, or Python.

Official site: TestArchitect Mobile Plus

  1. Appium

appium

It is an open source test automation framework that can drive native and hybrid iOS and Android mobile apps using the WebDriver JSON wire protocol. It supports a sub-set of the Selenium WebDriver JSON Wire Protocol, and provides extensions for automating mobile gestures like tap, flick, and swipe.

In order to automate iOS, it requires XCode and Apple Developer Tools (iPhone simulator SDK, command line tools). For Android automation Android SDK API >= 17 is required.

Official site: Appium

  1. Android/iOS Source Code Analysis

Android/iOS Source Code Analysis

It is a static code scanning tool developed by Checkmarx that provides the ability to detect vulnerable lines of code and enables to fix them, for Android and iOS app source code. Additionally, it is also capable of analyzing other code languages.

Official site: Android/iOS Source Code Analysis

  1. Soasta TouchTest

soasta touchtest

It is a functional test automation tool from Soasta that brings competencies for multi-touch, gesture-based mobile apps. It can validate tests by leveraging internal variable values and conditionally wait for internal application state changes.

It also facilitates testing for any mobile device across any location. The devices can be available in lab, used as part of an external test, or crowd-sourced.

Official site: Soasta TouchTest

  1. Adobe Edge Inspect

Adobe Edge Inspect

Inspect provides the user a preview of the content across multiple mobile devices. It wirelessly pairs multiple iOS and Android devices to the computer, captures screenshots from any connected device, and helps the user observe real-time results from changes to HTML, CSS, and JavaScript. It can be used for Android, iOS, Kindle Fire.

Official site: Adobe Edge Inspect 

  1. SeeTestMobile

SeeTestMobile

It is a mobile application test tool developed by Experitest Ltd. It is built for iOS, Android, Blackberry and WindowsPhone. It is used for both emulators as well as real devices and covers visual testing, functionality testing and speed performance.

The test recordings can be done by using real devices, where real device can be plugged into the desktop via USB. It leverages self-learning diagnostic and matches algorithms and a modular self-enhancing image recognition technology.

Some other mobile-related tools available include SeeTest Cloud, SeeTest Network Virtualization, Mobile Addon for UFT.

Official site: SeeTestMobile

  1. Ranorex Cross-Device Mobile App Testing

Ranorex Cross-Device Mobile App Testing

It is a Windows GUI test automation framework developed by Ranorex GmbH to test diverse application types comprising mobile (iOS, Android, Windows8 UI) test automation, and mobile web test automation (iOS). It can directly record tests on the device.

The IDE comprises test project management, integration of all Ranorex tools (Recorder, Repository, Spy), intuitive code editor, code completion, debugging, and watch monitor.

Official site: Ranorex Cross-Device Mobile App Testing

  1. Robotium

Robotium

It is an open-source Android test automation framework developed by Renas Reda/Hugo Josefson/Jayway and is available as Java source or jar file. It can integrate with Maven or Ant to execute tests for continuous integration. It also supports Android 1.6 and higher versions.

Official site: Robotium

  1. MonkeyTalk

MonkeyTalk

It is a free open-source test automation tool developed by Gorilla Logic for functional test automation for native, mobile, and hybrid iOS and Android apps. It can be used for simulators as well as actual devices. It is a cross platform tool that can run tests interactively or automate completely.

It has a Cloud version that provides on-premises solution for mobile app testing across devices, where the authorized users can submit test jobs and run the tests automatically on the target devices.

Official site: MonkeyTalk

In Conclusion

Agile, DevOps, and many modern day software development approaches/concepts are endorsing the relevance of testing in the overall SDLC process. Software testing tools are catalysts and will determine the pace at which Testing takes on the new-age challenges posed by the marketplace.

The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.

37 Epic Software Failures that Mandate the Need for Adequate Software Testing

37 Epic Software Failures

Disaster is an understatement for any brand/organization/institution that has incurred losses due to an overtly miniscule but catastrophic software glitch. While technology and innovative applications have been empowering brands, there have been numerous disabling instances recorded by enterprises.

In this run on top software failures of 2016 -2015-2014, we take a stock of the debacles/glitches that have changed the face of software development and endorsed the role of testing in the overall SDLC process.

This is a list of software glitches/technical issues witnessed by brands and enterprises across diverse industries. Please note that the numbers 1-37 do not signify in anyway high or low impact of the software glitch on the brand/enterprise.

  1. Yahoo reports breach

Yahoo reports breach

Amongst the most recent data breaches, on September 22, 2016, Yahoo confirmed a data breach that exposed about 500 million credentials that date back to four years. It is being considered amongst the largest credential leaks of 2016. The company believes that this was a state-sponsored breach, where an individual on behalf of a government executed the entire hack. It further urged users to change their passwords and security questions. As a relief for the users, Yahoo stated that sensitive financial data like bank accounts and passwords was not stolen as part of the breach.

Source: Money.cnn.com

  1. Nest thermostat freeze

Nest thermostat freeze

Software update for the Nest ‘smart’ thermostat (owned by Google) went wrong and literally left users in the cold. When the software update went wrong, it forced the device’s batteries to drain out, which led to drop in the temperature. Consequently, the customers were unable to heat their homes or use any amenities.

Nest claimed that the fault was due to a December 4.0 firmware update, with related issues such as old air filters or incompatible boilers. Later it released a 4.0.1 software update that solved the issue for 99.5% of customers who were affected.

Source: Cio-asia.com

  1. HSBC’s major IT outage

HSBC’s major IT outage

In January 2016, HSBC suffered a major IT outage, and millions of bank customers were unable to access online accounts. The bank took almost 2 days to recover and get back to normal functioning.

HSBC’s Chief Operating Officer (COO) declared that it was a result of a ‘complex technical issue’ within the internal systems.

Source: Cio-asia.com

  1. Prison Break

Prison Break

A glitch that occurred in December 2015 led to over 3,200 US prisoners being released before their declared date. The software was designed to monitor the behaviour of prisoners and was introduced in 2002. The problem was occurring for about 13 years and on an average prisoners were released almost 49 days in advance.

Source: Cio-asia.com

  1. HSBC payments glitch

HSBC

In August 2015, HSBC failed to process about 275,000 individual payments that left many people without pay before a long Bank Holiday weekend. This occurred due to a major failure with the bank’s electronic payment system for its business banking users, affecting the individual payments. Bacs, a payment system used for payment processes across the UK, later picked up on this issue, labelling it as an ‘isolated issue’.

Source: Cio-asia.com

  1. Bloomberg cancels debt issue

Bloomberg cancels debt issue

In April 2016, Bloomberg’s London office faced a software glitch, where its trading terminals went down for two hours. This came up at an unfortunate time when UK’s Debt Management Office (DMO) was about to auction a series of short-term Treasury bills. Later in a statement Bloomberg declared that the services were restored and the glitch was a result of both hardware and software failures in the network, resulting in excessive network traffic.

Source: Cio-asia.com

  1. RBS payments failure

RBS payments failure

About 6 lakh payments failed to get through the accounts of RBS overnight in June 2015, which included wages and benefits. Bank’s chief admin officer stated it as a technology fault and there was no further detail on the real cause. In 2012, about 6.5 million RBS customers had to face an outage caused due to a batch scheduling software glitch, where the bank was fined £56 million.

Source: Cio-asia.com

  1. Airbus software bug alert

Airbus software bug alert

In May 2015, Airbus issued an alert for urgently checking its A400M aircraft when a report detected a software bug that had caused a fatal crash earlier in Spain. Prior to this alert, a test flight in Seville has caused the death of four air force crew members and two were left injured.

Source: Theguardian.com

  1. UK government’s new online farming payments system gets delayed

UK government’s new online farming payments

In March 2015, the UK government delayed the launch of £154 million rural payments system.  The system is an online service for farmers to apply for Common Agricultural Policy payments from the EU. This online service that was supposed to be up and running by May 2015 got delayed due to integration issues between the portal and the rules engine software. It was then not expected to be up even by 2016.

Source: Computerworlduk.com

  1. Co-op Food’s double charges

Co-perative Food double charges

In July 2015, Co-operative Food apologized to its customers and promised a refund within 24 hours. The reason was a ‘one-off technical glitch’ while processing the software that resulted in customers being charged twice.

Source: Computerworlduk.com

  1. John Lewis

John Lewis

Mispricing is a common headache faced by retailers due to system glitches, resulting in retail outlets offering customers excessively lucrative offers. John Lewis is a recent example, where the online retailer witnessed a price glitch on its website that erroneously advertised hardware at software rates.

Source: Money.aol.co.uk

  1. Tesco iPad pricing disaster

Tesco iPad pricing disaster

In March 2012, Apple iPads worth £650 got priced at £49.99. After the glitch got identified, Tesco cancelled the sale and did not respond to these orders, resulting in dissatisfaction with the customers.

Source: Mycustomer.com

  1. Marks & Spencer 3D TV glitch

Marks & Spencer 3D TV glitch

In January 2012, 50 inch, 3D TVs worth £1,099 went up on sale for a mere £199 on the Marks and Spencer website. Eventually, the company decided to sell the Plasma TV sets at a lowered price after it faced a customer petition. The online petition called ‘Marks & Spencer supply our tvs that we paid for’ compelled M&S to honour the orders.

Source: Thisismoney.co.uk

  1. Reebok’s free trainers

Reebok’s free trainers

In November 2013, Sports retailer Reebok trainers worth £100 were getting picked up for free from the online site, where the customers were being charged only for delivery. While the company did not honour the orders and apologised to the customer, they refunded the delivery charges and additionally gave 20% off on their next order. The pricing glitch went viral on Facebook and other sport and price deal forums, where shoppers rushed to get a grab of £99.95 CrossFit Nano Speed footwear for just £8.50 postage.

Source: Theguardian.com

  1. Tennessee County kills System Update worth $1Million

Tennessee County

After investing two years of labour and investment worth $1 Million, Rutherford Country of Tennessee, US called off a court software system update. The core reason was that the software glitches were identified right when the deal took place, where problems related to issuance of checks, errors on circuit court dockets and creation of hidden charges came up in the weeks after it went Live.

Source: 99tests.com

  1. Software Security Flaws Revealed in OLA’s Mobile App

Software Security Flaws - OLA

Ola, India’s largest taxi aggregator faced major security flaws within their system. The software bugs detected helped basic programmers to enjoy unlimited free rides – at the expense of Ola and at the expense of users. The issue went public when customers brought up the weaknesses in the system. Ola tried to fix bugs when the complaints soared up and it was alarming for the brand’s reputation in the marketplace.

Source: Economictimes.indiatimes.com

  1. Leeds Pathology IT crash

Leeds Pathology IT crash

In September 2016, Leeds Teaching Hospitals NHS Trust, one of Europe’s largest teaching trusts witnessed a pathology IT crash that resulted in a delay of operations for almost 132 patients. Leeds Teaching holds a budget of a £1 billion and employs over 16,000 staff. It serves 780,000 people in the city and provides expert care for 5.4 million patients. The outage further affected Bradford Teaching Hospitals NHS Foundation Trust, GP services in Leeds and a minor number of GP services in Bradford.

Now that’s the impact!

Source: Digitalhealth.net

  1. Cisco’s Email Security Appliances glitch

cisco’s Email Security Appliances

In September 2016, Cisco Systems released a critical security bulletin to announce an IT exposure that could allow remote unauthenticated users to get access to its email security appliances. The vulnerability is associated with Cisco’s IronPort AsyncOS operating system. The company further indicated that there is a way out of this that can stop this remote access to the email appliances.

Source: Threatpost.com

  1. Cisco Nexus Switches warning

Cisco Nexus Switches

Cisco again! In October 2016, Cisco Systems released several critical software patches for its Nexus 7000-series switches and its NX-OS software. Cisco’s Security Advisory declared that both the Nexus 7000 and 7700 series switches were vulnerable to this glitch. The vulnerabilities declared allowed remote access to systems that could enable a hacker to execute code on targeted devices. Cisco further declared that this bug (CVE-2016-1453) is a result of “incomplete input validation performed on the size of overlay transport virtualization packet header parameters”.
Source: Threatpost.com

  1. Cyber Attack on Nuclear Power Plant

Cyber Attack on Nuclear Power Plant

In October 2016, the head of an international nuclear energy consortium declared that disruption at a nuclear power plant during the last several years was caused due to a ‘Cyber Attack’. Yukiya Amano, head of the International Atomic Energy Agency (IAEA) didn’t drill the matter much in detail, but did alter on the potential attacks in the future.

This shows that disruption in nuclear infrastructure due to a Cyber Attack is not a ‘Hollywood stint’!

Source: Threatpost.com

  1. Volkswagen’s ‘Dieselgate’ scandal

Volkswagen’s ‘Dieselgate’ scandal

In September 2015, the US government in a dramatic move ordered Volkswagen to recall about 500,000 cars after learning that the company had deployed advanced software to cheat emission tests and allowed its cars to produce 40 times more emissions than the decided limit. The Environment Protection Agency (EPA) accused VW for installing illegal ‘defeat device’ software that substantially reduces Nitrogen oxide (NOx) emissions only while undergoing emission test. The company further admitted it and announced a recall as well.

Source: Theguardian.com

  1. Interlogix Recalls Personal Panic Devices

Interlogix

In October 2016, Interlogix, a wireless personal panic devices manufacturer recalled about 67, 000 devices due to its inability to operate during emergency situations. The probable cause for this glitch in operations was that the device was unable to communicate with the security system during an event of emergency. The way out was the manufacturer replacing the devices. Furthermore, the consumers could contact their professional security system installer and call for a free monitoring and if required free replacement.

Source: News.sys-con.com

  1. IRS E-File goes Offline

IRS E-File

In February 2016, the Federal Agency suffered from a hardware failure. IRS announced that the hardware failure has affected numerous tax processing systems that went out of service, including the modernized e-file system and another related system. Majority of the folks trying to file taxes online could not complete the process. Later IRS made amendments and worked to restore regular operations to get back to the routine.

Source: Newyork.cbslocal.com

  1. 911 call outage

911 call outage

In April 2015, Emergency services got stalled for six hours for seven US states. This affected 81 call centers, literally speaking about 6,000 people made 911 calls and were unable to connect across the seven states. The nationwide outage was the third major outage in three years across telecom operators of the 911 call system. This raised worries amongst federal regulators pertaining to the vulnerability of the country’s emergency response system.

Source: Wsj.com

  1. New York Stock Exchange halts trading

New York Stock Exchange

In July 2015, The New York Stock Exchange stopped trading due to an undisclosed ‘internal technical issue’, where all open orders were cancelled and the traders were alerted and informed that they would receive information later. While responding to the shut down, NYSE announced that there was no cyber breach within the system and it resumed operations after 4 hours.

Source: Money.cnn.com

  1. UK government’s online calculator glitch

UK government’s online calculator glitch

In December 2015, UK government found out that its online calculator for estimating the spouse’s financial worth got hit with a Form E fault, where calculations went wrong for thousands of couples who had got divorced over the past 20 months. Though the issue was prevalent since April 2014, it got noticed only in December 2015. The damage caused is yet to be estimated.

Source: Cio-asia.com

 

Let’s take a dip into some of the interesting software debacles of 2014

27. Nissan’s recall

Nissan's recall

For over 2 years Nissan recalled over a million cars, thanks to a software glitch in the airbag sensory detectors. Practically, the affected cars were unable to assess whether an adult was seated in the car’s passenger seat and consequently would not inflate the airbags in case of a crisis.

Source: Computerworlduk.com

28. Amazon 1p price glitch

Amazon 1p price glitch

One of the most known glitches in history, Amazon 1p price glitch, where third-party sellers listed on Amazon saw their products being priced at 1p each. While the products got delivered, numerous small time retailers had to appeal to the customers for returning the items.

Source: Computerworlduk.com

29. Screwfix.com glitch

Amazon 1p price glitch

In January 2014, every item in the Screwfix catalogue got priced at £34.99 that included items costing almost £1,599.99. Smart customers quickly collected goods worth thousands after the news was spreading across twitter. Eventually, the website had to close down.

Source: Telegraph.co.uk

30. Flipkart apologizes for Big Billion Day sale fiasco

Flipkart

In October 2014, Filpkart, India based e-commerce giant, sent a note to its customers apologizing for the glitches that took place on the Big Billion Day Sale. The site encountered a heavy rush, which it couldn’t manage, which resulted in cancellation of orders, delayed delivery, and much more that was beyond them to manage. While the sale helped the ecommerce giant garner a billion hits in a day, it was certainly a PR nightmare for the brand.

Source: Livemint.com

31. CA Technologies paid RBS ‘millions’ for role in IT fiasco

CA Technologies

In October 2014, CA Technologies paid ‘millions of pounds’ to the Royal Bank of Scotland. This payment was a part of the settlement agreement with Royal Bank of Scotland’s (RBS) IT outage in 2012. In 2012, a failed upgrade to CA7 batch processing software by RBS IT staff resulted in breakdown of systems that affected millions of customers. The customers were unable to access their accounts or execute any payments.

Source: Computerweekly.com

32. Chaos at UK airports

Chaos

On December 12, 2014, UK’s busiest airports got stranded due to a system glitch at the main national air traffic control center in Swanwick. Planes were grounded and passengers got delayed. The impact was enormous as the runways got closed at Heathrow, which is one of Europe’s busiest airports. The transport secretary called this ‘unacceptable’.

Source: Theguardian.com

33. Toyota Prius recalled over software glitch

Toyota Prius

In February 2014, Toyota Motor recalled 1.9 million newest-generation Prius vehicles worldwide due to a programming error that caused the car’s gas-electric hybrid systems to shut down. The Automaker mentioned that the problems were with the software settings on the latest Prius generation that initially went for sale in 2009 and could damage transistor in the hybrid systems. The identified problem could turn on the warning lights and trigger the vehicle to shut down the power on a fail-safe mode.

Source: Nytimes.com

34. Heartbleed the Web

Heartbleed

In April 2014, the IT gang woke up to its worst nightmare, an emergency security advisory from the OpenSSL project warned about an open bug ‘Heartbleed’. The bug could pull out a chunk of working memory from a server and run their current software. While there was an emergency patch for it, tens of millions of servers got exposed by the time the patch got installed. This left everyone and anyone running a server in a crisis mode. This notorious bug left biggies like Yahoo, Imgur, and numerous others exposed to Heartbleed.

Source: Theverge.com

35. Apple pulls iOS 8 update

Apple pulls iOS 8

In September 2014, Apple faced an embarrassment after it had to pull out its new iOS software update only after a few hours of its release. This was post complains from iPhone users about calls getting blocked post the upgrade. The tech giant pulled out the update after a storm of complaints on Twitter, Apple user chatrooms. The update further disabled the feature where people could unlock their phones with fingerprints.

Source: Mirror.co.uk

36. iCloud hack

iCloud hack

On August 2014, almost 500 private pictures of celebrities got posted on social channels and sites like Imgur and Reddit. The images were apparently sourced through a breach of Apple’s Cloud services suite iCloud. However, later it was found that it could be due to a security issue in the iCloud API that enabled the access and innumerable attempts from try passwords. However, there have been recent reports of similar hacks into iCloud.

Source: Dailymail.co.uk

37. Air India diverts Boeing 787 flight

Air India diverts Boeing 787 flight

During an emergency stunt in Feb 2014, Air India diverted Boeing 787 plane to Kuala Lumpur when the pilots noticed a software glitch while on a flight from Melbourne to New Delhi. The Engineers were flown down from Hong Kong to fix the glitch and worked with Air India to resolve the same. It has been reported that 787 has been suffering such glitches and Boeing was aware about it.

Source: Reuters.com

Gallop Solutions has collaborated with world’s leading and innovative organizations/brands across diverse industries. Enterprises globally have trusted Gallop’s independent software testing services and expertise for over a decade and have achieved speed to market, higher returns on investments (ROI), and enhanced quality deliveries in their overall QA initiatives. Connect with our experts to bring speed and velocity to your QA practices with the best ideas in the testing space.

Application and Software failures dilute the brand’s credibility that is built over the years. Together, let’s work towards further strengthening your brand’s positioning, integrity and faith by ensuring Quality @ speed.
The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.

10 Emerging Trends in Software Testing: Predictions for the next decade

10 Emerging Trends in Software Testing

The last decade has seen an overwhelming evolution of the software testing industry giving way to greener pastures. This rapid scale of development is keeping not just the developers, but also the testers on tenterhooks, making them to continuously strive to upgrade their skills. Even businesses today need to be even more aware of what is best in terms of performance and security. This disruption has been caused by the new technologies, and it is always challenging for testers to overcome the new issues posed by these upcoming technologies.

2015 saw the acceptance of testing as an early activity in the software development lifecycle. This was predominantly due to the widespread adoption of Agile and DevOps methodologies by organizations across the globe. The goal was to get their apps faster to the market. 2015 also saw an increase in the use of virtualization and service oriented architecture along with cloud computing that led to many testing tool vendors vying for the market space in the testing arena.

This post summarizes the Top 10 emerging trends/predictions for the next decade that may change the landscape of software testing. This is based on our observations and experience with leading Fortune 500 enterprises and industry analyst research reports. It is interesting to discover each of these trends and to know how enterprises as well as testing professionals can get leverage these trends, re-strategize or re-skill themselves.

  1. The Future belongs to Open Source Tools: The next decade (may be more!) will see a lot of Open source tools in action as more and more organizations will adopt them for proper implementation of Agile, DevOps, and Test Automation. Support communities for the open source tools can only become more and more involved and active.
  1. Quality @ High speed is the new mantra: Everyone wants the best products in the fastest possible time. This is making organizations focus on providing the best user experience along with the fastest time to market. The speed is only going to increase (and the quality better) with the latest technologies and tools at the disposal of teams.
  1. Software Development Engineers in Test (SDETs) will be in huge demand: SDETs have been existing among us since almost a decade, but their role was very different from traditional testing roles. That said, by early 2020, almost all testers will need to wear an SDET hat to be successful in the field of Test Automation, that is going to become mainstream.
  1. Agile and DevOps will rule the roost – TCoE is dead: According to Forrester, organizations are not looking at having centralized Test Centers of Excellence anymore. Test automation developers today are now a part of the agile teams. The erstwhile testing arena is making a shift towards quality engineering, and testing is intended to become more iterative, progressive, and seamlessly integrated with development.
  1. Digital Transformation is here to stay: With a majority of organizations making a foray in the digital world, the need for digital transformation will require a huge shift of focus towards digital testing. Robust strategies for digital assurance will be required for focusing on optimizing functional testing across channels.
  1. BigData Testing will become really BIG: We are sitting atop an explosive amount of BigData today and need to have a very strong strategy around BigData Testing. Testing datasets requires highly analytical tools, techniques, and frameworks, and is an area that is set to grow big.
  1. IoT: Heralding an era of Connected Devices: With IoT growing in leaps and bounds, more and more customers rely on IoT Testing before using the products. If the products are not tested, their functionality, security, and effectiveness – all will come under scanner. According to a HP study, 70 percent of devices in the Internet of Things are vulnerable to security problems.
  1. DevOps will drive Quality Engineering: DevOps ideology is based on seamless collaboration and integration between the different departments of an IT Organization – developers, quality professionals, and IT professionals. Testing plays a business-critical role as developers are involved not just in the correctness of their code, but also in the testing and overall Quality engineering aspects. DevOps thus is propelling businesses towards greater speeds of deployment and quality assurance and is thus helping them realize higher returns on investment and faster time to market in a cost-efficient manner.
  1. Performance Engineering is replacing Performance Testing: Repeating a cliché – “A good user experience is the key to a successful product”. Consistent performance across diverse platforms, OSs, and devices defines how much of a market can a product really capture. The need to provide the best experience to users is making organizations change their strategy. They are now moving away from just providing Performance tests to providing Performance engineering.
  1. The best news is that Software Testing Budgets will continue to grow: It is but obvious that with such huge focus and demand for high quality products, and with major IT trends such as BigData analytics, Cloud Technologies, Mobility, and Virtualization, Testing has become more than just a need. This will push the organizations towards allocating a bigger chunk of their IT budget (around 40%) for software testing and QA.

About Kalyana Rao Konda

Kalyan is the President & Global Head of Gallop Solutions Inc. With 17+ years of experience in IT Services, specifically software testing, Kalyan has led large QA teams of 2000+ people at AppLabs as VP-Delivery and had been in QA leadership roles with Virtusa&BaaN earlier. He has a rare mix of high technical understanding with a pragmatic approach to testing services delivery. A strong proponent of Testing-as-a-Service (TaaS) delivery model, Kalyan is a thought leader with hands on expertise in building large scale test automation suites, executing and maintaining them. He has a patent pending with USPTO for ‘iGenerate test Scenario’ and ‘Web Services Validator’ filed last year. Kalyan is a speaker at various testing conferences including DevOps East, StarEast& Agile Testing Conference in Boston. He was recently awarded the prestigious “40 Under 40 Award 2016” from the Philadelphia Business Journal.

Do not forget to access the On-Demand Webinar on Testing Trends here:

http://www.gallop.net/webinar-on-10-emerging-trends-in-software-testing

The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.

Getting Started with Risk-Based Testing

risk based testing, software testing, functionality testing, vulnerability testing, security testing, quality assurance testing, software testing services, software testing company, gallop solutions review, gallop solutions, operation risk testing, external risk testing, technical risk testing, product testing, performance testing, usability testing

What is Risk?

A Risk, essentially is a possible problem. That is, it is some event that may, or may not happen, depending on other variables. In the software testing arena, a risk may be defined as a potential occurrence (leading to loss) which is a result (usually undesirable) of the presence of an issue or a bug in the product. Testing for these unwanted, possible events is known as risk-based testing.

Additionally, the definition of risk is incomplete without introduction to mitigation and contingency.

  • Mitigation: Mitigation is the act performed that reduces the possibility to defects to show up.
  • Contingency: This is the backup plan of action to be performed in case a risk becomes a possibility, and which helps to reduce the impact.

Types of Risks

In theory, there may be innumerable risks. However, following is a list of the most commonly faced risks in different domains:

Business or Operational Risks

  • Over dependence on a specific system, subsystem, function, or feature
  • Business-Criticality of a feature or function, subsystem, including the cost of failure

External Risks

  • Security related loopholes
  • Integration failures – of product or website pages
  • Regulatory requirements
  • Failures of functions
  • Performance and Usability related failures

Technical Risks

  • Collocated development teams
  • Complexity of a product

What is Risk-based Testing?

Risk-based testing (RBT) is an organizational principle that helps to prioritize testing the features and functions of a software according to the probable risks of failure, the need of the function, etc.

RBT thus is a ranking of tests, and subtests, for functionality. Tools and techniques such as equivalence partitioning, state transition tables, decision tables, boundary-value analysis, Path Flow testing, all-pairs testing etc. help assess the most risk-prone areas.

As there usually is not enough time to test complete functionality of a product, RBT involves testing the functionality that has the highest probability of failure – and thereby biggest impact.

RBT, to be fully effective, must be started in the initial stages of product development. It involves:

  • Identifying risks to system quality and guiding the process of planning, preparation and execution of the tests.
  • Risk analysis that helps identify opportunities to remove or prevent defects.
  • Mitigation–testing (that reduces the possibility of high-impact defects) and contingency–testing (that identifies the possible work-arounds for the defects found).
  • Measuring the effectiveness of finding/removing defects in critical areas.

4 Phases of Risk Based Testing Process

There are four main phases to be kept in mind while executing RBT:

  1. Identify and define all the possible risks for all the functional modules of the application under test (AUT) and assign them to the responsible stakeholders.
  2. Prioritize the tests based on the criticality of the risk associated. Come to an agreement on the prioritization, and update the functional requirement document and shared with the stakeholders.
  3. Plan and define tests according to requirement prioritization.
  4. Execute tests according to the accepted functional document.

Advantages of Risk Based Testing

  • As all the critical functions of the application are tested, it improves the overall quality of the product.
  • Planned prioritization helps take care of the business-critical areas which ensures that the product even in case of a risk-impact, does not get impacted much. On the other hand, you must keep in mind to test even the low-ranked risks so that they do not become real and cause trouble.
  • Since the problem areas are discovered early, preventive measures can be started immediately – which ends saving a lot of time and costs during production.
  • In case of limited resources (time or team), it helps as a negotiating tool for prioritization.
  • Helps make testing a better planned and organized activity.
  • Continuous monitoring of risks helps focus on the complete testing strategy and goal throughout the testing life cycle.
  • Improves customer satisfaction.

That said, the main objective of risk-based testing is to perform testing in accordance with the best practices in risk management. This helps create a product that is properly balanced in terms of quality, features, budget and schedule.

At Gallop, we cover all the bases and ensure that effective testing is performed by the right set of experts. We ensure the best quality for your product and that your customers are happy. Our tool agnostic test automation frameworks ensure accelerated testing so that you get higher productivity and an enviable time to market.

Icon vector designed by Freepik

The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.

Rise of the Software Development Engineer in Test – SDET

software testing, software development engineer in test, sdet, qa engineers, quality assurance testing, web application testing, software testing services, test automation, gallop solutions, gallop solutions review

“Program testing can be used to show the presence of bugs, but never to show their absence!” – Edsger Dijkstra, Dutch computer scientist.

The industry today has awakened to the fact that testing is actually more important than programming. Testing apps requires a bigger budget for tools and resources compared to programming. Every organization today is hunting for the best possible talent. Earlier software developers used to write code and testers checked it for quality. This won’t suffice today.

Software Development Engineer in Test (SDETs) are skilled professionals who are adept in the arena of both QA Engineering and Software Development.

Though it’s great to have someone with skills and expertise that are high in demand, it also creates a bit of confusion as to what really comprises the duties and responsibilities of SDETs. A regularly faced ambiguity is regarding the difference between an SDET and a QA Engineer.

SDETs vs QA Engineers

While the roles of SDETs may seem very similar to those of QA Automation Engineers, with even most of the tools and language expertise required being same (Selenium, Java, and Jenkins), there are certain very obvious and clear differences between the two roles.

An SDET, in layman terms, is a developer who instead of working in the product development team, works as part of the test team. In essence, SDETs are responsible not only for writing code, but are required to test the code as well. SDETs are required to continuously write, test, and fix the code they write. Their roles and responsibilities are based on the Agile lifecycle model. SDETs usually are professionals who have very strong analytical, technical, and problem solving skills.

On the other hand, QA Engineers are testers who do not need to have any programming experience as they usually are not exposed to the code. This clearly creates a demarcation between the roles and responsibilities of SDETs and QA Engineers.

SDETs: The Need

With the need and importance of software testing accepted across the globe, what, when, and how to test are areas that have never stopped evolving.

Most of the products and apps today require end-to-end Test Automation – especially in the areas of Functional, Performance, and Security Testing. SDETs, with their dual abilities in the areas of code development as well as performing tests (such as those listed) are a great fit in todays’ digital age. They help improve the Code Quality by performing strict and detailed source code reviews along with checking for the Testability of the code.

Armed with specialized testing knowledge of multiple tools, techniques, best practices, and processes, SDETs today have become a crucial part of development ecosystems. Based on their development experience, knowledge of technical architecture and design, and their programming skills SDETs are required to write a code to test the code written by developers. In addition, they are also required to write unit tests and also perform white box testing.

Following is a list of a few tasks that SDETs are usually responsible for:

  • Building robust, scalable, and high quality test automation solutions for functional, regression and performance testing
  • Developing code for quality automation and ensuring extensive unit test coverage of the code
  • Building, customizing, deploying, and managing the environment/ test automation frameworks
  • Checking for product scalability, reliability, consistency, and performance
  • Participating in design and architectural discussions
  • Performing high-class debugging
  • Preparing Test Reports

In essence, SDETs are Customer Advocates who influence product design by understanding end user expectations. While functional and automation testers will always be required, SDETs may prove to be the all-rounder that most organizations are looking for.

Gallop Solutions has a decade of expertise as an independent testing services provider. Contact Gallop’s team of testing experts for your testing requirements.

software testing, software testing life cycle, stlc, software development life cycle, testing process, software testing company, quality assurance testing, software testing phases, gallop solutions, web application testing, security testing, devops testing, agile testing

The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.

Testing Metrics – What Gets Measured, Gets Done!

testing metrics, software testing, quality assurance testing, software testing company, software testing services, web application testing, gallop solutions, gallop solutions review , software testing metrics

“What gets measured gets done. What gets measured and fed back gets done well. What gets rewarded gets repeated.”

The above stated pithy statement relates very well to the need and importance of testing metrics. When an organization is able to clearly and explicitly define the testing metrics it requires, and then is properly able to analyze them and use the analysis to fix the existing issues, it will invariably be treading on the right path towards success and growth.

In normal parlance, most organizations follow the well tread path of plan, do, check, act (- better known as PDCA) when beginning any new venture or project.

As per WhatIs.com, “PDCA (plan-do-check-act, sometimes seen as plan-do-check-adjust) is a repetitive four-stage model for continuous improvement (CI) in business process management. The PDCA model is also known as the Deming circle/cycle/wheel, Shewhart cycle, control circle/cycle, or plan–do–study–act (PDSA).”

Plan: In terms of a product/software development/testing lifecycle, Planning refers to defining and laying out specific Business Goals and gaining a thorough understanding of the need for the planned application. At a later stage, this also includes testing the product, collecting statistical data, identifying and ascertaining the root causes of the issues being faced, and planning for fixing them.

Do: This is the stage where organizations define and decide upon the multiple measurement variables and metrics. These metrics will help understand the effectiveness of the product as also help measure the quality of the product. This stage also involves developing and implementing solutions for the identified issues.

Check: This stage is used to analyse the reasons as to why a product is behaving in the way it is, as also to compare the data before-and-after a fix has been made. This stage requires you to document the observations, inform the team about any changes to the process, and also recommend changes that need to be made.

Act: As the name implies, this stage involves taking Corrective Actions and fixing the product to come up with a quality product.

As seen above, one of the most important link between all the 4 activities – PDCA – is metrics and its measurement. The question to ask is – what should we measure, and when should we start measuring?

What sort of metrics need to be collected and analysed must be decided in the planning phase. A few metrics that matter – especially when testing a product for quality – are as follows:

  • User Story Coverage
  • Planned vs Done Story Cards
  • Test Automation Coverage
  • Automated Test Effectiveness
  • Mean Time Between Failures (MTBF)
  • Mean Time To Repair (MTTR)
  • Overall Equipment Effectiveness (OEE)
  • Defect Rejection Rate
  • Production Defect Leakage
  • Defect Severity Index
  • Defects by Sprint
  • Deployment Lead Time

Unless we specify what needs to be measured, we will never really be able to give due attention and focus to completing it to the best of our abilities. Measuring the correct variable, analysing it correctly, and then performing the required tests will lead to creation of better products and make them more reliable and robust.

“Not everything that counts can be counted, and not everything that can be counted counts.” ~ Albert Einstein

That said, what is even more important than measuring metrics is the follow up – what is the actual improvement we have made in the product based on our findings. Identify your top-most, business-critical priorities, analyse the metrics against them, and then make the fixes required.

Zooming in to the top-most priorities and then working against the related metrics will help an organization provide better quality products, reduced go-to-market time and hence improve their ROI.

Experts at Gallop can help you understand what needs to be measured and tested to get the optimum outputs. Get in touch with us for meeting all your testing requirements.

software testing, software testing life cycle, stlc, software development life cycle, testing process, software testing company, quality assurance testing, software testing phases, gallop solutions, web application testing, security testing, devops testing, agile testing

The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.

5 Drivers of Differentiation which you must test

testing, software testing, quality assurance testing, web application testing, software testing services, risky businesses must tested, mobile testing, application testing, security testing, agile testing, software testing company, gallop solutions, gallop solutions review, functional testing, test automation, performance testing, ERP Testing, globalization testing, medical devices testing

Risk-free businesses don’t exist – not even in the wildest of fantasies. If you have a business to run, it will have risks involved, and to survive – and thrive – you will need to face and overcome these risks. Next generation product and services industry will heavily focus around Social, Mobile, Analytics, Cloud and more importantly, Internet of Things and Virtual Reality.

Let us try to understand a a couple of these which are already contributing to an organization’s growth, sustenance and differentiation. Subjecting these to intense, planned testing is what is needed to ensure the sustained differentiation.

  1. Mobile Apps: There’s hardly any industry left today that is untouched by the presence or use of mobiles. It is only obvious the kind of risks this involves – all features of all products must function seamlessly across all types of all mobile devices on all platforms at all times. One can only imagine the kind of risks involved in keeping all these activities running smoothly.
  2. Digital Transformation: The need to go digital and have a global digital imprint is making organizations resort to a lot of online activities that leave a big scope for hackers to make hay. More so, in the hands of the wrong people, digital security-related records of governments can pose a big security threat.
  3. Big Data & Analytics: A lot of companies are making a foray into the world of big data and analytics today. This is helping to generate a lot of statistical data that can be utilized for creating a better standard of life. Just imagine what may happen if the data collected – or analyzed – is incorrect, and the same data is implemented. Big risk, right?
  4. Automation: Automate everything seems to be today’s buzzword. Newer and better versions of automation tools and automated products are hitting the market every day. As long as the tools and products behave in the manner expected, it’s great. But what if the automation tool goes wrong and actually breaks a product? Effort will be lost, time will be lost, and go-to-market and ROI delayed.
  5. Internet of Things: With the increasing adoption of IoT there will be a tremendous opportunity for IoT testing (devices and software) in 2015 and years to come. Technology analysts Gartner suggest that IoT is currently at the ‘peak of inflated expectations’. IoT has also been identified as one of the emerging technologies in IT as noted in Gartner’s IT Hype Cycle.

So how do we create a risk mitigation plan for dealing with the challenges that come packaged when running either of the businesses listed above? Having a specialized team of experienced testing professionals with experience in each of these testing domains will ensure you leverage industry best practices in Quality Assurance.

In essence, if organizations use either in-house talent, or approach specialized independent testing service providers for performing thorough testing of apps, tools, or devices, they can take care of most of the risks.

Get in touch with us at Gallop to get further guidance on getting your products tested for a better ROI.

software testing, software testing life cycle, stlc, software development life cycle, testing process, software testing company, quality assurance testing, software testing phases, gallop solutions, web application testing, security testing, devops testing, agile testing

The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.

Testing is a Process, not just a Phase

software testing, software testing life cycle, stlc, software development life cycle, testing process, software testing company, quality assurance testing, software testing phases, gallop solutions, web application testing, security testing, devops testing, agile testing

Should Testing just be another Phase?

We all accept that Testing forms an important part of the software development life cycle (SDLC). However, the reason why a lot of organizations fail is the fact that they segregate testing as a single unit – a phase. When Testing is treated as just another ‘phase’, the implementation of this business-critical task suffers. Organizations try to club all sorts of tests and try to test their product for all possible areas at the fag-end of the development cycle. This naturally is impacted by the looming deadlines of the product launch, as also other pressures due to which the testing is not fool-proof. A product that has not been tested fully will naturally not be robust. There will always be a doubt regarding its security, performance, and functionality. So what can these organizations do to improve their testing process?

The answer is actually fairly simple.

Instead of trying to overburden the tester to test for completeness just before launching the product, plan your test related activities. While planning the product development phases, identify the types of tests you need to execute for your product. Then, allocate specific time and resources for performing tests related to the different phases. This also helps verifying and validating the product, as also drastically reduces the number of bugs that may otherwise be found later. Performing tests specific to a phase helps save a lot of time and efforts of the developers as the issues found are far easier to fix.

In essence, if we treat Testing as a process that supports the entire development process – instead of just a single phase – we can ensure products that are far more dependable and robust.

What to Test under each phase?

software testing, software testing life cycle, stlc, software development life cycle, testing process, software testing company, quality assurance testing, software testing phases, gallop solutions, web application testing, security testing, devops testing, agile testing,

Based on the common experience shared across industries, following are a few tests that can be executed under the different phases:

  1. Requirements Gathering phase: The main focus of this phase is to gather the business requirements such as who will use what type of data in what manner. Thus, it is only common sense to test and confirm (read validate) the basic requirements for the creation of a product before actually starting the development process. This ensures that we create what we initially planned to create – and not something else that got created while on the way due to unclear and ambiguous requirements. The output of this phase is a Requirement Specification document that acts as the guideline for the product development.
  2. Design phase: The Design phase uses the Requirement Specification document to prepare the layout of the system and software design. If a comprehensive and end-to-end test plan and strategy is thought of and implemented in this phase, it will help build a stable system architecture. Testing the ease of design will also help establish what and how to test in the product.
  3. Development phase: As the name suggests, the development phase involves the actual writing of code for the different modules. As a lot of code is generated in this phase that covers implementation of different features, it makes sense to test the features being developed. It also is a good time to implement regression testing on the code generated so far to verify that the software being developed performs correctly even if it is modified or interfaced with other software.
  4. Deployment phase: The deployment phase usually has two sub-phases – Beta-deployment, and Final Deployment. In the Beta-deployment phase issues are caught before a product is launched to the market. This is the time when you can implement tests related to product usage analytics, Real User Monitoring, & Automated smoke tests. Based on the results of these tests, and the other issues reported, the development team will make the final changes before the final deployment of the product.

As seen above, if Testing is implemented across all the phases of SDLC, the end result will be a product that is stable, reliable, and supports features and functions that will grab the attention of the end user. It is no surprise that the more a product is liked and appreciated, the better ROI an organization gets.

software testing, software testing life cycle, stlc, software development life cycle, testing process, software testing company, quality assurance testing, software testing phases, gallop solutions, web application testing, security testing, devops testing, agile testing

The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.

Non-Functional Testing – Quality is more than Validating Features

quality assurance testing, non-functional testing, usability testing, performance testing, efficiency testing, security testing, software testing, gallop solutions, gallop solutions review, software testing, services, software testing company, Non-Functional Testing – Quality is more than Validating Features

The term non-functional testing is commonly used to refer to testing the features not specific to functions. This includes testing for performance, usability, efficiency, security, the breaking point of the software, and many more.

In essence, most of these tests help us to understand the quality and reliability of the product. Quality is a term that has a widespread implications, and when thought about in the context of non-functional testing, it covers the following major areas:

  • Functionality (quality in terms of Features, Business Processes, and Integrations)
  • Security (quality in terms of Application, Data, Network, and Compliance)
  • Performance (quality in terms of Speed, Resource Consumption, Scalability, and Sizing)
  • Usability (quality in terms of Navigation, Aesthetics, Flexibility, A/B Testing, and Documentation)

Quality in terms of Functionality

Even though there’s a whole range of tests available for testing the Functionality of a product in terms of how to use it, when related to non-functional testing, it relates to the ease of use of the product and the experience of a consumer, and how happy is someone while using the same.

Apart from this, the seamless performance of all the processes also indicate that a lot of thought and planning have been put into the product to make it achieve the required levels of quality. This quality experience is what will put your product on top of the buyers list – thus ensuring higher ROIs for yourself.

Quality in terms of Security

The higher the levels of Security, the better is the quality of a product. With so much personal and business critical data at stake in this highly digitized world thorough, end-to-end quality testing of areas such as Application, Data, Network, and Compliance is imperative.

Not checking for the quality of security in either of the aforementioned areas may cause huge (at times not recoverable) losses in terms of finances, credibility, etc. – and in a lot of cases may result in severe security threats and legal complications as the impact of security loss may be felt on human lives across the globe too.

Quality in terms of Performance

On a different front, the quality of a software with regards to its Speed, Resource Consumption, and Scalability matters a lot while trying to retain and attract new customers. The digital world is getting smaller with a lot of apps being used on mobile platforms, tabs, laptops – and customers, with the huge array of choices before them, naturally try to buy the products that give them the best returns for their cost. They need tools that run the fastest, consume the least data charges or electricity, and are capable of working on any platform, OS, or device – all the while providing all the features seamlessly.

Quality in terms of Usability

Quality in terms of Usability refers to the ease a customer enjoys while navigating through the various features of the product, how easy is the product to learn the first time it is used; the comfort levels a customer has with the aesthetics of the product, that is, whether the colours used on the UI are too jazzy for him, or if the whole look and feel has been created thinking about what most users may enjoy looking at when using the product; and whether or not all the test cases documented earlier have been tested for practical use and cover all scenarios possible. It also means that the customers enjoyed using the product so much that they want to re-use the product and do not need to spend time again to learn the basics of the product and can quickly accomplish the desired tasks.

Thus, as can be safely deduced, that in terms of non-functional testing, good quality of a product will always help you gain (and retain) more customers rather than just focussing on testing the features of a product.

Gallop’s teams have successfully performed Non-Functional Testing for our numerous clients. Get in touch with us if you are looking to test Non-Functional Aspects of your business applications and we will be glad to do a free assessment for you.

The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.