Testing Machine to Machine interactions in IOT World

Testing Machine to Machine interactions in IOT World


M2M or Machine to Machine interactions have been around for quite some time now and with IoT getting into mainstream, the machine to machine interactions are at the forefront again.

So what makes IoT and M2M so interesting?

As per GSMA Intelligence, there are now more than 7.82 billion devices, including M2M devices, which is way above the number of humans pegged at 7.4bn. We are working in a world where the importance of connected devices is getting more relevant and with the amount of automation happening, we are going to see a huge jump in the connected machines by next decade. As per Cisco, “the Internet of Things (IoT) will have up to 50 billion things (or devices) that will be connected to the Internet by 2020; or, the equivalent of 6 devices for every person on the planet.”

Though M2M has been there since 1930s, when the British military first invented radar to detect aircraft, it is getting into every household these days. M2M is at the heart of Internet of Things and with connected devices getting so common, in future, it might be at every corner of your house. IoT is all about connected devices communicating with each other in form of data which is being analyzed and action being taken in real-time with/without human intervention. Given the current pace of things, human intervention is going to get less every year.

As more and more devices start taking decisions without human intervention, testing and certifying such devices will become more critical for the organizations. Some of the factors which needs to be considered are:

  • Skill set:

This is going to be the most important factor for testing M2M/IoT applications as the resources are supposed to have domain knowledge as well as systems engineering understanding. Having the right resources with good skillset is going to be critical for success of IoT projects.

  • Test environment management:

Having the right test environment for testing the devices, applications around it, communication between machines is going to be difficult as the number of devices rises. Once the devices with different versions and upgrades emerge, mimicking and simulating the test environment will be a challenge.

  • Test data management:

Here comes the real challenge. M2M usually generates terabytes of data for different processes, and which may also be behaviorally different. And given that there will be different types of devices which will be communicating and generating different types of data, getting it right becomes even more important.

  • Security testing:

Data privacy and application security are the non-avoidable types of testing, given that there will be lot of loopholes when multiple devices interact over multiple channels. This becomes even more important because m2m devices usually don’t have specific identities (currently), and testing thus them for vulnerabilities is must because the amount of data which gets exposed.

  • Compatibility testing:

IoT has given rise to different operating systems, devices and messaging protocols. Making sure that different devices communicate properly while maintaining the standards is key to success of the IoT.

  • Performance testing:

Performance becomes critical in the case of IoT as the response time between machines is critical to the success of the business scenarios which run over these devices. At the same time, measuring the vital statistics of devices like power usage, memory usage, endurance testing, disaster recovery testing is also important.

  • Accessibility testing:

Testing for accessibility is going to grow as we see more interconnected devices around us. With smart cars talking to smart buildings and in turn passing the information to smart wearables, this is going to be compulsory type of testing to be considered.

  • Regulatory compliance testing:

We are going to see standards, protocols and compliances emerging in the IoT space and making sure that regulatory requirements are adhered to will be crucial.

These are few of types of testing which we feel are important. But there will be many more types of testing which needs to be considered to cover the end-to-end functionality of the IoT devices.

It is going to be interesting to see how this space evolves and if any new standards, tools and, processes becomes part of the IoT software delivery pipeline. Whatever it will be, we will keep you updated with the latest in IoT testing world.

The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.

The Need for Testing the Internet of Things

internet of things testing, iot testing, IoT Security testing, iot analytics, testing software for iot, iot product testing, software testing services, software testing company, quality assurance testing, gallop solutions, gallop solutions review

Kevin Ashton, co-founder of the Auto-ID Center at MIT, which created a global standard system for RFID and other sensors, coined the phrase “Internet of Things” in 1999. IoT encompasses a world where living and inanimate things are connected wirelessly and serve the purpose of machine-to-machine communication.

“The IoT demands an extensive range of new technologies and skills that many organizations have yet to master,” said Nick Jones, vice president and distinguished analyst at Gartner. “A recurring theme in the IoT space is the immaturity of technologies and services and of the vendors providing them. Architecting for this immaturity and managing the risk it creates will be a key challenge for organizations exploiting the IoT. In many technology areas, lack of skills will also pose significant challenges.”

The above statement very clearly lays down the need, an importance of testing all things IoT. It is obvious that while IoT will have a strong, positive impact on making our lives easier, on the flip side, it will need the organizations providing such services pay a lot more attention to security, risk, designing, and the basic architecture of their products.

With the recent focus on IoT-enabled devices, there’s a huge surge in the products related to household, healthcare, utilities, cars, etc. on a Year-on-year basis. Additionally, enterprises the world over are also rapidly migrating, developing, or rolling out newer IoT-enabled products into the market.

As listed very accurately in a PR by Gartner, following will be the business-critical areas that most organizations will need to pay a very strong attention to in the coming years:

  • IoT Security
  • IoT Analytics
  • IoT Device (Thing) Management
  • Low-Power, Short-Range IoT Networks
  • Low-Power, Wide-Area Networks
  • IoT Processors
  • IoT Operating Systems
  • Event Stream Processing
  • IoT Platforms
  • IoT Standards and Ecosystems

Testing software and systems for IoT also involves physical, sensory and emotional interactions because today’s digital devices not only work based on keystrokes, but also respond to our touch, motion, and voice. This makes “Human Experience” testing also a very crucial part of testing IoT.

Testing IoT also will need to involve testing all possible devices for all possible permutations and combinations of OSs and software supported. Emulators and simulators will need to be used in a large scale to test all such products end-to-end. This will require making the use of all-encompassing Test Labs.

Internet of Things, thus, is all pervading. Each and every aspect related to the production of an IoT product needs to be thoroughly tested and verified before being launched for public consumption. Not doing so can lead to severe and serious repercussions.

In order to master and implement IoT, organizations need to work closely with mature vendors to overcome key hurdles. Testing IoT in addressing data management, security and privacy concerns helps in offering trusted products. Gallop experts provide in-depth skills to identify potential bugs and failures avoiding disastrous failure of products. Testing often and testing continuously will prove to be a crucial factor in identifying and fixing the discrepancies of the IoT products.

Gallop Solutions has a decade of expertise as an independent testing services provider. Contact Gallop’s team of testing experts for your testing requirements.

Icon vector designed by Freepik

The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.

IoT, Testing, Outsourcing, and Automation – The 4 Horsemen of the Future Software

IoT, Testing, Outsourcing, and Automation – The 4 Horsemen of the Software Future

Right on the heels of the Mobile App Software explosion and closely related, we have the Internet of Things (IoT). If predictions come true there will be billions (more) of new IP addressed devices running countless pieces of software. We will have them in anything that has electronics e.g., cars, HVAC systems, entertainment devices, robots, computers, wearables, transport systems, etc. and many more things that, to date, have not had electronics until we put it in them (e.g., shoes, clothes, pets, furniture, hardcopy paper, back-of-your-neck, etc.) Of course, we will need software developers, and on the flip side people to assess the qualities of these devices i.e., testers. IoT will offer us many opportunities.

Certainly aspects of traditional testing will apply, and there will be “new” challenges that were found in Mobile-Smart app software. But, what are some of the “new” concepts IoT testing is likely to encounter? Here are a few thoughts on this future (to be taken as on any prediction with some grains of salt).

First, there will be companies moving into “high-tech” development, that have minimal or no experience with software. These companies may have IT departments that cared for and supported computers, but this is not new product development. These companies will have product domain development experience e.g., how do you develop new clothing styles or shoes. However, they will not know much about software development and even less about testing it. They may think software is easy and testing is not needed.

These kinds of companies have several paths they can take in learning IoT software development, including:

  1. Training existing staff to become software developers and testers.
  2. Hiring new staff to create in-house software development and test capabilities.
  3. Outsourcing the software and testing to external companies who already have expertise.
  4. Or a mixture of the above items.

The path taken will be company and context unique. There is no “one way,” best path and, every company will learn as they go. Many organizations will likely learn that they will need some in-house software development skill/knowledge supported by outsourced companies and/or contractors. Currently the “outsourcing” trend is common in many segments of industry, but even so, IoT can create challenges to companies. Companies have challenges and opportunities.

Next, IoT will present many employment and growth possibilities for existing developers and testers. Companies should consider a variety of factors about their employees, including:

  • Knowledge and skill of the staff are important to success (these are different capabilities).
  • Communication between all development parties (developers, testers, management, stakeholders-users, hardware, manufacturing, systems, contractors, etc.) must be fostered throughout a product’s lifecycle.
  • While the first two items are important, many times some degree of contracts and/or legal documents (e.g., legal agreements and requirement (“shall”) documents) should be put in place to support and define and its lifecycle product.
  • Focusing on working software and performing required activities must be balanced by the team, i.e., we must balance software functionality with the support activities like configuration management, testing, and documentation used to help create and maintain the software.
  • Agile and lean software development ideals need to be worked by a team that includes software and product domain knowledge i.e., the software people need to listen to the product domain staff who understands clothing (as an example) and vice versa.
  • Outsourcing companies will be used for much of the future work in IoT, but the owning company needs to control the complete product and this includes having the skills to understand what the outsource providers will be doing i.e., the owning company will need systems, software, hardware, and testing skilled staff.
  • Automation and use of tools (for development and test) used by skilled staff are needed to support many aspects of an IoT project.

Each of these can be the subject of books and/or articles by themselves, but the list gives teams starting ideas.

Regarding the last item, tools and automation, some care and thought must be given by the companies moving into IoT. Software people and humans in general, are tool users/makers. It is very tempting for management to think a tool or automation will “save” an IoT project–like a magic bullet. And while tools/automation can help, here are some tool/automation keys for the IoT projects to consider:

  1. Tool and automation, even in testing, must address tasks that are repeated at least 3-to-5 times before any “cost savings” are likely to be realized. Likely automation candidates include: regression testing, configuration management for continuous integration and builds, modeling, data analysis, etc.
  2. Tools and automation must help in tasks which humans dislike or are not good at. Examples here include: planning, risk assessment, test data generation, design of code and tests, documentation retention.
  3. Tools and automation need management and staff support including schedule, budget, and skilled human resources, to be successful. For example, just buying a tool will not help if the staff receiving the tools does not have the training or skill to use it, much less are not given the time and money to implement the automation.
  4. Tools and automation must help in areas such as project knowledge retention or generation. Examples here include capturing code and test results, test documentation, mind-maps, or models of team thinking.

This list is not comprehensive either, but gets teams thinking about the scope of the problems we all must face in IoT development, testing, and automation.

IoT should become a brave new future for software and testing. Enjoy and grow.


john-hagar-300x300About the Author: Jon Hagar is a systems-software tester consultant and trainer supporting software product integrity, verification, and validation with a specialization in embedded, IoT, and mobile software systems. He has worked in software, particularly testing for over thirty five years. Jon is the author of the book: Software Test Attacks to Break Mobile and Embedded Devices. He is lead editor/author on numerous international standards including model-based testing, IEEE, and ISO testing. Jon has a web site at: http://breakingembeddedsoftware.com/
The opinions expressed in this blog are author's and don't necessarily represent Gallop's positions, strategies or opinions.