All About Functional Testing of IoT Cloud Applications
Key takeaways:
- Business case for functional testing
- What does functional testing of IoT Cloud applications include
- Shortcomings of current approaches
- Solution
Functional testing of IoT Cloud Applications can be a daunting task given the complexity and variety of “things”, applications, and scenarios that need to be tested. This article focuses on the need for functional testing, what it includes, and the shortcomings in the current approaches followed by the testing teams.
Business Case – why it is critical to do that?
Functional testing is important to ensure that the application is behaving as expected for different scenarios arising out of the things and environments being monitored.
- Ensure the right computations – OEE, utilization, predicted failures
- Reduce high-impact consequences by generating on-time alerts and events
- Right visualization and reporting of the data
- Improve user adoption and utilization
- Reduce recalls and field visits for servicing through device management and remote fixes with functionalities like FOTA
- Ensure that the edge and cloud integration work fine beyond just the protocol and payload agreements to avoid last-minute surprises in the field tests or sometimes only on the field deployments
What does functional testing include in the context of IoT Cloud Applications?
Functional testing in the context of IoT Cloud Applications includes the testing of the scenarios, such as user actions on the applications, data and events coming from the devices, and user actions spanning across the applications on the cloud and the device.
Application validated include:
- Web & mobile app
- API
- Database testing
- Device Management
- Data Ingestion Scenarios
- Real-time data processing
- Batch data processing
- Reconciliation of old stored data coming from devices
What needs to be tested?
Testers need to test the data flow for happy and unhappy paths/scenarios which includes sequencing of different kinds of payloads. The data values to be tested include:
- One or more related parameters impacting the functionality
- One or many iterations
- Going beyond or below thresholds
- Getting misses or becoming junk
Additionally, testing needs to be executed for the communication characteristics arising out of network conditions, such as payloads arriving late, payloads getting missed out, duplicate payloads being received, and payloads arriving in a different order.
Lastly, the testers need to check for the system-level behavior of the intelligent edge and cloud application for the different aspects of distributed systems, such as split-brain.
What are possible shortcomings that one might have in the current approaches?
|
Testing Approach |
Key Shortcomings |
|
Test with physical devices in the lab
|
Different data scenarios that be generated with a physical device are very limited, especially for unhappy paths Need for specialized knowledge and physical signal generators Time consuming Repeatability and consistency issues due to manual activity
|
|
Use protocol simulator tools |
Laborious activity, where the tester is forcing self to behave like a device Takes a lot of time, patience, and attention to do a good job of testing The very nature of the activity demotivates, limits the testers to cover the range of scenarios they would ideally like to cover Lack of repeatability and consistency |
|
Build custom simulators and scripts
|
Perhaps, a good solution that addresses a lot of the requirements.
The complexity of building these is a challenge and needs good programming skills and knowledge of the nitty-gritty of the IoT Protocol and the Platform. Simulation of the device behavior for network conditions, bidirectional data communications, etc., is difficult to conquer. Testers will spend valuable time on an enabler instead of focusing on the core activity of validating features and building automation. In times when the software engineering resource supply is scarce and costly, this isn’t an ideal situation for the business. Even if the 1st version of the simulator is created, maintaining it continuously over the lifetime of the product (AUT) across releases can become very cumbersome. |
When dealing with Cloud and intelligent Edge applications, none of the above options enable the kind of comprehensive end-to-end validation that is required.
Doppelio – Purpose-built to test IoT applications
Doppelio’s patent-pending approach leverages the concepts of virtualization and simulation to give you a testbed that is a very close replica of the real world. Test scenarios are easy to create with Doppelio Web Application without having to write even a single line of code.
Click here for more details on how Doppelio approaches functional testing of IoT cloud applications. Write to us if you want to see a quick demo.
Rajesh K, Co-Founder & Head of Products