Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

#ATAGTR2020 Presentation - Adopt Non-Functional Testing to realise the full potential of Connected Devices


Published on

Niruphan & Karthikeyan delivered a session on "Adopt Non-Functional Testing to realise the full potential of Connected Devices" at ATAGTR2020

ATAGTR2020 was the 5th Edition of Global Testing Retreat.

Niruphan is a Senior Non-Functional Testing Consultant with over 10 years of experience. Currently he is pursuing his IT career in London, United Kingdom

Karthikeyan is a Non-Functional Lead with 8+ years of experience. Has excellent technical skills with a focus on continuous improvement and client centricity

The video recording of the session is now available on the following link:

To know more about #ATAGTR2020, please visit:

Published in: Technology
  • Be the first to comment

  • Be the first to like this

#ATAGTR2020 Presentation - Adopt Non-Functional Testing to realise the full potential of Connected Devices

  1. 1. Adopt Non-Functional Testing to realise the full potential of Connected Devices Niruphan Rajendran Cognizant Worldwide Limited Karthikeyan Lakshminarayanan Cognizant Worldwide Limited
  2. 2. Introduction IoT Ecosystem and Challenges A View & A Deep-Dive on IoT NFT Solution – An Overview Solution – Non Functional Requirements Benefits AGENDA
  3. 3. Introduction How companies can offer services that ensure efficiency, safety of devices, smooth operations and intuitive interface design? How people can operate safely at home or office without much impact caused by smart IoT devices? IoT ‘Hot topic for decades, but radically improved to solve societal challenges’ NFT ‘Drive quality at speed with Customer User Experience’ According to Gartner, the number of IoT devices is set to reach 25 billion by 2025 and worth of $1.1 trillion by 2026
  4. 4. IoT Ecosystem and Challenges Connecting IoT devices [Device to Device] and testing its performance & security in near real time scenario is a key challenge. There is a need for handling device messages every milli second in a real-time scenario is crucial aspect of Non-Functional testing. KEY CHALLENGES 1 Non-functional validation of end to end IoT Ecosystem 2 Firmware validation and verification across IoT Devices 3 Setup the continuous performance test suite on IoT Devices 4 Industry Benchmarks and KPIs to test & validate IoT Devices Device / Client Layer UI/Network Responsiveness iOS Android End User Mobile App IoT Devices Validation between Hub to DevicesPlatform to HubPlatform Verification & Validation Cloud Platform APIs API GatewayDynamoDBLambda Functions Cosmos DB API Management APIs ZigbeeIoT Hub SMART Devices NFT Pain Point
  5. 5. A View & A Deep-Dive on IoT NFT Hotspot: ▪ Test for peak hour, validate scalability to peak volume ▪ App Unresponsiveness, Fragmentation ▪ Battery Consumption & Network provider bandwidth NFT Focus: ▪ Better page response and loading time ▪ Scalable & Reliable ▪ Availability & Maintainability ▪ Application Stability & Durability Focus on End User first Phase1 Device / Client Layer App Validation and Data Consumption iOS Android End User Mobile App Storage & Persistence Cloud Platform APIs API GatewayDynamoDBLambda Functions Cosmos DB API Management IoT Devices IoT Device and Data Ingestion/Simulation APIs ZigbeeIoT Hub SMART Devices Tune backend systems next Phase2 Hotspot: ▪ Server Capacity on each component ▪ Custom code for data processing ▪ API & Query performance ▪ Data Handling, Resource Consumption ▪ Processing Millions of messages NFT Focus: ▪ Data consumption per second ▪ High Availability ▪ Better processing and staying powerful ▪ Data Management Deep-dive into IoT devices Phase3 Hotspot: ▪ Data Ingestion & Simulation using different protocols ▪ Protocol Transmission ▪ Data processing & secure validation within Device to Device NFT Focus: ▪ Scalability, Stability, Security & Sustainability Is it possible to validate at IoT device level? • Turn on/off bulbs • Change colour of the bulb • Turn thermostat/heater on/off • Turn on/off smart plug etc Use Cases
  6. 6. Solution – An Overview Debug Adapter Sniffing Automated transactions Simulate IoT Devices Sniffer helped to get a view on below ▪ Custom code had built in, to pair & configure the device ▪ Simulate the messages (ex: Device ON or OFF) under different network conditions ▪ Sniff the calls using custom chip between two IoT devices to ensure the minimum edge nodes are connected to give better response time ▪ Using customized monitoring tool check the routing mechanism to understand if request process meets the NFRs ▪ Monitor the encryption techniques to see if it meets the security standards ▪ Overall breakdown analysis on Device to Device communication Performance Phase3 ▪ Security testing approach that uses binary code analysis to ensure the security of device firmware ▪ Reversing the firmware binaries using open source tools & customized code to extract and to perform automated scan to identify the common threats ▪ Implementation of continuous source-code quality & security assurance in IoT ecosystem Security
  7. 7. Solution – Non Functional Requirements NFT Category NFT Test Type Non-Functional Requirements Performance Load Simulation Response time SLA b/w Centralized Device & IoT Device Network loss between D2D Server Attributes Frame Control in Zigbee Network Poll Period Transmission Fall Poll mode Device Stability Different & Maximum number of devices to be connected Network Transmission & Connectivity Validate the Frame Control Ensure Network provider bandwidth Caching Compression Node & Routing Mechanism Node processing Call transmission between client and server Security Encryption Encryption methodology Network Stability Firmware Analysis IoT Device Security Static Device Firmware Analysis Firmware Assurance Prevention of exploitation of user space processes This table illustrates different set of Non-Functional requirements at Device level which will evaluate performance and security at higher standards
  8. 8. Benefits Aided 100% Reusability due to Automated Regression suite set up on IoT Ecosystem to support frequent changes Outline 25+ Non-Functional Requirements at Device level to meet the standards Security related issues and bad practices implemented in firmware files (both source & binaries) Best-in Class framework to assure different sets of Non-functional tests This diverse solution helps across several industries like Home Automation, Healthcare Industries etc., Authorize Resource Planning & Optimization across landscape Features can help IoT companies to track, manage assets and create contactless user experience
  9. 9. Q&A THANK YOU