Overburdened field tests could hurt your Connected Vehicle business
IoT testing gets complex and important than ever before. It needs a different approach. Here’s a use-case for connected businesses.
Gaurav Johri, Co-Founder
The market launch date has been fixed, sales and marketing plans are ready to roll, millions of dollars and several person-years of effort is about to see the light of day. Standing in the way is the field test, that final examination that will nit-pick all defects and issues in the solution. Software and hardware teams have worked in parallel but they have never been able to test the integrated solution in the real world. Surprises are bound to be there. A few surprises will be manageable but if there are too many, the costs will be enormous as the cost of fixing defects increases exponentially with each life cycle stage. If too many defects are found in field tests, product launches are delayed costing millions of dollars.
Field Trials are a time-boxed exercise and a frequent trade-offs between test coverage and time is made. It is the coverage that is usually sacrificed which means that some defects will sneak through into the market. If they are serious issues, millions of dollars will be spent on recalls.
Software accounted for only 4% of recalls in 2010 model cars. For 2019 models, they accounted for 46% of the recalls. Things are about to get worse. According to a Mckinsey report, today’s cars have about 100 million lines of code; by 2030, many observers expect them to have roughly 300 million lines of code. To put this into perspective, a passenger aircraft has an estimated 15 million lines of code, a modern fighter jet about 25 million, and a mass-market PC operating system close to 40 million. In 2023, 1 in every 3 cars on the road will be connected. By 2030 they will be everywhere.
The rapidly increasing software footprint on the vehicle and the cloud will power the vehicle of the future. The richness of the features delivered by the software will be a key determinant of the success and failure of a product. So the market pressure to keep cranking out new features will be relentless. The car will be like a smartphone with new apps and updates launched almost every day. The task of testing the software will be far more challenging and the coverage-time tradeoff way more excruciating.
But why are we faced with this trade-off? At the heart of the problem is that testing before the field trial has very low coverage. Today we see coverages ranging from 30-75%. This means that 25-70% of the test cases are tested for the first time during field trials. That is way too many and way too late compared to software best practices. Ideally, not more than 10-20% of the test cases should be tested for the first time during field trials. So the solution is to increase the lab test coverage to 80-90%.
The only way to do it is to have a testbed like Doppelio that enables testers to simulate data streams for a variety of real-world conditions in the Lab. The testbed should have the capability to support different connectivity protocols, network condition simulations, high load, sensor faults, test automation, etc., to test the software on the edge, cloud, and edge & cloud in an integrated fashion to get the maximum coverage and confidence.