The Most Common Pitfalls in Verification and Validation

Every year, problems with airplanes, trains or medical devices increase manufacturers’ operational costs and cause significant brand damage. Quality assurance is a must but it is a real challenge for mission-critical industries due to the increasing complexity and safety-integrity requirements of embedded components and systems.

To guarantee the required levels of safety, industry standards address the planning and development of safety-critical systems for each of the respective industries. In the case of the most demanding standards, formal Verification and Validation (V&V) activities account for more than 40% of the total project effort.

Getting embedded software systems right first time is crucial, and that hangs heavily on getting verification and validation activities right. We gathered ten of the most common mistakes so you can avoid these pitfalls and have a better chance of successfully launching your new software without incurring unexpected costs and difficulties.

Poor documentation

The worst thing that can happen is not having a clear starting point for your V&V testing activities. If a team fails to correctly define the requirements of their new product, it means they will have poor and incomplete documentation and lack a clear sense of the product’s design. Design problems are one of the most commonly overlooked errors when developing new software and it can come at a great cost. Define your software requirements so you know exactly what to test and how to drive your V&V activities, otherwise you may end up ignoring industry standards and safety critical issues.

Not using V&V at early stages

Techniques such as dependability, safety analysis, certification and qualification support should be applied at the early phases of development to try to detect defects before they are introduced into the system. The cost of remedying an error increases with time and the software’s lifecycle stage. Applying and anticipating these techniques early in the development lifecycle can provide significant early error detection and have a dramatic impact on development costs and production time. The cost of error-remedying can be as much as 100 times greater in the final phases of the development lifecycle.

Not validating your test tools or methods

All types of formal V&V tool sets (including design modelling tools, formal proofing tools and model checking tools) should be the subject of research and development. Meaning that you need to check if your V&V tools are working properly. Otherwise you will be risking the data from those tests. The same goes for your V&V techniques methods; you need to agree on the methods and manage the documentation and instructions before executing the tests.

Not giving independence to whoever is doing the evaluation

To ensure unbiased, reliable and secure conduct of these activities, a level of independence from the development team is usually recommended. Independent verification and validation adds value to systems and software development even when certification by a third-party is not mandatory.

Not testing the final product

This ties with the first pitfall we mentioned. If you change the design of your software half way through, this means you are probably also changing the requirements that should be driving testing. The first step should be to update the requirements avoiding incorrect and missing information, but you should also be open to repeat the V&V testing or risk having tested a completely different product.

Ignoring that something is wrong

When it comes to V&V there are few signs that should raise red flags. We all know that V&V takes a lot of resources when developing new products but it’s not an excuse to avoid having clear responsibilities assigned, or for undertaking design tests in a hurry, not preparing ahead and ignoring standards or safety critical issues, sometimes just because they are harder to test for or more expensive. If any of these are happening in your product or embedded software testing, it will undermine your chances of success.

Using inexperienced teams

V&V activities are quite complex and can make the difference between a more cost-efficient, faster time to market or a never-ending waste of resources. The risk of using inexperience teams to do V&V is that they tend to underestimate the resources and time needed to prepare and do the tests -just the preparation can take months. And this increases dramatically with the complexity of the product.

Understanding these pitfalls comes from the more than 20 years’ worth of experience we have at Critical Software, creating V&V plans for every phase of the development lifecycle and assessing the most efficient strategies (plans, methods, tools and strategies, risk assessment and management solutions). If you would like to learn more about our unique processes and offers, click the button below.