Commercial test tools are helping automotive software vendors save time, money, and refine the development lifecycle, but these off-the-shelf solutions also must be qualified to safety-critical requirements. As ISO 26262 grows in importance as a means of demonstrating best practices and due diligence, the ability to produce compliance documentation will become increasingly critical in the absence of regulatory oversight.

Recent concerns over automotive software quality and security spawning from ODB hacks, brake software malfunctions, and even emissions rigging, along with accelerated demands on time to market, have made software verification and testing an imperative for auto manufacturers and their Tier 1 suppliers. As a result, validation and verification testing previously left to the backend of the development right before designs were handed over to production are now being integrated into every stage of the software lifecycle, from early concept phases through production test to maintenance and support.

Subsequently, the responsibility of testing and certifying software products has been delineated across various points of development and production. Rather than utilizing homegrown test methodologies as was common in the past, however, the expansion of test processes has led software development organizations to leverage commercial test automation tools such as National Instruments’ TestStand and BullseyeCoverage in a whole lifecycle approach, with toolchains collaborating across product lifetimes. The benefits are non-trivial, as the approach provides insights that flow both upstream and downstream, saves time, and allows vendors to maximize their tools investment.

But there is an inherent disadvantage, namely that commercial test tools used in safety-critical software development must be qualified, which can be an expensive, laborious, and time-consuming task in and of itself. Tool qualification is required to ensure that test outputs are correct, and typically involves creating a qualification plan, defining operational requirements, verifying those requirements, and maintaining tool artifacts in a configuration management system. Especially when considering the generally long lifecycles and growing complexity of safety-critical automotive systems, qualifying tools to meet the requirements of ISO 26262, for example, can add up quickly as the task requires the expertise of a knowledgeable, seasoned engineer.

“50 percent of engineering cost is spent on verification and validation testing in safety-critical systems. Any seasoned manager will smile and nod when you say that,” says Jeff Gray, CEO of CertTech, an embedded test tools and services company that specializes in certification for safety-critical applications. “The Tool Qualification Kit is intended to make it efficient and cost effective to show compliance with tool qualification requirements.”

CertTech’s Tool Qualification Kit (TQK) is a fully automated method of formally qualifying tools to the requirements of industry standards such as ISO 26262, and can be customized based on functionality added to off-the-shelf test tools. The TQK includes a generic tool qualification plan (TQP), as well as generic tool operational requirements (TORs) that define the development environment, coverage measurements, code constructs, and other characteristics of a given tool. For requirements-based verification, the TQK includes an application framework that exercises the tool platform by running individual sequences that test various tool features, and then generates documentation and reports that demonstrate compliance with functional safety standards, which can be submitted to clients and/or regulatory bodies. In certain cases, the CertTech’s BullseyeCoverage TQK has been able to reduce the time and cost associated with tool qualification by as much as 90 percent, Gray says.

Despite lack of regulation, ISO 26262 increasingly important as liability defense mechanism

Although ISO 26262 has grown into a near de facto set of development guidelines for the automotive industry, it lacks the punch of standards in other markets because there is no official body enforcing its use. In the medical industry in the U.S., for example, the FDA is responsible for certifying that electronic devices have been designed to safety and security requirements, but no such process exists for automotive software.

Until such a time as regulatory bodies are formed and certification is mandated, however, the onus of certifying tools, processes, and procedures will fall to auto manufacturers and suppliers, if for no other reason that to protect themselves from liability claims down the road.

“One of the serious challenges with ISO 26262 is that it’s a new concept for companies, and it’s being refined as they update the suite of documents,” says Gray. “The lack of a formal submission process, or someone at the end of the row checking, will be something that needs to be addressed. It leaves a lot to open-endedness and interpretation. There’s currently no one to gate you from fielding those systems.

“It’s a challenge the industry doesn’t have an answer for because there isn’t going to be NHTSA or DoT regulation until lawsuits start flowing through,” Gray continues. “Last year at the Car Training Institute Symposium, the first four speakers were lawyers. Five-to-ten years after development and deployment, those lawyers will be looking to see that you showed due diligence during the development process,” he adds.

For more information, visit www.certtech.com.