Applying Theorem Proving for Safety-Critical Vehicle Software

JTEKT CORPORATION
Application of theorem proving for
safety-critical vehicle software
S. Yoneki, K. Zanma, K. Toyozumi / JTEKT corporation
J. Kanig/AdaCore
1

JTEKT CORPORATION
Background
• For current vehicle, loss of specified function is not acceptable.
– ISO 26262-10:2018 addresses this topic but mostly random hardware failure is focused. Software
systematic faults also face the same issue but very limited guidance is provided.
2
12.4 Software development phase
12.4.1 Software fault avoidance and tolerance
Safety-related availability requirements for software can be addressed by two approaches: fault avoidance (12.4.2) and
fault tolerance (12.4.3).
12.4.2 Software fault avoidance
The methods for fault avoidance are intended to reduce the overall occurrence of systematic faults. The necessary amount
of fault avoidance can be achieved by developing software elements using ISO 26262-6:2018.
12.4.3 Software fault tolerance
Techniques for fault tolerance try to keep the item operational despite the presence of software systematic faults. Some
fault tolerant mechanisms are mentioned in ISO 26262-6:2018, 7.4.12, NOTE 2 and NOTE 3.
From ISO 26262-10:2018
Can we claim that fault leading to loss of specified functions is avoided?
For the software fault tolerance, we need additional safety mechanisms such
as backup (degraded) control, diversified software.
But this is not always possible depending on many reason
(performance limitation, resource limitation, etc.)
How to deal with it?

JTEKT CORPORATION
• Comparison with software safety design for conventional EPS system and the
requirements for coming EPS system
Safety requirements on the main function
for a conventional EPS system:
• The system shall prevent unwanted output
from the main function in order to avoid
hazardous event.
Current implementation
3
Main function
QM (D)
Safety mech.
ASIL D(D)
Systematic fault
Risk mitigation is made by combination of main
function and shut-down function.
Safety requirements on the main function
for the latest EPS system
• The system shall maintain a specified
function provided by the main function in
order to avoid a hazardous event.
Main function
QM(D)
Safety mech.
ASIL D(D)
Systematic fault
Example:
However, this is very difficult (or impossible) in some case (e.g. I/O control, state
machine) !!!
SHUT
DOWN
SPECIFIED
FUNCTION
Example:
Risk mitigation is made by combination of main
function and shut-down function.

JTEKT CORPORATION
Key for solving the issue
• Currently no specific criteria for sufficiency of measures during development.
4
Start
For each architectural element, how
failure modes caused by systematic
faults will propagate is analyzed.
The criticality of the fault mode is
decided based on possibility of violating
the safety goals or safety requirements
Is the possibility of the failure mode
sufficiently mitigated during the safety measures
taken during development?
Introduction of
appropriate safety
mechanisms
Provide rationale for sufficiency
(including introduction of
additional measures)
YESNO
Typical chart to decide the necessity of safety mechanism
In general, it is difficult to demonstrate
this sufficiency to the customer since no
well established criteria.
 There are several supporting
evidences (but they are not "silver
bullet")
• Code metrics
• Test coverage
• Proven in use
 Based on the situation, criteria for
acceptance is not the same for all
customers
Can we demonstrate it by "formal verification"?

JTEKT CORPORATION
Formal verification
• Generally speaking, it is hard to claim that the software is bug-free by test.
→ Testing only demonstrates existing of bug.
• Formal verification is recommended by ISO 26262-6.
Not highly recommended because it is difficult to apply it.
However….
– Formal verification may demonstrate that there is no systematic failure that leads to loss of
specified function.
– If this is true, then formal verification is considered as strong tool.
5
From ISO/FDIS 26262-6

JTEKT CORPORATION
Model checking vs Theorem proving
• Two approaches for formal verification; model checking and theorem proving.
– Model checking:
The software to be proved is translated into state transition model and all states and transitions
are explored in the model.
– Theorem proving:
From the software and its specifications, mathematical proof is made. The truth of proof implies
conformance of the system to its specification.
6
Model checking Theorem proving Remark
Number of
states to be
handled
finite Not limited Theorem proving verifies program itself. Therefore, number of states is
not related to the capability of theorem proving.
On the other hand model checking is hard to apply for numeric
calculation because it easily results in state explosion
Automation Auto Semi-auto Model checking can proceed automatically via a software tool once
model is created.
Ability to
represent target
software
Low High For model checking, software has to be translated in the state transition
model. In order to prevent state explosion, software has to be
abstracted. During such abstraction process, systematic faults in the
software program may be hidden.
On the other hand, in theorem proving, source program is translated
one-by-one.
Difficulty Low High Because theorem proving usually needs extra information for the proof
by a human, expertise is high
Simple comparison of model checking and theorem proving w.r.t. finding SW bugs
Our target is to demonstrate "bug-free" for a very specific software component to
which safety mechanism is hard to apply.
Because of this, "theorem proving" is selected for the study.

JTEKT CORPORATION
What is the "theorem proving"
• Theorem proving is:
– Like "code review" with logical (or mathematical) proof
– Possible to trace how to proof based on logical equation
• Mathematical (or logical) proof that if the pre-condition is satisfied, then post
condition after program run is also satisfied.
7
Pre condition:
Conditions that are satisfied before execution of
program to be verified.
Post condition:
Conditions that are satisfied after execution of
program to be verified.
Program
(e.g. C source
code)
Program is theoretically translated and checked that
if pre condition is "true", then post condition is also
true.
Post condition can be defined at the arbitrary level
based on what is to be proved, such as:
• Don’t output unwanted value;
• No access outside the array;
• No divide by zero.
Other assumptions are made:
• No interference with other programs or
hardware
• Called functions are correctly specified
• …

JTEKT CORPORATION
Simple example
• Pre condition
The property that the input
parameters (x, y) have
{ 3x - 2y ≥ 0 }
• Program
y := x – y +10 … S1
x := x + 2 * y … S2
• Post condition
The property (e.g. safety
threshold) that the resulting
value has to possess.
{ x ≥ 20 }
8
Pre condition is transformed
as :
x + 2(x – y + 10) ≥ 20
Proof for S1:
Substitute y of intermediate post condition
{x – 2y ≥ 20} with x - y + 10, then
{x + 2(x – y + 10) ≥ 20 }.
This is equivalent with precondition. This
means that post condition of S1 is always
satisfied if pre condition is satisfied.
For S1, pre condition is
{x + 2(x – y + 10) ≥ 20 }, and
post condition is
{x – 2y ≥ 20}
For S2, pre condition is
{x – 2y ≥ 20}, and
post condition is
{ x ≥ 20 }
Proof for S2:
Substitute x of post condition
{x ≥ 20} with x + 2 * y, then
{x – 2y ≥ 20 }.
This is equivalent with precondition. This
means that post condition of S2 is always
satisfied if pre condition is satisfied.
If pre condition of the program is satisfied, post condition is always satisfied.
From proof of S1and S2

JTEKT CORPORATION
Difference between conventional testing
• The approach to demonstrate how a bug is not included in the software.
9
Testing Theorem proving
Experimental approach:
• It might be difficult to claim that the
software is 100% tested.
• Test coverage can be measured by,
for example, structural coverage to
demonstrate that the software is
tested as far as reasonable practical.
• High coverage is good rational that
the risk of bug is quite low, such the
overage for safety critical software is
acceptable or not depends on the
customer.
Logical approach:
• It can be possible to claim that software
does not contain any bugs with evidence.
• A safety critical software unit is
demonstrated as bug-free in logical
way.
• This make us possible to convince all
our customers that the software unit
will not cause a safety critical event.
Necessity of safety
mechanism depends on
customers.
Safety mechanism for the
software unit is not necessary
any more for all customers.

JTEKT CORPORATION
Weak points of theorem proving
• Timing issue
– Theorem proving does not verify calculation time. Therefore, it is difficult to
demonstrate if output is acquired within the specified time flame.
• Hardware (or ISA) dependent issues
– It can be possible to deal with but much effort is needed for, such as:
• Bit manipulation
• Address calculation
• Big endian/Little endian
10
This has to be confirmed by other methods such as worst-case
execution time analysis.
This has to be confirmed by, for example, static code analyzer
Theorem proving is not perfect solution but give additional confidence
for correctness of software from logical point of view.

JTEKT CORPORATION
Case study
• We tried to apply theorem proving to actual software element to get better
understanding of this method.
– Purposes for the case study are:
1. To understand the method "theorem proving".
– The method has not been well know to automotive industry (at least in Japan).
– Therefore, if we apply this method, we have to give clear explanation to our customers.
2. To investigate feasibility to apply this method to automotive E/E system
software.
– Since this method is quite new for us, we did not have clear image about how it works.
– We need to know how we can use this method for our purpose.
11
SAMPLE for the case study - Software element for autonomous driving.
• Function:
To decide whether acceleration pedal or break pedal is operated to release deceleration
control by halt break
• Safety requirement:
− if halt brake switch is not active, deceleration done flag shall not be "ON".
− if acceleration pedal position is smaller that halt break cancel position, declaration
done flag shall not be "ON"

JTEKT CORPORATION
Case study: source code
void dec_vo_JdgDecCtrlDone( void )
{
if ( u1_tDecCtrlInitReq == KU1_ON ) {
dec_u1_DecCtrlDone = KU1_OFF;
}
else {
if ( ( u1_VehicleCanDiagMaskReq == KU1_OFF )
&& ( u1_tEBC1CanRcv == KU1_ON )
&& ( u1_tEBSBrakeSwitchDataRangeErrStatus != DEC_KU1_ChkErrDeterminedState )
&& ( u1_tBrakePedalPositionDataRangeErrStatus != DEC_KU1_ChkErrDeterminedState )
&& ( u1_tEBSRedWarningSignalDataRangeErrStatus != DEC_KU1_ChkErrDeterminedState )
&& ( u1_tABSEBSAmberWarnSignalDataRangeErrStatus != DEC_KU1_ChkErrDeterminedState )
&& ( u1_tHaltBrakeSwitchDataRangeErrStatus != DEC_KU1_ChkErrDeterminedState )
&& ( u1_tEBC1TimeOutErrStatusStatus != DEC_KU1_ChkErrDeterminedState )
&& ( u1_tHaltBrakeSwitch == DEC_KU1_HaltBrakeSwitchActive ) ) {
if ( ( u1_tBrakePedalPosition >= DEC_KU2_KHaltBrkCancelBrkPedalPos )
|| ( ( u1_tEEC2CanRcv == KU1_ON )
&& ( u1_tAcceleratorPedalPosition1DataRangeErrStatus != DEC_KU1_ChkErrDeterminedState )
&& ( u1_tEEC2TimeOutErrStatus != DEC_KU1_ChkErrDeterminedState )
&& ( (u2)u1_tAcceleratorPedalPosition1 >= DEC_KU2_KHaltBrkCancelAccelPedalPos ) ) ) {
dec_u1_DecCtrlDone = KU1_ON;
}
else {
}
}
else {
}
}
}
12

JTEKT CORPORATION
Case study: Safety requirements
Interpretation of two safety requirements at code level
− safety requirement 1:
This means the situation that
“if HaltBrakeSwitch is not active, DecCtrlDone != KU1_ON”.
− safety requirement 2:
This means the situation that
“if AcceleratorPedalPosition1 ＜ KHaltBrkCancelAccelPedalPos, DecCtrlDone != KU1_ON”.
13

JTEKT CORPORATION
Case study: Verification strategy
• In the program, if-statements are nested.
• Therefore, theorem proving is applied for step-by-step manner.
14
if
…
else
…
If
…
else
…
Software portion 1:
This is verified with internal if
statement (software portion 2
below) as black box.
This means that all possible
outcome is assumed as pre-
condition.
Software portion 2:
This program is verified
independently.
NOTE:
The purpose of strategy is to do the case study.
Depending on the tool capability, it is possible to
make proof for complete software without splitting
portions.

JTEKT CORPORATION
Case study: Proof of SW portion 1 (1)
{
}
else {
}
else {
}
}
else {
}
}
}
15
Scope of verification

JTEKT CORPORATION
{
}
else {
DO SOMETHING BUT INDEPENDENTLY FROM "u1_tHaltBrakeSwitch" (to be verified separately)
}
}
else {
}
}
}
16

JTEKT CORPORATION
• Pre-condition and post-condition for the scope of verification
17
Program:
DO SOMETHING BUT INDEPENDENTLY FROM "u1_tHaltBrakeSwitch" (to be verified separately)
}
}
else {
}
Pre condition：
{true}
Post condition：
{u1_tHaltBrakeSwitch != DEC_KU1_HaltBrakeSwitchActive → dec_u1_DecCtrlDone != KU1_ON}

JTEKT CORPORATION
• If branch is taken
18
Pre condition：
{true ˄ }
Program:
DO SOMETHING BUT INDEPENDENTLY FROM "u1_tHaltBrakeSwitch"
( u1_VehicleCanDiagMaskReq == KU1_OFF )
&& ( u1_tHaltBrakeSwitch == DEC_KU1_HaltBrakeSwitchActive ) )
Post condition
According to the pre condition, (u1_tHaltBrakeSwitch != DEC_KU1_HaltBrakeSwitchActive) is false. Therefore, post-condition
is always true.
Therefore, the portion of the program is correct.

JTEKT CORPORATION
• If branch is not taken
19
Pre condition：
{true ˄ }┐( u1_VehicleCanDiagMaskReq == KU1_OFF )
&& ( u1_tHaltBrakeSwitch == DEC_KU1_HaltBrakeSwitchActive ) )
Pre condition modified：
{true ˄ }( u1_VehicleCanDiagMaskReq != KU1_OFF )
|| ( u1_tEBC1CanRcv != KU1_ON )
|| ( u1_tEBSBrakeSwitchDataRangeErrStatus == DEC_KU1_ChkErrDeterminedState )
|| ( u1_tBrakePedalPositionDataRangeErrStatus == DEC_KU1_ChkErrDeterminedState )
|| ( u1_tEBSRedWarningSignalDataRangeErrStatus == DEC_KU1_ChkErrDeterminedState )
|| ( u1_tABSEBSAmberWarnSignalDataRangeErrStatus == DEC_KU1_ChkErrDeterminedState )
|| ( u1_tHaltBrakeSwitchDataRangeErrStatus == DEC_KU1_ChkErrDeterminedState )
|| ( u1_tEBC1TimeOutErrStatusStatus == DEC_KU1_ChkErrDeterminedState )
|| ( u1_tHaltBrakeSwitch != DEC_KU1_HaltBrakeSwitchActive ) )
Pre condition modified again：
{true}
The pre condition after this modification is weaker (allows more situations) than previous one.
Therefore, this modification is valid for the propose.

JTEKT CORPORATION
• If branch is not taken(continue)
20
Program:
dec_u1_DecCtrlDone = KU1_OFF
Post condition：
If the pre condition is satisfied, the post condition is also satisfied
Pre condition modified again (from previous page)：
{true}

JTEKT CORPORATION
{
}
else {
}
else {
}
}
else {
}
}
}
21

JTEKT CORPORATION
• Pre-condition and post -condition for the scope of verification
22
Program:
}
else {
}
Pre condition：
{true}
Post condition：
{u1_tBrakePedalPosition < DEC_KU2_KHaltBrkCancelBrkPedalPos → dec_u1_DecCtrlDone != KU1_ON}

JTEKT CORPORATION
• If branch is taken
23
Pre condition：
{true ˄ }( u1_tBrakePedalPosition >= DEC_KU2_KHaltBrkCancelBrkPedalPos )
&& ( (u2)u1_tAcceleratorPedalPosition1 >= DEC_KU2_KHaltBrkCancelAccelPedalPos ) )
Program:
Post condition：
Pre condition modification：
{true ˄ (( u1_tBrakePedalPosition >= DEC_KU2_KHaltBrkCancelBrkPedalPos) ˅ }( ( u1_tEEC2CanRcv == KU1_ON )
&& ( u1_tAcceleratorPedalPosition1DataRangeErrStatus
!= DEC_KU1_ChkErrDeterminedState )
&& ( u1_tEEC2TimeOutErrStatus
!= DEC_KU1_ChkErrDeterminedState )
&& ( (u2)u1_tAcceleratorPedalPosition1
>= DEC_KU2_KHaltBrkCancelAccelPedalPos ) )
Post condition modified：
{dec_u1_DecCtrlDone == KU1_ON → u1_tBrakePedalPosition >= DEC_KU2_KHaltBrkCancelBrkPedalPos}
dec_u1_DecCtrlDone == KU1_ON in case
that the conditions shown as right side.
Because of this,
post condition is "false"
( ( u1_tEEC2CanRcv == KU1_ON )
An issue was found!

JTEKT CORPORATION
• If branch is not taken
24
Pre condition：
{true ˄ ┐ }( u1_tBrakePedalPosition >= DEC_KU2_KHaltBrkCancelBrkPedalPos )
Program:
Pre condition modified：
{true ˄ }( u1_tBrakePedalPosition < DEC_KU2_KHaltBrkCancelBrkPedalPos )
&& ( ( u1_tEEC2CanRcv != KU1_ON )
|| ( u1_tAcceleratorPedalPosition1DataRangeErrStatus == DEC_KU1_ChkErrDeterminedState )
|| ( u1_tEEC2TimeOutErrStatus == DEC_KU1_ChkErrDeterminedState )
|| ( (u2)u1_tAcceleratorPedalPosition1 < DEC_KU2_KHaltBrkCancelAccelPedalPos ) )
Post condition：
If the pre condition is satisfied, the post condition is also satisfied

JTEKT CORPORATION
Results of case study
• Positive points:
– It is demonstrated that the method can be used to demonstrate logically the
correctness of program. Therefore, if the proof is successfully done, it can be claimed
that there is no bug as far as logical point of view
• Negative points:
– It is very "tough" work for the software engineers to do this because of
• Lack of knowledge of theorem proving;
• Lack of time to do it. (this is very time consuming)
25
If the negative points are overcome, it can be understood
that this method have high potential for safety critical
portion software component.
→ For the proving itself, tool support is expected.

JTEKT CORPORATION
Possibility to use supporting tools
• In order to overcome such difficulty, the usage of commercial tool can be
considered.
• For this purpose, we tried to use SPARK.
– SPARK is:
• Programming language for safe and secure software
• Very readable
• Strong type-checking
• Formal verification tool
• Proves absence of access to uninitialized data
• Proves absence of runtime errors:
– Values within expected ranges
– Buffer overflow
– Division by zero
• Proves user-specified properties
– Results:
• Same conclusion but very short time.
• There is still difficulty to define appropriate set of pre and post conditions.
26
This is used in the case studies.

JTEKT CORPORATION
Steps to use SPARK
• Theorem proving with SPARK by following steps.
1. Transform from C to SPARK:
– Since the automotive E/E system software is developed as modification from existing
one, it is assumed that the software is written by C language.
– For most of the automotive E/E system, one-by-one interpretation can be possible.
– Of course, this is not necessary if software is developed by SPARK from scratch.
2. Define property to be proved
3. Execution
27
This is the most important task
for this method.

JTEKT CORPORATION
How to integrate theorem proving?
• In order to successfully perform theorem proving, safety properties (pre-
condition & post-condition) have to be determined at the necessary level of
detail.
– At least, each software unit has its own safety property if it is safety related.
28
Software architectural design
(Clause 7 of ISO 26262-6)
• Software architectural design
• Safety analysis
• Safety requirement refinement
Software unit design & implementation
• Software unit design
• Software unit implementation
Software unit verification
• Software unit static verification (review, code
analysis)
• Software unit dynamic verification (or testing)
• Based on the result of them, expected behaviour shall be
defined for safety related SW elements.
• Decision making for application of theorem proving.
− Safety properties are extracted for each SW unit.
• Theorem proving is done as a part of SW unit verification

JTEKT CORPORATION
Decision making for application
• Currently SW FMEA is enhanced to add such information.
– (1) to (8) are done in the same way as conventional FMEA
– If the software element is judged as safety critical based on the result of analysis ((5) to (8)),
decision is made for necessity of applying theorem proving.
There are two points for this decision.
• Point 1: The feasibility to apply safety mechanism with respect to the failure mode:
If it is hard to apply safety mechanism to meet safety requirement with respect to the failure
mode assumed, we have to rely on safety measures during development. Theorem proving
is one of the powerful methodology for this purpose.
• Point 2: The risk of systematic failure in the specification for the software element:
Theorem proving on source code level can detect coding error. However, this method does
not address specification error. Therefore, it is a prerequisite that systematic faults in
specification has already been eliminated.
– If it is decided to apply theorem proving, properties are identified.
29
Item
SW
function
SW
element
Failure
mode
Effect
Theorem
proving
applied?
Properties
Operating
mode
Locallevel
System
Level
ASIL
Pre-
condition
Post-
condition
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11)
Since the methodology is not mature for us, area of application is still limited.

JTEKT CORPORATION
More systematic approach
• Our absolute goal is to claim that our software does not need any safety mechanism for
software systematic fault!
• To achieve this, mode systematic approach from specification to implementation is
needed.
30
Theorem
proving 
Formal description
language + Model
Checking??
Formal description
language ??
WearetryingtoapplySPARK
fortheseareaasacandidatesolution

JTEKT CORPORATION
Conclusions and future steps
• Theorem proving can be used to demonstrate that software is free from
systematic fault during coding.
– Thanks to this, it can be possible to claim fault avoidance is achieved.
– As a result, no additional safety mechanism is necessary!!
• For the next step, JTEKT will investigate to apply similar proof for upper
abstraction level to reach safety mechanism-free software design.
– Logical proof from specification to implementation.
– This will need additional effort for software design at high level, but
• Reduce software testing:
Since no safety mechanism, test for it is not necessary anymore.
• Reduce required performance:
Since run-time checking is no more implemented, it is not necessary to spent
hardware resource for it.
• Reduce effort for justification of safety:
Since everything is theoretically explained, we don't need to rely on in-direct
evidence (such as code metrics, test case and test coverage) for safety. 31
JTEKT has started to apply theorem proving to safety critical
software elements.

Applying Theorem Proving for Safety-Critical Vehicle Software

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Applying Theorem Proving for Safety-Critical Vehicle Software

Similar to Applying Theorem Proving for Safety-Critical Vehicle Software (20)

More from AdaCore

More from AdaCore (20)

Recently uploaded

Recently uploaded (20)

Applying Theorem Proving for Safety-Critical Vehicle Software

Editor's Notes