A3.1  As noted in the main text, reliability claims for computer based SSs to which SAP ESS.27 is applied can range from of the order of 1E^-1 (interpreted herein as 0.3) to 1E^-4 pfd. The following table provides indicative guidance on the reliability claim that NII would expect to be applied, given application of the multi-legged procedure of ESS.27 including use of techniques and measures as outlined in the IEC 61508 standard (i.e. where the SS is implementing the safety function). ESS.27 need not be applied for claims higher than 0.3 pfd, though licensees should still apply appropriate safety standards such as IEC 61508 SIL 1, so far as is reasonably practicable.

Table 1 - Safety Integrity levels: target failure measures for a safety function and reliability claims for SSs

IEC 61508 Safety Integrity Level (SIL)	IEC 61508 Probability of failure on demand (pfd) Range	Minimum acceptable nuclear safety case probability of failure on demand (pfd) value
4	≥ 10-5 to < 10-4	10-4
3	≥10-4 to < 10-3	10-3
2	≥ 10-3 to < 10-2	10-2
1	≥ 10-2 to < 10-1	10-1

A3.2  The above table is in agreement with public statements on justifiable claims for computer based systems in nuclear reactors ², ⁴, ⁹, ¹⁰ and ¹³ It also represents a conservative and cautious approach which reflects the high standards expected within the nuclear sector. The maturity of current techniques, their practicability of application to real life SSs and other factors such as computer based system complexity, configurability, novelty and maintainability also have to be considered. Taking all of the above into account, a probability of 1E^-4 failures per demand is considered to be the best that can justifiably be claimed for computer based SSs  used in circumstances where the consequence in the event of failure of the SS could potentially involve very large releases of radioactive material. However, it is recognised that advances in system design and software engineering techniques might lead to a situation where a strong case could be made for a lower figure. Such a case would not then be ruled out of consideration.

A3.3  Assessors should consider applying similar reliability claim limitations (i.e. when referring to the levels stated in IEC 61508-1:1998 Table 3) to SRIs where they are subjected to ESS.27, for example, as shown in table 2 below. ESS.27 need not be applied for claims higher than 0.3 failures per year, though licensees should still apply appropriate safety standards such as IEC 61508 SIL 1, so far as is reasonably practicable.

Table 2 - Safety Integrity levels: target failure measures for a safety function and reliability claims for SRIs

IEC 61508 Safety Integrity Level (SIL)	Minimum acceptable nuclear safety case probability of dangerous failure per year (pdfy) value
4	10-4
3	10-3
2	10-2
1	10-1

A3.4  In some software development projects there might not be a significant difference between the techniques and measures adopted at the different IEC 61508 SIL levels. In addition, the selected confidence building measures might add significant weight to the safety case. Assessors should, therefore, consider the techniques and measures adopted by the system producer in relation to those of the next higher IEC 61508 SIL, the rigour with which they were applied and the strength of the independent confidence building measures. Following this assessment the assessor should determine whether the nuclear safety case reliability claim (see tables 1 and 2) of the next higher level could be substantiated.  For example, an IEC 61508 SIL 1 system that substantially and rigourously adopts the requirements of SIL 2 combined with a rigourous confidence building programme might allow a nuclear safety case claim of 10^-2 pfd.  

A3.5  It should be noted that tables 1 and 2 above apply to specific types of systems important to safety and specific circumstances where the highest level of confidence of achievement of target reliability figures (pfd/pdfy target) that it is reasonably practicable to achieve is required. The factors affecting the decision to apply the tables include both:

1. circumstances where the consequence in the event of failure of the system important to safety could potentially involve very large releases of radioactive material, and
2. systems important to safety that are very complex (which would include all software based systems).

For those circumstances and situations falling outside (1) and (2) above, the adoption of the pfd/pdfy of the lower target failure measure at the boundary of the IEC 61508 SIL should be considered in respect of the effectiveness of what has been done to control or avoid systematic failures. However, the licensee should provide appropriate evidence that it had achieved this by effective and rigourous adoption of all appropriate techniques and measures for the SIL and that this is appropriate given the circumstances of the specific case.

A3.6  In assigning reliability values to a computer based system it is appropriate to consider the hardware and software aspects separately since their behaviour is quite different. Hardware failures are predominantly random, hence coincident failures have a low probability of occurrence unless occasioned by a common cause. Hardware reliability can, therefore, be improved by the use of simple redundancy, although a limitation is imposed due to the incidence of common cause failures.  Software failures are due to systematic faults.  Their occurrence depends upon the values of input and stored parameters causing paths containing faults to be executed. Here simple redundancy gives no reliability improvement since each program sees the same input values.  The software equivalent of hardware redundancy is achieved by software diversity, since only by such means can coincident failures be rendered less likely.  Where a claim is made that very high reliability has been achieved through software diversity then the assessor should consider the guidance provided in Appendix 4 and the document “Licensing of safety critical software for nuclear reactors.  Common position of seven European nuclear regulators and authorised technical support organisations [736KB]”.

A3.7  The overall system reliability in terms of failures per demand is the sum of the separate software and hardware contributions.

A3.8  Where credit is claimed for self-revealing fault detection and automatic testing in reliability calculations, the contribution attributable to system and operator response, and MTTR should be specified and justified. Failures which are not self-revealing should be deemed to exist until the next test that would reveal them.

A3.9  The claimed effectiveness of the fault detection system should be justified. A claim of 100% should be rejected.

A3.10  When evaluating the numerical reliability claimed for the hardware of a computer based system, limited credit (depending upon the software production method employed) can be claimed for the diagnostic software designed to detect hardware faults.

A3.11  Licensee numerical reliability claims would be enhanced by the application of statistical testing techniques. An important consideration here is whether the system facilitates statistical testing and the generation of direct evidence to support reliability claims. A review of research into statistical testing should be undertaken so as to inform the steps needed to generate a convincing statistical reliability claim. Note that the number of required tests (representative operational transients randomly selected from the input space) could run to tens of thousands (e.g. of the order of 50,000 tests with no failure for a 1E-4 probability of failure on demand demonstration to 99% confidence).