Criterion 3.1 "The safety report should clearly describe how the Operator uses risk assessment to help make decisions about the measures necessary to prevent major accidents and to mitigate their consequences."

The purpose of this criterion is to help the Assessor determine if the Operator's approach to risk assessment is suitable and sufficient ie.(proportionate and systematic). To this end the following questions and answers may prove useful:-

Q: Has the Operator a policy on risk assessment?

This is an important point because the Operator must demonstrate a risk-based approach to his activities and to the production of the safety report. Failure to provide adequate evidence on this point may be viewed as a failure to comply with both the Management and the COMAH Regulations. The section of the safety report dealing with the major accident prevention policy (MAPP) will inform the Assessor on this issue.

Companies that manage their business with the aid of risk assessment might refer to the use of risk assessment in areas of safety management such as COSHH, commissioning (HAZOP) and cost benefit analysis. In these cases there may be reference to one or more formalised methods of determining risks such as event tree, fault tree and FMEA, and the use of risk assessment will probably not be confined to major accident analysis, but be detectable throughout the report. Assessors should not forget that risk does not necessarily involve quantification and that qualitative risk assessment has its place in the demonstration of safe operation.

Examples of non-quantified approaches that are acceptable include:-

• Hazard studies.
• Job safety analysis.
• Reference to industry standards.
• Safety reviews.
• Human error identification.

Q: Does the safety report summarise the methods of risk assessment or quantified risk assessment that are used in the report?

Since the regulations call for a risk assessment, the safety report should describe the approach adopted. If a QRA has been undertaken, the information that should be presented includes:-

• The extent of the analysis (plants/processes addressed).
• The method of identifying major accident event sequences (HAZOP).
• The analytical approach (event tree, fault tree, FMEA).
• The source of the base failure rate.

If a non-quantified approach is adopted, the basis for demonstrating that the residual risks are both tolerable and ALARP should be given. One or more of the following is acceptable if supported by well reasoned argument:-

• Industry standard good practise.
• Regulatory guidance.
• Industry association guidance.
• Historical data.

Q: Does the safety report summarise the criteria for use with the risk assessments or quantified risk assessments that are used in the report?

Operators should summarise the criteria used to judge when risks are tolerable/ALARP. Ideally this should appear near the beginning of the report so that the Assessor can make the following judgements:-

• Are the criteria appropriate?
• Does the safety report demonstrate compliance with its own criteria?

Q: Does the safety report state the basis for judging whether all necessary measures have been taken to prevent major accidents and to limit their consequences?

This question is related to the previous one, but in this case a basis for judging if the safeguards systems are adequate can be suggested:-

• Demonstrating that the risks are negligible (risk of death of an individual <10^-6/year).
• Demonstrating that risks are tolerable (risk of death of an individual <10^-4/year).
• Societal risks are shown to be tolerable or broadly acceptable.

Assessors should not expect to see detailed cost benefit calculations in a COMAH safety report, but Operators should list possible practical improvements and justify why they are not implemented.

Q: Has the Operator demonstrated a routine and general application of risk assessment in different aspects of operations, or has a limited amount of quantified analysis been carried out for the sole purpose of the safety report.

The safety report should convince the Assessor that the Operator understands risk assessment and routinely uses it to reduce risks at all levels and in all aspects of site operations. The complexity of such uses and level of detail given in the safety report should be proportionate to the risks involved.

The tone of the safety report and the way it is written will be a reliable indicator of the Operator's use and understanding of risk assessment. Assessors should look to the MAPP for evidence of a risk assessment culture rather than the accident analysis that may have been carried out by a consultant.

Criterion 3.1.1 "It should be clear that human factors have been taken into account in the risk analysis."

When making a judgement about compliance of the safety report with this criterion, Assessors should pose the following questions:-

Q: Has the Operator demonstrated that the risk assessment he has carried out to aid decision-making on the measures necessary to prevent major accidents and to mitigate their consequences includes allowance for human factors?

Risk assessment should not focus exclusively on random failures of hardware, but should also consider all types of operator error that can result in a major accident or a dangerous situation. The Operator should describe the role operatives play in controlling hazard and show that their potential errors are identified. He should also describe measures that have been taken to reduce their probability and how they are accounted for in the major accident analysis. The safety report should demonstrate that his systems and procedures are fit for purpose and incorporate adequate attention to human factors. This may be described in the management section dealing with staff training, competence assessment, and the way incidents and near misses are dealt with.

Accounting for human error in risk assessment is not straightforward because some human reliability literature data are not universally applicable. Assessors should primarily be concerned with checking that human reliability is included in the analysis rather than with the accuracy of the data used.

Q: Does the safety report consider an adequate range of human failings?

Inclusion of human factors in risk assessment does not necessarily mean simply accounting for process plant Operators opening the "wrong" valve or failing to control the process properly. Events such as corner cutting, unauthorised absence and even sabotage may warrant consideration.

Examples of the types of event which may warrant consideration are:-

• Failure to successfully carry out an operation that is part of normal duties.
• Bulk tank filling and discharging errors made by operators.
• Erroneously carrying out an operation that is not part of normal duties.
• Failure to respond correctly to an alarm situation (failure to control or making a situation worse).
• Deliberate of inadvertent degradation of the safety of a plant (eg switch an alarm off, or bypass a safety system),
• Deliberate rule flouting (eg smoking in a non-smoking area).
• Failure to detect failed components during testing.
• Introduction of failures by damaging equipment or leaving equipment mis-aligned during testing or maintenance.

In practice many safety reports will not address human factors as thoroughly or with as much rigour as engineering issues. This can be understood in the light of traditional approaches to safety and safety reports, but cannot be justified where human reliability plays a critical role.

The following are examples of common omissions in safety reports:-

The potential for an Operator to override designed safety features has not been covered.

There should be some mention of 'violations' or 'breaking the rules' as well as 'human error'.

The hazard analysis process failed to identify anything more than errors of omission (the Operator failing to act).

Most safety reports need to consider errors of commission (an Operator making an action but the wrong one), or decision making errors.

The role of people other than as front-line Operators (eg maintainers, supervisors) is not considered.

Many human failures are the result of actions, omissions and decisions taken by other people including designers and managers. For example, the potential for a maintenance error on a safety related system may not be addressed in the RA process.

There was no consideration of the possibility of a hardware failure with a simultaneous human error.

Some appreciation that when the hardware of a protective system fails the Operator may also not respond in the intended manner.

The Operator is being asked to do a critical task that would probably be more reliably done automatically.

There appears to be undue reliance on an Operator to identify and respond rapidly to an alarm condition.

If so, we would need some justification of the human error probability included. This should be justified in relation to the specific design of the system interface they have on site rather than a generic value taken from a table.

There is reliance on 'heroic' acts by Operatives to recover situations eg going back to the control room when suffering from effects of toxic gas.

Q: Does the safety report show how human factors are included in the risk assessment?

Data tells us that human failures contribute up to 80% of industrial accidents. Even in oil refineries, which are highly capitalised and automated, the figure is 50%. The implications of this run throughout the safety report and through many of the assessment criteria, so they will need to be considered by several members or all of the assessment team.

The safety report should consider in a rigorous and proportionate way how Operators may contribute to the initiation of a major accident (see Criterion 3.4.4). It should also describe the part Operators play in controlling hazards and risks. If an Operative is required to take certain actions following an alarm, the risk analysis will need to make assumptions about the likelihood that the correct action is taken. For example, if the economic consequences of emergency shutdown are great, the Operator may very well hesitate or fail completely to press the button.

If a task is critical to the prevention of a major hazard and an unrealistically high level of human reliability has to be assumed to make the risks ALARP, this may not be acceptable as it places an undue burden on the Operator. Instead automatic control and protection systems can be used to reduce the reliance on the Operator to intervene correctly. To achieve the required reliability it may be necessary to build redundancy and diversity into the control systems.

Not all safety reports will need to quantify human reliability. The focus should be on demonstrating the quality of the training and supervision. If a human reliability figure is used in a fault tree, the Assessor should check that the top event is not sensitive to the value adopted.

Q: Does the safety report describe how the probability of Operator error is reduced?

In the context of Operator error and how the company ensures that it is minimised, the safety report should:-

• Describe how Operator errors are identified.
• What measures have been taken to reduce their probability.
• How they are accounted for in the major accident analysis.
• Demonstrate that the systems and procedures for selection, training and supervision of Operators are fit for purpose.

Criterion 3.1.2 "Any criteria for eliminating possible hazardous events from further consideration should be clearly justified."

This criterion deals with the Operator's limitation of accident analysis in the safety report and can be judged by reference to the following:-

Q: Have any major accidents been discounted on probability grounds?

Operators are obliged to demonstrate that low frequency events with severe consequences are adequately controlled - that all necessary measures have been taken to prevent their occurrence. However, most safety reports are unlikely to determine the consequences and frequency of all possible accident scenarios, but it is essential that the risk dominating accidents are dealt with comprehensively. Very improbable accident initiators such as a meteor strike, simultaneous multiple failures of reliable systems and terrorist activity can usually be neglected, but cold catastrophic failure of vessels and guillotine rupture of large diameter pipe work should not be discounted.

Assessors should recognise that the COMAH regulations do not call for QRA. Frequency evaluation for highly improbable accidents does not need to be as detailed as that for risk dominating sequences and can be based on historical data, industry standards and regulatory guidance, etc.

Q: Does the safety report unjustifiably eliminate 'small scale' releases?

It is reasonable for the Operator to reduce the number of release cases by defining a scale of event that will not lead to a MA. For example, the consequence assessment may show that any failure resulting in a release smaller than that equivalent to a 10 mm diameter hole does not produce a hazard to on-site or off-site populations. This provides a basis for defining major accident hazards. However, Operators may need to take account of smaller flammable releases into confined spaces, which might ignite and explode and trigger a more severe accident. The Operator should also consider any known or foreseeable changes to the sensitivity of the surrounding environment, eg future dwellings which may be built nearer to the site boundary as these can affect the decision. Such changes should be also considered whenever the risk assessment is reviewed.

In situations where this 'protection' based approach is not sufficiently limiting, ie the hazard ranges from very small releases extend into population, a risk based approach may be needed. This requires the contribution to the residual risk of releases of different sizes to be considered so that a justifiable 'cut-off' can be decided. All contributions to release likelihood need to be taken into account otherwise, the 'cut-off' may be overly optimistic.

Q: Has the Operator grouped the consequences of several accidents together?

It is reasonable for Operators to describe in detail the consequences of only a relatively small number of representative accident sequences, provided all significant accidents are identified and ranked according to the risk they pose. Thus, for example, if six different accidents resulted in a similar rate of release of methane, with similar duration and type of fire, the consequences of only one of them need be described in the safety report. The relative likelihood of the others should be evaluated in order to demonstrate that the risks are ALARP. Operators have discretion on the way this is done and Assessors should not insist on a particular approach, but the arguments presented must be robust.

Q: Is adequate justification provided for dismissing major accidents on the grounds of low probability?

A safety report may describe the consequences of a representative set of accidents, but it must determine the frequency of all credible major accidents. In particular it should describe the risk from all accidents that the Company has taken measures to prevent occurring. The frequency determinations do not necessarily have to involve the application of formalised methods such as fault tree analysis, reference to appropriate source material/documents, industry standards etc is likely to be the norm.

The safety report should also demonstrate that risks from accidents, for which no preventative measures are taken are tolerable. In general these will be low probability events initiated by an off-site event such as aircraft impact or an earthquake.

Incredible accidents are not clearly defined in this context, and Assessors are expected to use common sense and professional judgement about events that can be neglected. Examples include meteor strike, terrorist activity and simultaneous failure of several diverse and redundant safety systems.

Q: Has the Operator determined or ranked the frequency of all major accidents?

Assessors should recognise that the COMAH regulations do not call for a full QRA. Frequency evaluation for highly improbable accidents does not need to be as detailed as that for risk dominating sequences and can be based on historical data, industry standards and regulatory guidance, etc. However, the statement - 'the probability of this accident is judged to be less than 10^-6' is not acceptable if they are not backed with supporting evidence. A poorly documented or sparsely detailed frequency analysis that appears somewhat optimistic may be judged as failing to comply with the assessment criteria.

Operators are obliged to demonstrate that low frequency events with severe consequences are adequately controlled, ie that all measures necessary have been taken to prevent their occurrence. If precautions have been taken to reduce the probability of an accident, then the consequences of the event must be assessed so that they can be balanced against the precautions.

If the Operator has not attempted to quantify accident frequencies, but builds a case based on terms such as high, medium and low probability, he should rank the accidents according to their perceived severity. Without any quantification it is difficult to determine if an accident that kills a few people with "medium likelihood" is worse than one that kills many people with "very low likelihood". In such cases, the Operator should determine that both risks are tolerable.