This website uses non-intrusive cookies to improve your user experience. You can visit our cookie privacy page for more information.

United Kingdom Interdepartmental Liaison Group on Risk Assessment UK-ILGRA)

ILGRA SUB-GROUP
Report on Department of Environment Methodology for Setting of Safety Standards

ILGRA Logo

Contents

1. Introduction

1.1 Background
1.2 DoE Guidance and Standards
1.3 The Case Studies
1.4 Method of Analysis
1.5 Main Findings

2. ACRE and ACNFP Recommendations on Approving the Marketing of Genetically Modified Maize

2.1 Background
2.2 Divergent Value Judgements and Ignorance
2.3 The Precautionary Principle
2.4 Assessment of Benefits
2.5 Summary

3. ERM's Risk Assessment of Dioxin Releases from Municipal Waste Incineration Processes

3.1 Background
3.2 Divergent Value Judgements
3.3 Alternative Options
3.4 Uncertainty
3.5 Multi-attribute Analysis
3.6 Summary

4. EPAQ's Recommendation for National Standards for Ambient Benzene Concentrations

4.1 Background
4.2 The Role of Value Judgements
4.3 1ppb as an "indicative aim of policy" and the use of Costs and Benefits
4.4 The Role of Standards
4.5 The Role of Expert Groups

5. Conclusions and Recommendations

5.1 Genetically Modified Maize
5.2 Dioxins
5.3 Benzene
5.4 The DoE Guidance

Members of the Sub-Group

1. Introduction

1.1 Background

1.1.1 The ILGRA sub-group on the setting of safety standards was set up following a 1996 Treasury report, 'The Setting of Safety Standards'. That report looked at the economics of health and safety standards in general and did not assess any specific standards, except for illustration.

1.1.2 The ILGRA sub-group aimed to build on that report, by examining samples of the methodologies used to set departmental safety standards and the guidance and principles on which they are based.

1.1.3 This report sets out the findings and conclusions of three cases from the Department of the Environment. The work was a joint HMT/DOE exercise, with input also from the Cabinet Office Deregulation Unit, the Environment Agency, HSE, academia and industry.

1.2 DoE Guidance and Standards

1.2.1 In 1995, the Department of Environment published its 'Guide to Risk Assessment and Risk Management for Environmental Protection'. This guide was aimed mainly at policy makers and it sought to "explore some of the underlying principles of assessing environmental risks".

1.2.2 In examining the DoE Guidance, the sub-group had two main objectives: to consider the quality of the guidance, and to examine how it is applied in practice. This was done by assessing case studies, selected and conducted with the help and cooperation of DoE.

1.3 The Case Studies

1.3.1 Three case studies were considered. First a recent case was considered, where a multinational company, Ciba Geigy, had applied to market genetically modified maize in the EU. In the UK, MAFF advised that the presence of a gene for resistance to antibiotics could pose a serious threat to human health, which resulted in the UK objecting to the unrestricted marketing of the product in the EU.

1.3.2 Secondly, the risk from dioxin emissions from municipal incinerators was looked at, drawing on a 1996 report by Her Majesty's Inspectorate of Pollution (HMIP). This case was not concerned with setting a standard since one already existed, but it was informative because the HMIP study adhered closely to the DoE Guidance.

1.3.3 Finally, the standard recommended by the Expert Panel on Air Quality Standards (EPAQs) for concentration of benzene in ambient air was examined.

1.4 Method of Analysis

1.4.1 The sub-group comprised three external experts, plus officials from DoE, the Environment Agency, Cabinet Office, the Health and Safety Executive and HMT. The issues were examined through a review of both published and unpublished documentation and discussed by the sub-group at a series of meetings in the second half of 1996. The sub-group benefitted from being able to talk to officials involved in the actual decision making in the selected case studies.

1.5 Main Findings

1.5.1 The sub-group's key conclusions were:

the DoE Guidance has much to recommend it but greater effort should be made to promote wider usage;

continuing development is needed to tackle the difficulties involved in establishing methodologies and institutional arrangements to best take account of the available scientific data, uncertainties, people's preferences and costs of risk reduction;

particular emphasis should be placed on ensuring the assumptions and factors taken into account in setting or recommending safety standards are made transparent to policy makers and the public.

Back to top

2. ACRE and ACNFP recommendations on approving the marketing of Genetically Modified Maize

2.1 Background

2.1.1 This case study examined Ciba Giegy's application to market genetically modified maize in the EC. Ciba Giegy applied to the French Competent Authority who forwarded the proposal to all other Member States via the European Commission. Member States were given 60 days to put their view to the Commission on whether the application should be accepted or rejected, under the terms of the EC Directive on the Release of Genetically Modified Organisms (GMOs) into the environment.

2.1.2 The maize was genetically modified for resistance to the European Corn Borer which is a pest which significantly affects maize crops. The insect resistant gene implanted into the maize is derived from Bacillus thuringenises, a soil bacterium widely used for biological control of insect pests, and is toxic to corn borers which chew the plant tissue.

2.1.3 As part of the process, the maize had two other "marker" genes inserted, to make it possible for the plant breeder to identify the insect resistant maize plants. One gene was used to make it tolerant to a type of herbicide (glufosinate ammonium), for identification and selection during breeding in the field, and the other was for resistance to ampicillin antibiotics, to identify and select bacteria in the laboratory which contain the transformation plasmid. Under the terms of the Directive, the risk of damage to the environment or to human health had to be assessed for each gene.

2.1.4 In the UK, two advisory committees had a remit to provide advice on the proposal. The Advisory Committee on Releases to the Environment (ACRE) advises the Secretaries of State for Environment, Scotland, Wales and the Minister for Agriculture, Fisheries and Food on the risks posed by genetically modified organisms to the natural environment and human health (which includes risks to marketed products). In assessing Ciba Geigy genetically modified maize against that remit, ACRE concluded that there was no significant risk to human health or the environment through deliberate release of the organism into the environment.

2.1.5 In addition, as the product was to be used as food and animal feedingstuffs in the UK, clearance was also needed from MAFF's Advisory Committee on Novel Foods and Processes (ACNFP). This Committee has a narrower, more specialist role, advising MAFF on the safety of genetically modified organisms when contained in food or animal feed. Their members have particular expertise in genetic modification as it relates to novel foods. They advised there might be a problem resulting from the use of unprocessed maize as animal feed. This was because (i) the antibiotic resistance gene might be transferred to micro-organisms in the gut of animals fed the unprocessed material (considered by the Committee to be a small but finite possibility); (ii) if such a transfer took place the gene would become immediately active, because it was accompanied by a bacterial promoter; (iii) the gene could be reproduced far quicker than anything found in nature, and so would be expected deactivate a normal dose of antibiotic extremely quickly. The Committee also noted the importance of ampicillin antibiotics in treatment of animal and human disease.

2.1.6 MAFF, as the lead government Department on food safety, accepted the more cautious adivice of ACNFP. DoE, as the Competent Authority, accepted MAFF's decision and forwarded an objection to Brussels. Subsequently, the corresponding European expert committee examined all the evidence, including new evidence obtained by MAFF in response to ACNFP's advice, and concluded that the risks identified by ACNFP were too slight to warrant withholding consent from the product. Consequently Ciba Geigy are now permitted to market their maize in the EC but may face labelling requirements and other restrictions.

2.2 Divergent Value Judgements and Ignorance

2.2.1 A prominent feature of this case was the difference of views between the two expert groups, ACRE and ACNFP, on the basis of the same scientific evidence. It was agreed that the maize only posed a risk in terms of accelerating, rather than initiating, antibiotic resistance, but ACNFP did not share ACRE's confidence that the existing problem to human health would not be significantly aggravated. Whilst both Committees took the same approach to risk assessment, (although the DoE guidance relates only to environmental risks, whereas in this case the critical issue was a food safety one, hence ACNFP's involvement) ACNFP were more cautious, in an area where uncertainty is accepted, and placed a higher weighting on the risk of gene transfer. This example illustrates the necessary role of subjective judgements in interpreting scientific evidence, as both the estimation and evaluation stages often depend on judgements concerning the relative importance of different aspects of risk.

2.2.2 In addition, while ACRE and most other advisory bodies include representatives drawn from the scientific community and environmental interest groups and may even have one or two 'consumer members,' they do not necessarily reflect the opinions of the general public, who themselves hold a wide range of views.

2.2.3 Full consultation with the general public on every case is impractical, but more attention needs to be given to developing ways of reflecting public attitudes and values in risk assessment, particularly when placing subjective weights on reducing particular kinds of risk. All relevant information about applications to ACRE are already made available to the general public through a public register which is indexed on the Internet, which should offer the general public an opportunity to respond and let their views be known, should they make a point of monitoring what is going on in this field.

2.2.4 Subsequently DoE hosted a National Conference (March 1997) to stimulate a broader debate about the regulation of biotechnology.

2.2.4 All these features suggest that, where practical, consultation should be conducted on a wide basis, eliciting input from all interested / affected parties. This may have the disadvantage of lengthening the time taken by the approval procedure, and it does not get round the existence of divergent value judgements. But it is important that value judgments are systematically and transparently taken account of wherever they are used in the setting of safety standards. Wide consultation would also help ensure (though could not guarantee) that all relevant risks had been considered.

2.2.5 Better and more systematic consultation might have offered benefits in terms of refining and validating the assumptions used in analysis of the risks, and would also have provided an additional and potentially valuable opportunity for 'peer review' of the frame of reference imposed in the initial assessment by ACRE. Consultation can also be beneficial in that it generates public support for policies, as well as being a feature of open government. On the other hand, wider consultation has the disadvantage of imposing a delay on reaching the decision and in this case the need to comply with the time periods specified by the European Directive was an important constraint. The costs of extensive consultation would also need to be taken into account.

2.2.6 One possible method of dealing with diversity of opinion both within and between groups of experts and the general public is multi-attribute analysis, which is discussed under the dioxins case study.

2.3 The Precautionary Principle

2.3.1 The decision not to grant approval for marketing the maize was taken in a context of uncertainty surrounding the new technology and the final decision can be perceived as an illustration of a precautionary approach at work. The Precautionary Principle states that, when potential damage to the environment is uncertain but potentially significant, the government should be prepared to take precautionary action, even where scientific knowledge is not conclusive, if the likely balance of costs and benefits justifies it. The Precautionary Principle implies that had MAFF supported granting approval and DoE opposed, on the basis of environmental arguments, the government would still have adopted the same position.

2.3.2 Some members of the sub-group felt that the true meaning of the Precautionary Principle was unclear, and various possible interpretations were identified. The government published its definition of the Precautionary Principle in its sustainable development strategy. The DoE's "Guide to Risk Assessment and Risk Management for Environmental Protection" states the Precautionary Principle should be invoked "if the likely balance of costs and benefits justifies it". But the Precautionary Principle is mostly relevant in contexts where great uncertainty surrounds the nature, magnitude and /or probability of serious adverse consequences, i.e. exactly those situations where the balance between costs / risks and benefits is most difficult to judge.

2.3.3 A number of different interpretations in addition to those of the government were identified. Some saw the Precautionary Principle within a cost benefit framework as a reflection of the view that greater uncertainty requires a higher premium to be placed upon the benefits of proceeding cautiously. The appraisal of risky options should attempt both to identify the potential costs and benefits and assign some (often broad-brush) estimates to the magnitudes and probabilities involved, while at the same time giving as good an idea as possible of the degrees of uncertainty associated with these estimates. On this basis, decision makers should make explicit judgements (aided, perhaps, by sensitivity analyses) about the premia to be paid as "insurance" against uncertainty. The Precautionary Principle guides decisions towards caution in the face of uncertainty.

2.3.4 A minority of the sub-group (non-officials) interpreted the Precautionary Principle as standing outside the cost benefit framework. For example, it was argued that a precautionary approach should change the burden of proof in a risk analysis, requiring proof or strong evidence that an action was "safe" before undertaking it.

2.4 Assessment of Benefits

2.4.1 DoE Guidance recommends that decisions on permitting new technologies onto product markets should involve a rigorous assessment of the costs and benefits. ACRE, which focuses exclusively on risks, has no counterpart charged with considering the benefits of proposed innovations. The case of genetically modified maize appears to display a relatively thorough examination of the costs involved without giving equally sufficient emphasis to the benefits. This could in principle be remedied by requiring explicit consideration of benefits, either by another body, or by widening ACRE's remit and composition to "internalise" consideration of these issues. However, the Directive on Release of GMOs into the Environment requires that decisions on these issues be taken on grounds of safety, and makes no mention of other benefits, such as reduced loss of crops.

2.4.2 A more balanced judgement about the Ciba Geigy application would have taken account of at least two other things: first, how much benefit is expected to derive from resistance to the corn borer (a pest which destroys 7 percent of the annual maize crop), which might not be realised by some other means; and second, what costs would be involved in removing the ampicillin resistance gene prior to transforming the maize? DoE had access to both these sets of information, but could not employ them for decision making purposes under EC legislation.

2.5Summary

2.5.1 The assessment of genetically modified maize was not wholly in line with DoE guidance. Both ACRE and ANCFP assessed the issue in terms of the potential risk presented by the GMO without reference to the costs of regulating that risk to an acceptable level or, by implication, benefits presented by its use. In this particular instance DoE were constrained by the EU Directive covering such considerations. However, it does illustrate the need to ensure that, where possible, a balance is struck on considering both sides of the risk equation. This could be achieved through careful consideration of the remits of all bodies involved in such judgements.

2.5.2 Restructuring remits will not provide a complete answer because of the role that such remits play in focussing advisory groups on the particular areas of risk, in this specific instance ACRE on the wider environment, ANCFP on food. Therefore the assessment and advisory process would also be assisted by ensuring that value judgements made by such groups are clearly stated and that there is an opportunity for participation by the general public, particularly when a major issue, such as GMOs, is first considered.

Back to top

3. ERM's Risk Assessment of Dioxin Releases from Municipal Waste Incineration Processes

3.1 Background

3.1.1 The Environmental Resources Management (ERM) study, 'Risk Assessment of Dioxin Releases from Municipal Waste Incineration Processes', which was the focus of the dioxins case study, was not undertaken in order to determine a standard; instead, it was carried out with the aim of

3.1.2 The examination showed incinerators to be the most significant source of dioxin emissions; while many other sources were identified, such as coal combustion and asphalt mixing, MWIs contributed an average of 70 percent to total emissions from industrial sources. Whilst the source of dioxins is environmental, the main route of exposure to man is via the food chain.

3.1.3 The report, in most respects, closely follows the DoE guidance, which is itself sound.

3.2 Divergent Value Judgements

3.2.1 The DoE Guidance notes that both the 'estimation' and 'evaluation' aspects of risk assessment are contingent on value judgements concerning the relative importance of different aspects or 'dimensions' of risk. However, the ERM report does not address this aspect. The sub-group suggested that future guidance might give more explicit attention to techniques for eliciting, testing and exploring the implications of perspectives of different affected parties.

3.2.2 For example, the ERM report highlights the issue of the relatively high levels of dioxin exposure borne by breast fed infants, exceeding the Tolerable Daily Intake set by EU legislation. The question of what constitutes a 'fair' or 'acceptable' distribution of risk across groups is a fundamentally subjective issue. The degree of significance to attach to this particular result obtained in the ERM report is one issue where it would be informative to consider value judgements by interested parties, and on which formal guidance would be useful. Consultation may also be helpful in defining site-specific, population-specific, behavioural or operational factors for use in setting standards, such as the concept of the "hypothetical most exposed individual" in the report.

3.3 Alternative Options

3.3.1 A broader examination of dioxin issues could give explicit and systematic attention to the relative performance of alternative functional options. The ERM report was only intended to examine dioxin impacts, not reduction strategies, and although it took account of a range of possible plant unit sizes, the remit did not require consideration of other disposal options such as landfill and recycling. Such detailed examination, along with a systematic quantitative comparison of different abatement options (eg electrostatic precipitators and fabric filters) would normally fall to operators under a BATNEEC type regime rather than to the regulatory authorities. Examining more options in more detail obviously takes more time and cost, and is constrained by data availability. However, it is not clear that in this case the balance was closely examined.

3.4 Uncertainty

3.4.1 The ERM report contains a limited sensitivity and uncertainty analysis of the predictions of the methodology. The approach is probabilistic, rather than deterministic in character and the ERM team recommend further development and application of this type of probabilistic technique. The DoE Guidance does not specify recommended techniques for uncertainty or sensitivity analysis in any detail. It may be that attention might fruitfully be given to more explicit guidance on this important issue. There will be instances where a study of uncertainty is helpful, although this will depend on the judgement of those involved.

3.4.2 Probabilistic approaches are useful where probability distributions are at least roughly normal. They can however be misleading when probability distributions are far from normal as is the case for some important variables associated with waste incineration. The precision with which the results of this analysis are expressed is, in these circumstances, not a fair reflection of their accuracy.

3.4.3 However, other models exist which can handle non-standard distributions, and where input parameters are not necessarily assumed to be independent of each other. The latter form of Monte Carlo modelling is also "probabilistic", in that it uses replication to generate probability distributions for the output variables, but avoids the objections outlined above.

3.4.4 The deterministic approach also has the advantage of being simpler than probabilistic techniques.

3.5 Multi-attribute Analysis

3.5.1 The problems created by differing value judgements and insufficient information, discussed above, suggests there may be merits in examining a technique which allows an explicit combination of science and value judgements, such as multi-attribute analysis. This is a general approach to making choices between options, whether regulatory, financial, investment options etc. Multi-attribute approaches could have a role to play in environmental safety but are not currently included in the DoE Guidance.

3.5.2 The first main step in multi-attribute analysis is to identify options and their significant attributes. Then for each option in turn, each attribute receives a value/utility score. Each attribute is then assigned a weight according to its judged importance. By combining weights and scores, each option is given an overall weighted average of its attribute scores. Ranking options according to these weighted averages provides guidance about which is the most preferred of the possibilities.

3.5.3 In the specific case of the ERM report on dioxin releases from municipal waste incinerators, a multi-attribute framework might have assisted in eliciting and exploring value judgements relative importance of different pathways (e.g, meat, fish, vegetable, soil, water, breast milk or inhalation) or modes (e.g, voluntariness, controllability, intensity, intermittency) of exposure and different receptors (e.g, employees, neighbours, children, nursing mothers, unborn individuals, animal and plant life). Judgements might also usefully be clarified about the weight to be placed on the contrasting health effects of different toxic agents likely to be contained in emissions, as well as those of other comparable pollutants.

3.5.4 This technique allows for subjective aspects to be fully taken into account by a quantified mapping expressed in terms of sensitivities, and offers a potentially fruitful basis for a rigorous approach to wider participation in public decision making over risk. Multi-attribute analysis is not circumscribed and allows any dimension of impact to be considered. It demands that results are expressed in sensitivities, because the weightings employed are subjective. The key advantage claimed for this approach, compared with cost benefit analysis, is that it incorporates value judgements while still allowing these to be separated from the technical appraisal.

3.5.5 However, although sophisticated computer software exists to support multi-attribute analyses, the assumptions which underpin this technique are quite restrictive, as with conventional cost benefit analysis. In particular, it is usually assumed that attributes can be evaluated independently of each other, which, when checked, may be hard to defend. Moreover, there are often doubts about the properties of the scores assigned on each attribute, and whether it is really appropriate to combine these scores in a weighted average.

3.5.6 Finally, there remains the issue of whose scores and weights are to be considered. Often it is "experts" - e.g. scientists, engineers, economists, etc. - who are the participants in the time-consuming and demanding process of eliciting these numbers. However, there is now much evidence that the values, perceptions and judgements of experts of this kind may diverge markedly from those of members of the general public who, ultimately, are liable to form the bulk of the population exposed to any risks and who also incur the costs of any measures to reduce them. Once again, this draws attention to the desirability of developing methods for incorporating public values and perceptions into decision making.

3.6 Summary

3.6.1 The ERM report used and benefitted from the available DoE guidance. However, some refinements to the approach adopted might have been worthwhile: divergent value judgements could have been elicited through more consultation; alternative options could have been given a more rigorous assessment; and it may have been fruitful to have considered the merits of multi-attribute analysis, which helps overcome some conventional risk assessment problems.

Back to top

EPAQS Recommendation for National Standards for Ambient Benzene Concentrations

4.1 Background

4.1.1 The Expert Panel on Air Quality Standards (EPAQS) was set up by the Secretary of State for the Environment in 1991 following the undertaking, in the Environment White Paper "This Common Inheritance" to establish an expert panel to advise the Government on air quality standards.

4.1.2 The terms of reference for EPAQS are:

"to advise, as required, on the establishment and application of air quality standards in the UK, for purposes of developing policy on air pollution control and increasing public knowledge and understanding of air quality, taking account of the best available evidence of air pollution on human health and the wider environment, and of the progressive development of the air quality monitoring network."

4.1.3 EPAQS assess the scientific and medical literature and recommend levels of ambient air pollutants at which, in their view, effects on public health, including sensitive sub-groups, are very small or non-existent. This process is deliberately separate from any considerations of achievabilty of the relevant level or the costs and benefits of reaching it. It is then for Government to decide how best to apply this advice to air quality policy.

4.1.4 The Environment Act of 1995 required the Government to publish a strategy for air quality and a the National Air Quality Strategy (NAQS) was published in March 1997, following consultation. This took the advice of EPAQS (and, where relevant, recommendations the WHO) and government began the risk management process by proposing air quality objectives, informed by that advice and by setting out the extent to which current policies would achieve the objectives and the further policies which might be required. It is at this initial stage of the risk management process that questions arise of feasibility, the appropriate degree of compliance in terms of the proportion of time a given standard is to be achieved, and dates at which the objectives might be achieved. Where possible costs and benefits associated with the achievement of the objectives have been set out, although in most cases information on exposure-response relationships is not available and more work will be carried out in these areas before the review of the NAQS in 1999.

4.1.5 The initial risk assessments, leading to recommendations on levels at which risks to health are very small or non-existent are a matter for EPAQS. Subsequent steps in the process, including what is essentially the risk management, is not within EPAQS' remit but has been initiated by Government in the consultations and development of the National Air Quality Strategy.

4.1.6 For pollutants where a threshold concentration is identified below which effects are unlikely in healthy adults/subjects, EPAQS have used that threshold with the addition of a series of safety factors to recommend a standard. However, for pollutants such as benzene which are genotoxic carcinogens, no such threshold is currently identifiable. EPAQS therefore tried to derive a standard at which "the risk to public health would be exceedingly small". EPAQS published their recommendation on benzene in 1994:

"The Panel recommend an Air Quality Standard for benzene in the UK of 5 parts per billion as a running annual average. We also recommend that this Standard be reduced to the lower level of 1 part per billion running annual average, and that the Government set a target date by which this be achieved."

4.1.7 The starting point for deriving this standard was EPAQS' conclusion based on previous epidemiological studies that "the increased risk of leukaemia in cohorts of workers exposed to 500ppb of benzene over a working lifetime would be too small to detect in any feasible study". In order to take account of the difference between working lifetime and chronological life, the figure of 500ppb was then divided by 10. A further division by 10 was used to reflect the fact that the epidemiological studies had been carried out on "healthy workers" and did not necessarily include the full range of sensitive individuals such as those exposed to other causes of leukaemia, young children and those with impaired defence mechanisms. The safety factor of 10 was used because it was analogous to factors used in regulatory toxicology for non carcinogens, in the absence of any scientific evidence on inter-individual differences. Thus the recommended air quality standard for benzene was 5ppb, measured as a running annual average.

4.1.8 The National Air Quality Strategy distinguished between Standards and Objectives. Standards "are set purely with regard to scientific and medical evidence on the effects of the particular pollutant on health, or in the appropriate context, on the wider environment, as minimum or zero risk levels". Standards are to be used as the benchmarks or reference points for setting of air quality objectives and do not have a timetable for achievement. Air quality objectives are policy targets. In principle, they represent the Government's best judgment of air quality targets which are achievable and justified in terms of costs and benefits within a risk management framework. Specific objectives for a number of pollutants including benzene were set out in the draft National Air Quality Strategy. The air quality objectives are to be laid in regulations for the purpose of implementing the system of local air quality management throughout the UK.

4.1.9 The draft National Air Quality Strategy proposed the higher recommended standard of 5ppb as a specific objective to be achieved by 2005. This standard is expected to be achieved in this timescale under existing policies. The draft Strategy also proposed that the tighter standard of 1ppb should be treated as an "indicative aim of policy", but did not set a target date for achievement.

4.2 The Role of Value Judgments

4.2.1 Professional judgements by EPAQS were necessary at various stages in the process of deriving the standard for benzene. The setting of standards is thus never a "purely scientific" exercise, in so far as this implies a process free from judgement.

4.2.2 A factor of 10 was used by EPAQS to allow for sensitive individuals. This factor is commonly used in toxicology and the sub-group thought it reasonable that such allowance be made. Nevertheless, the identification of any particular value for this factor is ultimately a matter of judgement.

4.2.3 The standard of 5ppb for benzene would represent an exceedingly small risk to health, and for non sensitive individuals, ie the vast majority of people, this concentration would be less than a tenth of the level at which no perceptible effect could be observed. However, EPAQS went on to recommend that a tighter standard of 1ppb be set. The justification for this was that, because benzene is a genotoxic carcinogen (and therefore there is no threshold below which no effects are identifiable), "in principle, exposure ... should be kept as low as practicable." But it is unclear how this final dividing factor of five was derived, nor is it clear what "practicable" means in this context. It is not within EPAQS' remit to define "practicable". However, some members argued such a procedure was entirely consistent with a principal of 'precaution' and as an acknowledgement of divergent judgements, such as those concerning the protection of unborn individuals.

4.2.4 The different contexts of risk are an important factor which could influence the way in which risks are assessed and managed. They are likely to affect the perception and valuation of risk by individuals. In arriving at their benzene recommendation, EPAQS noted the estimated exposures associated with different sources of benzene, such as road traffic and smoking as points of reference, though these played no part in the derivation of the 5ppb standard. The National Air Quality Strategy states that standards are set "... purely with regard to scientific and medical evidence...". However, the appropriate balance to strike between (i) involuntary risk borne by the general public, (ii) informed occupational and (iii) voluntary public risks is not a 'pure scientific or medical' matter, and is not within EPAQS' remit. A systematic procedure for eliciting and exploring the perspectives of interested and affected parties (including the general public) might better address this latter aspect - in this case the perspectives of interested and affected parties (including the general public) were canvassed through the consultation of the National Air Quality Strategy.

4.3 1ppb as an "Indicative Aim of Policy" and the use of Costs and Benefits

4.3.1 The 5ppb standard was proposed as a specific objective in the consultation draft National Air Quality Strategy on the grounds that no further measures would be necessary to achieve it by 2005. However, the 1ppb standard (which would require further measures to fully achieve it) was proposed as an "indicative aim of policy", without any explicit assessment of the costs and benefits which might attach to achieving this aim. Some members of the sub group took the view that dividing by a factor of 5 to derive the 1ppb objective was arbitrary, and not justifiable without the costs and benefits of achieving that objective being established. Others felt that this is in principle a reasonable way of reflecting value judgements concerning the character of the risks involved, although it would be better to make such adjustments on a stronger empirical basis, were the data available.

4.3.2 In general, the costs of setting particular emissions standards are easier to estimate than the benefits which they achieve. The health benefits of changes in air quality standards cannot easily be quantified, let alone valued. However, any quantitative air quality objective will reflect either an ethical judgement about the maximum tolerable risk to which people may be exposed or, probably more often, it will imply some monetary valuation, or range of values, for the health benefits. Failure to be explicit about these ethical limits or valuations will lead to inconsistencies, and hence to lower standards of health than could otherwise be achieved with the same resources.

4.3.3 The Guide to Risk Assessment states (para 2.8): "It follows that explicitly or implicitly a price must be put on human life and the quality of life and that risk assessment must acknowledge and design to a specific probability for the preservation of human life." It is not entirely clear what the last part of that sentence means. Nor is any other central guidance offered on what 'price' levels for quality and length of life might be appropriate. Appendix C in the 1991 Policy Appraisal and the Environment booklet does have some (now rather outdated) discussion of some relevant techniques, but falls short of providing clear and comprehensive guidance.

4.3.4 On the costs side, estimates of compliance costs based on surveys of those companies likely to be affected by new requirements may be an overstatement of actual compliance costs. This is because when new regulations or other instruments are introduced, companies may innovate, producing new technologies which permit them to comply with tighter standards at lower costs. It would be desirable to incorporate such dynamic effects into any consideration of costs and benefits.

4.3.5 Further work is needed on valuing the health and safety benefits of improvements in air quality. This is an extremely difficult area, but work could build on the approach taken in the National Air Quality Strategy, which could usefully be further developed. Two issues need to be addressed. One is how to establish the circumstances in which trade-offs between health benefits and the costs of achieving them are in principle relevant. The second is how such trade-offs might be quantified in practice. Work is ongoing to explore the contextual factors which influence valuations and on the broad techniques used to elicit money values. Standards and objectives in the National Air Quality Strategy will be subject to review in 1999, by which time further work on the costs and benefits of achieving the objectives will have been undertaken. Further work might also be done to assess the effects of innovation on dynamic costs of compliance.

4.4 The Role of Standards

4.4.1 The sub group discussed the role of standards as proposed by EPAQS. It was noted:

(i) For some pollutants, there is no "critical threshold" level of exposure below which medical and scientific experts feel confident there is no risk to health. The sub-group noted that standards set at a level below which there is no detectable risk to health would vary according the scientific ability to detect the effects of very low exposures to particular pollutants. As science develops, the level at which the risk to health is "... unlikely to be detectable by any practicable method" will tend to fall. In setting an air quality objective, all costs and benefits need to be taken into account and subject to regular review, in order to make the process consistent with the DoE guidance on risk assessment and Policy Appraisal and the Environment.

(ii) Even where it is possible to define a threshold where there is minimum or zero risk to human health, this can be no more than a benchmark for policy making. The policy objective itself must be justified, in several ways, including cost-benefit terms and in terms of aggregate as well as individual detriment, in order to be consistent with the DoE guidance on risk assessment. So the main purpose of such a benchmark is to define a level of concentration of a particular pollutant below which there is judged to be no health benefit from further improvements in air quality. It is not necessarily crucial to the setting of policy objectives if sufficient information on the costs and benefits of achieving those objectives is available.

4.4.2 It might therefore be helpful for policy making if expert groups such as EPAQS were asked to provide explicit information on dose-response relationships, where available, which could feed directly into an assessment of costs and benefits of a range of air quality improvements. Such an assessment might then be used to inform or derive particular objectives in a transparent way, within a risk assessment framework.

4.4.3 An additional issue is the fact that public expectations are inevitably created by the publication of a recommended standard. The National Air Quality Strategy distinguishes the EPAQS' recommended standard from the policy objective, but once an expert group's recommendation is published, public pressure will be created for government to accept that standard - even if it is judged to be completely unjustified in cost benefit terms.

4.5 The Role of Expert Groups

4.5.1 The sub group concluded that EPAQS' role is not "purely scientific", since value judgments, which are not always easy to distinguish from professional judgements, are inevitably involved in recommending air quality standards, nor was it based on a risk assessment/cost-benefit approach. EPAQS' terms of reference may suggest more of a policy role than is consistent with DoE guidance, and some clarification of this might be useful.

4.5.2 Care should be taken to set the terms of reference of expert committees in such a way that it is clear that the role of the group is to draw on their particular expertise to provide information which will assist the decision makers to come to a decision using a risk assessment framework, taking into account all appropriate costs, benefits and value judgments.

4.5.3 An appropriate approach might be to ask expert groups to address the following questions explicitly:

a) Is there broad consensus that there is a level of exposure below which there is no reason to expect any significant adverse effects on any exposed individual? If so, what is this level (it might be zero)? What is known about the dose-response relationship above that level? What is known (or believed) about the impact of exposures in the workplace, in the atmosphere, for any particularly vulnerable types of people, and with what degree of consensus/confidence is such knowledge (or belief) held?

b) If there is no broad consensus about some or all of the above questions, what is the nature and extent of current uncertainty/disagreement? What kind/volume of evidence would be required in order to resolve particular uncertainties/disagreements? (eg use power calculations such as those epidemiologists use in deciding on target sample sizes in clinical trials).

4.5.4 In presenting the expert group's conclusions, care should be taken to clarify the status of any recommendations. It should be made clear that EPAQS do not consider economic factors but that such factors must be weighed in determining policy commitments.

4.5.5 Such an approach should help to make explicit any value judgments involved in an expert group's advice, and allow these value judgments to be discussed openly and assessed by policy makers if appropriate. In this latter task, policy makers would be greatly assisted by a greater empirical evidence on the weights (or range of weights) that the public place on the various dimensions of individual policy decisions.

Back to top

5. Conclusions and recommendations

5.1 Genetically Modified Maize

5.1.1 Divergent value judgements can lead to significantly different conclusions in setting safety standards, even where the standards are based on scientific evidence alone. This should be acknowledged and explicitly considered when taking decisions. This is applicable to all three case studies.

5.1.2 The Precautionary Principle, as presently drafted, is open to various interpretations: for example, some members perceived it as within a cost-benefit framework, reflecting a view that higher uncertainty warrants a higher premium on risk; others saw it as outside such a framework and perceived its role as being to change the burden of proof in a risk assessment.

5.2 Dioxins

5.2.1 More attentions should be paid to alternative options in risk assessment such as, in the dioxins case, recycling or landfill. Data may be limited but it could be worthwhile devoting effort to overcome these problems.

5.2.2 When uncertainty surrounds the available data, there may be advantages to using stochastic, rather than deterministic, techniques. These should be investigated further.

5.2.3 When choosing between potential options, multi-attribute analysis may be useful. Some regard this tool as being best able to cope with the combination of science and value judgements in the safety field.

5.3 Benzene

5.3.1 Ethical considerations are bound to have an important role in any standard setting process, even if based on purely scientific and medical grounds. There is a case for taking into account informed, though inevitably subjective public preferences.

5.3.2 The costs and benefits of the indicative policy aim for benzene should be evaluated in the first review of the National Air Quality Strategy, since public pressure will inevitably be created to achieve it as a result of its publication.

5.3.3 Estimates of the costs of complying with safety regulations should try to take into account the technological progress which may arise; firms, subject to the regulations, have an incentive to discover the cheapest method of compliance. This has already been identified by the NAQS' economist group as a potential area for further work. However, this is a complicated issue and estimating the likely effects of technical progress is not easy.

5.3.4 Expert groups' terms of reference should be framed in order to make clear that their role is to assist decision makers. It might be helpful if EPAQs were asked to provide clear information, where available, on the dose response relationships of particular air pollutants to feed into the assessment of costs and benefits used in setting policy objectives.

5.4 The DoE Guidance

5.4.1 The DoE Guidance has much to recommend it. While it could benefit from possible amendments (perhaps in the form of revised guidance or as new, accompanying material), high priority should be given to disseminating and promoting the use of the existing document. Although the document is only a guide and the procedures it recommends are not mandatory, standard setting should apply the principles described. Alternative approaches may often have merits but, when used, should be comprehensively justified.

5.4.2 Future guidance might give more explicit attention to the procedures and techniques for eliciting, testing and exploring the implications of the perspectives of the different interested and affected parties

Back to top

MEMBERS OF THE SUB-GROUP
Ms Nicola Baker (Cabinet Office) Professor Graham Loomes (University of Newcastle)
Mr Mark Beatson (HSE) Mr Peter Matheson (HM Treasury - Secretary)
Mr Gursh Bimbh (HM Treasury) Mr Richard Read (British Petroleum)
Ms Sally Burlington (HM Treasury) Dr David Slater (Environment Agency)
Mr Norman Glass (HM Treasury - Chairman) Dr Linda Smith (DoE) 
Dr Peter Hinchcliffe (DoE) Dr Andy Stirling (Science Policy Research Unit)
Dr Martin Hurst (DoE) Ms Caroline Turk (HM Treasury)

The following were not members but attended discussions:

Added to HSE website 31 August 2000

Updated 2011-05-04