Webfram: higher tier refinement of drainflow exposure
Webfram is a web-based risk assessment tool that can be used as a higher tier refinement option. In refining the risk it uses the current risk assessment models but incorporates both uncertainty and variability. Webfram performs multiple simulations (sometimes millions) to produce outputs as probability distributions. The outputs are different from conventional risk assessments and therefore the aim of these information and the step-by-step guide is to outline how to carry out a Webfram assessment and what information to include in a regulatory submission. In addition it provides some guidance on how to interpret outputs and use them to make regulatory decisions. The information presented here may be updated as CRD's experience of utilising Webfram for making regulatory submissions increases.
Webfram was developed from a suite of defra funded Research and Development projects. It is one of a range of possible higher tier refinement steps available for UK pesticide product authorisations. It may also be useful in refining some UK biocide product authorisations for very specific use patterns (eg exposure of surface water following disposal of contaminated manure in PT3). All modules have been peer reviewed, validated, are freely available and user friendly. The website can be found at www.webfram.com . The tool has been available for use in regulatory submissions since 2009 and an updated version was released in 2014. The updated version has significantly greater functionality to aid refinement of drainflow exposure. The drainflow module is likely to be the most frequently encountered part of Webfram and is therefore the focus of this guidance. When addressing variability and uncertainty within the drainflow module, the key factors considered are as follows: timing of application, amount of spray intercepted by the crop, degradation and sorption properties, soil pH and organic matter content, field capacity period and clay content (as an indicator of vulnerability to preferential flow). The drainage module is based on current risk assessment methods and utilises a meta-model of MACRO v4.3. Links to the underlying R&D that supported the development of Webfram and that provide more background information on all of the modules are included at the end of the guidance and also at Research & development.
Role of Webfram in regulation
Webfram is only one of a range of options available and an Applicant is free to choose which option is the most appropriate for their assessment. Advantages over alternative approaches (eg stand alone higher tier assessments using MACRO – see Higher tier drainflow from MACRO) are that Webfram contains a set of standardized scenarios and outputs, where the user only selects the substance properties and GAP. The ability to quantitatively account for variability1 and uncertainty2 in a fully probabilistic manner is also an advantage over alternative deterministic models. However the incorporation of uncertainty and variability in environmental regulatory risk assessment is still relatively new and care should always be taken when using and interpreting Webfram outputs to ensure we maintain the same degree of consistency between assessments and maintain the appropriate level of protection or conservatism. Further guidance on this is provided in the sections below.
Since Webfram is only one of the options available it may be used on its own to refine a Tier 1 drainflow exposure assessment, or may also be provided alongside a standalone MACRO assessment. Based on limited current experience, there is no standard tiered approach to refining drainflow. What this means is that for some substances and use patterns, Webfram will provide a greater level of refinement (ie higher tier) than standalone MACRO, and in other cases Webfram will provide a lower level of refinement (ie lower tier). Since either approach is accepted, provided that the subsequent aquatic risk assessment is acceptable, the use of either approach would be accepted by CRD. It is not necessary to pass both methods. Based on limited experience, Webfram may give a greater level of refinement for moderately sorbed substances with relatively large regulatory datasets for DT50 and/or Koc. It may give a lower level of refinement for more mobile substances with standard data sets. This latter finding may be linked to the ability to quantify variability and uncertainty.
Basic model guide
Further Information on running the model can be obtained via the available links within the Webfram model itself, or clicking on the various 'here' or the '?' symbols throughout. Further detailed guidance on specific aspects of the model are provided in the additional sections below.
How to deal with uncertainty
Currently in regulatory risk assessment uncertainty is not dealt with explicitly as it is assumed to be incorporated in the uncertainty factors applied to the ecotoxicological effects endpoints. These do not account for any uncertainty in the exposure estimate. However, the output from Webfram highlights the degree of certainty via the use of confidence intervals, ie it gives an indication of how sure we are about any of the outputs. The key issue here is how should we deal with this issue?
It is proposed that the median of any chosen exposure value should be used in regulatory risk assessment. In addition it is proposed to not use the confidence intervals provided by the Webfram models directly in making a regulatory decision. However since the confidence intervals do provide useful information on the level of certainty in any particular assessment, it is proposed that these are always reported as part of a regulatory submission. These may provide useful supporting information when making a regulatory decision. For example, an exceedance of the Regulatory Acceptable Concentration (RAC) in one scenario of 25% with confidence intervals of 23 to 27% would be different to an exceedance of 25% with confidence intervals of 2 to 98%.
How to present and use in a regulatory submission
The following outlines what should be submitted and considered when a regulatory submission includes a Webfram based risk assessment.
Input data: It is necessary for Applicants to submit all the input data that has been used in the risk assessment. Details of the GAP assessed and how this relates to the label should be provided (ie confirmation that the modelled GAP represents a reasonable worst case use pattern as per the label). As regards the choice of application date, this should be justified in terms of the disease, weed or pest being treated and should generally reflect the most vulnerable timing of application with regards likely drainflow exposure. Where a wide application window may be expected based on the label recommendations, it may be necessary to run multiple simulations to cover both early and late applications. Confirmation of all other parameters (eg percentiles, number of iterations used in each calculation, whether any relationship between sorption, degradation and pH was assumed, relevant RAC etc.) should be provided automatically as part of the input data submission.
All relevant outputs used to make a regulatory decision should be submitted with any submission, along with underlying studies unless these have been previously evaluated (eg at EU level or as part of another UK product assessment in which case COP number references are required). The key output from the drainage model is most likely to be the details of the Exceedance Statistics for each scenario. This broadly matches the standard scenario years type approach detailed in higher tier drainflow from MACRO. Here it will be critical to report the individual level of exceedance for each scenario for comparison against regulatory triggers (for example, where the RAC is based on effects against primary producers a maximum of 60% exceedance for any single scenario must not be breached). The overall exceedance must also be less than 10%.
For risk assessments where the RAC is based on effects against fish or aquatic invertebrates the maximum 60% exceedance level for any single scenario is not appropriate. This is consistent with the approach to assessing risks to these groups in other forms of higher tier drainflow modelling eg using MACRO. However an alternative approach based on exceedances in Webfram modelling can be derived based on the individual 90th percentile PEC values for each scenario. Given that the risk from spray drift is regulated on the 90th percentile PECs, it is considered that the same approach can be used to assess the risk from drainflow exposure on these soils, ie consider the 90th percentile PECs against the RAC. It should be noted that Webfram calculates a single maximum PECsw value after the first drainflow event in each model iteration – so the distributions represent maximum values that effectively account for both the spatial and temporal aspects in an appropriately conservative manner. The use of a 90th percentile value for direct use in the risk assessment is therefore considered reasonable in this case. If the 90th percentile PEC in Webfram is less than the RAC, the risk may be considered acceptable. If the 90th percentile PEC in Webfram is above the RAC, it does not necessarily mean that the use is unacceptable. However additional modelling and analysis (eg using MACRO) may be necessary in order to determine the acceptability of the use.
In addition copies of the key exposure distributions or risk graphs may be useful for further interpretation of the outputs. One drawback of the Webfram approach is that it is not possible to provide a detailed assessment of exposure profiles (eg to consider duration of exposure events or intervals between exceedances and potential for recovery). However the advantage of the Webfram simulation in providing a quantitative estimate of uncertainty and variability is useful when considering the outputs in a regulatory decision making context.
Where the regulatory submission provided is insufficient to allow HSE CRD to validate all aspects of the submission, the missing information will be requested.
How to download input data
A summary of the input data can be obtained by going to 'Saved Models', locating the relevant model and right clicking on 'data' (see red circle), click on 'save target as' and save this as word document.
Once in word, open the file and it should look something like this:
Input data for Aquatic Model using an exposure distribution in surface water arising from drainflow.
Model Title: Test 3
Chemical Name Test 3
Toxicity data inputs.
Units: Micrograms/Liter (ug/L)
Toxicity Data: Species Name Data Label
18 Lemna EC50
Exposure Data via Drainage
Application Type - Page ID:104
Multiple Application Parameters
Crop Type: Potatoes
Crop Type and Growth Stage: potatoes BBCH 10-18
Application Rate: 30
Application Date: 15/04/2013
Application Interval: N/A
Crop Type and Growth Stage: potatoes BBCH 21-29
Application Rate: 30
Application Date: N/A
Application Interval: 15
Calculated outputs accepted
General Information - Page ID:72
Selected crop type and growth stage: potatoes BBCH 21-29
Application Rate: 30
Residual Mass at final application: 24.708
Application Date: 30/04/2013
Taxonomic Group: Plants
Modeling Options - Page ID:106
Sorption consideration: Assume instantaneous sorption equilibrium
Relationship between sorption, degradation and pH: None
Degradation Values - Page ID:73
Entry Number 1
Entry Number 2
Entry Number 3
Entry Number 4
Sorption Values - Page ID:74
Entry Number 1
Entry Number 2
Entry Number 3
Entry Number 4
Calculation Parameters - Page ID:52
Variability iterations: 100
Uncertainty iterations: 100
Percentile 1: 90
Confidence Interval 1: 95
Choice of crops and risk envelope
Within the current Webfram model the range of available crops is limited to some of the most important UK crops where refinement of first tier drainflow risks are most often required (eg winter and spring cereals, winter oilseed rape, potatoes, maize and peas). This list does not cover all of the crops for which at least a first tier drainflow risk assessment would be required. If there is an unresolved risk assessment issue with a crop which is not included in the pull down list, it may still be possible to use the Webfram model to try and refine the risk assessment. One option would be to select an appropriate surrogate crop. For example to model an early spring or late autumn application to grassland you could select spring or winter wheat as a surrogate crop. The Webfram model will automatically estimate crop interception depending on the growth stage selected on a later page. Although the model assesses variability around crop interception at time of application, this feature could be disabled when simulating a surrogate crop by selected a pre-emergence application and manually amending the application rate to take account of appropriate levels of crop interception. Note this would effectively disable the assessment of this specific source of variability and this should be considered when preparing the submission.
Another option to addressing risks from crops not covered by the standard list in Webfram would be to use the risk envelope Risk envelope suitability approach for GAPs covered by the model. For example, if you had a product that had uses on cereals and grassland at similar rates and timings, you may be able to perform a Webfram assessment of the cereal use only. Provided this demonstrates an acceptable risk, you may be able to use the risk envelope approach to argue that the risks arising from the grassland uses are effectively covered by the cereal risk envelope. It should also be noted that the simulation of crop rotations is not possible within Webfram. Within the drainage model of Webfram, all applications are made in the same season for a single crop. The model estimates exposure occurring from a single, maximum drainflow event within this single season. The single event is simulated multiple times, taking into account all the sources of variability and uncertainty included in the model to produce the distribution of exposures that are ultimately used in assessing the risk.
Multiple application GAPs
Note that when inputting data on the GAP, on the same page you can also select the period between application and drainflow for calculating the first tier risk assessment (used in the threshold approach). Since the threshold approach is no longer recommended for use in assessing the acceptability of the output of the Webfram model, this can be left as the default value of 0 days.
When simulating multiple application GAPs, because of the way that the Webfram model works by simulating a single maximum drainflow event in a single season, the inclusion of multiple applications adds a degree of complexity to the model simulations. The complexity arises from the need to ensure that the model appropriately estimates the maximum drainflow event from multiple application GAPs. Further background and guidance is provided below.
The option to automatically include multiple applications of a single substance within a season was added to Webfram as part of the further development of the drainage model. The model accounts for any carry-over of previously applied pesticide and adds the residual mass to the rate of the last application before calculating losses via drainage. Webfram calculates the mass of pesticide present just before the last application based on first-order degradation kinetics. The user must specify a single DT50 value (for example the geometric mean of all available DT50 values consistent with the selection of a FOCUS groundwater or surface water modelling input parameters) and a Q10 value. The updated model allows the user to select the appropriate Q10 value. This will normally be consistent with the value used to either normalise the DT50 values, or used in the FOCUSgw modelling (ie typically either 2.2 or 2.58) as per the original Annex I approval. The DT50 value is then corrected for the temperature between the applications. The user must also enter the rate of each application, the date of the first application and the interval between the applications. The crop growth stage at each application must be selected. Once the data is entered the Calc. Outputs option must be selected to calculate the aeric mass remaining before the final application.
An example of the mass of the pesticide simulated with Webfram for multiple applications is shown in Figure 1. Five applications are made with intervals of 21, 21, 42 and 42 days between them. Webfram calculates the aeric mass in soil up to the time just before the fifth application, accounting for interception depending on the crop growth stage. The residual mass is displayed on the screen (728 g/ha in this example). If the user accepts this value, then the residual mass is added to the rate of the last application (after correction for interception). A Webfram simulation of concentrations in surface water is then performed with the adjusted aeric mass.
Figure 1. Aeric mass of pesticide in soil simulated with Webfram for successive applications
In most cases it will be appropriate for the user to simply accept the output of the multiple application calculation by selected the 'Accept' option. However two specific situations are possible where it may not be appropriate to accept the output of this calculation. One exception is for very rapidly degrading substances, the other is for multiple applications where at least one application is made within the main drainflow period and the final application is made outside of the main drainflow period (ie after the period of soils at field capacity ends). These two situations are further discussed below.
Entry of the pesticide into the ditch via drainage is only simulated after the last application. This approach seems justified where all applications are within the field capacity period. In this situation, a larger mass of pesticide is present in soil after the last application due to carry-over, thus leading to greater losses in drainflow. An exception is for very fast degrading compounds. The mass of these pesticides may decline to low levels before the next application is made (depending on the application interval). The mass of rapidly degrading pesticides in soil can be smaller at the time of the last application than after previous applications due to an increase in crop interception. If this is the case, even for multiple applications it may be more appropriate to simulate a single application which will represent the worst case (ie the highest effective soil loading during the field capacity period).
A further important consideration is the fact that the simulation of drainflow losses only after the last application will not give conservative estimates where one or more of the early applications are within the field capacity period and the last application occurs after the field capacity period ends (ie when the final application is outside the main drainflow period). The onset of the drainflow event is set to a default of 3 days for applications within the field capacity period whereas there can be several months between applications just after the end of the field capacity period in spring and the onset of drainflow in the following autumn. The end dates of the field capacity period are sampled from distributions within Webfram. The latest possible end dates are 11 May, 31 May and 14 June for the dry, medium and wet weather scenarios, respectively. An alternative approach must be followed where at least one application is made before these dates and at least one application is made after these dates:
- Simulate concentrations in the ditch arising from losses via drainflow for the first application with the Webfram drainage module;
- Simulate the mass remaining in soil just before the second application using the multiple applications tool (set no of applications to 2);
- Add the residual mass to the rate for the second application and simulate concentrations in the ditch arising from losses via drainflow with the Webfram drainage module;
- Repeat this procedure for all subsequent applications in a step wise manner;
- Report the largest of the concentrations calculated for the various applications.
A further useful feature of the updated model is the option for users to test for relationships between sorption, degradation and soil pH. This feature is only possible in combination with the assumption of instantaneous sorption. Users of Webfram have the possibility to include one of four linear relationships between key variables - logDT50 and log Koc; log DT50 and pH ; log Koc and pH; and logDT50 and log Koc and log Koc and pH. The pH values in water should be used. The user must decide whether the relationship between the measured data is strong enough to justify inclusion in the modelling. WEBFRAM undertakes an analysis of variance and reports the p-value for the F-statistics. It is recommended that the F-test should be significant at the 5% level, ie p must be <0.05. Further information on this aspect of the model is provided via the link on page ID: 106. If no statistically significant relationship is found, or if the user knows that no relationship exists (for example as a result of the Annex I review or EFSA conclusion) the 'No relationship' option can be selected. Note this feature of the model could be used to assess the significance of pH dependence for other parts of the exposure assessment, or during an Annex I assessment of a new or existing substance.
Input data issues
It is necessary to enter the full range of DT50 values to allow the model to then sample from a log-normal distribution of values during each run. In general the full set of Annex I agreed endpoints should be used for Webfram simulations. It should be noted that the DT50 should be derived via single first order kinetics corrected to FOCUS reference moisture and temperature (ie 20°C and pF2). The Q10 can also be selected on this page and should be consistent with the value used to normalise the original degradation rates as per the Annex I agreed endpoints. If the substance has been shown to degrade via bi-phasic kinetics, the DT50 value entered should be conservative pseudo first order DT50 derived according to FOCUS kinetics (eg slow phase of DFOP or HS or FOMC DT90/3.32). The easiest way to decide which endpoints should be added here would be to include all values that were used in deriving the mean input parameter for standard FOCUS groundwater modelling. For example, if the FOCUS groundwater modelling input parameter was a geometric mean value derived from 8 soils, the DT50 values from each of the 8 soils used to the derive the geometric mean should be included in Webfram. Laboratory or field data can be used in line with other exposure assessment areas. A minimum of two unique DT50 values is required. In the event that only a single DT50 is available, the user should enter the same value as a duplicate entry. In order to get Webfram to accept this approach, two unique values must be entered either side of the actual single value. So if the single available DT50 if 15 d, the user should input 14.9 and 15.1 d for example. Users should be aware that entering duplicate values in this way will not give quite the same result as if a single value were entered and sampling from a distribution were disabled. Since addressing uncertainty around the substance parameters is an important feature of Webfram it is preferable to maintain this feature even for such limited datasets. The use of Webfram in such situations where limited datasets are available is still considered acceptable. This approach is considered defensible because for those substances where limited data sets were accepted for the purposes of Annex I approval, it is generally the case that these are assumed to be lower risk substances (eg plant extracts, List 4 type compounds etc.) where a partial data set was justified. However the applicability of this approach in cases where more standard substances have limited data sets should always be considered on a case by case basis.
Following on from the guidance on DT50 values, these data should be the Annex I agreed endpoints and all values used in deriving mean FOCUS groundwater modelling inputs should be included. Further details regarding Koc and 1/n can be found by clicking on either the '?' symbol or 'here' on the respective pages.
Combined risk assessments in Webfram
Approaches to combined exposure assessments using Webfram are still under development using the same basic principles as outlined in Risk envelope suitability for other 1st tier and higher tier approaches.
Webfram 1: Web-integrated framework for addressing uncertainty and variability in pesticide risk assessment – this is basically an overarching project that developed the statistical methodology and the website.
Webfram 2: Modelling the effects of pesticides on non-target aquatic organisms to improve risk assessment. This module developed landscape based scenarios that enables the risk to aquatic life from both drainflow and runoff to be assessed.
Webfram 3: Addressing uncertainty and variability in pesticide risks assessments for birds and mammals. This module dealt with developing risk assessment models for birds and mammals that followed current guidance as well as incorporating the issues of uncertainty and variability.
Webfram 4: Methods of addressing variability and uncertainty for improved pesticide risk assessments for non-target invertebrates. This module has developed a risk assessment that investigates the issues of uncertainty and variability for non-target arthropods.
Webfram 5: Development of a web-based pesticide risk assessment module for below-ground invertebrates. This module has developed a risk assessment that investigates the issues of uncertainty and variability for below ground soil organisms.
Webfram 6: Acceptability of pesticide effects on non-target species – this project explored the issue of acceptability.
Webfram 7: This project explored the issues associated with assessing the long-term risk to birds and mammals.
Due to the novel nature of the research associated with Webfram 4 and 5 it was considered that more work would be required before these were implemented in to regulatory risk assessment. Note that Webfram 6 and 7 were not aimed at producing Web-based models.
Experience gained with using the drainage model of Webfram identified several useful refinements that could significantly increase the flexibility and realism of model outputs. Further developments of the Webfram drainage model were therefore made under a separate project: See Full details of this project.
Note that since the initiation of the project in 2003 guidance in several of the risk assessment areas covered by the original Webfram tool have been further developed in other forums. This has resulted in some of the Webfram projects being no longer compatible with current 1st tier risk assessment methodologies. For example, significant developments have been made in the area of bird and mammal risk assessments, largely as a result of the work of EFSA. Webfram 3 was extended in a further project PS2362 which specifically considered the recommendations from the EFSA bird and mammal guidance (EFSA, 2009). This extension project concluded that the Webfram models were of similar conservatism to the EFSA Tier 1 assessment, which clearly limits their usefulness for conducting refined assessments: See Full details of this project.
Variability within Webfram is considered to be an inherent property of natural systems that cannot be reduced by further measurement (examples would be that sorption or degradation varies between soils or exposure varies in space and time).
Uncertainty within Webfram is, crudely, the sum of what we do not know (examples would be extrapolating sorption or degradation from a small number of soils to others, or sampling bias or measurement error).