Do location specific forecasts pose a new challenge for communicating uncertainty?

: In the last decade, the growth of local, site-specific weather forecasts delivered by mobile phone or website represents arguably the fastest change in forecast consumption since the beginning of television weather forecasts 60 years ago. In the present study, a street-interception survey of 274 members of the public a clear first preference for narrow weather forecasts above traditional broad weather forecasts is shown for the first time, with a clear bias towards this preference for users under 40 years. The impact of this change on the understanding of forecast probability and intensity information is explored. While the correct interpretation of the statement ‘There is a 30% chance of rain tomorrow’ is still low in the cohort, in common with previous studies, a clear impact of age and educational attainment on understanding is shown, with those under 40 and educated to degree level or above more likely to correctly interpret it. The interpretation of rainfall intensity descriptors (‘light’, ‘moderate’ and ‘heavy’) by the cohort is shown to be significantly different to official and expert assessment of the same descriptors and to have large variance amongst the cohort. However, despite these key uncertainties, members of the cohort generally seem to make appropriate decisions about rainfall forecasts. There is some evidence that the decisions made are different depending on the communication format used, and the cohort expressed a clear preference for tabular over graphical weather forecast presentation.


Introduction
The ways in which weather forecasts are delivered to the general public have undergone a significant change in the last 5 years. In common with other forms of content consumption there has been a shift from broadcast media such as radio, television and daily newspapers towards more flexible and personalized 'narrow-cast' consumption. In the narrow-cast style of communication, consumers expect and attain some control over the information they receive (Swatman et al., 2006;Hirst and Harrison, 2007). One example of this process, which is particularly relevant to weather forecasting, stems from the rapid market penetration of smartphones in the United Kingdom. By the middle of 2014, 75% of the population was expected to have access to a smartphone (http://www. marketingmagazine.co.uk/article/1216797/iab-engage-smartphone-penetration-reach-75-2014). Almost all new smartphones come packaged with a weather forecasting application and there is wide uptake of additional enhanced weather forecasting applications. For example, the Met Office launched apps for iPhone and Android phones in 2012, which have been subsequently downloaded more than 5.5 million times (http://www. metoffice.gov.uk/services/iphone).
Both smartphone and other web-based applications provide regularly updated and highly location-specific forecasts of weather variables. Many forecast providers give forecasts down to individual post-codes (typically around 0.15 km 2 ), representing a change not only in forecast specificity and availability, but also in resolution (although it is important to be clear that the forecast data used often have much coarser resolution). This study attempts to understand in detail if this change in forecast presentation poses a new challenge to forecasters who seek to communicate the uncertainty inherent in forecasts at this hyper-local scale. The focus is on forecasts of precipitation because this is the most studied variable in the literature on public understanding of forecast uncertainty and is particularly pertinent for consumers in the United Kingdom.
For some time there has been interest in understanding how well different methods of communicating uncertainty translate into operational weather forecasts (see, e.g. Murphy et al., 1980). Recent studies have used large surveys to develop a clearer picture of the communication process. End-users typically infer a background level of uncertainty in weather forecasts, even if this is not stated explicitly (Morss et al., 2008) and they have a good heuristic understanding of both the decreasing skill of weather forecasts with increased lead time and in the different levels of skill for different forecast variables (Morss et al., 2008;Joslyn and Savelli, 2010). However, when members of the public are asked to interpret probability of precipitation (PoP) forecasts, they are more likely than not to misinterpret the measure of uncertainty provided by the forecast 555 (Gigerenzer et al., 2005;Morss et al., 2008). It appears that people who do not correctly interpret PoP forecasts may tend to risk averse interpretations (Joslyn et al., 2009), although there is increasing evidence of a complex relationship between probabilistic forecasts and end-user decision making, which does not conform to simple cost-loss models (Morss et al., 2010). One under-explored aspect when considering end-user decision making under uncertainty is the extent to which descriptors of the likely intensity of precipitation are interpreted correctly by the general public, or if these influence decision making under uncertainty.
It is hypothesized that the rapid penetration of narrow-cast weather forecast information in the United Kingdom may have had an influence on the way the general public consume PoP forecasts. To test this hypothesis, a street-interception survey of the general public in and around Reading, UK, during July and August 2013 was performed. Two recent studies have also used similar methods to investigate public understanding of forecast uncertainty in a group of undergraduates at the University of Manchester (Peachey et al., 2013) and weather enthusiasts in the Republic of Ireland (O'Hanrahan and Sweeney, 2013). This study builds on these studies to provide a new view of forecast consumption for UK consumers. By using the street-interception method, it was possible to survey a broad demographic range of consumers (see Section 2 for further discussion).
The aim of the study is to answer the following questions: 1. do the UK public have the same level of understanding of PoP forecasts as reported in other studies? 2. do the UK public understand descriptive information about rainfall intensity? 3. when combined, how do these factors influence understanding and decision making? 4. how does the way in which forecast information is presented influence perception and decision making?
The study methods are presented in Section 2. Section 3.1 presents results showing that narrow-cast technology is a significant source of weather forecast information, particularly amongst those under 40. Section 3.2 then analyses understanding of PoP forecasts both in terms of probability and precipitation intensity in the study sample. Section 3.3 then shows how these factors influence forecast preference and decision making. Finally, Section 4 presents conclusions and suggestions for further work.

Participants and procedure
A total of 274 people responded to the questionnaire (144 females; 128 males; 2 participants did not record their gender). The participants had a mean age of 40.6 years (range 13-92 years). A total of 237 participants identified their nationality as British with 37 participants saying that they were from elsewhere.
The survey took place during July 2013 in two main locations: at a range of public events in and around Reading Town Centre and at an open day at the excavations at Silchester Roman Town an archaeological site open to the public, approximately 14 km southwest of Reading. Completion of the survey took 5-10 min and all participants provided informed consent and were told that they were free to withdraw at any time.

Study tasks and materials
The survey was split into six sections. The sections were as follows. Section 1 asked for basic demographic information (age, gender, nationality, educational level). In this section, four further questions on why participants usually consulted weather forecasts, what sources they used and with what frequency, which source they preferred and why were also included. Section 2 presented participants with verbal descriptions of a range of rainfall estimates that varied in their probability (40% chance of rain; 60% chance of rain) and in their intensity (no intensity information; light rain; moderate rain; heavy rain). On the basis of each of these estimates participants were asked to judge how likely they would be to change their plans to attend an outdoor event. Section 3 presented participants with a choice of times at which they could choose to carry out a particular outdoor activity. Each timeslot was presented alongside information about the likelihood of rain and its intensity. Likelihood information was given either as percentages (e.g. 20 and 60%) or as verbal probabilities (low, medium and high). Intensity information was presented either just verbally or with an additional visual cue (in terms of progressively more strongly hatched box indicators on a scale). The information was either presented to participants verbally in a tabular format or graphically using a scale layout (with probability scales arranged either horizontally or vertically). By design, none of these formats directly replicates presentations common in the UK media in order to avoid biases in user preferences towards a familiar presentation, but they do have much in common with typical ways in which forecast organizations present time-based forecasts (see, e.g. temperature ranges at http://www.metoffice.gov.uk/public/weather/forecast). Figure 1 shows the presentation formats used. For each choice, participants were asked to select which of the timeslots they would prefer. Section 4 presented participants with the four different presentation formats used in Section 3 (verbal, verbal numeric, graphic vertical and graphic horizontal) and asked them to choose the one they preferred. Section 5 assessed participants' understanding of verbal intensity levels used in weather forecasts. For each of light, moderate and heavy rain, participants were asked to provide assessments of the amount of rain that was likely to fall (in mm h −1 ), of the length of time that it would take for puddles to form (in minutes) and of the things they might see or experience with each type of rainfall. Section 6 asked participants to rate their current experience with weather forecasts. They rated their confidence in forecasts in general and their satisfaction with the forecasts that they currently receive. Participants were additionally asked if they had consulted a weather forecast the previous day, and, if so, what source they had used and to rate how accurate they believed the forecast had been. Finally, as part of this section, participants were asked a question to assess their understanding of uncertainty in rainfall forecasts based on that used by Gigerenzer et al. (2005). Participants were asked to select the correct interpretation of the phrase 'a 30% chance of rain' from a selection which included the correct interpretation (the 'days like this' interpretation), along with two common misunderstandings (the 'time' interpretation and the 'area' interpretation). Participants were also free to provide their own explanation.
Our aim across the different sections of the survey was to get a picture of participants' understanding of and preference for different kinds of uncertainty and intensity information.

Study design
Four variants of the questionnaire were prepared in order to balance question content across the different rainfall forecasts in Section 2 and to allow comparison of the tabular formats (verbal and verbal numeric) with the graphic (vertical and horizontal scales, respectively) in the selection of times for activities in Section 4. Participants were assigned at random to each of the four questionnaire variants.

How do users get forecast information?
Before discussing the ways in which end-users in this cohort understand and interpret uncertainty it is important to understand the means by which they access forecast information, the reasons why they make these choices and the trust they place in current forecast information. Table 1 shows the primary source used by respondents for gathering weather information. The clear majority (68%) prefer narrow-cast channels (website and mobile phone) to traditional broadcast media (television and radio). This is a considerable contrast to an analysis of a US sample in 2009 by Lazo et al. which showed that 90% (albeit of a wider population) rarely or never used electronic devices for weather information (Lazo et al., 2009). Splitting the cohort into two age categories shows that 75% of those whose primary source of information is mobile phone were under the age of 37 and no users in this group picked radio as their primary source. In contrast, three times as many users older than 40 as those of 40 or younger picked television as their primary source of weather information. The cohort was split at age 40 because this divided the study group almost equally in two.
To get a broader picture of forecast consumption, participants were also asked about frequency of use of a range of different sources. Comparing the behaviour of groups with an expressed preference for phone and web versus television and radio forecasts reveals that forecast use on phones is often supplemented by other sources such as television (>50% of the phone/web group still use television forecasts at least twice per week). In contrast, a clear majority of those preferring television (65%) rarely or never Table 1. Preferred source of weather forecast information for respondents (sample size: 265) split into respondents whose stated age is 40 or below and whose stated age is above 40.
Preferred source for weather forecast information use mobile phone forecast apps. This difference may present a challenge in the future for forecast providers to present a consistent and clear message across different forecast formats. Because of the contrast between age groups and apparent change in behaviour in the last 5 years it is important to try to understand how users choose their primary source. Figure 2 shows responses to question 1.9, which asked users to pick the most desirable aspects of weather forecasts for them. More than 50% of respondents cited ease of access as the most desirable aspect of forecasts. This is consistent with the apparent shift of use towards narrow-cast information, particularly that provided via mobile phone. Participants were asked for their levels of satisfaction with and confidence in current forecasts. A total of 79% of respondents indicated they were satisfied or very satisfied with the current forecasts they use but only 44% had very high or high confidence in the accuracy of their forecasts. This contrast is consistent with earlier studies (Morss et al., 2008(Morss et al., , 2010 which indicate that forecast users in the general public have a sophisticated appreciation of the limitations of weather forecasts and match their expectations of forecast performance to this. Both the level of satisfaction and level of confidence in forecasts were similar for respondents who expressed a preference for narrow forecasts and those who preferred broad forecasts.

How do users attribute probability in precipitation forecasts?
To compare the understanding of probabilistic forecast information for this cohort with previous groups, respondents were asked a standard question (about their interpretation of probabilistic information) that was used by several previous studies (Gigerenzer et al., 2005;Morss et al., 2008;Peachey et al., 2013): Imagine that the weather forecast predicts 'There is a 30% chance of rain tomorrow'. Please indicate which of the following is the most appropriate interpretation of the forecast? Figure 3. Interpretation of the statement 'There is a 30% chance of rain tomorrow' by respondents. Possible answers were 'It will rain in 30% of the region'; 'It will rain for 30% of the time'; 'It will rain on 30% of days like tomorrow'; 'I don't know' and 'other'. Responses are expressed as percentage of total number of answers (sample size: 271).
The correct interpretation of the statement is that this will occur on 30% of days like tomorrow. Figure 3 shows the frequency of the different interpretations given by respondents.
In common with previous studies, a majority of respondents did not interpret this statement correctly, and a substantial fraction answered 'other' (again in common with Morss et al., 2008;Joslyn et al., 2009;Peachey et al., 2013) which suggests widespread difficulty in interpreting probabilistic forecast statements. In this sample, interestingly, of the three categories (region, time and days) in which respondents could indicate that they understood what the statement meant, the correct interpretation (days) was the most common answer (27%) indicating some understanding of PoP forecasts. One important caveat here, which may also be true in other studies, is that a significant proportion of respondents answered 'other' and that often these respondents did potentially demonstrate some understanding of the PoP forecast by restating the question posed (see Table 5). Gigerenzer et al. (2005) hypothesized that increased familiarity with PoP forecasts improved the accuracy with which the public interpreted them (their study showed greater accuracy for respondents from New York compared with several European cities, where PoP forecasts are not commonly employed). In the United Kingdom, provision of PoP forecasts (as opposed to deterministic forecasts) is mixed, but increasingly PoP is provided in narrow-cast forecasting services, such as smartphone apps. To test if differences in forecast consumption might influence the accuracy with which users interpret PoP, the sample was segregated by several different criteria. Comparing interpretation of the PoP forecast by respondents who preferred narrow forecasts with those who preferred broad forecasts showed no significant difference in interpretation between the two groups (Table 2). This suggests that the relatively recent introduction of narrow weather forecasts in the United Kingdom has yet to influence people's comprehension of probabilistic information, although the variety of presentation techniques used by forecast providers shows that not all end-users will have seen probabilistic representations.
However, when the respondent group was split into two sub-groups based on age (above and below 40) there was a significant difference in their responses (using a chi-squared test, 558 S. Abraham et al.  Table 3), with those in the younger group more likely to give the correct response. Similarly, when the respondent group was split into those with degree-level education and above and those without, the sub-group with the higher level of educational qualification were also more likely to give the correct response (Table 4). These results suggest that there may be an effect of exposure to probabilistic information when users interpret PoP forecasts, but that this is likely related to exposure to thinking about uncertainty, generally, during their educational career. Given the large rise in the proportion of people going on to higher education in the United Kingdom after the age of 16 over the last 40 years (8.4% in 1970 and 33% in 2000, House of Commons library) the results in Tables 3 and 4 are unlikely to be independent.
Finally in this section, write-in responses for participants who selected 'Other' were explored (see Table 5). For this survey, it was found that the range of write-in responses was smaller than that of Morss et al. (2008) and similar to Peachey et al. (2013).
As in previous studies, almost all of these answers focussed on re-writing the probabilistic statement in some way rather than specifying what users thought the probability measure referred to.

How do users understand precipitation intensity descriptors?
In addition to information about likelihood of rain, forecasts also typically give information about expected intensity of precipitation. This is usually given in verbal descriptors such as 'light',  'moderate' and 'heavy' or equivalent graphical signifiers of these classes (e.g. the number of rain drops below a cloud). How users understand these intensity descriptors and, subsequently, how these two parts of the forecast (intensity and likelihood) influence decision making were investigated. Respondents were asked for their understanding of rainfall intensity descriptors in three different ways: as a numerical estimate of rainfall rate, the amount of time they would expect for puddles to form on road surfaces and a descriptive comparison of what they would expect to see. Because interpretation of verbal descriptors is inherently subjective, a second, small survey of seven academic experts in meteorology was performed at the University of Reading to compare their understanding with that of the general public. Current Met Office practice is to split rainfall verbal descriptors into drizzle, rainfall and rain showers classes and then use additional descriptors ('slight', 'moderate' and 'heavy') within each class. Numerical values for these classes are assigned as follows. For rain (other than in showers), 'slight' is <0.5 mm h −1 , 'moderate' is 0.5-4 mm h −1 and 'heavy' is <4 mm h −1 . For rain showers, 'slight' is <2 mm h −1 , 'moderate' is 2-10 mm h −1 and 'heavy' is <10-50 mm h −1 . An additional class (violent) is used for showers but is not discussed further here. Figure 4 shows mean estimates and 95% confidence intervals for numerical estimates of rainfall intensity and the time for puddles to form for the expert and the general public cohort.  . Mean estimates of rainfall intensity (mm h −1 ) and time (minutes) for puddles to form for three rainfall descriptors for general public (light grey) and expert (dark grey) groups (95% confidence interval for each mean estimate is shown by the error bars). Sample sizes, general public group for rainfall amount in millimetres (light = 218, moderate = 216, heavy = 217) and time for puddles to form (light = 241, moderate = 250, heavy = 250).  (90)  245 Raw numbers of responses are shown in brackets. There is a highly significant difference between light and moderate rainfall (df = 3, X 2 = 33.4, p = 0.000) and between light and heavy rainfall (df = 3, X 2 = 15.31, p = 0.001) and a marginally significant difference between moderate and heavy rainfall (df = 3, X 2 = 7.73, p = 0.052).
There is a significant difference between experts and the general public in their estimates of both rainfall rate and the time taken for puddles to form in all rain rate categories and in the moderate and heavy time for puddles to form categories (see Table 6). For rainfall rate, the public generally estimate much higher rainfall rates than experts, with the expert estimates consistent with the official definitions for rain showers. Importantly, the standard deviation of estimates from the general public was higher than that of the experts for the light and moderate categories, which indicates wide variance in understanding of rainfall rates, although this is true for both experts and the public for heavy rainfall. Similarly for estimates of the time taken for puddles to form, there is a wide variation amongst members of the public and experts for light rainfall. There were also a large proportion of the public survey returns that did not make any estimate for these categories (∼20% for estimates of rainfall rate in mm h −1 and ∼10% for estimates of the time for puddles to form), which indicates a general lack of confidence in the understanding of rain rates among the public cohort. The sign of the difference between the expert and the public cohort for estimates of time for puddles to form and the estimate of rainfall rate for the moderate and heavy classes is different, with the public estimating much longer times for puddles to form than experts.
To further explore how members of the public interpret rainfall intensity descriptors, the survey asked respondents to describe in words what they might experience or see for light, moderate and heavy rain. These responses are categorized into those that indicate a behavioural change (example answer: 'Wouldn't want to be out in it'), those describing an impact on the person (example answer: 'Sodden clothes'), an impact on the environment (example answer: 'Lots of puddles and mud') and direct observation of characteristics of the precipitation (example answer: 'Bigger droplets'). As shown in Table 6, the results of this analysis indicate significant differences between the types of descriptor used for the three different rainfall intensity classes. The majority of respondents describe light rainfall in terms of their direct observations of the rainfall (often commenting on changes to visibility or the sound of the rainfall in addition to discussion of the size and frequency of rain drops). When describing moderate and heavy rainfall there is a significant shift towards choosing descriptors based on the impact on the person or the environment. This result further highlights the challenge faced by weather forecasters and forecasting organizations in communicating rainfall intensity to end-users because of both, the broad range of interpretation of rainfall intensity descriptors and the difference in the interpretation of light, moderate and heavy rainfall.
In summary, there is widespread uncertainty amongst our general public cohort in understanding both probability and intensity descriptors commonly used for precipitation forecasts. Although this is the first large-scale survey of the UK public to assess these two factors for forecast communication with the same group, it appears that this group has a similar understanding of the probabilistic part of PoP as other groups in previous studies, which suggests that the results in the following section should be broadly relevant to the forecast communication problem. The next section seeks to understand how, given this general lack of understanding, members of the cohort express preferences for the display of complex precipitation forecast information.

How do people combine probability and intensity information in decision making?
Section 2 of the survey investigated how the ways in which the combination of intensity and probability information is communicated to the public influences the ways in which they make decisions about how to act upon weather forecasts.  Although not significantly different (chi-squared test, df = 4, N = 273, X 2 = 8.51, p =0.075) there is a suggestion of a different pattern of results for the two formats. Mean likelihood to change plans is calculated as the product of the probability of precipitation and the mean likelihood to change plans for all respondents.
Although respondents were asked to try to quantify the extent to which they would change their plans, it was recognized that this expressed likelihood may not be well quantified by respondents, unlike the studies of Roulston et al. (2006) and Morss et al. (2010), which ask respondents to make monetary decision based on probabilistic forecasts. Respondents were asked to quantify the likelihood of their changing plans in a range of scenarios with different PoP forecasts, either 40 or 60%, and different intensity descriptors (light, moderate and heavy). A control case was also included, where no intensity information was given. By combining results from the four different survey variants, the influence of probability and intensity on decision making irrespective of the given decision-making scenario was quantified. Figure 5(a) shows the mean likelihood that respondents would change their plans for all forecasts issued with each of the four different intensity descriptors. For light, moderate and heavy rain these mean likelihoods are well separated and show a difference of ∼25% likelihood between light and heavy rain. The relatively low likelihoods for all intensity descriptors are also interesting and likely to be strongly dependent on local climatic conditions. In the control condition, where no intensity information was given, participants interpreted the forecast similarly to the 'moderate' or 'heavy' intensity conditions. By comparing the mean likelihood to change plans for each intensity descriptor for cases with PoP of 40 and 60%, the extent to which the increased probability influences behaviour ( Figure 5(b)) could also be quantified. The mean likelihood to change plans is similar for the 'moderate' and 'heavy' descriptors (around 10%) and in the control case. However, the expressed likelihood to change plans for the 'light' case is much smaller and not distinguishable from zero. This is interesting because it suggests, as in the results in Table 6, that end-users consider 'light' rain differently to 'moderate' or 'heavy' rain.
Taken together, the two results in Figure 5 suggest that end-users in this localized sample are relatively insensitive to precipitation forecasts, because it is only when there is a 60% probability of heavy rain that they are more likely than not to change their plans and, 'light' rain aside, an increase in probability of rainfall makes only a small difference to their likelihood to change plans.

Do people make different decisions based on the format of the presentation?
In Section 3 of the survey, participants were asked to make judgements about how they would change their behaviour if forecasts for precipitation probability and intensity over an 8 h period were presented to them in a range of different formats. Subsequently, participants were asked to express a preference for one of the different formats.
Before presenting the results of how users make decisions based on each different forecast presentation, the context to this result is given by discussing user preferences for each forecast. It should be noted that in the study, users were asked for their preferred forecast, followed by questions about decision making and that users had the opportunity to use all four different forecast presentation types.
Amongst this cohort, there was a clear preference for tabular methods of presentation (86% of respondents preferred tabular formats), and furthermore a preference for the verbal-only (non-numeric) format (50% of end-users preferred this format). There is also a marginally significant difference (chi-squared test, df = 3, X 2 = 7.19, p-value = 0.066) in preference between the cohort who expressed a preference for narrow-cast methods and those who expressed a preference for broadcast methods. Although in both groups a clear majority preferred the tabular (verbal and combined verbal numeric) format (84% narrow, 91% broad), there was a much smaller preference for the verbal format amongst the narrow-cast group (44% narrow, 59% broad) There is a highly significant difference between the two distributions based on a chi-squared test (df = 4, N =273, X 2 = 16.92, p = 0.002). Mean likelihood to change plans is calculated as the product of the probability of precipitation and the mean likelihood to change plans for all respondents. and consequent increases in preference for the verbal numeric and graphic horizontal formats. As with the interpretation of a probability statement, discussed in Section 3.2.1, this suggests an influence of past experience on people's preferences. Given the complexity of the task involved in communicating both forecast probability and intensity, the lack of understanding of both amongst the general public and their preferences for different presentation formats, it is likely that the method of presentation of forecast information may change the way in which people make decisions. To test this idea, participants were asked to indicate their preferred time to complete a range of activities based on forecasts communicated using each of the four different forecast presentation formats described above. To test the differences between verbal and graphic presentations, participants were provided with identical forecasts issued using the verbal and graphic vertical methods (Q3.1 and Q3.4) and the verbal/numeric and graphic horizontal methods (Q3.3 and Q3.2).
To quantify the perceived likelihood of changing plans of respondents for the 5 forecast hours shown to participants, the results from Figure 5 are used.
For each intensity descriptor, Figure 5 shows an estimate for the mean likelihood of survey participants to change their plans ( Figure 5(a)) and an estimate of how a 20% change in forecast probability (from 40 to 60%) would change this likelihood ( Figure 5(b)). Using simple linear extrapolation, how the respondents would react to a forecast with any probability for each intensity could then be estimated. As an example, for an 80% probability of heavy precipitation, the mean estimate of likelihood to change plans (49%) was taken. A factor is then added on taking into account the 80% probability. This factor is the 30% difference between 80 and 50% (0.3) multiplied by the difference between the mean likelihood to change plans for 40 and 60% probability (0.13) divided by the 20% difference between the two categories (0.2). This gives a difference of 19% and a final estimate of the likelihood to change plans of 68%.
The comparison between the graphic horizontal and verbal/numeric presentations is shown in Table 7. There is little difference in the responses to the two presentations. However, for both presentations there is a clear bias towards the two categories with lowest likelihood to change plans (20% moderate and 40% light), suggesting that, in both cases, respondents are making appropriate decisions based on the information presented.
The comparison between the verbal and graphic vertical formats is shown in Table 8. In this comparison, there is a significant difference between responses for the two different presentations. Differences between responses to the two formats are found in the number of respondents who chose the 60% light category and the 20% heavy category, with the 60% light category more frequently chosen when presented in the verbal format and the 20% heavy category more frequently chosen when presented in the graphic format. This is an interesting result, because it suggests that the verbal presentation format emphasizes the intensity descriptor and the graphic presentation format emphasizes the PoP for the end-user. It is not clear why this effect should have been stronger in the verbal versus graphic vertical comparison than in the verbal/numeric versus graphic horizontal condition and further work is needed to confirm it, although one might speculate that the histogram-style appearance of the graphic vertical condition distracted participants from the intensity descriptor or that the combination of numeric and verbal format was more demanding for participants to process than the verbal-only format. Given that the 60% light, 20% heavy and 20% moderate categories have similar mean likelihood to change plans, it again seems that the large majority of respondents are able to use the information presented in both formats effectively.

Conclusions
The aim of our study was to investigate the ways in which members of the public in the United Kingdom understand uncertainty in weather forecasts. An important change in the way the general public consume weather forecast information was discovered, with a clear first preference for those under 40 for mobile phone and web-based forecasts. For all respondents, ease of access was by far the most important feature when choosing a forecast method and so it seems obvious that, in particular, smartphones will likely be a dominant route to forecast information. A further finding was that the groups who expressed a preference for narrow versus broadcast methods used television forecasts as much as those who expressed a preference for broadcasts. This suggests that the use of narrow-cast methods is as a supplement to traditional methods of forecast communication and that, in general, smartphone and web forecasts have increased the amount of weather forecast information end users collect.
Given this increased access to forecast information, whether this has influenced the way in which end-users consume forecast information has to be understood. In common with studies beginning with Gigerenzer et al. (2005), a general misinterpretation of probability of precipitation (PoP) forecasts was found, with most end-users unable to state correctly that the probability refers to the likelihood of a rain on days like the current day. There was no difference in this misconception between groups who expressed a preference for narrow-cast or broadcast methods. However, there was a significant influence of age and educational background on the ability of respondents to correctly interpret a PoP forecast with those under 40 and with education at degree level or above more likely to interpret it correctly. It is not known if other studies that have shown this influence exist, and it is important to see if this effect is replicated in other samples. For the future, this suggests that the general trend towards providing forecast uncertainty in the form of probability is likely to be beneficial to an increasing part of the population.
In addition to testing the ability of the general public to interpret the probabilities in precipitation forecasts their ability to interpret rainfall intensity information was also tested. In comparison with both the official definitions of rainfall intensity descriptors and the assessment of meteorological experts, the general public overestimate the intensity of rainfall described by 'light', 'moderate' and 'heavy' intensity descriptors. There is large standard deviation amongst the group which is also reflected in the wide variety of write-in descriptors provided via survey participants. The challenge of communicating rainfall intensity is reflected in the variation of the type of descriptor used for 'light', 'moderate' and 'heavy' rainfall which shifts from physical characteristics of the precipitation for the 'light' category towards descriptors based on the impact on the person or the environment for 'moderate' and 'heavy' descriptors.
Given these two significant uncertainties, how this survey group made decisions based on probabilistic forecasts with different intensity was then studied. There is a clear difference between the mean likelihood to take action for different rainfall intensity categories and in general a low likelihood to change plans based on rainfall forecasts in the survey group.
Using the mean likelihood to change plans expressed by the survey group for different intensity descriptors and probabilities of precipitation, the interpretation of different rainfall forecast presentation formats was then tested. In general, for all forecast presentations the survey group interpreted the forecasts and made decisions which were consistent with their previously expressed likelihood to change plans for test forecasts. This gives confidence that, despite the uncertainties in the interpretation of both probability and intensity information amongst the survey group, they are still able to extract key information from probabilistic forecasts. While the ways in which individuals respond to uncertain forecasts is likely to be complex and personal (Morss et al., 2010), for forecast providers this suggests that there is value in providing probability and intensity information to end-users. The study showed a user preference for verbal formats for presenting probabilities and intensities over graphic formats and, at least within the context of this questionnaire, better performance with verbal and numeric formats than graphic formats. The likely shift towards narrow-cast information provides exciting opportunities to present probability and intensity information with increasing specificity across location and time, and also, as is already the case in some forecasting apps, for consumers to choose the format of presentation which suits them best.
Further reassurance for forecast providers comes from the general high level of satisfaction with weather forecasts reported in this study, despite only 44% of respondents suggesting they had high or very high confidence in forecasts. This suggests, as other studies have found (Morss et al., 2008;Joslyn and Savelli, 2010), that this survey group has a realistic, heuristic interpretation of the weather forecasting problem and its inherent uncertainty. This relationship has also been found in other fields (Fischhoff, 1995;Epstein et al., 2004;Gardner et al., 2011) and importantly does not mean that accuracy is unimportant to the public. In this study, the Spearman rank correlation between expressions of forecast confidence and forecasts satisfaction is 62% (highly significant, S = 1252966, p-value = 0.0000). Therefore, any increase in forecast accuracy and resulting confidence in forecasts amongst the general public is likely to lead to increased satisfaction in forecasts. For future studies, it would be interesting to test the relationship between the skill of forecasts and its influence on forecast confidence amongst the general public. The large diversity of forecast providers in, for example, the mobile phone app market also makes exploration of any link between the skill of the forecast provided and how people interpret and respond to the forecast an important topic.