Radical
Statistics
Stats Watch - Monitoring press comment on official statistics
John Martyn
Introduction
The Statistics Users' Council for many years provided its members with first-hand knowledge of the vicissitudes and cutbacks affecting government statistics as well as information on developments and improvements. This activity confirmed the view that statistics are an important public good. This view prompted the launch of Stats Watch in 1995, which recognised the immense importance of the national press and the value of monitoring its coverage of official statistics. Monthly Stats Watch bulletins drew attention to and reproduced newspaper cuttings which were judged to be of particular interest to those users who sought to improve the quality and availability of official statistics in the UK. This monthly monitoring continued for over three years, having begun thanks to a grant from the Joseph Rowntree Charitable Trust linked with administrative back-up from Roehampton Institute. The initiative was from the beginning supported by some leading users of statistics - Ian Maclean, Chairman of the Statistics Users' Council, Professor Roger Jowell of the London School of Economics and Social and Community Planning Research (SCPR) and Bob Worcester of MORI in particular. Other supporters included the Market Research Society, Radical Statistics, the Social Research Association, the Campaign for Freedom of Information, the London Research Centre and the Trade Union Council. The Office for National Statistics also gave assistance. The data contained in the monthly bulletins, including the supporting cuttings, were then mailed to some 40 subscribers over a period of 31/2 years. They were not systematically analysed until this was commissioned by the Royal Statistical Society (RSS) earlier this year as part of a project to improve the reporting of statistics in the media.
For practical reasons it was decided to focus Stats Watch on official statistics, using measures of press coverage such as the number of papers reporting, and column inches devoted to an item, to assist selection for their inclusion in the bulletins. National papers were concentrated on because their reporters and commentators are generally first and foremost in drawing attention to government abuses and questionable aspects affecting the quality of official statistics. This is as it should be because they have a daily audience of millions, compared with the thousands who may eventually be reached by learned journals. Moreover, news in daily papers has an impact and immediacy which government and other would-be opinion-formers well recognise.
At the outset it was hoped to monitor radio broadcasts, though this proved impractical. Next came consideration of which papers were to be monitored on a daily basis and, here too, there were cost constraints along with consideration of the time needed to scan, read and abstract relevant material from the selected newspapers. In the event, using estimates of national daily and Sunday papers1 and a knowledge of the type of material appearing in each, five daily and four Sunday papers were selected. All were broadsheets. The dailies were The Times, The Daily Telegraph, The Guardian, The Financial Times and The Independent, whilst on Sundays, The Sunday Times, The Sunday Telegraph, The Observer and The Independent on Sunday were examined. To these were added The Evening Standard and The Economist. The former, whilst not a national paper, has an important and influential readership in London and the South East and is often the first newspaper to carry news from important press releases issued early in the day. It thereby provides early warning and alerts interested readers to the publication of further reports on the same topic in the following day's broadsheets.
The findings in the following section are taken from a report entitled Coverage and presentation of statistics in the national press prepared for the RSS. It is hoped that a fuller account based on this report will be published in the Society's journal later this year. In this short paper some of the findings are presented with commentary.
Selected findings
Over July 1995 - June 1998 inclusive some 625 items had been listed and details entered in the monthly Stats Watch bulletins. The main subjects mentioned were: the economy (184 mentions), health (128), unemployment (58) and education (42), followed by market research (36), crime (27), government (27) and social security (26).
The type of official statistics which feature most prominently in the daily papers are thus those which relate to the economy and business. With the notable exception of labour market statistics, their publication has seldom been attended by much criticism of the statistics themselves. This is rarely the case with other subject areas such as crime, health and education. Reports on such statistics featured regularly in the monthly Stats Watch bulletins and often contained more politically-charged comments. This is not surprising when it is remembered that health, for example, includes smoking, hospital waiting lists, BSE and CJD and the Gulf War syndrome.
Classification of the comments made in press reports by type of comment puts the importance of the statistics themselves being reported on at the top of the list, followed by calls for their improvement and the drawing of attention to questionable statistical procedures. The importance of the latter is borne out by the exceptionally large amount of coverage and comment that was devoted to unemployment statistics over the period covered.
The headlines used by papers to capture attention for statistical stories are revealing. The word 'figures' was used about twice as often as 'statistics'. The latter word had some uncomfortable associations, being linked with 'weapons', 'analysis' and 'official'. 'Statistical' fared little better, being used in conjunction with 'tricks of the trade', 'U-turns' and 'magic'. Some of the most attention-arresting headlines were used by The Financial Times and The Economist, and one interpretation is that the word 'statistics' is not readily associated by the press with authoritative figures and pronouncements.
The press coverage of selected items was also used to select 168 more noteworthy items from the 625. Using the number of broadsheets reporting as a measure of coverage, international competitiveness and the 'true jobless figures' came top, both being reported on in six broadsheets. The first of these was based on the World Competitiveness report, produced by the World Economic Forum (WEF) in Geneva.
The second item securing high press coverage (also 6 papers) was the 'true jobless figure'. Here a report by HSBC, the parent company of the Midland Bank, estimated in April 1997 that the true unemployment figure was four million, rather than the official 1.7 million at that time. Five broadsheets reported on the CSO task force on the measurement of unemployment and the same number on the RSS report on school league tables, the SCAA report on school tests for 11 year olds, the suspension of the General Household Survey and an increase in hospital waiting lists.
A second, even cruder measure of press coverage, is column inches, based on 'running the rule' over the press cuttings attached to the Stats Watch bulletins. On this basis the HSBC report on the (then) latest unemployment figures came top with 112 column inches. Next, the following month, unemployment came top again with 75 column inches, then 'inflation' with 69 column inches accompanied by the headline 'EU cuts inflation at the stroke of a statistician's pen'. Press comment discussed so-called 'harmonisation' and its implications. The European Union's Harmonised Index of Consumer Prices (HICP) is lower than the Retail Price Index because it excludes certain housing costs and includes other measures not in the RPI. Were it to be used as an index for welfare benefits it would have huge implications for their level.
Inevitably, most of the value of press comment on official statistics is lost when attention is limited to the subject and the headline. This is particularly the case in the few instances which discuss the inherent variability in statistical results. In this connection attention is drawn to Philip Howard's article in The Times ('They may average out right, but some statistics seem less than vital' 5/4/96), and then to an important contribution in The Economist entitled 'Damned lies. Economic statistics are in a bad way' (23/11/96). In the latter, there is reference to the 'shifting nature of economic activity making some statistical measures irrelevant' and there are some pertinent remarks about 'official number-crunchers being ultra-cautious, by training and temperament'.
Disputed figures (claim and counter-claim) were examined separately. Unsurprisingly, they peaked in the run-up to the 1997 general election. In early 1996, under the headline 'Parties wield statistical weapons in economic battle', Janet Bush in The Times (5/1/96) reported that 'the Government and Labour had locked horns in what both believe will be the critical battle of the election campaign: the economy'. Later Michael White in The Guardian (31/1/96) was to write 'PM's policy claims fail lies test' challenging his statements that crime was falling, that university funding was rising, and that the UK's economic performance was outstanding. Very usefully too, the Institute for Fiscal Studies examined the 'truth' about incomes and taxes with findings published in The Evening Standard and The Financial Times (28/11/96). Welfare dependency and the scale of poverty in Britain were two other highly contentious areas.
International comparisons - Britain's place among nations, as indicated by well-established independent organisations specialising in international comparisons, also featured prominently among the items obtaining major coverage. Measures of international competitiveness were among these. Chief among them were the annual reports issued by the World Economic Forum (WEF) and the International Institute for Management Development (IMD). The WEF, in June 1998, ranked Britain fourth out of 48 economies in world competitiveness, whilst the IMD placed Britain 12th out of 15 economies in the first rank. The difference in ranking was noted, as was the comment by Larry Elliot in The Guardian (22/4/98) that 'Britain is in danger of slipping into the second rank'.
Other measures reported on international corruption and the UK Development Index. The first of these said that in the 'sleaze' league of commercial corruption Britain comes a lowly 43rd out of 54 countries, with Nigeria top (The Guardian 19/4/98). The United Nations Index has eight tests of a nation's progress. Britain passed four, but failed in terms of equality, family structure, crime and pollution.
Finally, under this heading, The Guardian (29/1/98) reported 'UK lagging on social security spending'. The finding came from the 1998 edition of Social Trends2 which put Britain 9th of 11 European Union states ranked by expenditure on social protection benefits. Eurostat had also reported in April 1997 that Britain has more children living in poverty - one in three - than any other European country (The Guardian, 28/4/97).
Conclusion
Stats Watch and the monitoring of press comment on statistics were prompted by numerous cutbacks to official statistics with many instances of deliberate manipulation and withholding of sensitive information by successive Conservative governments. The Labour Party in opposition in the run-up to the 1997 general election declared its commitment to 'Freedom of Information' and a unified National Statistical Service independent of government.3 Two years on the jury is still out on both these measures. Should New Labour renege on its pledges then there would be an even greater need for authoritative and independent monitoring which draws attention to government abuses.
NOTES
- JICNARS National Readership Survey.
- 'Social protection benefits' (1998), Social Trends (9th edition), Stationery Office: London.
- Straw, Jack (1995), 'Labour to establish unified National Statistical Service independent of government', RSS News, May, and an Address to the Royal Statistical Society 27/4/95.
John Martyn
School of Sociology and Social Policy
Roehampton Institute London
Southlands College
80 Roehampton Lane
London SW13 5Sl
Tel/Fax: ( 0181) 788 4867 (home)
