Radical
Statistics

The Journal

The Subjects

The Books

News

Links

About

Home

Census fieldwork: the bedrock for a decade of social analysis

Ludi Simpson

Introduction

A population census provides multi-faceted data for every small area. It involves the design and management of a very large field operation. In the UK, planning the fieldwork starts soon after the previous ten-yearly census, and for the 2001 Census major changes were designed to make data collection and processing suited to current social and technological conditions. Apart from its UK coverage of small areas, the unique feature of the census is that the data are available to the public who provided them, in almost as equal measure as to the government that manages the census collection. The census is a representation of the people that has great democratic value.

The main purpose of this article is to examine the experience of census enumerators and their local managers - those recruited temporarily to 'number the people'. The UK Census of population and housing on 29th April 2001 completed its fieldwork in May amidst foot and mouth restrictions and a general election campaign. Neither turned out to be significant barriers to the collection.

The fieldwork experiences described here highlight reasons why the fieldwork hit such snags as to push the Census £ 9million over its budget by mid-July 2001, but more importantly they highlight the ways in which the fieldwork organisation managed to respond to the difficulties and succeed in numbering the people. Fieldworkers raise serious questions regarding management of large parts of such a complex exercise through commercial contracts, and make positive suggestions for the design of future censuses. The experiences also provide indications of the quality of the data, which field workers expect to be more variable than in the past, and underline the need to report that quality to users of census data.

The first section below describes the method of interviews and focus groups that inform the article and comprise a unique record in the UK. The fieldworkers' experiences are reported regarding two key aspects: the response to the Census 'on the street', and the collection of the forms by 'post-back' which caused such problems for enumeration. Brief sections summarise what the interviews tell about recruitment and preparations before Census day, and management issues raised by the fieldwork, which will be addressed in a full account elsewhere. The article ends with a summary and discussion of the issues raised.

Method and sources

During the fieldwork for the UK population census of April 29th 2001, I recorded face-to-face interviews, telephone interviews and two focus groups with temporary census staff in the rural areas of Haworth and Wirksworth, the urban areas of Keighley, Shipley, Queensbury, Calderdale, Kirklees and Bramley in West Yorkshire, Chesterfield, Milton Keynes, Cambridge, and Oxford, the inner city areas of Manningham and West Bowling in Bradford, and finally Hackney in Inner London. These interviews were supplemented by reports from Camden, Redbridge and Lanarkshire which were partly already synthesised from various fieldworker experiences. More than sixty pages record experiences of more than two dozen individual fieldworkers (Bradford Council, 2001). They provide an approximation to an ethnographic account of the 2001 Census fieldwork.

Each focus group and interview followed the same semi-structured pattern, asking for descriptions of:

• Recruitment of field staff
• Listing of addresses in the area to be enumerated
• Response from the public to the census as a whole, then to the form and particular questions in it
• The collection by enumerators at addresses from which a form had not been received by post
• Management support
• Any other issues that those interviewed wished to raise.

The discussions were allowed to stray as the interviewees wished, but were prompted to come back to these headings. Each interview lasted between 30 and 90 minutes, while the two focus groups lasted 90 and 150 minutes. I took detailed notes by hand at the time the interviews took place, and transcribed them to a record as close to the words used as I could manage, shortly afterwards.

There are two limitations to this method of assessing the census fieldwork. First, while covering a variety of circumstances, the interviews do not cover a purposefully representative sample of areas. For this reason the comments reported in this article are those that were repeated in spirit by more than one fieldworker, or were in other ways verified as not being an exceptional experience. Second, fieldworkers have a tendency to blame the organisation for the ills they have to deal with, and to take credit for the solutions that they implement. Most fieldworkers are unaware of the research and consultation that preceded decisions about the census form, its content, and the fieldwork design. The heartfelt criticisms that fieldworkers make of some parts of the census operation must be interpreted with respect both for the personal effort that they expended, and for the possibility that there isn't an easier way of undertaking a census.

The issues raised in this article are best seen as prompts for a more comprehensive review with access to all the planning documents and information held internally, and access to those centrally involved in the management of the Census. At the time of writing (August 2001) the census office at ONS has not been able to estimate response for separate areas within England, nor do they expect to do so before the Autumn of 2002 when the coverage survey has been fully analysed. Nor have they shared the feedback gained from ONS interviews with Census Area Managers in June and July 2001, again preferring to wait for their own 'overall evaluation'.

Perhaps the clearest role for this feedback from field staff is to help set an agenda for the review of the Census that the national Statistics Commission (2001) has announced, and for ONS' own internal review.

Staff recruitment, address listing, form delivery

Recruitment, address listing and delivery all highlighted problems with the management of such a large operation, which will be discussed later in this article where they arose in other connections. Delivery was achieved successfully but entailed considerable effort in some areas. The enumerators' first task was to check a pre-printed list of addresses derived from the Ordnance Survey product Addresspoint. These lists were most deficient in areas of new housing and where houses had been converted into multiple flats. Many District Managers were concerned that "Some enumerators must have taken the pre-printed list and not bothered to look if there were any more." Any addresses not listed after the enumerators' check will then have been missed from the enumeration unless the householders themselves demanded to be counted.

Response to the census and quality of responses

This section reviews the response of the public both to the census as a whole and as far as can be told to individual questions. Field staff voiced criticisms of the layout of the census form. Less checking of quality was done by enumerators on the doorstep than in past censuses, which probably led to a reduction of completeness and quality of each form returned. Only date of birth, sex, and marital status were checked for valid responses from all forms. Students as usual were a problem for enumerators, and enumerators gave useful feedback on the enumeration of Asian communities.

The form itself aimed to ask a considerable number of questions in simple form, and to route children and those aged 75 or older quickly past the questions they did not have to answer. The form is inevitably more difficult for some than for others, and drew criticism from most field staff.

The questions are unnecessarily complicated. I think quite a lot of information on the forms was quite poor and that's because the form was badly designed. The questions were complex. (DM8)

One thing that was really annoying was putting 3 pages for each person. It should have been two or four. It was really hard finding yourself in the form. (DM8)

In the very first batch, I got back 3 forms from the same street, I know the area and they are all the same sort of houses. The question that says are all rooms behind one outside door, one said yes, one said no and the other hadn't filled it in. You could tell they hadn't understood it. I found it very hard to fill in myself. (DM7)

Many forms had comments on about how many times was it necessary to complete the names of residents - in the case of the householder 4 times does seem a bit excessive! (TL13)

It needed clearer instructions as to when to fill in the form and when to send it back, they were not told to send it back promptly. Also clearer instructions to include children on the form, in Table 1 as a Person. (EN12)

I have seven people in my house and I filled it in. It took me 45 minutes and wasn't easy. The relationships table was particularly hard. (DM8)

The most complex question - a new series of tick box columns on relationships between each person in the household - was felt to be a major problem by many field staff. 'They didn't do it; you haven't got a cat in hells chance of telling relationships from that question' (DM6). Particular problems may be encountered during processing when matching the relationship question to individuals' answers later in the form: 'Yes, if there's five or six people, they'll be listed in the first table, but in a different order on the form. So long as they were all there I just left them' (DM6). 'I had loads like that' (DM8) and the same issue was raised with concern 'in many cases' (TL13).

Neither religion nor ethnic group raised problems noted by field staff. The employment questions were highlighted by enumerators as likely to be the least well completed. The quality of forms was generally considered a problem, though it is impossible from these interviews to say whether this is worse than in the past.

In general I would say the forms were badly filled in, particularly where text was required in boxes, many ignored the boxes so I wonder how well these will scan in. (TL13)

Illiteracy is more of a problem because of postback, and there's a lot of it. (DM1)

The reference to postback highlights that the enumerator in 2001 had less opportunity to help householders fill in their forms. DM8 honestly declared 'We didn't have the contact to know if any of the questions were difficult to fill in'. All forms posted back were checked only for valid answers to the first questions - date of birth, sex and marital status for each person listed at the start of the questionnaire. Failing forms were returned to enumerators for follow-up. The rate of failure of the 'three questions' test among posted-back forms varied a great deal, often around 10%, but over 50% in some District Managers' areas. Often the failed forms were missing date of birth, particularly for children.

When we went back, we knew that only questions 2, 3, and 4 were necessary as a last resort; these will be the only ones that are well filled in. (DM2)

There's no way of checking back for the questions once they have passed the first three, age sex and marital status. I was told to just make sure that those were filled in, and even to fill them in myself if I had to. I did that just for one form where they had listed seven people on the form but only given one form. When I went back and asked who were the names that weren't completed, the children told me it was the parents, and I knew enough to work out most of the answers, over 60, not working, born in Pakistan, Muslim, probably worked in textiles up to 15 years ago, so I could fill it in. (EN8)
[LS notes: There is no suggestion that this intelligent imputation of details was repeated by other enumerators, but it is not unlike the automated imputation during census processing, referred to in a later section.]

The 3-question checking of posted-back forms undoubtedly greatly improved the quality of key questions on a significant number of forms. During processing the rate of missed other questions and inconsistency will be measured and reported. The indications from fieldworkers suggest it may be higher than the value of under 1% that was reported for most questions in 1991 (OPCS/GRO(S), 1994).

The only category of people that fieldworkers expected were missed off inadvertently was children. They were aware from their checking that many households felt they did not have to complete all details for children. There was no way that enumerators could identify forms where children had been missed off altogether.

In areas of relatively easy enumeration, refusal to fill the form was not a big problem, with 'Only one refusal in S Derbyshire', 'Few refusals and all taken in the end' (TL9, Milton Keynes), and 'Some people were gruff, but no problems at all' (EN19, Cambridge). On the other hand, in areas of harder enumeration, other problems of retrieving forms meant that outright refusals were not high on fieldworkers' agenda, but were a problem.

I had 20 refusals in the end, out of 50 in the beginning. I'm going to suggest they all get prosecuted. They were nasty, very aggressive and they threatened enumerators. (DM8)

I had 2000 forms unaccounted for, and you don't mop up that number of forms. No way could I suggest anyone for prosecution. (DM12)

Open refusals would normally lead to a household missed from the enumeration. Those refusing to be included on a returned form would not be so noticeable to the fieldworkers. Both households and individuals missed from the enumeration are a problem for analysts. Records approximating those missed are to be added to the census database as part of the One Number Census procedures described later.

For the first time, students were counted as residents at their term-time address for the census, though they could be included at their vacation address too but not as residents. This is consistent with the treatment of students in government population statistics. The enumeration of others in communal establishments was not a significant problem according to field staff.

Of the District Managers interviewed, two dealt with what might be considered student areas, with mixed results. The detail is held within the interview scripts, but the main difficulty, as foreseen in the census planning, was the overlap of census fieldwork with vacation dates. In Bradford the University term did not start until May 7, so the delivery of forms was not relevant for those in halls of residence, which were closed. Enumeration of halls of residence was very dependent on co-operation and effort by university managers. In one University this was not forthcoming until a change of staff well after census date, when a well-organised last effort brought over 90% of the forms expected back. In Leeds, the Metropolitan University refused to do the work asked of managers of communal establishments, insisting that all flats in halls of residence be treated as households that ordinary enumerators would enumerate. While the enumeration may have been successful, the number of Leeds' households sharing amenities will need to be treated with some caution when compared to other cities where halls of residence would not be counted as households at all.

Concerns were raised about capturing the varied family arrangements encountered, which could also apply more widely than the South Asian communities from which these comments were derived.

If there's 13 people in a house, there's no way they'd all be got. (DM3)

People move a lot between family houses leaving some empty. (DM2)

One family I took a continuation form because he said there were seven of them. When I took it he wasn't there but the woman said there were eleven, the man didn't know, so I left two extra forms. But next time someone different was there and said there were just 10. So I've taken one blank form back and there's 10 filled in. (EN16)

There's one with 2 adults but three children born within three months, well that's not possible, or maybe there's grandchildren and they're cousins, well I can't ask can I, its not for me to question that kind of thing. (EN16)

Often its almost like there is a house that they sleep in and a house that they live in during the day. My advice was to divide the family into two and put some in each house. That way they didn't need a continuation form. (DM8, who had insufficient continuation forms - see later)

This evidence points to a likely under-coverage and poorer quality of data from extended families. The Census and more generally almost all household surveys assume that households are complete units that live within a single dwelling. This is not the case for many extended families in the British South Asian communities. Nor is it the case for many dual households where children have shared care by parents living in different households.

Collection by postback and enumerators

While postal return of Census forms was taken up by a majority of households, delays in their return caused the most severe problems of all for the enumerators.

For the first time in the UK, householders were asked to post their completed forms in envelopes addressed to the District Manager, to whom Royal Mail was to deliver all forms received. The District Manager and Team Leaders were to check all forms and let enumerators know those that had been returned and checked, in time for the follow-up by enumerators of those not received, which was to begin on 9th May. One of the interviewed Census Area Managers had done the job during the 1999 Census Rehearsal, and summarised the experience of 'postback' then and in the Census itself.

The rehearsal recommended that the deadline be extended before returning enumerators to the field. In particular we said not to put the day for Post Office returns as the day after a bank holiday, because that was what happened in the Rehearsal. There was a solution that we suggested at rehearsal - leave the collection to a later date. The enumerators could have had a three week rest and then concentrated on a smaller number of non-returns. ONS allowed one week and 2 days, to 9th May, for the forms to be received and sorted, returned to the District Manager, checked and then each enumerator to be informed of the forms that had been received. With a second class post contract, there was never any expectation that forms would come back on 30th April or 1st May. Added to that, the 5th, 6th, and 7th were out, the Post Office were not working Saturday or Bank Holiday Monday. So they were overwhelmed on the 8th May. (CAM2)

It appears that Royal Mail did not know of the importance of delivering posted forms to District Managers before the Bank Holiday weekend, and ONS did not foresee the financial implication now described by this Census Area Manager, a very large one when multiplied across the country.

Imagine an average ERB with 400 addresses. In my own area, less than 50% were ticked as returned by the District Managers. So the enumerator got an average £ 200 extra - £ 1 for each not returned. (CAM2)

The choice of day for the Census is an extremely difficult one, attempting to avoid Easter holidays, and later summer holidays, for the whole period of census fieldwork, including the follow up Coverage Survey. The experience of both the rehearsal and the census suggests that a more expensive contract with Royal Mail, or an extended collection period, was needed for the postback strategy to be a success. Fieldworkers found it a 'disaster' in ways that will impinge on coverage and quality of results, as well as the Census budget. The problem was not the number of postal returns - which exceeded the ONS expectation. Fieldworkers were not all against postback of itself, but felt it could not be done in the hurry they experienced. The main problem was the delay before fieldworkers received forms that had been posted back, which affected every field worker interviewed without exception.

I got the list of non-return forms quite late, the Post Office had problems with the envelopes, with the forms. It meant that when I was going round they said 'I sent it back 10 days ago', or 'two weeks ago'. Some people get a little shirty. Really you just had to believe them. That was a problem And because you then had a short time left to get round, you had to put one leaflet in saying you haven't posted back, and only a couple of days later you gave the one 'You have a legal duty' and so on. So that wasn't very good. They should have started the process a bit earlier, but I had to wait for the Team Leader to give me the list of non-returns. I got the list two days after we were supposed to start. It meant that after that I was out every night, following it up and it took more time than I thought. (EN10)

The postback was 'really, really slow, terrible', 'a big disaster', 'a complete fiasco', 'The Royal Mail made a thorough pig's ear of it'. 'It was a massive disappointment. Follow-up consisted of turning up on the doorstep to say 'We haven't got your form', to be told 'We already sent it back mate'.

The first impact was to delay the return of enumerators to the field in many districts, wherever significant numbers of envelopes had not yet been received from Royal Mail. In many areas this was a delay of two, three or four days. Even this was only achieved by a tremendous (and expensive) effort by District Managers and their Team Leaders to sort and check the received mail, within a tightly squeezed time scale. The following are typical descriptions of the intensity of the work that postback created at this stage.

They delivered approximately 5400 forms which I was expected to check. This was, in my opinion, an unbelievable task given the timescale and the pay involved (I think my hourly rate worked out less than that of some of my enumerators)! Some days I spent 12 hours plus opening, sorting and checking forms, obviously mistakes will have been made when it was necessary to spend such long periods on a task such as checking. (TL13)

Post back meant me and 3 Team Leaders had to count 8,000 forms. It didn't take long for one of them to say that if the average time to check one form is 3 minutes, this isn't on ... It became clear I could use enumerators for sessional hours to do this; by that method I got into follow up only 2 days late. (DM12)

When enumerators did return to the field, they found that many householders whose form had not been received insisted that they had already posted their forms. A very great number of forms were 'stuck in the post'. There were different approaches to this problem, and some concern at what enumerators were asked to do.

It was difficult because they went back accusing people for not doing something - not sending it back - which is quite challenging. ONS said we should ask people to fill in another form - it was only at the mopping up stage with team leaders that ONS said not to chase those who've said they've sent them back. (DM8)

Time that enumerators and other field workers had earmarked for census collection now had to be re-arranged because the fieldwork was delayed by up to a week in many areas. Enumerators having developed an understanding of their Enumeration District, they never had a final sense of how well they had done as there were still forms 'in the post' at the end of their contract. The effects on fieldworker morale and effectiveness were a serious problem.

Apart from the difficulties of dealing with the unexpected delays in Royal Mail work, the ambiguity in the field may have led to more refusals. 'They cottoned on they could just say 'I've posted it' and there was nothing you could do. You are unable to prove whether they have' (DM2).

There were continuing delays in receiving forms. One District Manager explained in a briefing note to his enumerators that 'Your frustration is shared by those of us who have spent three weeks trying to prize postbacks out of Royal Mail' (DM12). As a consequence, in many areas the enumerators were out in the field later than the planned 18th May, when they were supposed to finish all their paperwork and end their contract on 21st May. Some reported a few days extra whilst others worked until the 26th, and one until the 30th May. A mop-up of remaining households was then to take place, by Team Leaders and the most successful enumerators. Extra costs were incurred to pay the enumerators their later work, over and above the £ 1 per unreturned form referred to above (this rate as others was more in London, less in areas easier to enumerate). A further consequence was some interference with the Census Coverage Survey that went into the field on 24th May.

Postback may have been a success if measured by the percentage of forms eventually received through the post. But the unexpected delays led to ambiguities in the collection procedures, intense pressure on field staff with detrimental impact on census coverage and quality assurance, and a very significant financial contribution to the £ 9million overspend on the Census budget in 2001/2.

Management issues

Some doubted whether ONS have the experience to set successful contracts for a major operation for which timeliness was essential.

Previously it had all been in-house: pay roll, production and distribution of forms, co-ordination and delivery of all the myriad items of stationery; collection of forms, scanning. ONS was a naïve and inexperienced contract organiser. As soon as things went wrong, the contractors, who were not naïve, said 'you asked for something, which we have done; you can't change it now'. So the service was less than was required, or ONS had to pay more. (CAM2)

As with postback, contracting the pay roll and the helpline both involved organisations whose staff and management were committed to a contract rather than to the Census aims, leading to a lack of control by ONS, and higher costs than expected when things did not go according to plan.

In addition, there seems to have been an unresolved tension between quality-assured national standards set by ONS and the essential need for a timely and appropriate resolution of any problems that arose. ONS management was perceived as inappropriately 'paper heavy'. '2001 was worse that 1991 and 1981. This time we were wrapped up in red tape and bureaucracy. I'd like to say I've enjoyed it but I haven't' (DM1). Examples of inappropriate red tape were drawn from recruitment procedures, stationery re-supply, pay claims and translation facilities.

In most cases those that suffered from these management difficulties were the field staff and in particular the enumerators, who worked many more hours than they had expected under frustrating conditions. Many thousand enumerators were paid late due to a failure of the pay roll contractor's systems. The resolution involved several hundreds of staff days by ONS, and hardship by enumerators who could not cover expenditure with the wages that they had earned and been promised. Over one thousand enumerators were still awaiting completion of payment at the end of July, six weeks after their last payment had been due.

Discussion

In the UK, published feedback from census operations has in the past been limited to tests before the main census itself (e.g. ONS, 1987; Moss, 1999). The interviews and reports from field staff reported here provide a rich understanding of how the UK 2001 census was taken. They are entertaining in themselves, and will strike empathy with anyone who has been involved in survey or census fieldwork. How much value do they have for improved census processing and for social analysts and planners?

Along with other feedback, the interviews can be used to help identify, monitor and prepare for possible difficulties in the scanning of census forms and their processing through the One Number Census procedures. They highlight the likelihood of higher rates of missing households and missing items within forms than in the past, the difficulties with the relationship question and the likelihood of many households having completed more than one form after delays in Royal Mail processing. The observation that many forms were completed poorly, including answers outside boxes, suggests monitoring how such cases are dealt with during the scanning and coding of the census forms. The Census Office has a copy of the full transcripts of all the interviews.

Feedback from field staff has also highlighted issues that the current reviews of the Census could take on board. If the Census no longer gains commitment from a very high percentage of the public, the simplicity of the questions should have a premium. Field staff made a variety of suggestions, including removing difficult questions and ensuring that each person's questions are placed in the same position in the questionnaire. Experienced census field managers felt that the 2001 Census resources and management were not flexible enough to achieve its aims of reducing differential undercount, nor to cope with the unexpected problems from new procedures of postback and pre-printed address listing. They felt strongly that both the contracting of major parts of the census operation to agencies with little commitment to the Census, and the bureaucratic style of ONS management had negative effects on Census coverage and quality which could and should be overcome in future censuses.

The fieldwork suggests considerable variability in the coverage and quality of the data, which will concern users of census data. It is too early to know whether this variability will be greater than in the past. The Census Coverage Survey and the One Number Census procedures will measure this variability and go further than in the past to reduce it. If the One Number Census achieves an accurate correction for coverage, and if the editing procedures fill gaps in the questionnaire responses, then the picture we gain from the census will be on average correct, and not biased.

There will nonetheless be greater uncertainty in the census results for some groups than for others, including greater uncertainty for poor and socially excluded groups but also for young men generally, for the significant number of people living in locked blocks of flats, and for young children. Users of census data should expect that ONS make available for review records of the quality of census data for each Enumeration District, to measure the geographical variation in this uncertainty. These records would contain the number and type of invalid items and missing items on returned forms, the numbers of duplicated forms, the number of households recorded by the enumerator as residential but not returning a form and the number and types of people and households imputed by the One Number Census procedures. Analysis of such records would give information to census users on the quality of census data, and the variability of that quality. It would also identify the extent to which the One Number Census can be successful for small geographical areas.

These are all issues that the ONS and the Statistics Commission could address both to enhance the 2001 Census and to improve future censuses.

Acknowledgement

Thanks to the 2001 Census fieldworkers, all of whom gave much more than can be reported here.

NB: In the article, quotes are attributed as EN (enumerator), TL (Team Leader), DM (District Manager) or CAM (Census Area Manager) followed a by an identifying number.

References

Bradford Council (2001) Collated fieldwork experiences from the UK 2001 census. Research and Consultation Unit, Jacobs Well, Bradford BD1 1RW. Bradford: City of Bradford Metropolitan District Council.

Moss, Celia (1999) Selection of topics and questions for the 2001 Census. Population Trends 97 (Autumn).

ONS (1998) 1997 Census test evaluation, Census Division Advisory Group paper AG(98)01. Titchfield: Office for National Statistics.

ONS (2000) Timetable for recruitment of field staff for 2001 Census in England and Wales. Census Advisory Group paper (00)05, February 2000, available from www.statistics.gov.uk , or from ONS.

ONS/GRO(S) (2001a) Census News, August 2001.

ONS/GRO(S) (2001b) Guide to the One Number Census. Titchfield: Office for National Statistics.