CReSC: The Social Life of Methods

Please visit our blog at our new CRESC website at

http://www.cresc.ac.uk/news/blog.

We have designed two images for the front and back covers of our (almost final) conference programme (click to view larger image). They are visualisations of all the conference paper titles produced using a creative commons tool at www.tagxedo.com. Tag or word clouds are a popular method of quantifying and visualising text. Typically, the font size of a word increases with its frequency of occurrence. However, the relationship between frequency and size can be manipulated such that more frequent words are emphasised less (there is a technical formula for this which you can read about at the website). Using this technique we produced the front cover using an emphasis of 60%, while for the back cover it is 0%. We also set an upper limit of 300 words and chose the shape of a conversation bubble to visualise the word clouds.

Not unexpectedly social, science, research, methods, data and digital dominate in the former but then most of them disappear in the latter (due to the random and equal probability of the selection of words). But many more versions are possible as it is possible to manipulate many of the settings, decide on what is important and what to emphasise. That is the key point. In addition to visually representing the contributions to the conference we think the tool nicely (and perhaps too neatly) illustrates some of the concerns about methods: that they are not neutral tools but along with many other actors necessarily construct, shape, configure, frame and make up social worlds.

The CRESC Annual conference is getting closer.

It’s at St. Hugh’s College in Oxford, from 31st August-3rd September 2010, and it promises to be a high-energy, high-profile event, with around 160 papers, plus keynote speakers including Andrew Abbott, Engin Isin, Patti Lather, Katie King and Celia Lury.

You’ll find the full conference details here.

And if you’re interested in the exciting PhD workshop (during the day on 31st August, just before the conference opens), please visit this link for further details.

But what is the ‘social life of method’ (or SLOM)?

CRESC is in the process of updating its web pages,  and we’re spelling the SLOM idea out there. But here’s what we’ve written for our new SLOM front page .

Please read on if you’re interested – or if you want to debate.

Our Question: What do Methods Do?

What do social research methods actually do? And how are they shaped by the social world? These are the core questions we are exploring in the Social Life of Methods (SLOM).

These two questions aren’t simply technical. Methods aren’t a neutral toolkit that can simply be picked up or put down. Indeed, they aren’t narrowly methodological. The technicalities are important, but there’s a  bolder claim here. It is that research methods are vital players in the social world: that they help to create society. And we want to know how.

This is the focus for SLOM.

There are Many New Methods

Social research methods have proliferated in recent decades. They used to be based in the universities, and in government research agencies. This is where sociologists, demographers and anthropologists traditionally worked.

But lots of new methods now come from the private sector (marketing, or track-and-trace technologies). They’re also created in government (focus groups and security surveillance systems.) In addition, digital data have exploded in importance. Much of this data is freely available on the internet (think of Google). So too are open source software and applications.

Some say that this means that social research methods are being democratised. Perhaps this is right. Perhaps it isn’t. But one thing is sure: research methods and ‘the social’ are both changing. And those methods aren’t ‘innocent’: they are helping to change the social world.

The Social is Being Remade in those New Methods

Professional social science does lots of innovative work on methods. But we’re worried that it’s missing out on this broader trend.

Our worry isn’t professional jealousy. It’s because the social is being remade by new research methods. Indeed, this is happening before our eyes. But social science isn’t tracking this properly. Here’s the core SLOM point. If we want to understand socio-cultural change, then we need to understand changes in research methods too. And we need to be involved in those changes ourselves.

This is why we’re interested in the social life of method. We want to understand how research methods construct the social world, and also how the methods themselves are shaped by the social changes around them. And we want to intervene in that process.

This year’s CRESC annual conference will focus on the Social Life of Methods (for more information visit http://www.cresc.ac.uk/events/conference2010/index.html) . The agenda for the conference is summarised below.  We have set up this page for the purposes of discussing, debating and developing this agenda further and hope to continue the conversation at the conference.

Summary of Conference Theme

During the past century and longer, social scientific methods have come to be extensively deployed in government, administration and business, as well as in academic research. Maps, enumerations, surveys, interviews, indicators, software and visualizations proliferate. The aim of this conference is to consider how we can best understand the role of social science methods in both shaping, and themselves being affected, by economic, social and cultural change, both historically and in the current context when digitalization poses challenges to established social science methods.

Mindful of the ideas developed within Science and Technology Studies, which show how objects in the natural and medical sciences can be social agents, we seek to broaden this agenda to focus more particularly on methods within the social sciences and humanities. Papers are invited from interdisciplinary audiences addressing the following issues:

• Is it useful to explore how agency can be located in certain kinds of social scientific methodological repertoires?
• What kinds of methods succeed and which fail? What are the respective powers of different sorts of qualitative and quantitative forms of analysis? How can we explain why certain sorts of methods become hegemonic in particular domains?
• What is the role of the visual in social science methods? How is this changing?
• With the proliferation of digital data, are we currently seeing a crisis of standard social science methods based around the sample survey and the interview, and what does this portend for our understanding of socio-cultural change? Does the idea of a descriptive turn offer a useful way of grasping the role of these new methods?
• What is the transformative and critical potential of social science research methods, both historically and today?

We are interested in reflecting theoretically about how actor network theory, genealogy, complexity theory, feminist theory, anthropological studies of expertise, ecological studies of knowledge, political economy and field analysis can be used to understand and illuminate these issues. There will be four themes which will structure the sessions of the conference:

1: The device: what kinds of device have come to play an important historical role, and which have failed? How can we better understand the histories of nations, social groups, individuals and organizations through a focus on devices?

2: The challenge of digital data: what is the implication of the proliferation of digital information for the ordering of economic, social, political and cultural knowledge?

3: Envisaging the visual: how have visual methods historically competed with textual and numerical methods, and how is their role changing in the current context?

4: Transformative practice: history, discipline and movements: how can methods be mobilized to critique and challenge dominant methodological repertoires, focusing especially on the role of historical analysis, ethnographic, feminist, and subaltern methods?

Continuing the topic of polls as narrative devices, John Law writes that ‘Stories, political stories, depend on narrative devices such as polls. The polls, so to speak, give particular political stories substance. They help to make some real and others less real.’ Numbers and narratives are indeed bound up together and in the world of journalism ever more so now that government data has been ‘liberated’ from the confines of administrative offices. Several government websites have been launched in the past year, which make data publicly available on the Internet along with devices (applications and tools) for people to do their own analyses of ‘raw data.’ From the US Data.gov and San Francisco DataSF.org to the UK Data.gov.uk and London’s Datastore, governments are embracing Web 2.0 technologies to change the traditional ways that they communicate and disseminate official statistics. Transparency, democracy, accountability, engagement, collaboration, participation – these are just a few norms that politicians and administrators attach to the project of providing open access to government data along with devices for analysing, visualising and interpreting that data.

In many ways Gov 2.0 can be understood as a logical step in the development of eGovernment policies and projects, which promise better government through the use of information technologies to improve citizen access to information, the quality of service delivery, cost savings and to widen political participation. With eGovernment interactive web-based platforms citizens have been able to manage their services, access information and communicate with various state agencies. Gov 2.0 constitutes a new dimension of eGovernment that engages citizens not as customers, service recipients or information gatherers but as data analysts and interpreters. It not only liberates and opens up easier access to government data, it also draws on existing open source tools and circulates tools and methods for analysing and visualising that data.  All of the various government sites also engage IT developers and software designers in the making up of new and innovative applications and tools such as APIs, visualization software and mash-ups that can be used to configure, mediate and organize the way data is understood and interpreted. As such, Gov 2.0 introduces new information gatekeepers, organisers, and interpreters who more than ever can challenge the jurisdiction and authority of ‘traditional’ experts.

Several news media and journalists are leading the analysis and interpretation of government data, such as the Guardian’s on-line Datablog which aspires to be the gateway to government data, applications, and visualisations. Simon Rogers, the editor of the Datablog, states that ‘data has become trendy’ and that these government databases have spurred a new breed of analysts called ‘datajournalists.’ Of course, Rogers is not alone in recognising the potential of Gov 2.0 datasets for the media. A brief survey of sites and on-line discussion forums reveals how the news industry and individual journalists envision data as one of the answers to the current restructuring pains of the media in the new digital economy.

With the 2010 UK election we can now watch how datajournalists mobilise data in what Rogers suggests may be the world’s first data election. Indeed, journalists need not draw on the data analyses of other professionals (academics, civil servants, researchers) but can now narrate the election by doing their own crunching, mashing and visualising of datasets -  see for example the Guardian’s ‘factfile’ on crime. This will surely provide lots of material for studying how data travels and is mobilised, and as John Law writes, gives substance and ‘makes real’ particular narratives.  But then, if democracy is its devices, what are the politics of these new data devices?

Narratives elicit tensions, says Mike Savage. And narrative devices like polls perhaps embed (as he puts it) ‘a recognition of dramatic uncertainty’. They are incomplete. So what completes the narrative? Resolves the tensions?

One answer is: ‘the real election’. Yes, of course. But/and. The real election is a narrative device too. It can be told in this way and that, so there is still tension in the air.

Look at the UK Polling Report’s Polling Average. This is handy if we are overwhelmed by the number of polls 15 days off from that ‘real election’ because it weights the results of recent polls and combines them into a single set of figures. There’s a methodology at work (or a set of assumptions) here. It (I quote):

‘takes in polls from the last 20 days and gives them weightings based on various factors, including how recently they were conducted, the past record of the pollster producing the figures, the methodology used, the sample size and how many polls have been produced by a single pollster.’

All of this can, as its author Anthony Wells notes, be debated and discussed. But what I want to get to is its average projection for the share of the vote. I’m writing this at 4.25 on Wednesday 21 April, and the current figures are: Conservatives 33%; Labour 27%; Lib Dems 29%.

So what to make of this? The parties are narrating this one way or another. Non-innocently of course. But the plurality of the story-lines from the commentators too tells us that no one really knows. Here’s George Parker in the Financial Times (Wednesday April 21, page 3).

‘Nick Clegg’s surge in the opinion polls has transfixed Britain’s political classes, leaving the election campaign in a state of suspended animation: something extraordinary has happened but nobody knows what the final reckoning will be.’

Now move one step forward. Because the pollsters are also busy picking through the entrails and trying to work out what this might mean for the ‘real election’. This form of translation is an art – Mike Savage talks about this. But if I stick with Anthony Wells’ admittedly crude uniform swing projection the answer at the moment of writing this blog is: Conservatives 255 seats; Labour 267; Lib Dems 97; and others 32. The overall verdict? ‘Hung parliament: Labour short by 59’.

We can haggle about the figures. But most versions of the story agree that unless the Lib Dem bubble bursts, we’re into hung parliament territory. So what is being said about this? Well, you pays your money, you picks up your newspaper, and you reads the narrative of your choice. Here’s Polly Toynbee (The Guardian, Tuesday 21 April, page 35):

‘David Cameron is dead right when he warned yesterday: “Vote Clegg, get Brown.” The shocking prospect is that Gordon Brown could come third and still emerge with the most seats, sailing back into Downing Street as unelected as he ever was. But this time voters would be in a state of revolutionary outrage.’

Behind this what’s being widely said is that a ‘real election’ is a device very likely to produce a decisive result in a two-party race. It did this in the 1940s, the 1950s and the 1960s (even if it sometimes got it ‘wrong’ in terms of the total party vote). But in a three-way contest its decisiveness turns to something that is much more easily narrated as arbitrary. (The effect is exacerbated if nationalist, regional, or local candidates are also picking up seats.) Polly Toynbee may be over-egging the pudding – surely we are some way off guillotines and tumbrels – but her general story-line is scarcely unique.

Here’s my point. Stories, political stories, depend on narrative devices such as polls. The polls, so to speak, give particular political stories substance. They help to make some real and other less real. They allow us to distinguish between possible political realities (a Conservative government) and political fantasies (a UKIP administration). With the poll surge in the Lib Dems we’ve also seen that they can change the probability of different story-lines. In short, opinion polls have what we might think of as realist effects. Polls, however, are themselves embedded in other stories, political, statistical and otherwise. Their credibility as ‘reality devices’ is interwoven into and depends in part upon a hinterland of those other narratives and their practices. As we know, for instance, a small sample size diminishes the standing of a result.

But with minor modifications, all this logic can be applied to the ‘real poll’ as well. This is too is sustained in a narrative hinterland. And this is why I have picked on Polly Toynbee’s story. If her account were to become popular the ‘real poll’ wouldn’t last as a ‘real poll’ very well for long. Something would give. I agree that this may be a narrative too far. But, two points. One, who knows? And two, less apocalyptic versions of the same story-line are possible with similar results: for instance, in the form of a Labour-Lib Dem coalition and a subsequent system of proportional representation. The tensions never really go away, and new uncertainties proliferate.

How do we explain the compelling power of the opinion poll device? Why are they so readily reported and why do political parties, and significant numbers of the general public alike pay heed to them? Why indeed does it become interesting to follow the polls avidly as they present their findings each day during an election campaign? I want to pick up the thread that Evelyn Ruppert and John Law have laid down regarding their character as narrative devices to reflect further on how they entice and enthral.

It is important to remind ourselves that although the device of the opinion poll dates back till the 1930s, it took 30 long years, until the 1960s, before they were widely reported in the national media and came to be regarded as effective devices by political parties, newspaper editors, and the general public. Although they were carried out regularly at general elections since 1945, they occupied little column space, and attracted only minor comment until the later 1960s.

This lack of appeal was not because they were ‘primitive’ or inaccurate in these early days. Famously, they correctly predicted that the Labour Party would win the 1945 General Election – though no one paid them any regard and Churchill’s defeat was greeted with widespread astonishment. The statistical basis of sampling was reasonably well developed by this point. And, although poll forecasts were sometimes proven wrong, as in 1955, such errors have continued to more recent years, most famously in 1992 when they did not predict the Conservative victory of that year. It would therefore be erroneous to somehow see contemporary polling as more ‘accurate’ than in the days when they attracted little attention.

Part of the popularisation of the poll is due to their effective mobilisation by key agents who could explain the translation of percentage points (as revealed by the poll results) into actual outcomes. The rise of television allowed visual devices new kinds of opportunities to persuade the public of their effectiveness. Thus the ‘swingometer’ (as deployed by pundit Bob Mackenzie) was a device which translated numerical indications into expected outcomes (in terms of seats won and who would become the governing party). But the swingometer became popular not through publicising opinion polls so much as assessing how early results could be scaled up to predict the final outcomes.

We perhaps also need to see the effectiveness of polls as lying in their capacity for dramatisation. It is not incidental that the poll became visible at the same time that political parties themselves – especially Harold Wilson’s Labour government of 1964-1968 – became more strategic in the timing of elections. Just as we have seen the power of polls to shape short term fluctuations in voting intention following the leaders debate last week, so they also became key players in the dramatic rise of the Liberal Party’s vote in the 1974 elections. The rise of the poll might thus be seen as part and parcel of increasing voter volatility.

In reflecting on the reasons for this, we might stress the fact that that the popular awareness that polls make mistakes is an important part of their appeal. No one believes a poll is ‘true’, an accurate reflection of voting intention. It is at best, a pointer, a staging post on the way to the next poll, or to election day itself. Just as every narrative elicits a tension around its future resolution, so every poll now carries with it a sense of uncertainty of how its findings will be borne out by the results. Thus, the poll now embeds a recognition of dramatic uncertainty. In the American presidential election in 2010, it was a matter of debate whether the polls would overestimate Obama’s vote because white voters would not admit to their un-preparedness to vote for a black man. In the British election of 1997, no one expected the Labour lead in the polls to actually lead to a Labour landslide. Today, there is an ongoing discussion about whether polls tend to systematically overestimate the Labour vote, and considerable speculation about the difficulty of translating poll figures into actual seat distribution because of local factors such as tactical voting.

The appeal of polls might thus be said to lie exactly because they generate suspense. They draw on the recognition that they are prone to error and are invitation for further polling as means of resolving this suspense. Hence, polls can be done in more exact ways – for instance on key marginal seats – and in ever more shortened time periods. As John Law notes, the poll results of the party leaders’ debate were announced minutes after the debate had been concluded. Exit polls are reported seconds after voting booths close. This attempt to close the gap between the poll and the outcome is rather similar to Walter Benjamin’s discussion of how we continue to seek the aura of the sacred object even though in a mechanical age this is a hopeless quest. The poll thus testifies to its own incompleteness, so that it becomes part of a story whose conclusion lies beyond it.

Evidence from surveys does not figure extensively in other parts of the election campaign, and there is now a well known argument that the appeal of surveys is declining as a result of the proliferation of new kinds of digital data. Perhaps the visibility and appeal of the opinion poll might be read, counter-intuitively, as further evidence for the partiality of contemporary survey methods.

Reporting the Debate

The timing was awkward for the daily papers. Ending at ten pm on Thursday night, Friday morning’s newspaper accounts of the election’s first leadership debate were rushed and more or less insubstantial. The Financial Times’ front page report (April 16th), squeezed under a picture of a smiling Peter Mandelson ballroom-dancing at Blackpool, highlighted the clash on immigration, describing the debate as a whole as ‘sparky’, ‘free-flowing’ and ‘surprisingly abrasive’. ‘Mr Cameron,’ it continued, ‘… often found himself under attack from both sides. But his answers … scored well with viewers’, while Mr Clegg ‘tried to distance himself from his two opponents’ and ‘sought to exploit a rare chance to gain credibility’.

There was no talk of winners and losers here, but all-in-all one was left with the very mildest impression that Mr Cameron had acquitted himself well. ‘Lord Mandelson’, it noted, ‘has conceded the Tory leader has better debating skills than Mr Brown’. Mr Cameron, it added, was ‘deemed by opinion polls … to be the most likely to deliver the knock-out blow.’ And then again, in the way the FT reported it, the business of spin was important too. The paper commented on the accompanying party attempts to ‘influence coverage’ in the blogosphere and beyond in what it also noted had become ‘a more presidential style of politics’.

Next day, the 17th, was a big news day for finance. The FT’s front page was dominated by the story that Goldman Sachs was to be prosecuted by the US’s Securities and Exchange Commission for alleged fraud in its CDO dealings with the hedge fund, Paulson and Co. The second story was headlined ‘European airline shares fall as volcanic ash worsens travel chaos’. With the eruption of Eyjafjallajökull and the consequent grounding of much of Europe’s air traffic and airline shares, Iceland was proving once again that when push comes to shove it can punch way above its international weight.

The result was that the big leadership debate was squeezed into three column inches on the front page. Below a head-and-shoulders photograph of a clench-fisted Cameron, the FT told us that: ‘Tory leader David Cameron… was urged to turn his election guns on Nick Clegg after the Lib Dem leader scored a clear win in the UK’s first televised prime ministerial debate. Polls taken after the ITV debate … suggested Mr Clegg was seen as the winner – reaching 61 per cent in one poll – with Mr Cameron second and Gordon Brown third.’

The full FT report, appearing on page 4, developed these themes, and added that the parties were all wondering whether this was a flash-in-the-pan ‘Ross Perrot’ moment:

‘In the 1992 US presidential contest …. [Perrot] was deemed to have won the first three-cornered television debate against George W. Bush and Bill Clinton, trading on his status as an independent outsider. His campaign then fizzled out.’

It reported, too, in a table of putative election losses and gains whose provenance was far from transparent that (this was the headline) … ‘the Lib Dems still face an uphill battle to make real gains.’

The Guardian and the ICM Poll

The Guardian on the same day reported the results of one of the polls in a double page spread on pages 9 and 10 under the banner headline ‘The Big Story. Clegg: Propelled from a leaders’ debate into ‘the stratosphere’’. Not, presumably, the same stratosphere as the volcanic ash. (A related article appears on The Guardian website). The article described an ICM randomised telephone poll of 505 people interviewed both before and after the debate. The responses to three questions appeared in a story and in three pie charts on page 10.

• ‘Putting aside your own party preference, who do you think won the debate?’ The answers were: Clegg 51%; Cameron 20% and Brown 19%.
• ‘Based on their performance in the debate, who do you think would make the best PM?’ For ‘female respondents’ the answers were: Clegg 31%, Cameron 29%, and Brown 29%. (Men were more favourable to Cameron.)
• And three: ‘Did the debate change your mind about who to vote for?’ No, reported 62%. Yes, said 23%.

At the top of the page there was a long thin graph with three wriggly lines, one for each leader. The long x axis was time – from before the debate to shortly after its conclusion. The lines depicted ‘positive and negative responses as recorded by readers of guardian.co.uk.’ Starting together these lines slowly, erratically, but in the end consistently diverged, with Clegg at the top, Brown following below, and Cameron tailing off into negative territory. Obviously, we don’t really need to remind ourselves as we look at this, that The Guardian is scarcely the established organ of Toryism.

Just a little more detail. Look at the ICM questions about the character of the party leaders. These weren’t directly reported in the article where the story-line is pretty confusing. But here’s what we do learn from the article (for the results themselves and the questions, go to the ICM website):

• ‘[Clegg’s] apparent on-screen likeability meant that 43% of those who watched thought he appeared able to work with cabinet colleagues’ (Cameron 32%, Brown 20%);
• for Labour supporters he had ‘changed his party for the better’ (44%) more than Brown (34%);
• Brown was ahead as ‘the leader most likely to be respected around the world’ (Brown 40%, Cameron 34%, Clegg 20%).
• Cameron was just ahead on decisiveness (34% as against, Brown 33%, and Clegg 27%).
• And Brown was reported as being ‘most likely to make the right decisions when the going gets tough (Brown: 39%; Cameron 34%; Clegg 23%).

There was lots more in these two papers on the morning after, and the morning after the morning after, but that’s more than enough for the moment.

Is Democracy its Devices?

My colleague Evelyn Ruppert writes of ‘devices’. ‘To the masters of polling,’ she observes, ‘we now have added a multitude of mediators – from newspaper and television media to political organisations and individuals – who are packaging and interpreting data and creating unique data devices.’ She asks: ‘What kinds of mediators are these devices?’ Her response (I quote her again) is that ‘Indices are metrics that select and order data in way that configures comparisons.’

She’s spot-on. Look again at The Guardian’s wriggly lines. This is a device. It configures a metricated comparison of the performances, debating interventions, and the attributes of the leaders. (What the y-axis represent isn’t, I agree, entirely clear). We learn, for instance, that after 1 hour and 28 minutes Clegg gets a bounce when he ‘Namechecks many of the questioners in summing up.’ And the pie charts and the statistics are all about comparison too: as we’ve seen, the ICM-polled men like Cameron better than the women, while Brown leads Cameron as most likely to take the right decisions when the going gets tough. These, then, are devices that allow us to compare. Compare leaders.

But there is more. If these devices are mediators, then note that there’s a real sense that we don’t actually know what to think about Brown, Cameron and Clegg in the debate, we don’t know what happened, until they (the devices) have got to work. That’s why I quoted the flat-footed initial response of the FT. Given the paper’s early printing deadline, what’s important is not so much that it (more or less) got it wrong. That’s understandable. What’s significant is that it reveals there was no received answer to the question as to who was winning the debate until those devices started generating data.

Perhaps, then, it follows that Clegg only did well because the devices tell us so. You or I might think he did well (or otherwise). So might the leader writers (Peter Preston makes this point The Observer on 18th April – see the website) but this doesn’t really count. But then the same logic applies just as much to the standard opinion polls. And (we’ll need to return to this thought in due course) to the ‘real poll’ on May 9th. All of these are mediating devices, and the first-past-the-post constituency-based electoral system is no exception. Democracy depends upon its devices – as those who seek electoral reform or direct democracy are always reminding us, albeit in different language. Indeed, democracy is its devices.

Devices and Their Narratives

But there’s something else. This is obvious, but it bears repeating. Quantifying devices and narratives (narrative devices?) go together. Indeed, it doesn’t really make much sense to try to prise them apart.

‘Knives out for Nick Clegg after dramatic poll boost: Election thrown open as quarter who watched TV debate say they will switch vote.’

That was The Guardian’s front page headline the morning after the morning after (17th April). The narrative grew out of the polls (not just the ICM poll but the comparable results of others published in rival newspapers).

It looks, then, that without the polls, there would have been no ‘big story’. This suggests that polls can be thought of as devices for collecting stories, stories about (or from) lots of people: that those polls depend on narratives. They’re devices for eliciting more or less implicit stories. It’s obvious, really. You can’t phone people up without some patter. You can’t get them to say the kinds of things you want them to say without prompting them. Unfortunately ICM doesn’t tell us about its opening patter, but it does tell us how it worded its questions. Here, for instance, is question 2:

‘Putting aside your own party preference and only basing your answer on what you saw or heard during the debate, as far as you are concerned which one of the three leaders do you think won the debate?’

So what to make of this? We’ve no reason to suppose that the ICM respondents didn’t understand what they were being asked. But then if we look at what the interviewees needed to know in order to respond properly, the list is nearly endless:

• they needed to know there was an election on – and what an election is;
• they needed to know that there were (at least) three national parties;
• they needed to know that those three parties were represented in the debate (the nationalists’ views on their omission from debate were withering!);
• they needed to know about the party leaders;
• they needed to have some idea what a debate is; and
• they needed to have an idea – implanted if not there beforehand – that debates are there to be lost or won.

All this tells us that polls are devices that work by gathering and collating stories. But it also tells us that they work, too, as devices for framing story-possibilities. Some possibilities are opened up in polling devices (there’s an election on, there are parties, there are leaders, and electors are people who chose between parties and their leaders); and others are closed down. Take your pick, but versions of the political world according to the nationalists, the BNP, UKIP, anarchism, syndicalism, plebiscitary democracy, deep green feminism, theocracy and the revolutionary dictatorship of the proletariat all get sidelined in these kinds of polling devices. In greater or lesser degree they don’t fit within the framing of opinion polling devices and the narratives that these reproduce.

All this may seem a trifle abstract. Perhaps it is. In the real world of British representative democracy and its devices – the opinion polls and the ‘real poll’ – these are stories that fit more or less badly. But here’s the point: these devices generate stories that are probable, stories that are possible, and lines of storytelling that are so improbable they become more or less unthinkable. Though sometimes things shift. Suddenly, after this first leadership debate, it became possible to argue that the UK was in a three-horse rather than a two-horse electoral race. That’s what the debate devices told us. That’s the story-reality that they conjured up. For the time being they re-shaped the narrative possibilities and probabilities.

Narratives of Leadership

Go back to the ICM poll for a moment. Look at something else that it was doing. It was making it possible – nay, important – to talk of leaders (and the FT’s ‘more presidential style of politics’). And it was narrating a particular kind of leadership fulfilled in varying degrees by Brown, Cameron and Clegg. I’ve touched on most of this already:

• leaders are people able to work with cabinet colleagues;
• they are able to make the right decisions when the going gets tough;
• they are able to change their parties for the better;
• they have substance and don’t depend on spin;
• they are respected round the world;
• they are decisive;
• and then, of course, if they are any good they are also able to win.

That’s a pretty strong set of assumptions, a pretty strong (dare I say gendered?) story about the proper character of leadership. It’s also a pretty strong set of assumptions about the contestatory character of politics and the need for leadership.

All this might be good, it might be good in parts, or it might be bad. We can debate. But my root point is that without any discussion at all, this set of assumptions – this kind of leadership reality – was being assumed and reinforced in this ICM poll: the universe of political stories and styles was being set. Of course, in the abstract this is not a complaint. All devices work this way whatever their character. They have to frame the universe one way or another. That’s the nature of mediation. The other opinion polls, the debate, and the ‘real poll’ itself, all of these are devices that frame possibilities and impossibilities. They are carrying story-line assumptions, they are propagating them, and they are reproducing them. But I do think that it’s useful to poke around and think about the hidden agendas of devices, to ask about the work that they are doing in the background. The world may (or may not) have changed for Nick Clegg, but the realities of ‘leadership’ were roundly reinforced in the panoply of debate-related devices.

What kind of agent is a ‘voter power index’?

There are many forms of analysis mobilised during modern election campaigns. One form involves the deployment of data to predict outcomes and the electoral chances of individual MPs and parties. For decades, opinion polls have been the main technique of creating and deploying data to intervene in elections and much debate has occurred around their influence.  With new on-line and Web 2.0 digital techniques data is being mobilised in new ways, raising questions about how data can intervene in election outcomes.

We now have BBC’s Poll Tracker, an interactive display that tracks opinion poll support for Labour, the Conservatives and the Lib Dems since 1983. There is the Daily Telegraph’s interactive and dynamic political map that illustrates all constituencies along with 2005 election results, track opinion polls, play with a ‘swingometer’, and access information on candidates and government performance measures by constituency (in relation to improvement or decline) based on health, crime and education data. There are also many sites operated by individuals such as Electoral Calculus, designed by Martin Baxter, a financial analyst who specialises in mathematical modelling. The site is regularly updated with predictions of election results  based on his modelling and analysis of opinion polls and electoral geography. There are many, many more.

To the masters of polling we now have added a multitude of mediators – from newspaper and television media to political organisations and individuals – who are packaging and interpreting data and creating unique data devices. What kinds of mediators are these devices? Take, for example, the Voter Power Index (VPI), which was designed by Nic Marks, a fellow with nef (the new economics foundation). Marks describes how he came to develop the index after the last election when he realised that his ‘vote wouldn’t really count towards the result as I live in a safe seat.’ Incensed by this and as part of the nef’s report Spoiled Ballot, Marks thus created the VPI. He based it on two factors: how marginal a constituency is (the more marginal the more power of a vote) and the second the number of registered voters (fewer voters means each vote counts more). He then looked at as many elections as possible to figure out what was the probability of a seat changing hands at different levels of marginality. By creating a probabilistic model he then estimated this probability for every constituency and calculated the VPI. Basically it is a measure of how votes in marginal constituencies ‘count more’.

Since 2005 the VPI has been buried in the Spoiled Ballot report but now, thanks to web designer Martin Petts, an interactive web-site enables people to ‘see’ how much their vote is ‘actually worth.’ The VPI site enables you to find the index for your parliamentary constituency and also identifies the most and least powerful constituencies in the UK. Arfon in North Wales (1.308) has the highest index and Coatbridge, Chryston & Bellshill in Scotland has the lowest (.004). The site produces an easy-to-read display of a constituency, using color-coded pie charts and a power barometer. Both Marks and the nef see the VPI as a tool for advancing electoral reform, arguing that it is the first past the post system that leads to an unequal distribution of power and leads to each person not having one vote in the UK.

What are we to make of these additions to the repertoire of mediators engaged in election campaigns? All of these tools – from the VPI and swingometer to performance indicators -constitute and elevate a particular kind of political calculus. Part of the ‘power’ of the VPI is its capacity to produce an metric that reduces assumptions and calculations into a simple number that can be visually arrayed in a manner that is easy to read and entertaining too! Indeed, much fun can be had playing with the various interactive tools.  As you move through the various examples cited above (and many others) it is clear that different web designers and hosts are falling over each other trying to come up with the most interactive and powerful numbers and visual effects.

Indices are metrics that select and order data in way that configures comparisons. Of significance is the word ‘select’  – there are numerous possibilities and ways of ordering and indexing data. For example, we could take apart the VPI and challenge the underlying assumptions and criteria and offer alternative ones. But more than this there is another troubling aspect of VPI and all of the other election tools and metrics. I tend to think that these are more agents of distraction than forms of political debate and engagement. They perform and draw our attention to a calculus without substance. Indices are metrics that are nicely packaged, seductive and perform for us in ever more technically savy and flashy ways on websites. Perhaps they rightly belong to the same politics that we will watch on television in the form of the ‘historic’ leaders debates. The results of the ‘debates’ will likely hang more on the staging and performance of the three candidates for Prime Minister rather than any revelations of political substance. After all of their coaching by consultants and their memorisation of soundbites to pre-submitted questions, will the debates be more than packaged performances? Will grooming, performance, positioning, reaction, and body language be the new metrics of success?

20 July 2009

Evelyn Ruppert and Mike Savage

The row over MPs expenses reflects much about the current practice of identifying and knowing people on the basis of their transactions, on what they do rather than what they say.  It also shows in practice how new techniques such as Crowdsourcing, Google Docs and visualisation software can be used to analyse databases and mobilise collective projects. It indicates the potential of new mechanisms and devices for detecting and mobilising the popular.

Democratic political regimes have always depended on devices for eliciting ‘popular’ views. The most visible of such devices – the ‘vote’ – is only one, rather marginal device, though it remains the most fundamental to democratic legitimacy – take it away, and claims to democratic politics look thin.

Since the 1930s, it has also become clear that the delineation of ‘public opinion’ has become critically dependent on the mobilisation of evidence from polls and surveys, which can be construed as machines for arraying popular feelings in a ‘flat’ format, reduced to cell counts in a contingency table.  By the early 21^st century, however, a new potential for eliciting popular views has emerged in the form of digitised data sources. In this contribution we argue that these sources have the potential for construing democratic accountability in new and powerful, yet in unappreciated ways. Using the example of digital data on MPs expenses, we show how new kinds of popular mobilisation become possible. Our aim is to argue against naïve views that such data sources somehow allow more accountability. Rather they offer a new kind of politics, which alter the nature of political stakes.

Transactional data is put to many uses, one of which is to classify people. For example, based on our shopping patterns Tesco identifies groups according to lifestyle segments from the healthy or traditional consumer to the ‘price sensitive’ or ‘finer foods’ shopper.  In government, digital traces can be matched up to identify ‘benefits thieves’, tax dodgers, patients at risk of re-hospitalisation, security risks or frequent migrants.  By linking transaction patterns with postcodes, geodemographic classification software such as Mosaic can be used to categorise people further into groups such as ‘New Urban Colonists.’ Such data sources can themselves be resources that people use to reflect on and strategise their market behaviour, as they log onto websites advising them where they might live, and what kind of areas they are located in.

And now with the digitisation of MPs expenses we can categorise MPs according to their transaction patterns.  Who is the most expensive MP or the one who spends the highest on office expenses or travel or earned the largest windfall from a property transaction? At an amusing level, we can imagine categorisations such as the ‘profligate spender,’ ‘frequent diner,’ ‘excessive nepotist,’ ‘world traveler’ or ‘wannabe property developer’.  And just as data about ourselves can be used to detect fraud and criminal activity so too can MPs transactions be used to do the same.  The gaze has been turned from the governed back onto the governors. Clearly, and for understandable reasons, many of them don’t like it. This is perhaps the most telling indication of the value of these methods for extending democratic scrutiny. However, this is not our major concern here. We are much more interested in the forms in which this kind of informational politics comes, and what it makes both possible, and impossible in the name of democracy.   

We should recognise that this categorisation of MPs conduct is not new—for several years, the website TheyWorkForYou (run by volunteers and part of the charity mySociety) has been tracking MPs voting patterns, policy positions, committee attendance, public appearances and has compiled a numerology of debates and written questions. The site has also added tabulations of MPs expenses. The expenses scandal is therefore only the culmination of a much longer process of digital scrutiny.

One of the major issues which has already come to light in the high profile Daily Telegraph expose has been the labour intensive nature of combing detailed data, in this case with every item of MP spending itemised. Yet it is also apparent that it is possible to mobilise amateurs to amalgamate this data in a way that is certainly cheaper. While some 25 journalists at the Daily Telegraph spent several weeks detailing these accounts, The Guardian used Crowdsourcing to engage over 20,000 participants in the review and digitisation of MPs expenses. Crowdsourcing is a method that enables large, undefined and unsupervised groups of people to collectively perform tasks.  As of the end of July, over 200,000 out of about 500000 pages of MPs expenses had been reviewed, producing a publicly accessible database on GoogleDocs.

Let us consider in detail what can be done using such data. Table 1 is extracted from The Guardian database and represents a standard crosstabulation of aggregate categories (Labour, Conservative and Lib Dem). This is, in effect, treating data on expenses analogously to that from a survey in order to reveal aggregate findings. From this we can learn that Conservatives spend much more on carpets and on mortgages than do Labour MPs. So far, so stereotypical. It turns out that Labour MPs have more of a relish for running up a food bill, Lib Dems are especially keen on expensive TVs, whereas Conservatives prefer cheaper tellies and Labour MPs are somewhat in the middle. Lib Dems are much keener gardeners than those from the other two parties. But in fact, the general message from this Table is actually not very interesting. Aggregate differences largely balance out between the parties: one cannot say in general terms that Tories are more on the make than are Labour or Lib Dem MPs. In fact, there is not much of a story here.

This all looks rather different, however, when more advanced visuals are deployed, so that specific individual MPs can be highlighted. This of course, is where the Daily Telegraph expose has been most effective, not in placing generalised blame but in fingering particular MPs. This is a politics of ‘whole populations’, one in which individuals can be singled out vis-a-vis their peers and their idiosyncracies revealed in all their mundane splendour.

Let us consider this through seeing how the database has been analysed using IBM’s on-line Many Eyes Wikified visualisation software to produce bar charts, block histograms, scatter plots and maps. Visualisation software enables the detection of patterns that might not otherwise be apparent in the Google Docs spreadsheet.  Such visualisations reduce all of the detail and ‘noise’ of data to simplified patterns. One visualisation is a ‘heat map’ of Britain showing those areas with the highest (hottest) expenses down to the lowest (coolest). Another illustrates the location of each MP and their total travel expenses thus illustrating MPs who appear to be claiming disproportionately more than other MPs with constituencies in a similar area.

These first visuals give remarkable detail on the spending of every MP, on a range of different headings. By scrolling the cursor across the figure, every MP can be highlighted. Again, the overarching impression is that there is not very much of a story here. There is in fact rather little variance in terms of MPs expenses excluding travel. The most miserly MP is Holbourne who claimed £40.4k, and the most spendthrift are Borrow and Wallace, with £158k – but the bar chart shows very little variance around the mean figure of £90k or so. However, by distinguishing more specialised categories, apparent differences can be detected. Thus airfare is highly variable with McNeil charging £28k but most MPs claiming nothing. Rail fares give a more linear distribution, with Milburn topping the list with £13,650.

These are just a few of the possible kinds of analysis and visualisations of digital transactional data made possible by the mobilisation of thousands of people. In this way the same digital methods used by commercial and government authorities can be mobilised for popular political engagement. But we wish to highlight that methods of identifying and classifying people are not neutral and benign but involve choices about what is deemed important and about how people and things are categorised.  For example, the website TheyWorkForYou states that the quantity and type of activities of MPs reveals little about the quality of their work (and acknowledges that they do many other things not counted). Users are advised to read actual speeches and attend meetings and thereby use the site only as a ‘gateway’ to more detailed inquiry. But such gateways organise data in particular ways, make decisions about what should be included and excluded, and form the basis of comparisons and rankings and in doing so have consequences. Similarly, the categorising and analyses of MPs expenses involve the same kinds of decisions: different techniques and visualisations reveal different versions of expenses. The point that we wish to highlight is that digital data and methods do not simply provide different perspectives on expenses but rather mediate how we come to know the social world and intervene in politics. Thus, we not only need to scrutinise MPs expenses but also the very methods involved in such scrutiny and the differences that they enact. This is one meaning of the social life of methods: it challenges us to engage critically with the politics of new forms of digital data and analysis to provide reflexive accounts of the work that methods do.