Systematic Reviews in the Social Sciences

There is a tendency amongst some students looking for an approach for their undergraduate dissertation to claim they want to do “library based research”, by which they mean they are going to look through whatever literature is available to them on a particular topic, ie whatever is available in the university library or easily accessible on the internet, and write a kind of extended essay or literature review on the subject. I wouldn’t wish to undermine the value of such desk-based research, and indeed when done well some very good dissertations have been completed in this way.

There are, however, different ways of approaching desk-based research or the extended literature review. One approach is to produce a piece of theoretical research. This does not mean “in theory this will be a good dissertation but I haven’t written anything yet”, but might mean the application of a particular theory or body of theoretical literature to a particular case, a feminist critique of the UK counter-terrorism policy for example.

When it comes to the extended literature review, it may also be possible to produce what is known as a systematic review. A systematic review is an approach to reviewing the existing literature on a subject which involves the adoption of a transparent and replicable system in the identification and analysis of the literature. A detailed description of the method then forms a part of the study itself. The notion is that this is more thorough and less subject to bias than traditional literature reviews which tend to involve the researcher picking and choosing those works, and themes they think are important.

Systematic reviews have been common in some fields, such as medicine, for a long time, but have recently been applied more widely, particularly in social policy. They provide a means of reviewing all the evidence on a particular policy. Systematic reviews must be comprehensive and cannot, by definition, be limited only to the literature available in the nearest library. Consequently they involve a lot of work and are very challenging but they can result in a rewarding and useful study.

The Centre for Reviews and Dissemination at the University of York have been at the forefront of systematic review and have produced some useful guidance. There is inevitably a journal devoted systematic reviews which provides an invaluable editorial on the nature and growing use of systematic reviews. Although not widely covered in the research methods literature there is a useful section on systematic reviews in Bryman’s Social Research Methods, 3rd edition, pp.85-91, while Sundberg and Taylor-Gooby’s systematic review of comparative studies of attitudes to social policy is a good example of the approach.

Posted in Uncategorized | Tagged , | Leave a comment

Writers on writing: Stephen King

IMG_0065I’ve just finished reading Stephen King’s, On Writing: a memoir of the craft. It is part memoir and part guide book for the aspiring writer. I have observed in the past that while, one hopes, there may be significant differences between what goes into a work of fiction and a carefully researched piece of social research, when one sits down in front of a blank screen to communicate ideas the challenges can be remarkably similar. King’s book is full of useful advice. While some of it obvious, it is no less valuable for that, and it is considerably more engaging than staid self-help guides one might read.

Almost all writing, fiction and non-fiction, is a creative process and as such there can be a tendency to imbue it with certain mystical qualities. How many times have you heard someone say they have to be in the mood to write. This is something I have observed in students, and of which I have also, on occasion, been guilty. King’s book seeks to demystify the process by encouraging us to treat writing like any other form of work, something which should be done on a regular (daily) basis but which doesn’t need to take over our lives. Sitting around waiting for inspiration to strike is likely to be much less productive than setting aside several hours each day to sit down and write. As King advises:

Don’t wait for the muse… he’s a hardheaded guy who’s not susceptible to a lot of creative fluttering. This isn’t the Ouija board or the spirit-world we’re talking about here , but just another job like laying pipe or driving long-haul trucks. Your job is to make sure the muse knows where you’re going to be every day from nine ’til noon or seven ’til three. (p.157)

One can help this process by creating the right environment. King is not overly prescriptive about this, some people will work best in the library (although King advises against this), others will take themselves off to a motel, a quiet cottage in the woods or at least a shed in the garden. J.K. Rowling famously wrote the first of the Harry Potter novels in a café in Edinburgh. King tells us that he wrote his first two published novels, Carrie and Salem’s Lot, ‘in the laundry room of a doublewide trailer, pounding away on my wife’s portable Olivetti typewriter and balancing a child’s desk on my thighs.’ He adds that while the space can be humble, (‘you don’t need an Early American rolltop desk in which to house your writing implements’), it should have one thing, ‘a door which you are willing to shut.’

The closed door is your way of telling the world and yourself that you mean business; you have made a serious commitment to write and you intend to walk the walk as well as talk the talk. (p.155)

He does suggest that, ‘for any writer, but for the beginning writer in particular, it is wise to eliminate every possible distraction.’ There should be ‘no telephone in your writing room, certainly no TV or videogames for you to fool around with’. On writing was published in 2000, but one might now add, disconnect the internet or at least switch off social media. Although King does admit that he works to loud rock music like AC/DC, Guns ‘n Roses and Metallica! Once the door is closed, King suggests a daily target and that the door stays closed until that goal in met. As with physical exercise, to avoid discouragement he suggests setting a low target to begin with. His recommendation to begin by aiming for 1000 words a day is fairly standard in guides to writing.

Sticking with the writing as work analogy, King also recommends that all writers should build up a toolbox to enable them to do their work. He narrates a story from his childhood in which he accompanied his uncle, a carpenter, to carry out a job on the other side of town. As this turns out to be a relatively simple job requiring little more than a screwdriver and a handful of screws, King is puzzled as to why his uncle chose to carry a heavy toolbox across town. Of course as any craftsman knows, when embarking on a job, one never knows what tools one might need, and his uncle naturally responds:

“I didn’t know what else I might find to do once I got out here did I? It’s best to have your tools with you. If you don’t, you’re apt to find something you didn’t expect and get discouraged.” (p.114)

The writer’s toolbox, King suggests, is not a physical thing, it has nothing to do with pencils, pens and notebooks, and may therefore be less physically demanding to carry, but it should be no less complete and failure to have a well-stocked toolbox is likely to be discouraging. It is comprised of those intellectual tools which will make the process of writing easier. Like the carpenter’s toolbox, the writer’s toolbox should have several levels and the tools within it should be well maintained, ‘if some are rusty (as they may be if you haven’t done this seriously in awhile), clean them off.’ The most commonly used tools, King suggests, should be kept at the top. The first two levels should be comprised of vocabulary and grammar, with additional levels, drawers and boxes containing information on style, paragraphs and sentence structure.

King also has much to say about the process of drafting and redrafting. Perhaps the most significant and succinct piece of advice was offered to King by the editor of a provincial newspaper early in his career, ‘write with the door closed, rewrite with the door open’. The first draft is for you, but once you’re reasonably happy, or as happy as you can be, then let others read your work, and redraft it accordingly. As with many guides, he recommends putting a piece of work aside between drafts. On returning to it you will often (hopefully) find that it is better than you remembered. He also passes on a useful formula, provided by his High School English teacher, for dealing with what is, in my experience, one of the most common preoccupations of students – word count. The formula, which King still claims to apply, is ‘2nd draft = 1st draft – 10%.’

What the formula taught me is that  every story and novel is collapsible to some degree. If you can’t get out ten per cent of it while retaining the basic story and flavour, you’re not trying very hard. The effect of judicious cutting is immediate and often amazing – literary Viagra. You’ll feel it and your… reader will, too. (p.223)

There is much else of interest in King’s memoir/guide, but for those who are not interested in how King became a writer, or how he got back to writing after a one-sided encounter with a truck in 1999, I would strongly recommend the chapters on the writer’s toolbox (pp.111 -137) and the section on paragraphs (pp.129-135) in particular, and the sections on reading (pp.145-154), the writing room (pp.155-162), revising (pp.208-220) and the ideal reader (pp.220-227).

If all of that sounds a bit much, remember the following

If you want to be a good writer, you must do two things above all others: read a lot and write a lot.

Write with the door closed, rewrite with the door open.

2nd draft = 1st draft – 10%.

Stephen King, On writing: a memoir of the craft, New York: Scribner 2000. The book was serialised in The Observer in 2000, there are also, inevitably, a huge number of interviews with King on the subject including this and this from The Atlantic and this from Open Culture.

Posted in Uncategorized | Tagged , | Leave a comment

Beware the alibi of photocopies

IMG_2694There are many intellectual aspirins in Oliver Burkeman’s weekly, ‘This column will save your life’, in the Saturday edition of The Guardian. The column ‘What unread books can teach us’ is particularly pertinent to the budding researcher.

In it Burkeman, drawing on observations by the writers Umberto Eco and Nassim Taleb, defends the tendency of writers, and indeed academics, to surround themselves with groaning shelves of books, many of which they will never read. While some assume that this is simply showing off, Umberto Eco points out that books are a research tool, and like any good toolbox our shelves should be filled not only with those we will use on a daily basis, but others which may serve very particular but no less important function at some point in the future. Similarly Taleb argues that the real aim of a personal library should be to fill it with the books you have not read, rather than those you have. According to Burkeman, Taleb advises that your bookshelves should contain ‘as much of what you do not know’ as finances allow. There is a simple logic to this. If we assume that there is no way that we will ever be able to read everything, then there is always more to be learnt from the books that we haven’t read than from the relatively small number that we have. As Burkeman observes ‘unread books are where the action is.’

However, Burkeman’s column also contains a warning from Eco that we should not assume that accumulating unread work is the same as accumulating knowledge. Eco refers to this as ‘the alibi of photocopies’. By this Eco identified a practice which is surely familiar to all of us. That by the simple act of photocopying or printing a book chapter or journal article we sometimes, perhaps often, assume that we have read it and internalised it.

It is important to remember that accumulating data, whether that is books, articles or research data, is not the same as having read or analysed it. The pile of books and journal articles sitting on the corner of your desk does not constitute research. Research is what begins to happen when you read them. Moreover, as Burkeman points out, Eco was writing about the alibi of photocopies in the mid-1970s. Developments in information technology have significantly expanded our capacity to accumulate unread texts, but have had little impact on our capacity to read them. There is then real value in unread books, as long as we don’t make the mistake of assuming that we’ve read them.

Posted in Uncategorized | Tagged , | 1 Comment

Writers on writing: Ian Rankin

On the face of it there may be little similarity between the social sciences and crime fiction, but I often find that there is a great deal to be learnt from professional novelists. The one thing we do have in common is the need to communicate ideas through writing. One of the reasons why successful novelists are successful is their ability to write and in many cases to write quickly. Moreover, one often finds that successful writers have given considerable thought to their craft, to the process of writing, something which data-obsessed social scientists perhaps do less of. In this context I particularly liked this interview with one of my favourite novelists, the Scottish crime writer, Ian Rankin, in The Globe and Mail in which he outlines five habits which can lead to successful writing. Not all of these are applicable to research in the social sciences. In my own writing I have yet to encounter the knotty problem of writing a sex scene, and in non-fiction writing the ending is rarely much of a surprise. Nevertheless, Rankin’s advice about the importance of getting into the habit of writing, is a lesson from which we could all benefit:

Just keep writing

If I’m working on a novel, I try to write every single day. It may not be much, but it’s seven days a week. I think that by writing quickly you inject pace. If a book is written quickly, it tends to be a quick read as well. My first draft will be very rough, but it won’t take me much more than 30 or 40 days. I remember one writer that I met couldn’t start chapter two until he got chapter one just right. That’s just not how I work. My rule is to just get the thing down on paper. Even if there are mistakes, misjudgments, I’ll ignore that and know that I can go back later. I remember I was talking with an agent one time. He asked how things were going and I said they were going great, but that I was just about to take a break because I have to research this one thing. He said no, no – just make a note to yourself to do the research later and keep writing. I save a lot of time these days by doing the bulk of the research between the first and the second draft. Research can be a rabbit hole and you don’t come out for weeks. Once you have a first draft, you know what you need and that saves a lot of time.

It is important to remember, as Rankin acknowledges, that there are different approaches to writing and that some people would baulk at the notion of writing a quick first draft before the research is complete. But it is hard to dismiss the notion that when one has some momentum it is probably better to carry on writing than to stop to check every fact, as long as one remembers to check them later. As someone who can spend a great deal of time chasing ever more data, Rankin’s characterisation of research as ‘a rabbit hole… you don’t come out of for weeks’ is particularly striking. We’re not writing fiction so there are times when we can’t just make it up, but there are also times when we all need to avoid the rabbit holes and just keep writing.

Posted in Uncategorized | Tagged , | Leave a comment

On the importance of literature reviews

In a recent article in The Washington Post, Daniel W. Drezner, professor of international politics at the Fletcher School of Law and Diplomacy at Tufts University, neatly summarised the importance of literature reviews in academic papers:

Done well, a literature review states what the extant research has to say about a topic. Is there a scholarly consensus on the question or not? Are there contending schools of thought? What puzzles persist? What data controversies are there?

Peer referees like to see literature reviews in papers, because it sends a signal that the author is keenly aware of what preceded his or her article. An inadequate literature review can be the kiss of death to a paper if the author then proposes an argument or test that the referee knows about but goes uncited.

More importantly, a literature review is the way for someone who is not an expert on this particular topic to digest the current state of play. As someone who is a bit of a generalist in international relations, I find literature reviews extremely helpful, because they let me get up to speed quickly on a new area of research.

More advice here from the Enago blog.

Posted in Uncategorized | Tagged | Leave a comment