Putting Quality Into Qualitative Research
What, exactly, is qualitative research, and more to the point, how can it be best deployed? Maybe those are the wrong questions to start with, suggest the authors a new textbook, Successful Qualitative Research: A Practical Guide for Beginners. Instead, say Virginia Braun and Victoria Clarke, it might be better to figure out what you want it to achieve and not become fixated immediately on the method. Qualitative research is not just “one thing,” they said in a recent interview; it’s a diverse ecosystem that’s capable of achieving many things. “And it’s often trying to do something quite different to quantitative research, so understanding what it is trying to do is really important.”And so their 382-page book and its companion website (which comes close to equaling the amount of words in the book) set out to answer that question, the definition question, and many more, in an engaging but still pedagogically robust way. The authors are both psychology professors -- Braun, an associate professor at the University of Auckland and Clarke, an associate professor at the University of West England – who admit to loving qualitative methods, but are leery of committing ‘methodolatry.’
And yet, others idolize Braun and Clarke’s presentation. (That's them to the side: Virginia Braun, left, and Victoria Clarke) The book, published by SAGE, has gained its share of kudos already: Last month the Association for Women in Psychology gave it the 2014 Distinguished Publication Book Award, while it’s on the shortlist this month for the British Psychological Society’s annual book award. “Reading it,” Ithaca psychology professor Carla Golden, who chairs the AWP awards committee, wrote the authors, “and made me almost wish I was a younger scholar about to take on a major research project so I could implement all that I learned from it, and/or that I taught a methodology course at my institution.”Below, the authors discuss the making of both successful and unsuccessful qualitative methods, the seemingly eternal tension with quants, and what they think makes their book special. If there’s one lesson that learners can take away from the volume, it’s this: “To be thoughtful, careful, critically-reflexively-aware qualitative researchers, who creatively deploy data collection and analytic approaches that best suit their research questions.Note: When ‘I’ is used the speaker is Virginia Braun.In the book you’re unabashed about your love—your word—of qualitative methods. Why is that?I love language – I love the beauty of it, the creativity of it, and people’s creative use of it. Language is amazing if you start to look at it closely, and start to look at it as more than just a straightforward communication tool. Doing qualitative research allows you to keep a focus on language, if you want to.Another reason I love qualitative research is that it offers you a very different window into people’s worlds, and social worlds, than other approaches to social science research. It captures beautifully the patterning, as well as the contradictions, of experience; of perspectives and understandings; of how people explain their lives… And qualitative approaches give the researchers tools to focus both on the unique specific aspects, as well as the patterned aspects…And to understand these as telling us a whole range of different things. If, for instance, you think language is really important as a ‘tool’ by which realities and experiences are shaped or created, and molded, and reworked, and dismantled and recreated, then working with language makes complete sense – because you can explore those sorts of things ‘in action’ as it were.There’s a joy and exciting potential to be found in qualitative data, that I don’t feel with numbers – so the way I relate to it is both affective, and intellectual. Qualitative research ‘makes sense’ to me in lots of different ways. Partly, as well, because it usually (but certainly not always!) fits best for the sorts of research questions I have.What does your book do that earlier volumes did not?We wanted to do a few things with this book that we felt were unique and missing from many other qualitative textbooks.First, we wanted to be extremely practical. The teaching of qualitative research can often be somewhat ephemeral, and we are not unique in that our learning qualitative research fit the ‘learning a craft skill’ analogy which lots of writers have noted – just without any knitting patterns! We had excellent qualitative researchers throughout our studies, and we owe a lot to what they have given us. But we see how our students – both in taught classes, and especially research students – can really struggle with the lack of practical ‘guidance.’We also got a lot of feedback from our 2006 thematic analysis paper where people said they so appreciated its very practical approach (though we worry that sometimes people look at the practical aspects, and ignore the other more theoretically informed discussion, such as not treating themes as emergent! But that’s perhaps another discussion). So we wanted the book to emphasize the practical steps, the dilemmas that you might encounter, and the practical side of things, in starting to do, and doing, qualitative research.Second – and closely related – we wanted to disaggregate the practical aspects of qualitative research from theory. We had both been through a ‘need to know the theory first’ model of qualitative research learning, and I wholeheartedly still believe that good qualitative research needs good theoretical knowledge behind it. We need to know theory, and our research needs to be theoretically grounded. But the theory around qualitative research – especially if it’s of the more constructionist, poststructuralist varieties – can be really mind-boggling for students, particularly if they are, as they often are in psychology, taught within a very positivist/experimental research tradition. We have noticed that in the classroom, or in supervision, this theory can provide a stumbling block and an impediment to learning. We wanted to ask the question – can we teach a certain range of skills for qualitative research, which give students a sense of understanding and mastery, before we immerse them in theory. Can we flip the usual directional model of ‘teaching qualitative research’ on its head a bit? So this is what we’ve tried to do – have enough theory in the book to locate qualitative research, but emphasize the practical – yet still highlight the need for a return to theory!Third, we wanted to provide a swathe of materials which show what qualitative research actually looks like – in all aspects. Some of these are in the book; many more are on the companion website. Having the website allowed us to really boost the resources we could provide (for students, but also for teachers) – and there were nearly as many ‘words’ on the website as the book itself!Finally, we wanted it to be fresh, engaging, real, and hopefully even funny in places. We wanted to reveal the messy truths, the mistakes, the dilemmas – as well as the joys – that are part of qualitative research.I’m a journalist. You are qualitative researchers. Are we the same? Since I suspect not, how exactly are we different?I would say no, we’re not the same. Though I did want to be a journalist many aeons ago - before I did psych, I even applied to study journalism at university.I think we’re similar in that we both seek understanding of a phenomenon, and we seek to understand it through a practical of research that often gathers information through language… and we report back, publicly, and we present our ‘stories’ through language as well.We are different in that we have different expectations and codes that guide us (I don’t know yours enough to comment on those specifically) and I think of qualitative research as a more directed and designed activity than journalism. Although we (also) have the scope within qualitative research to ‘follow the story’ to some extent, I think we tend to define our parameters earlier (we’re required to, often, with ethics).We also have a system where quality is judged by expert peers, who refer to both existing standards in the field, as well as the individual product in front of them. This is, I think, a fairly unique aspect of academia and the intellectual endeavor. I think also the outcome processes are different, too – I think we potentially have more control over our product and outputs, though this depends massively on what and how you are working, both as an academic, and as a journalist.That said, I know more about my field than yours, and I’m wary of making grossly generalized statements that are based on ignorance!There always seem to be some quantitative practitioners who are dismissive of qualitative methods, and vice versa. How do you address that tension? Or what do you say to such critics?My basic response is ‘please be open-minded’! And know what’s driving your judgment, if you have it. And to know what you’re talking about. Don’t just dismiss research – qualitative or quantitative – out of hand.I think this practice varies considerably with discipline (as well as by place), so I’ll talk to the situation in psychology. ‘Qualitative research’ has always been part of psychology, from some of the earliest days. But as the discipline evolved, it became strongly quantitative, and even more, associated with experimentation and methods that allowed for the testing of hypotheses and the assessment of causative relationships; the development of explanations and models.Qualitative research in psychology, then, emerged in a field which was – and still can be – fairly hostile to it! So many qualitative researchers have had to battle against the dominant paradigm(s), often in a context where they understood the parameters of quantitative research (having been trained in it), but where quantitative researchers hadn’t necessarily been trained in qualitative research. And given certain values associated with the dominant paradigm(s), qualitative research is often dismissed out of hand. But this is ridiculous – a bit like saying dismissing an apple as terrible because it’s not an orange – instead of judging the quality of that apple on its own terms – recognizing that you can get a very poor apple, and you can get an excellent one (and the whole spectrum in between). And if you just hate apples, full stop, and will never like an apple, you generally would recognize that as your own personal quirk – not a shared universal truth.Any research (qualitative or quantitative) can range from the excellent to the poor, and research needs to be judged first on its own terms of reference, and second on how well it serves its purpose. There are bigger questions behind this – around how we theorise the individual and the social – which has, for instance, was part of the so called ‘crisis in social psychology’ and the development of diverse approaches within social psychology. So these broad theoretical frameworks do cluster together approaches which may be appropriate or not, and questions which may be suitable or not, for a discipline. But beyond that, we need to ask, ‘Can this research generate information that answers the research question?’, and if so, does it do that in a way we consider to be of good quality?What I find very problematic is when quantitative standards get treated as the unquestioned universal standard, and applied to qualitative research. I see this all the time. It happens in peer reviewing of papers, of research proposals and grants, of PhDs, etc. It happens with journal editors. I frequently get emails asking ‘how should I reply to this critique?’ – for instance, not having ‘inter-rater reliability’ calculated for coding of data, through using two trained independent coders. Well, that criterion is hugely problematic, but lots of people who don’t know better, treat it as a gold standard, and that if you have it, and get a good IRR score, your research is somehow validated.As a qualitative researcher, if you’re trying to get grants, and trying to publish in journals that are mainly quantitative, and thus dealing often with reviewers who aren’t expert in qualitative research, you often have some hurdles to jump though. I do find it astounding that I can have an article reviewed by a reviewer who admits that they aren’t that knowledgeable about qualitative research, and yet they can still make judgments about it, that I have to justify – and my expertise is somehow take at the same level as their ignorance around qualitative research (and sometimes even questioned!). But it does happen.
I do find it astounding that I can have an article reviewed by a reviewer who admits that they aren’t that knowledgeable about qualitative research, and yet they can still make judgments about it, that I have to justify – and my expertise is somehow take at the same level as their ignorance around qualitative research
This can make you grumpy, and defensive! Especially as it’s not the case that the reverse applies, where quantitative research gets judged on qualitative terms (well it would certainly be very rare in psychology!). It means publishing can be more restricted, and more of a battle, than if you’re a quantitative researcher. But it’s a battle I think is worth fighting, as it has the potential to shift the discipline, and it’s also educative – as long as those who read our responses accept them. And there’s a lot of ignorance and incorrect assumptions and information about qualitative research. That said, it’s also really important to note the diversity of the qualitative field, and lots of qualitative research does get done in ways that conform to lots of the expectations of a quantitative paradigm – which Louise Kidder and Michelle Fine have referred to as ‘small q’ qualitative research. Qualitative research is not one thing (another sometimes assumption); it’s incredibly diverse. And it’s often trying to do something quite different to quantitative research, so understanding what it is trying to do is really important.Your book is titled Successful Qualitative Research. What are hallmarks of UN-successful qualitative research?There is qualitative research that might be seen as successful by the researchers themselves, but which qualitative researchers might judge to be poor quality – and therefore unsuccessful. Then there’s research which is unsuccessful because it fails to deliver, in a whole host of ways, on what it aimed to do. And then there’s research which is unsuccessful, because it doesn’t get completed. All of these I would consider UN-successful.In calling the book ‘successful’ we were trying to capture two aspects of ‘success’: the skills needed to complete a project, and the skills needed to design (and complete) a project that is of high quality and (best) enables you to answer your research question. We wanted to get away from the idea of the interview (or maybe the focus group) as the ‘method par excellence’ of qualitative data collection (as others have described it), and think about treating qualitative research as an activity informed by your questions as much as your skills. I think we also believe that it’s important to take a ‘best practice’ approach to qualitative research. Although we don’t want to promote methodolatry, as Stephen Reicher has warned of, and although ‘quality criterion’ are debated and some contest them, we do feel that it’s important that researchers are aware of good and bad practice(s) in what they’re doing, and enact good practice…But also that they have the knowledge to defend their practice (which relates to the point below)…So what might count as some examples of ‘unsuccessful’ qualitative research? Here are a few examples:- Poor quality data that don’t appear well suited to answering the research question?- A mismatch between theoretical position, or even methodological approach, and the sorts of claims that are made. Or no theoretical position acknowledged, at all.- Very basic descriptive research that offers little beyond the participants’ words.- Research unreflexively based within a quantitative paradigm that appears to have little or no awareness of what qualitative paradigms require. How would you assess the current state of qualitative research in psychology (i.e. used appropriately, overused, underused, poorly reported, etc.)?Overall, it feels like a massively growing approach to psychological research, but it varies a lot, country by country (and university by university)… In our locations (New Zealand; England) qualitative research is quite widely practiced, and fairly well accepted. In other places (e.g., USA, Australia) it’s far less integrated into the discipline. There’s definitely scope for a lot more qualitative research, but I would like to see more of better quality. Currently, there’s a real mix out there. If qualitative research was a totally accepted part of the psychology research cannon, then we’d have better teaching, and a richer exploration of approaches, etc. – so the discipline would benefit. And I think that would mean we’d get far better standards overall.