Qualitative Research & Policy Studies: Interview with Editors

JS. In the introduction to this special issue, you state: "These political and discursive interventions in education, however, are not solely to blame for the dismissal of qualitative inquiry. Some of this blame must be apportioned to researchers themselves." Can you make suggestions for researchers who want to conduct qualitative or mixed methods research, but fear they will face the kinds of criticisms you highlighted? 

​MK & ATJ: A great deal of criticism of qualitative research and mixed methods research (MMR) does emerge from positivist perspectives on the purpose of research and appropriate methods, while MMR also suffers from critiques from methodological purists. However, it would be remiss of us to not also acknowledge that we ourselves contribute to a poor perception. We came to this conclusion primarily through the publication review process. Many times, we've reviewed articles where the methods were insufficiently described, poorly attributed (if at all - it's frustrating to see how many researchers fail to acknowledge the whole body of literature on research methods in their procedures), and poorly reasoned. It is so important to carefully and thoroughly answer the questions: Why did you choose those methods? Why were they appropriate? Both qualitative inquiry and MMR have their own expectations for quality, rigor, and trustworthiness in the reporting process; we need to live up to them. Practically, this adherence to these standards of rigor also enables researchers to push back on unfair and inappropriate critiques.

Furthermore, many researchers neglect to address the constructivist nature of qualitative inquiry - this is what sets our epistemology apart and yet we fail to acknowledge the role this plays in our work through discussions of positionality and reflexivity. By engaging in all of these issues more thoroughly, we can demystify our research process and more properly manage perceptions of opacity. We don't think this contributes to "rigor mortis" but instead honors the foundations of these methodologies, while still allowing for creativity in the construction of methods.

JS: Do you have suggestions for those who teach research methods?​

MK & ATJ: As methods instructors, we are responsible for the messages our students receive about rigor. First, as instructors we should take it upon ourselves to learn about other strategies of inquiry or at least engage with other researchers whose epistemology differs from our own. A siloed approach to our work only perpetuates myopic perspectives rather than advancing methods as a whole for the next generation of researchers. Second, if we do not adequately engage our students in these conversations, then how can we expect them to appropriately apply techniques that would ensure the dependability, credibility, confirmability, and transferability of their research? For example, doctoral students in education may engage with data produced from a local schooling context, but this engagement is often troubled by messages about evidence and data within the broader context of the technologization of education policy (Ball, 1998). Without sufficient guidance, students may manage the tension between evidence and context by essentializing phenomena in pursuit of the unnecessary requirement of generalizability as a standard of validity, ultimately weakening their research and its implications. We need to teach them various approaches to ensuring the rigor of their work and instill in them the value of engaging these approaches consistently.

JS: You note that those writing about policy-oriented research need to communicate in ways that policy-makers can understand. Qualitative research often generates stories that bring social context to life and give voice to those whose situations are under scrutiny. What practices for analysis and reporting are most effective? 

MK & ATJ: We are cautious about identifying what is “most effective” but rather we advocate for narrative, visual, and other forms of storytelling to convey quantitative, qualitative, and mixed methods research findings in a compelling and engaging way. Researchers are now beginning to realize the value of effective infographics or visual displays for sharing information concisely while also being consumable so that findings can inform decisions and actions. In addition, narratives that include research participants’ own words remain powerful because of their ability to embody complex ideas and evoke a response in the reader. As researchers engage with policymakers, the power of narrative helps to capture the interest of, and promote understanding among, policymakers.

JS: What roles or practices do you recommend for journal editors and reviewers who want to give voice to a wide range of researchers, rather than "silencing researchers on the margins—queer, feminist, disabled, and/or indigenous"?

MK & ATJ: This question reveals challenges for two different, though overlapping, groups: first, researchers who are on the margins as a function of their own identity; and second, researchers whose scholarship focuses on the margins. Editors and the editorial board are crucial for both. Prior to choosing a publication as a home for a manuscript, we review who sits on the editorial board. In our field, we often see that these boards are made up of quantitative researchers, which then conveys the message that researchers on the margins may not be welcome. Quantitative methodologists are often perceived as allowing very little room for the voice of participants or the researcher - voices that are critical to the stories on these margins. Journals need to do a better job of both diversifying their editorial boards to include researchers marginalized by (post)positivist assumptions about research and encourage more methodological variety. Our professional associations can also play a role in getting this message out and holding journals accountable. It is also a socialization issue. For example, Inside Higher Ed reported a gender citation gap, where men cite women much less than their peers. To what extent should editors be responsible for the citation practices of their authors? Moreover, when we teach, do we stress the importance of embracing diverse, often marginalized, research(ers)?  These are tough questions that we've yet, as a field, to really grapple with.

We also encourage editors to consider to the guidelines set by APA Style for Journal Article Reporting Standards (APA Style JARS), which establishes guidance for reviewing standards for different methodologies. Notably, with respect to reporting findings from qualitative data, they recognize the challenges experienced by researchers and reviewers. It is incumbent upon editors to select appropriate reviewers to ensure authors are not held to inappropriate standards, which results in barriers to publication. This process also helps editors better evaluate reviewers’ responses to an article and ensure they are consistent with the strategy of inquiry.