Codebooks are not just for teams
by Angie Sibley-White, Senior Lecturer in Education, De Montfort University, and Gisela Oliveira, Senior Lecturer in Education, De Montfort University.
This blog is part of a short series of posts in relation to the use of codebooks in qualitative research.
There are two main reasons presented in the literature for the use of codebooks: one, to support/train a team of researchers on how to code; two, to improve the consistency of coding qualitative data, often within a team (e.g., Macqueen, et al., 2018). Both reasons highlight the existence of a group of researchers working together on the same project. However, codebooks are not just for teams. Here, we present some alternative benefits of codebooks that will make them attractive to a researcher working independently on a qualitative project. More specifically, this post will make the case that codebooks can (Oliveira, 2022):
Facilitate process and reflexivity
Facilitate early theorisation during coding
Codebooks, research process and reflexivity
One of our arguments is that codebooks can be artifacts of research practice and reflexivity. Indeed, when creating a codebook, researchers will take a structured approach that can bring clarity and ease to what is generally a messy activity. Through the codebook creation process, researchers will create codes, develop definitions for those codes that are supported by clear inclusion and exclusion criteria, and then apply those codes to the data set. This systematic approach, with a step-by-step process to follow will make the process more manageable for the researcher, but also be reassuring to the readers. It is evidence that the research was completed in a rigorous manner.
This reassurance, we argue, is particularly useful for researchers working independently in a qualitative project, because they will not have the benefit of showing intercoder agreement measures to convince readers of the quality of the coding. These are quantitative measures of coding consistency within teams (Milford et al., 2017) that are usually presented to show that the coding process was rigorous. When there is a single researcher coding the whole data set, this is not needed. However, the question on how to evidence rigour remains. This is where the codebook is fundamental, as it emerges as a living artifact of research process and quality that the readers of the research can audit.
In addition, we would also like to argue that the codebook process will include many useful prompts for reflexivity. For example, when creating and using codebooks, the researchers will have to consider methodological choices (e.g., to use or not use multicoding – see blog post 3 on this series), conceptual choices (e.g., should specific codes be created, merged or deleted), and processual choices (e.g., what steps will the data analysis include). Although these are considerations that all qualitative researchers are likely to engage with at some point in their projects, the benefit of the codebook is that it continuously prompts the researcher to consider these choices, and it codifies the researcher’s decisions into a research artifact – the various iterations of the codebook. These in turn, create an explicit audit trail of how the research was developed.
Codebooks and theory development
When developing a codebook, researchers must make decisions about the process of coding that will impact the analysis of the data. For example, researchers must decide if they will use multicoding or simultaneous coding. This is the option to assign different codes to the same section of the transcript. In practice, it means that the researcher might use different codes to code the same sentence or paragraph within the transcript.
Multicoding can make analysis more complex by expanding the amount of coded data under each code, but it can also assist the researcher in identifying patterns in the data set. This is because multicoding will reveal potential connections between different codes, when they are used to code the same segments of text. Through this process, multicoding can make implicit meanings explicit.
In our experience of using codebooks, multicoding is one of the most valuable strategies for early theorisation. Allowing for multicoding in our projects (Oliveira, 2022; Sibley-White, 2024) made us aware of specific connections between different codes and allowed the exploration of the meaning for those connections. Indeed, after multicoding reveals a connection, researchers can check if that connection happens across the data set, or just in specific transcripts/participants. They can investigate if there are additional connections with other codes, what are the similarities or differences between participants or their contexts, and, consider what might explain the patterns they found. Reflecting on the underlying reasons for the patterns that the use of the codebook revealed will take researchers beyond the mere mechanics of coding, and into the in-depth analysis of the data – they are now, not just coding, but theorising the data!
Overall, we argue that these additional benefits of using a codebook have not been explored enough in the literature that focused on their benefits for teams. By highlighting them here, we wish to encourage qualitative researchers working independently to consider the potential value of this tool to improve their projects in terms of their process, early theorization and rigour.
References:
DeCuir-Gunby, J., Marshall, P., & McCulloch, A. (2011). Developing and using a codebook for the analysis of interview data: an example from a professional development research project. Field Methods, 23(2), 136–155. https://doi.org/10.1177/1525822X10388468
Milford, C., Kriel, Y., Njau, I., Nkole, T., Gichangi, P., Cordero, J. P., Smit, J. A., & Steyn, P. S. (2017). Teamwork in Qualitative Research: Descriptions of a Multicountry Team Approach. International Journal of Qualitative Methods, 16(1), 1-10. https://doi.org/10.1177/1609406917727189
Oliveira, G. (2022). Developing a codebook for qualitative data analysis: Insights from a study on learning transfer between university and the workplace. International Journal of Research and Method in Education. https://doi.org/10.1080/1743727X.2022.2128745
Sibley-White, A. (2024). A critical evaluation of the impact of Government neoliberal policies on leadership and experiences in English primary schools. PhD Dissertation. De Montfort University.