Video Data Analysis: How 21st century video data reshapes social science research
by Nicolas M. Legewie and Anne Nassauer, co-authors of Video Data Analysis: How to Use 21st Century Video in the Social Sciences.
CCTV disguised as cubist art
Midjourney AI
In 2011, anti-government protests and uprisings erupted in Northern Africa and the Middle East in what is often called the ‘Arab Spring’. For the first time, smartphone recordings and social media posts played a crucial role in the spread of protests and their global visibility. Thousands of citizens filmed the protests with their phones, often capturing the same instance from multiple camera angles. Especially Egypt’s self-proclaimed ‘Facebook generation’ uploaded large amounts of these videos on social media platforms. As a result, everyone could see first-hand footage of the protests, taken from the perspective of those involved in the marches. Major media outlets began to incorporate such videos in their investigation and coverage to find out and convey what happened on the ground. As one journalist describes: ‘In most cases citizens capture the breaking news moments first. The Arab spring was really the tipping point when it all came together’ (Batty 2011). Since then, video footage from smartphones, surveillance cameras, and other non-newsmedia sources form a natural part of reporting on and our perception of uprisings and civil war, from the 2021 Storm of the US Capitol to the war in Ukraine and many other large-scale events. Ubiquitous video cameras also capture rare events such as mass panics and natural disasters, including the panic at a 2010 music festival in Germany that left 21 people dead and 652 injured. Thousands of videos on social media sites show behavior and interactions during floods and earthquakes. On May 25, 2020, the brutal murder of George Floyd was filmed by bystanders‘ mobile phones, CCTV cameras, and officers’ body-worn cameras.
But not only the extraordinary, dramatic, or terrible is being captured. People use smartphones to document more common events, such as weddings and funerals, or to capture mundane events, such as playing video games or spending time with friends. CCTV cameras capture everyday behavior in public places, on school yards, and even in workplaces. Parallel to this increase in third-hand video data, advances in camera and data storage technology also enabled new ways of collecting first-hand videos for research, by researchers. In short, we find ourselves in a new era of how social life is captured.
Video capture is everywhere, offering new opportunities for social scientists.
Use the code MSPACEQ323 for a 20% discount, valid until September 30, 2023.
Smartphone head
Midjourney AI
For social scientists, who are inherently interested in how social life ‘works,’ these videos can generate completely new insights. Over the past decade, using 21st-century video data has allowed researchers across sociology, criminology communication studies, anthropology, psychology, education research, political science, and other disciplines to make groundbreaking discoveries. Researchers can now use video data to find out how we talk to each other, how we express emotions, how we fight, or how we learn. We can use video to study what successful workplace meetings look like, how family interactions shape pre-school learning, or how people do racial discrimination in education or in the housing market. Thanks to video data, we can look at situations step-by-step and frame-by-frame to understand sequences, interaction routines, communication patterns, social hierarchies, or other aspects of culture and social life. Moreover, through video data, we have access to permanent first-hand recordings of situations and events that we did not necessarily observe ourselves. We can look at the same situation over and over again, as a team or alone.
Using these new sources of video data and building on established theoretical perspectives and methodological approaches, new lines of inquiry have emerged across the social sciences, in which researchers use video data to study situational driving dynamics of social processes and events. Situational dynamics refer to things that happen when people interact with one another, while being in temporal proximity and physical or mediated co-presence. Recent Studies use video data to seek to understand the situational dynamics of real-life events and processes by studying human behavior, interactions, and emotions in situations, and how they impact social outcomes (for a list of recent publications in the field, click here). From these developments, Video Data Analysis has emerged as a methodological framework in an effort to systematize methodological development, formulate open questions and solutions, and increase inter-disciplinary dialogue.
Video Data Across Disciplines
One of the most prominent among recent applications in sociology is Collins’ (2008) analysis of pictures and videos to study emotional dynamics in a variety of violent and near-violent situations. Collins focuses on the minutes and seconds before and during violent behaviors and identifies emotions in actors’ facial muscles and body postures. This approach allows Collins to challenge core assumptions of conventional theories of violence, by showing that situational emotions, instead of actors’ prior strategies or motivations, trigger violent behaviors. Visual data are instrumental in enabling Collins to develop his argument and corroborate his findings. His study suggests that “there is crucial causality at the micro-level” (Collins 2016), which can be uncovered using visual data. Further examples from studies on deviant behavior include analyses of massacres (Klusemann 2009) and protest violence (Nassauer 2019). Criminologists increasingly use visual data to study crime as it unfolds (Lindegaard and Bernasco 2018; Stickle et al. 2020). In the field of policing (McCluskey et al. 2019; Sytsma, Chillar, and Piza 2021), videos can capture how real-life police-citizen encounters unfold and thereby contribute to our understanding of policework as part of social order, the state monopoly of force, procedural justice, and police use of force. In law studies, scholars have used video data to, among other things, better understand the jury deliberation process (Diamond et al. 2003) and successful interrogation techniques (Alison et al. 2013).
In business and organization studies, scholars have researched behavior and team work in emergency call centers (Fele 2008; Mondada 2008) or nuclear power plant control rooms (Waller, Gupta, and Giambatista 2004). In primary care research, videos have been used to study topics such as staff-patient interactions in nursing centers (Caldwell and Atwal 2005), training and simulation exercises of health care workers (Hunziker et al. 2011), and cooperation in anesthesia teams (Burtscher et al. 2010; for an overview, see Heath, Hindmarsh, and Luff 2010:8ff; Asan and Montague 2014). Research in education, too, has a tradition of studying visuals to examine non-verbal and verbal aspects of social interaction, and videos are now a prominent tool in the field (Derry et al. 2010:4). Examples include classroom interactions (Andersson and Sørvik 2013) and peacemaking among children (Verbeek 2008). In political science, videos have been used to study the impact of candidate debates on audiences’ voting behavior (Brierley, Kramon, and Ofosu 2020) and cooperation and outgroup perception (Chang and Peisakhin 2019; Choi, Poertner, and Sambanis 2019).
As the examples illustrate, in Video Data Analysis researchers use video data to find out how we talk to each other, how we express emotions, how we fight, or how we learn. We can use video to study what successful workplace meetings look like, how family interactions shape pre-school learning, or how people do racial discrimination in education or in the housing market. Thanks to video data, we can look at situations step-by-step and frame-by-frame to understand sequences, interaction routines, communication patterns, social hierarchies, or other aspects of culture and social life. Moreover, through video data, we have access to permanent first-hand recordings of situations and events that we did not necessarily observe ourselves. We can look at the same situation over and over again, as a team or alone.
The main source of data in Video Data Analysis (VDA) studies are ready-made or custom-made videos that capture events or situations in real life (i.e., outside laboratory settings). Ready-made data are produced as by-products of non-academic activities. Custom-made data are collected by researchers for the purpose of a given study. Both have unique strengths and drawbacks for VDA, but both have a number of unique affordances: They comprise direct and detailed observations of driving dynamics at the micro-level during real-life social processes and events, are easy to re-watch and share with colleagues and readers, and often enable highly efficient data collection.
Studies of what happens “on the ground” during events or situations can take three perspectives (for details, see Nassauer and Legewie 2022): seeing such micro-level processes as consequences or manifestations of “structural” or “macro” phenomena (e.g., the education system); seeing micro-level processes as phenomena in their own right (e.g., to understand how we maintain order in everyday life); or perceiving micro-level processes as driving forces behind macro phenomena (e.g., micro-level processes may produce differences in the frequency of officer-involved shootings in the United States).
Social life is now ubiquitously captured on camera.
For researchers interested in social life, these recordings offer a new world of insights. Paraphrasing Carl Sagan, we are still at the shore of an ocean of possibilities that video data offer. Over the next decades these possibilities will benefit scientific research and our understanding of human behavior.
Video Data Analysis: How to Use 21st Century Video in the Social Sciences by Anne Nassauer and Nicolas M. Legewie (2022)
Video data is transforming the possibilities of social science research. Whether through mobile phone footage, body-worn cameras or public video surveillance, we have access to an ever-expanding pool of data on real-life situations and interactions. This book provides a flexible framework for working with video data and understanding what it says about social life. With examples from a range of real video research projects, the book showcases step-by-step how to analyse any kind of data, including both found and generated videos. It also includes a non-technical discussion of computer vision and its opportunities for social science research.
Use the code MSPACEQ323 for a 20% discount, valid until September 30, 2023.
More Methodspace Posts about Creative Methods
Sometimes taking a break from the keyboard to write by hand unleashes creativity.