Sage Innovators
A How-To Webinar Series
Teaching Research Methods to Prepare Students for Careers in Behavioral Science and Data Science with Gorilla
Teaching Research Methods to Prepare Students for Careers in Behavioral Science and Data Science with Gorilla Experiment Builder
In this webinar, Johanna Tomczak, Research Specialist at Gorilla Experiment Builder, shares insights into the most cutting-edge approaches and practical examples for teaching research methods, illustrating where and how these techniques can be applied. She discusses how online research has enabled both staff and students to reach larger samples and gather data more quickly, positively impacting both teaching and career preparation. By embedding research directly into teaching methods, students experienced the full scope of behavioral science research firsthand - from study design and participant recruitment to data analysis. Watch on to discover innovative strategies and best practices to create a dynamic learning environment where students are actively engaged in the research process, equipping them with valuable skills for their professional futures.
About the Speakers
Johanna Tomczak works as a Research Specialist at Gorilla Experiment Builder. She completed her degree in Mathematics, Computer Science and Cognitive Science at the University of Grenoble (France), before obtaining her PhD in Cognitive Psychology from the University of Leeds (UK) in 2022. Her thesis, which examined the impact of bilingualism and ageing on the communication between the cerebral hemispheres, has shown her the benefits of online research and she decided to join Gorilla to work on improving behavioral science as a whole.
Additional Resources
Teaching Research Methods with Gorilla Blog
What over 1,000,000 participants tell us about online research protocols
The Kings Business School class of 2021
UCL charitable giving project
Gorilla Grants
Gorilla Presents Webinars
BeOnline Conference
Support:
Previous Sage Partnered Gorilla Webinars:
Q&A
+ Does Gorilla provide tools to simplify data analysis for students?**
Yes, Gorilla’s platform is designed to make data analysis approachable. It encourages students to think about their research questions and hypotheses first, which naturally informs their data analysis plans. This approach can make statistics more intuitive, as students see how their choices relate directly to their research questions.
+ What are some best practices for engaging students in research through teaching?**
A hands-on approach, where students can create their own research tasks, tends to be effective. We’ve found that students enjoy the applied aspect of Gorilla, and it helps reduce "stats anxiety" by making data analysis more accessible and relevant to their research.
+ Does Gorilla provide resources for creating practical teaching simulations with dummy data?**
Yes, Gorilla offers curricula and guides, particularly in psychology. These materials help teachers set up courses, analyze data, and streamline the learning process. We also have an online support system that answers students’ questions, reducing the burden on instructors.
+ Can Gorilla capture audio files, such as gathering verbal responses from participants?**
Yes, Gorilla can capture audio responses. You can set up tasks to record participants’ voices, and once you download your data, these audio files will be included. Learn more here and here.
+ How easy is it to import experiments from other software, like E-Prime, into Gorilla?**
Currently, Gorilla doesn’t have an E-Prime integration. However, there is a Qualtrics importer for survey and questionnaire imports. We’re also working on integrations with other task-building tools. If there’s significant demand for E-Prime integration, users can suggest it for our development roadmap.
+ Can Gorilla be used in clinical research settings, like medicine or surgery?**
Absolutely. An example is a study where radiology students used Gorilla to analyze CT scans, identifying potential medical issues. Gorilla recreated the CT scan environment, allowing for interactive tasks similar to what they’d use on medical devices. This setup can be adapted to various clinical training scenarios.
+ Can Gorilla be used in GIS (Geographic Information System) applications, such as environmental health data collection?**
While Gorilla primarily supports cognitive psychology and related fields, it may support integration with GIS tools if they can connect via USB. For specialized setups, please contact Gorilla for more guidance.
+ How does Gorilla’s pricing work, and does it cover participant recruitment?**
Gorilla has different subscription tiers, including department and lab packages. Recruitment fees for services like Prolific or MTurk are separate. Alternatively, users can distribute a study link for unpaid participants at no extra cost.
+ Is there a university-wide subscription option available for Gorilla?**
Yes, Gorilla offers bespoke university-wide subscriptions in addition to department, lab, and individual researcher plans. Find out more about pricing here.
+ Does the researcher subscription limit data collection to 200 participants, and can additional responses be purchased?**
The researcher subscription includes data for 200 participants, but additional “top-up” packs for more participants are available.
Explore How To Webinar Series
Online research advanced tools: Multiplayer, gamified experiments and shopping simulations with Gorilla
Online research advanced tools: Multiplayer, gamified experiments and shopping simulations with Gorilla Experiment Builder
In this webinar, Johanna Tomczak, Research Specialist at Gorilla Experiment Builder, showcases powerful tools like Game Builder, Multiplayer, and Shop Builder. Discover how advanced online research tools are transforming the way behavioral science experiments are conducted! Learn how these tools create engaging, ecologically valid experiments without needing coding skills. Whether you're studying consumer behavior or cognitive processes, these features can help elevate your research by motivating participants and ensuring higher-quality data.
About the Speakers
Johanna Tomczak works as a Research Specialist at Gorilla Experiment Builder. She completed her degree in Mathematics, Computer Science and Cognitive Science at the University of Grenoble (France), before obtaining her PhD in Cognitive Psychology from the University of Leeds (UK) in 2022. Her thesis, which examined the impact of bilingualism and ageing on the communication between the cerebral hemispheres, has shown her the benefits of online research and she decided to join Gorilla to work on improving behavioral science as a whole.
Additional Resources
Gorilla Game Builder:
Gorilla Shop Builder (a winner of the 2023 Sage Concept Grant):
Gorilla Grants
For £50 research credit
Gorilla Presents Webinars
BeOnline Conference
Support:
Previous Sage Partnered Gorilla Webinars:
Q&A
+ How can researchers recruit participants for their studies on Gorilla?
Gorilla offers various recruitment services that integrate natively, such as Prolific, Amazon Mechanical Turk, and Sonar systems. If you prefer not to use an external service, you can create a link to your study and share it via social media or email. This method allows anyone with access to the link to participate in your study. Many students prefer this approach as it doesn't involve additional fees.
+ Can Gorilla be used for both qualitative and quantitative research?
Absolutely! Gorilla supports both qualitative and quantitative research. We have a survey tool for creating qualitative surveys, and participants can record responses via webcam. For quantitative research, Gorilla captures reaction times, accuracy, and interaction data, such as mouse clicks. All participant actions are logged as part of the dataset, providing a comprehensive range of data.
+ What research domains can benefit from using Gorilla?
Gorilla can be applied across many domains, including military research, teaching, economics, and security analysis. For example, we’ve had researchers study attention in military pilots using online experiments. The Shop Builder tool is great for examining consumer behavior, and Game Builder allows for gamified research. Social behavior can also be studied using multiplayer features. Gorilla can even simulate websites, such as online shops, to study user interactions and reaction times.
+ How is statistical significance determined when using Gorilla?
Determining statistical significance depends on your specific research design. For example, in a study comparing three different shop designs, you might calculate average reaction times and use ANOVA or t-tests to compare the results across participant groups. While the specifics vary by field, Gorilla provides guides to help users with data analysis.
+ Are there resources available for learning how to create gamified experiments in Gorilla?
Yes, we have a recorded webinar on our website that walks you through setting up a game using Gorilla’s Game Builder tool. This hands-on tutorial explains everything from start to finish. Additionally, our support page contains extensive resources on all Gorilla tools, making it easy to find the information you need.
+ What is Gorilla's fee structure?
Gorilla has a tiered pricing model. It’s free to use for building and testing experiments. When you start collecting data, pricing kicks in, and you're charged per dataset (participant). We offer a pay-as-you-go option or subscription plans for labs and departments. Many UK universities already have subscriptions in place, but pricing details are available on our website if needed.
Explore How To Webinar Series
Online Research Advanced Tools: Social media & Website simulations with Gorilla
Online Research Advanced Tools: Social media & Website simulations with Gorilla Experiment Builder
In our latest webinar, Johanna Tomczak and Max Mawby shared their expertise on combating misinformation through cutting-edge inoculation methods. Johanna, a Research Specialist at Gorilla Experiment Builder with a background in Cognitive Psychology, and Max, a leading practitioner in behavioral science with extensive experience in both the private and public sectors, provided valuable insights into their approaches. If you missed it, watch the full video recording and review the Q&A session to learn about their strategies for enhancing digital literacy and influencing positive change.
About the Speakers
Johanna Tomczak works as a Research Specialist at Gorilla Experiment Builder. She completed her degree in Mathematics, Computer Science and Cognitive Science at the University of Grenoble (France), before obtaining her PhD in Cognitive Psychology from the University of Leeds (UK) in 2022. Her thesis, which examined the impact of bilingualism and ageing on the communication between the cerebral hemispheres, has shown her the benefits of online research and she decided to join Gorilla to work on improving behavioral science as a whole.
Max Mawby is a leading practitioner in applying insights from behavioral science to drive positive change. He heads Thinks Applied Behavioral Science team with extensive experience of designing and running online experiments for government and industry clients. Max is passionate about applied behavioral science and how this can improve our lives. In particular, he is interested in driving positive financial behaviors, sustainable decisions, and using behavioral insight to make design inclusive for all.
Most recently, Max conducted an online randomized controlled trial (RCT) to explore the threat posed by election disinformation ahead of the 2024 UK general election. This focused on a behavioral approach to testing and addressing simple interventions with the potential to help limit the spread of mis and disinformation.
Additional Resources
Thinks Insight and Strategy: Addressing disinformation online case study
Due diligence support information
What over 1,000,000 participants tell us about online research protocols
Webinar about improving participant engagement
Webinar about gamification with Dr Drew McLaughlin
Previous Sage Partnered Gorilla Webinars:
Q&A
+ What is the best approach to rolling out an inoculation intervention to effectively reach people and change their behavior on social media?
The key challenge with rolling out inoculation interventions is effectively reaching people in a way that actually influences their behavior on social media. While online research offers a controlled environment, applying these findings in real-world settings remains complex. The real challenge lies in scaling and distributing these interventions to achieve meaningful impact.
+ Can you discuss the concept of confounding variables and how they can affect the validity of research findings?
Confounding variables are factors that might influence the outcome of a study but are not controlled or accounted for. In behavioral research, it's nearly impossible to control every variable, and some may not even be recognized until after the study is conducted. This inherent limitation means that researchers must acknowledge the potential for confounding variables and the compromises they make in controlling their research environment.
+ How can researchers address the challenge of confounding variables when conducting online research?
Online experimentation and simulation offer a controlled environment that helps minimize confounding variables. By using platforms like Gorilla, researchers can manage and control various aspects of the study, leading to cleaner data sets and more reliable results. Although some confounds may still exist, online research generally provides a clearer and more manageable setting compared to field studies.
+ How do online research methods compare to traditional lab-based research in terms of controlling the research environment?
Online research methods, while offering a controlled environment, often create an illusion of control compared to traditional lab settings. In labs, researchers can more directly observe participants and control variables. However, online platforms allow for a wide reach and can manage many variables effectively, making them a useful tool for obtaining clean data and conducting research at scale.
+ How do you ensure data quality and participant authenticity in online research?
Ensuring data quality and participant authenticity online involves implementing checks and attention measures within the research design. Despite not being able to physically observe participants, online platforms can use various techniques to verify participant identity and engagement, leading to more accurate and reliable data.
+ Are there tutorials on Youtube?
Yes, here.
Explore How To Webinar Series
Teaching Research Methods in a Digital World with Gorilla
Teaching Research Methods in a Digital World with Gorilla Experiment Builder
Dive into the future of research education with our latest webinar Teaching Research Methods in a Digital World with Gorilla. Led by Jo Evershed, Johanna Tomczak, and Dr. Miles Tufft, this session highlights innovative strategies and concrete examples reshaping research methodologies for the digital age. An essential watch if you’re an educator training the next generation of researchers: stay at the forefront of behavioral science education!
About the Speakers
Jo Evershed is the Founder CEO of Gorilla Experiment Builder, a powerful, flexible and intuitive platform for running behavioural research online that allows researchers to go far beyond surveys. She also convenes and hosts the Behavioural Science Online Conference. As an Innovate UK Women in Innovation Award Winner and Computer Weekly Rising Star Women In Tech, Jo is on a mission to provide behavioural scientists with tools to liberate their work from the lab and accelerate research initiatives that can be tested rigorously at scale. Jo leads a multidisciplinary team of software engineers and psychologists focused on creating powerful and accessible experimental research infrastructure.
Johanna Tomczak works as a Research Specialist at Gorilla Experiment Builder. She completed her degree in Mathematics, Computer Science and Cognitive Science at the University of Grenoble (France), before obtaining her PhD in Cognitive Psychology from the University of Leeds (UK) in 2022. Her thesis, which examined the impact of bilingualism and ageing on the communication between the cerebral hemispheres, has shown her the benefits of online research and she decided to join Gorilla to work on improving behavioural science as a whole.
Dr Miles Tufft is a Lecturer in Experimental Psychology and co-director of the Eyethink Lab at University College London. He graduated with a BA in Natural Sciences from the University of Cambridge and holds an MSc in Cognitive Neuroscience and a PhD in Cognitive Psychology from UCL. Prior to academia, he worked for over a decade in industry including seven years as a management consultant where he applied data science to inform strategic and organisational change. His research examines the perceptual and attentional mechanisms that underpin distributed cognitive systems. Currently, he investigates the ways in which information deemed to be the responsibility of others in joint contexts is actively prioritised or de-prioritised to optimise individual or team performance (social offloading). He has received a Provost Teaching Award from UCL and currently convenes research methods on the BSc Psychology programme. He also works extensively with industry partners to help embed enterprise and knowledge exchange activities into the undergraduate curriculum.
Additional Resources
Gorilla Grants - expression of interest
Due diligence support information
Webinar about improving participant engagement
What over 1,000,000 participants tell us about online research protocols
Teaching Faculty mentioned:
Gillian Brooks: Students’ Presentation and her Module
Miles Tufft: Open Teaching Materials
Rachel Theodore: Her Experience
Previous Sage Partnered Gorilla Webinars:
Q&A
+ Which operating system or browser do I need to use for Gorilla?
Gorilla operates seamlessly in web browsers, accessible from various devices, including smartphones, for both study creation and data collection. Comprehensive support resources are available, including tutorials on YouTube and documentation on the Gorilla website. These resources provide thorough guidance for users, ensuring smooth navigation and utilization of the platform.
+ Are participants restricted by location, or can you recruit internationally?
Participants can be anywhere globally.
+ Is Gorilla applied in qualitative research?
This platform offers a range of survey components for qualitative research, including open text responses, multiple choice questions, and image, audio and video stimuli. It allows for various data collection methods, such as talk-out-loud protocols and video recordings, even enabling the creation of sign language corpora. Subsequent analysis can be conducted outside the platform, offering flexibility for researchers.
+ How is data security ensured?
We meet stringent data security standards required by universities worldwide and are fully GDPR-compliant, ensuring robust protection for human data. With extensive experience across 1,000 universities, we undergo rigorous assessments and maintain top-tier servers subject to annual penetration testing. Our platform offers comprehensive data processing agreements, privacy policies, and tools for anonymous data collection and minimization. We prioritize data security, providing customizable options to safeguard sensitive information effectively.
+ How do you ensure that the participants are not bots or bad actors?
We offer various options to control for factors like attention checks in tasks, allowing researchers to filter out incomplete data effectively. Using Gorilla, you only pay for complete participants, minimizing costs associated with incomplete responses. Online research with Gorilla generally shows minimal evidence of checkbox or bot issues, ensuring data quality.
+ In using Gorilla for teaching, do you have sort of a user manual that people can use to get started?
Our platform supports student success with onboarding tutorials and resources, including the Gorilla Academy and Gorilla support pages. Many universities use Gorilla for undergraduate teaching, enabling students to engage in research projects and gain valuable skills. We offer ample samples and materials, alongside webinars teaching effective navigation and modification. Gorilla Open Materials provide additional content for exploration and extension of research projects.
+ How does utilizing Gorilla assist students in grasping research methods such as counterbalancing, randomization, and experimental controls? Specifically, how does this interaction facilitate understanding, and what benefits does it offer in terms of comprehension?
Engaging students with Gorilla effectively enhances their understanding of research methods such as counterbalancing and randomization. Rather than solely conceptualizing these complex ideas, interacting with them in a tangible way on the platform allows students to grasp them more deeply. By actively manipulating elements on the screen, students can experiment, make mistakes, and refine their understanding, leading to a more embodied comprehension of research design concepts. This approach is vital as it shifts the focus from mere mechanics to broader thinking about good research design and question formulation, essential aspects of experimental psychology. Ultimately, by offloading technical aspects onto the platform, students can concentrate on developing robust research questions and collecting meaningful data, which is crucial for their growth as experimental psychologists.
+ Can you give some more information about eye-tracking?
Yes, eye tracking in Gorilla is feasible through webcam technology. It allows researchers to measure participants' gaze location on the screen, albeit with some differences compared to in-lab eye tracking. Despite potential noise in the data, online eye tracking still yields comparable results to lab-based studies. This method is particularly useful for experiments involving visual stimuli, such as comparing participant gaze between different images on the screen. Comparisons between lab-based and online eye tracking data show similar trends, with variations in noise levels across different environments, like classrooms and home settings.
+ Are there tutorials on Youtube?
Yes, here.
Explore How To Webinar Series
Online Research for PhDs and PIs - New Possibilities - with Gorilla Experiment Builder
Online Research for PhDs and PIs - New Possibilities - with Gorilla Experiment Builder
Welcome to this post dedicated to exploring the transformative potential of online research, in partnership with Gorilla Experiment Builder. In the wake of the COVID-19 pandemic, the landscape of behavioral research has evolved dramatically, with digital platforms offering unprecedented opportunities for innovation and collaboration. Join us as we delve into the dynamic world of online research, highlighting the invaluable insights shared by experts from Gorilla Experiment Builder. From cutting-edge tools to addressing common concerns, this post serves as a guide for researchers seeking to harness the full capabilities of digital methodologies. Let's revolutionize the way we conduct behavioral science research in the 21st century.
About the Speakers
Jo Evershed is the Founder CEO of Gorilla Experiment Builder, a powerful, flexible and intuitive platform for running behavioural research online that allows researchers to go far beyond surveys. She also convenes and hosts the Behavioural Science Online Conference. As an Innovate UK Women in Innovation Award Winner and Computer Weekly Rising Star Women In Tech, Jo is on a mission to provide behavioural scientists with tools to liberate their work from the lab and accelerate research initiatives that can be tested rigorously at scale. Jo leads a multidisciplinary team of software engineers and psychologists focused on creating powerful and accessible experimental research infrastructure.
Johanna Tomczak works as a Research Specialist at Gorilla Experiment Builder. She completed her degree in Mathematics, Computer Science and Cognitive Science at the University of Grenoble (France), before obtaining her PhD in Cognitive Psychology from the University of Leeds (UK) in 2022. Her thesis, which examined the impact of bilingualism and ageing on the communication between the cerebral hemispheres, has shown her the benefits of online research and she decided to join Gorilla to work on improving behavioural science as a whole.
Additional Resources
Due diligence support information
Webinar about improving participant engagement
Previous Sage Partnered Gorilla Webinars:
Q&A
+ Is Gorilla applied in qualitative research?
This platform offers a range of survey components for qualitative research, including open text responses, multiple choice questions, and image, audio and video stimuli. It allows for various data collection methods, such as talk-out-loud protocols and video recordings, even enabling the creation of sign language corpora. Subsequent analysis can be conducted outside the platform, offering flexibility for researchers.
+ How is data security ensured?
We meet stringent data security standards required by universities worldwide and are fully GDPR-compliant, ensuring robust protection for human data. With extensive experience across 1,000 universities, we undergo rigorous assessments and maintain top-tier servers subject to annual penetration testing. Our platform offers comprehensive data processing agreements, privacy policies, and tools for anonymous data collection and minimization. We prioritize data security, providing customizable options to safeguard sensitive information effectively.
+ How do you ensure that the participants are not bots or bad actors?
We offer various options to control for factors like attention checks in tasks, allowing researchers to filter out incomplete data effectively. Using Gorilla, you only pay for complete participants, minimizing costs associated with incomplete responses. Online research with Gorilla generally shows minimal evidence of checkbox or bot issues, ensuring data quality.
+ In using Gorilla for teaching, do you have sort of a user manual that people can use to get started?
Our platform supports student success with onboarding tutorials and resources, including the Gorilla Academy and Gorilla support pages. Many universities use Gorilla for undergraduate teaching, enabling students to engage in research projects and gain valuable skills. We offer ample samples and materials, alongside webinars teaching effective navigation and modification. Gorilla Open Materials provide additional content for exploration and extension of research projects.
+ From a Research Ethics perspective, do you have automatic debrief material following the use of deception during research data collection, built into the software?
Researchers are responsible for debriefing participants appropriately after experiments, given the vast range of methods and experiments conducted. The Experiment Builder allows researchers to create debriefing materials to be presented at the end of the experiment, enhancing the participant experience and gathering valuable feedback. It's good practice to solicit feedback from participants on their experience to improve future experiments. Utilizing the debrief screen for feedback can greatly enhance the quality of research outcomes.
+ Do you provide official statements on participant confidentiality for IRB purposes?
Our data security policies and confidentiality can be found on this due diligence page.
+ How does Gorilla access participant data and is an additional disclaimer necessary for participant consent?
As a data processor, we only access data with explicit permission granted by users. This access is limited to situations where users request assistance, implicitly granting permission to our support desk. Users retain control over their data, and we only access it to support research purposes with explicit user consent. We adhere strictly to contractual agreements and ethical guidelines regarding data access and privacy.
+ When studying reaction time, how do you ensure/control for internet speed?
On Gorilla, researchers can restrict participants to have certain connection speeds to be able to access their experiments. The online timing accuracy page provides a useful resource to review the differences in terms of timing between online and lab-based research.
+ What are some innovative online research methodologies that you've found effective for PhDs and PIs?
Replicating lab studies online ensures data quality and comparison. Utilize resources like onboarding webinars for tool familiarization. Researchers can then explore theoretical extensions, like age or cultural differences, using multiplayer tools or gamified simulations to enhance research depth and applicability.
+ Can you share examples of successful online research projects conducted by PhDs or PIs?
A lot of them can be found on the BeOnline conference page, hosted by Gorilla.
+ What advice do you have for PhD students and PIs looking to integrate online research into their projects?
Exploring diverse experiments on platforms like Gorilla offers valuable insights. Utilize resources like onboarding webinars and support documentation to familiarize yourself with tools and capabilities. Start by replicating lab studies online to ensure data quality and comparison. Then, explore theoretical extensions using multiplayer tools or gamified simulations to broaden studies and enhance research findings.
+ Could I do a longitudinal study where the same participants log-in and complete a task once a week for many weeks and it can keep track of these participants (mixed model design, both between and within-subjects)?
Of course, this is definitely possible. Researchers can design multi-session studies on Gorilla and participants will be prompted to complete the next session once the time has come for them to be invited back.
+ It seems your tool can eliminate, for example, lab bias (people often behave differently in lab environments than in natural environments - and today, most of us feel naturally in front of a screen). On the other hand, the ability to control the course of the experiment could be minimal. How do we know the respondent conducted the experiment as it should have been? In other words, does Gorilla offer tools to eliminate respondents' biases? In the case of eye-tracking, how are measurement calibration and accuracy ensured?
A good starting point could be to think about the illusion of control we usually have in lab-based experiments. More specifically, there are several researchers who have already tried eye-tracking in the browser – successfully! Gorilla offers calibration.
+ Could we “manipulate” other parts of the 'shop builder' e.g. the Privacy notice?
Most things in Gorilla are customisable, including the Shop Builder settings. Have a look at the complete guide with Gorilla’s tools and services.
Explore How To Webinar Series
Gamifying Cognitive Tasks with Gorilla
How to gamify cognitive tasks to increase participant engagement - A Gorilla Masterclass
How to gamify cognitive tasks to increase participant engagement is an online tutorial that we held in May 2022. Here we share the webinar video with you. We have also added the questions that were asked during the live session and their responses. If you have a question, please send it through using the form below, and we will follow up with a response and any other resources.
More engaging experiments means better quality data - participants are more motivated and attentive and are less likely to underperform due to boredom or fatigue. Moreover, visually rich experiences help you verify your findings persist in richer (i.e. more ecologically valid) settings. With Gorilla's Game Builder, rich games and game-like experiences are accessible to research scientists without touching a line of code.
About the Speakers
Jo Evershed - Founder CEO
Jo is the Founder CEO of Gorilla Experiment Builder, a powerful, flexible and intuitive platform for running behavioural research online. An Innovate UK Women in Innovation Award Winner, Jo is on a mission to provide behavioural scientists with tools to liberate their work from the lab and accelerate research initiatives that can be tested rigorously at scale. Jo leads a multidisciplinary team of software engineers and psychologists focused on creating powerful and accessible experimental research infrastructure.
Nick Hodges - Founder CTO
Nick is the Founder CTO of Gorilla and has been building platforms for online research for nearly ten years. Before that he worked in the videogame industry on titles such as Call of Duty, Resident Evil and Lara Croft, and has always been passionate about enabling people to build rich and compelling interactive experiences.
Additional Resources
Participant Engagement Webinar
This is Sarah Jayne Blakemore's Director Game for studying ToM in Adolescence
For 20% discount on a 1 year subscription, use the code: GORILLASAGE2022
Q&A
+ How is analysis done with games in research?
You can create whatever game you need for your research questions. Analysis is the same as for non-game tasks. You'll get the data file of all the participant responses and then you can analyse them to look at accuracy and reaction time.
+ Does your platform handle audio files?
Yes - you can have audio playback, and (soon) also audio recording.
+ What kind of research questions you can answer with this game?
This is a classic go-no-go task - so often used to look at inhibitory control. The important thing is not the mechanics of this game, but how you can add images, control stimuli, add animations.
+ Can you apply gaming techniques to online survey completion?
Potentially yes - the example with the crime scene could work (e.g. even N questions they get another clue).
+ Can we gamify physical tasks? Are such gamification complicated (e.g., requiring IoT, etc)?
It would probably require your participants to have whatever hardware you require, which would likely be prohibitively complicated/expensive.
+ Is gorilla providing different gamified experience for every experiment/task?
Gorilla includes tools that allow you to build your own games - I'll be demoing them in the second half of this session. Stay tuned!
+ I would be interesting in exploring whether this could be applied to more complex thought processes. Can you run sequential or choice dependant programmes - leading to different outcomes?
Yes - you can create games that have more complex sequences of screens depending on the choices made. Reach out and we can chat further. Feel free to send a message to @EvershedJo on Twitter.
+ Will some of those games you’ve shown be available for replication?
The games I've shown belong to researchers, so I can't share them. Some have shared them to Gorilla Open Materials to make it easy for other researchers to use. This is Sarah Jayne Blakemore's Director Game for studying ToM in Adolescence
+ Does Gorilla provide templates for various cognitive tasks, or do we have to "hard-code" the tasks into the gamification?
There are lots of samples here.
And many researchers publish their tasks to Gorilla Open Materials
So yes - often you can find the cognitive task you need, and just change the spreadsheet.
+ Can you possibly do a demo for the multiplayer? Or a “step by step” tutorial?
Here's a video of an ultimatum game: https://www.loom.com/share/b9353962a2c04874b5b2cf0739789b56.
+ Is multi-character input (e.g. words, phrases) an option for responses you can collect?
Yes - you can collect a wide range of responses including words!
+ Please explain how you’d collect data/ demographics from the game.
When you collect online data you usually have anonymous or pseudonymous IDs for participants to alleviate data security concerns. So, you’d need to add a questionnaire before the game. Essentially each response is captured - so in the treasure game with the dragon, we log whenever they press the space bar (together with whether that was the correct response or not). If they don't respond, we log a 'no-go' response at the end of the trial. The researchers can then download a CSV file where each row is one response, and also contains timing information, the participant ID, etc.
+ How to get permission from the users to gather all that data, is it okay to use a disclaimer page so users click accept on them?
You normally put a consent form at the start of your study. You’d put that in first, before they start the game. You can wire this up easily in Gorilla.
Explore How To Webinar Series
Introduction to Online Behavioural Research with Gorilla
Watch our webinar, hosted by Dr Catriona Silvey from Gorilla Experiment Builder; Introduction to Online Behavioural Research: Successes, Challenges, and How to Get Started. The webinar covers the benefits of online behavioural research, how to maintain data quality and a practical demo showcase of the Gorilla tool.
Introduction to Online Behavioural Research with Gorilla Experiment Builder is an online tutorial on how to design and run your own experiments. Here we share the webinar video and slides with you. We will add some of the questions that were asked during the live session and their responses. If you have a question, please send it through using the form below, and we will follow up with a response and any other resources.
Learn about the benefits of online behavioural research, how to collect and maintain data quality when designing experiments. Dr Catriona Silvey shares a few fascinating research projects that used Gorilla, and walks through a practical showcase of the tool. This tutorial is perfect for PhD and postdoctoral researchers wanting to take Reaction Time tasks online quickly and easily without learning JavaScript or other programming languages.
About Gorilla
Gorilla Experiment Builder is a cloud-based research platform that allows researchers and students to quickly and easily create and deploy behavioural (reaction-time) experiments online. Get 50 tokens FREE when you buy 50 tokens with the promo code GORILLASAGE2021. Offer ends January 31st 2022. Find out more and follow Gorilla on twitter.
About the Speakers
Dr Catriona Silvey is a former academic researcher with 10 years of experience running linguistics and psychology experiments both in the lab and online. At Gorilla, she focuses on introducing new users to the possibilities of the platform and the benefits of online research.
Q&A
+ How can I start using Gorilla?
You can create an account and build an experiment for free with Gorilla, and use the promo code GORILLASAGE2021 to get 50 free tokens when you buy 50. With Gorilla you only pay for the data collection. See more on our flexible pricing here: https://app.gorilla.sc/pricing
+ Is there a user manual on Gorilla software?
We've got a wide range of support documentation available for users to get started. I'd recommend our get started guide or one of our onboarding webinars as great places to start.
+ Who has access to the actual data that is collected?
At Gorilla we are very aware of the importance of keeping your data secure. The data that's collected is owned by the researcher and we never access it (unless explicitly asked to do so by the researcher e.g. to answer a support question). We have a comprehensive due diligence information about data security here.
+ Is Gorilla GDPR compliant?
Gorilla is fully GDPR compliant, and you can find more about this here.
+ How do you ensure that the participants are really eligible to participate in your study?
Gorilla is a platform for creating your tasks and experiments rather than finding participants. However, we do offer integration with a number of popular recruitment services, such as Prolific, Sona Systems and MTurk. You can find more information about recruitment options here. If you were using Prolific, for example, you can check out this extensive step-by-step guide on how to connect it with Gorilla.
+ Is it possible to include survey questionnaires before the experiment?
Yes! Gorilla includes a Questionnaire Builder tool which allows you to collect a wide variety of survey data from participants. Find out more in our how-to guide.
As well as a survey tool, Gorilla also includes a task builder, code editor, game builder and shop builder. To create an experiments, the individual tools are combined using our experiment tree tool. This allows for an immense range of experimental designs, while making it easy to work with each bit of the experience.
+ Is there a way to run multi-participant experiments (i.e., economic games)?
One of our upcoming features in 2022 is Multiplayer, which will allow multiple participants to interact within the same task; if you are interested in trying it out, contact us on info at gorilla dot sc.
+ Does Gorilla support other languages?
Our documentation and guides for using Gorilla aren't yet available in other languages; however, if you are able to use Gorilla, we have localisation options so you can customize both the instructions and the automatic text within the Task Builder to be in the language your participants will be using. Find out more about localisation in Gorilla here.
We have researchers using Gorilla all over the world including: The Netherlands, Germany, France, Brazil, Israel, Japan, China and UAE. So we're confident that the Gorilla participant experience, can be customised for a wide range of languages and scripts.
+ Does Gorilla support two different tasks on one screen, for instance can a participant see an image and hear an audio file simultaneously?
Yes, you can present an image and an audio file on the same screen as multiple stimuli by adding both an Image Zone and a Web Audio Zone to the same screen in your Task. Find out more about incorporating different Zones in our Task Builder Zones Tooling Reference Guide.
+ Can I design my own tasks if these aren't available?
Yes! The Gorilla task builder has lots of different zones that can be configured in an wide range of different combinations to bring you study design to life. Essentially, we provide the LEGO bricks and you combine them to create what you need.
If we don't have the right 'lego brick', you can you extend the functionality of our tools using using scripting. So you'll always have a way through. An advanced webinar on this topic is coming soon!
You can also program your own tasks in the Code Editor. You can import third-party libraries to build your own task from scratch, or alternatively import tasks built in other frameworks (e.g. jsPsych and PsychoJS). Find our list of example tasks built in the Code Editor here.