Confident assessment in research-oriented classes, Part 1

by Rachel Forsyth author of Confident Assessment in Higher Education. Use the code MSPACEQ423 for a 20% discount through December, 2023.

Design and support of assessment.

Research methods was once described to me as the holy grail of course design. When you first start thinking about planning the course, it seems obvious that everyone should agree on the content, but as soon as you start planning, you realise that the course is going to have to be very specific to a particular approach. Hence the perfect course becomes a holy grail. I don't know where to start looking for that, so how about some tips on assessing a course to make it relevant and useful for students’ future research? These posts are based on the book, Confident Assessment in Higher Education.

It can be hard to develop true confidence as an assessor. There is a great deal of scrutiny of grades, both inside and outside the university, and getting grading wrong can have serious consequences for individual students. It can feel much easier to continue with a traditional, ‘tried and tested’ approach than it is to experiment with assessment. One problem is that ‘tried and tested’ isn’t necessarily the best model; recent research on grading, moderating and feedback suggests that it can be both ineffective and discriminatory (Bloxham, 2012; Bloxham et al., 2015; Campbell, 2022; Nieminen, 2022).

So how can you build your confidence to do things differently? One thing to try is giving yourself a new set of criteria to review the assignment tasks you set for students. Think about one of your existing assignments, whether formative (not leading to student grades) or summative (resulting in student grades), and think about these four factors:

1.      Validity: Does it let students demonstrate achievement of the learning outcomes?

2.      Manageability: Will it be straightforward to mark, give feedback, and moderate?

3.      Clarity: Will students understand what to do and see how this task fits into their course overall?

4.      Enjoyment: Will I look forward to marking it and seeing what the students have achieved?

If you can answer yes to all four questions, then you are probably already a confident assessor. That is good news, and I hope you are already in a position to have open discussions about assessment with colleagues, particularly those who are new to assessing.

The first factor, validity, is obviously essential in most modern higher education systems, which are based on student achievement of outcomes, but sometimes a close look at an assignment shows up anomalies. Maybe grades include attendance points, when the learning outcomes are actually about content. Maybe accurate referencing is in the grading rubric, but not in the outcomes. Maybe the learning outcome mentions designing and carrying out a project, but only the proposal is assessed. It is always a good idea to check carefully.

Figure 1: The assessment lifecycle (adapted from Forsyth et al, (2015).

The other three factors are much more about how you and the students spend your time together. Do you share a clear vision of the purpose of this work, and how it relates to future independent research? Is it possible to excite the students about research in this preparatory course, and for you to be excited by their ideas and enthusiasm?

If you want to work more on these four factors, you could try working through this simple assessment cycle (figure 1). Parts 1 and 2 are covered in this blog post, with parts 3-5 in a second post.

1.      Setting: the process of planning an assignment task. In many systems, this is done when the course is first proposed, and included in any quality assurance processes. This might mean that you cannot change it until the next review cycle, or you may be in a role where you feel unable to change it. You can still review the task, and try to discuss changes with colleagues. The first thing to consider is whether it is a constrained task, like an examination, or is it done over a period of time, like a project proposal? Which will better allow the students to demonstrate achievement of the learning outcomes? If you are preparing them to do independent research, can you show the value of a time-constrained unseen examination? If you are concerned about originality, then an oral examination or a poster might be a good way to prepare them for presenting their research concisely. Be imaginative: think about what you would like your students to take to an existing or potential research supervisor as a result of your course.

The question of validity also requires you to consider whether all students will have an equal opportunity to demonstrate their achievement. Nieminen (2022) proposes five principles for ensuring inclusive assess­ment: rethinking accommodations, anti-ableist work, celebration of human diversity, student partnership, and interdependence. I would replace ‘anti-ableist’ with ‘anti-discriminatory’, to broaden the scope of thinking, but these are all key principles for inclusion. The assumption should be that all assessments are accessible to all students as they are designed.

2.      Supporting: how will you get the students from where they are to being able to succeed in your assignment task? In other words, your teaching needs to be aligned to the eventual goal of successful completion of the assessment. This isn’t to say that the whole module or course needs to be fixated on the assignment, that you should be spoon-feeding students with a perfect model for the task, or that it isn’t possible to go off in different directions and take a more creative approach. It is just recognition that students will be concerned with the assignment, that they are likely to do better if they start preparing for it earlier, and that focusing on getting the students to the task will give you a good structure for planning your teaching.

A simple checklist would include:

i)       Explain the assignment brief, probably at least three times – at the beginning of the module, in the middle, and a couple of weeks before the deadline.

ii)      Talk regularly and naturally about the assignment and how your teaching is designed to prepare the students for success. Micari and Calkins (2019) found that a perception of teacher openness and students’ own approaches to help-seeking in class was correlated to grades, indicating that it is important to normalise talking about assessment and potential challenges students may have.

iii)    Provide opportunities to practice. This doesn’t have to be completion of a complete assignment which you have to give detailed feedback on, but can take many forms: asking questions which students answer; writing the introduction to the assignment; preparing an annotated bibliography; setting the students the task of writing an outline of their assignment; asking students to reflect on their own progress so far. You can also get them to review each other’s progress.

iv)    Use short tests and quizzes. Most learning management systems have functionality for setting tests which can be computer-marked, using variations on multiple-choice or very short answers. Even if you are setting a more complex assignment task, these tests can be useful for building students’ confidence in their own knowledge. Many textbooks include such questions, either in the books or online. See what’s available and make use of it, offering the students a chance to talk through their results in class or in tutorials.

Whatever else you do, believe that there is no perfect way to assess anything in higher education, any more than there is a perfect way to design a research project, and so review and reflection are essential to ensure the assessment is working well. If you are interested in finding out more about the underpinning for this advice and examples of assessment tasks and feedback strategies, do ask your library if they have a copy of Confident Assessment in Higher Education.

References

Bloxham, S. (2012). ‘You can see the quality in front of your eyes’: grounding academic standards between rationality and interpretation. Quality in Higher Education, 18(2), 185–204. https://doi.org/10.1080/13538322.2012.711071

Bloxham, S., den-Outer, B., Hudson, J., & Price, M. (2015). Let’s stop the pretence of consistent marking: exploring the multiple limitations of assessment criteria. Assessment & Evaluation in Higher Education, 1–16. https://doi.org/10.1080/02602938.2015.1024607

Campbell, P. I. (2022). ‘Pray(ing) the person marking your work isn't racist’: racialised inequities in HE assessment practice. Teaching in Higher Education, 1-15. https://doi.org/10.1080/13562517.2022.2119075

Forsyth, R., Cullen, R., Ringan, N., & Stubbs, M. (2015). Supporting the development of assessment literacy of staff through institutional process change. London Review of Education, 13(34-41).

Micari, M., & Calkins, S. (2019). Is it OK to ask? The impact of instructor openness to questions on student help-seeking and academic outcomes. Active Learning in Higher Education, 22(2), 143-157. https://doi.org/10.1177/1469787419846620

Nieminen, J. H. (2022). Assessment for Inclusion: rethinking inclusive assessment in higher education. Teaching in Higher Education, 1-19. https://doi.org/10.1080/13562517.2021.2021395

---

More Methodspace Posts about Assessment and Evaluation