The fifth session of the #EconTEAching chats organised by CTaLE and the Department of Economics at Warwick University took place on 8th July 2020. Fabio Arico (UEA) talked to us about viva voce assessment. The session live-stream can be found on the CTaLE YouTube channel.
I remember discussing the similarities between (West) Germany production system and the Italian one in front of a panel of four teachers. I was ten years old and that was my first viva examination. In Italy, viva assessments are very common, from primary school to post-graduate studies, viva voce are employed for assessing students of all disciplines. Dr. Fabio Arico, an economist of Italian origin based at University of East Anglia, has been largely inspired by this tradition. Actually, Fabio has re-designed his History of Economic Thought module’s assessment to encompass a viva-voce component.
During EconTEAching Session 5, Fabio magisterially illustrated the beauty and the challenges of viva voce assessment or “evaluative conversations in Economics”. The talk was highly interactive. The audience was extremely active in asking questions and making comments. The issues of scalability, capacity, anonymity have been deeply investigated. However, before getting to this, let me explain how Fabio has implemented his evaluative conversations.
In the past three years, Fabio has opted for mixed assessment consisting in group-work, an individual essay and a 20 minute-viva. In particular, the latter has been focused on an oral discussion of the feedback received for the essay as well as on answering some questions on the syllabus. It is important to note that the students were not provided the viva questions in advance. This is one of the key aspects of the overall exercise. Fabio wants his students to develop critical and creative thinking abilities as well as the capacity to verbally communicate ideas. When asked what learning outcomes are better served by vivas in comparison, for example, with presentations, Fabio is confident that vivas could help students in tackling the unexpectedness of questions as well as thinking quickly and creatively. These soft skills are particularly valuable for job interviews.
As viva voce assessments are quite similar to job market interviews, Cloda Jenkins, Careers Tutor at UCL Department of Economics, asked whether there is scope for having more than one person asking questions at the viva. The issue with this seems to be capacity in terms of work-load allocation. Keeping on constraints (we are economists at the end of the day!), another person asked how this innovative form of assessment could be implemented for large cohorts of students. Fabio currently teaches 50 students but he believes that viva-voce could be scaled-up if they are organized efficiently. In particular, it is very useful to prepare in advance a set of questions of varying levels of difficulty, design a clear and quite “rigid” marking rubric (link to Fabio’s presentation) to make sure the assessment is fair and unbiased and, finally, in these COVID times, scheduling on-line vivas ensures enough flexibility for both students and lecturers. Fabio’s final point was that marking an average essay, i.e. 1,500 words, does not take less than 20 minutes. True!
Another contentious point, especially in the UK, is “anonymity”. In particular, it is believed that non-anonymous assessments are more likely to be biased. In response to that, Fabio argued that bias-generating processes are quite spread in lecturers’ routines. For example, seeing students during office hours can generate biases. Recognising hand-writing could bias traditional written exams. Finally, vivas are less keen to plagiarism and cheating behaviour. All in all, Fabio also clarified that the quality of viva voce is assured through viva’s recording, internal moderation as well as external checks, as recommended by the Quality Assurance Agency.
Turning to students’ views on vivas, Fabio candidly admitted that some of them found it “the most scary thing they have done in their life.” Wow! In general, students feel nervous about having a conversation with a lecturer and appreciate having examples of good practice. In response, Fabio shares footage of vivas with students. Moreover, in 2020/21 he will introduce “mini-vivas” which are 1-question, 5-minute conversations that are aimed to prepare students for the final viva. Mini-vivas are summative assessment but they are low-stakes. Regarding the distribution of marks and student satisfaction with the overall module, both indicators are in line with other modules. Moreover, no single student has complained about the viva, nor the difficulty of the questions, nor about possible unfairness.
The conversation with Fabio has been very engaging and inspiring. Novel assessment, innovative teaching techniques, digital communication are all risky and not well explored. However, should we deprive our students of these opportunities only because there are challenges? As Fabio told us “it’s a risk-management exercise” and as cutting-edge economists we can dare that!
Thanks to Fabio for his inspiring talk and Parama for sapient moderation and to all the participants who kept the discussion lively and contributed with valuable comments and suggestions.
Centre for Teaching and Learning Economics 