In Spring 2020, just before the pandemic affected delivery of teaching at UCL, I (Prof Cloda Jenkins) agreed to join the teaching team for our first year Economics course, based on CORE, in 2020/21. By the summer it was clear that teaching would be largely online, or at least socially distanced if on campus. In September, after the upswing in A-level results, we also learnt that the cohort coming into first year would be nearly double what was the norm and that there would be just over 800 students in the first-year course.
I was teaching topics related to decision-making, strategic interactions and how to get individual decision makers to agree to socially optimal outcomes such as delivery of public goods. Our Adaptable Learning model involved providing the students with a weekly route-map that set out what parts of the textbook they should read and what practice questions to cover, leading into a live 1.5-hour online session. I needed to find a way to deliver an engaging, interactive live session online that would take student learning to a deeper level building on what they learnt from the textbook, The Economy. My inspiration came from a July 2020 CTaLE EconTEAching seminar, Engaging Students Through Experiments Online and in the Classroom, where I was motivated to use experiments as part of the teaching and learning strategy. A year on from that seminar, in this blog I reflect on my experience teaching the material online, remotely from home, using experiments for the first time. Maha Khalid, a first-year student who has since helped me look at the data from the experiment, also reflects on her experience of learning through experiments.
Reflections from Cloda Jenkins, Professor (Teaching) and new member of the first year Economics teaching team in 2020/21
Planning a lecture using experiments was easier than expected, thanks to the accessibility of the classex software, which Humberto Llavador had introduced in the EconTEAching seminar. It also helped that in parallel the CORE team was developing Experiencing Economics and allowed me to access the draft instructor resources. I am not a technical programmer by any means, nor an expert in experimental economics, so I was daunted initially going into the software. However, it turned out to be very user-friendly and there was lots of scope to adapt and play around with games yourself, including with robot players, to get confident in the set-up. There are many pre-programmed games to base your interactions with students on.
I was teaching a session on strategic interaction, free-riding and the public goods problem. I was at home, teaching using the ECHO360 platform, and students were also at their own homes. I could not see them, but we did have the opportunity to interact through the chat in ECHO360, polls and the experiment itself. The students were relatively new to university and did not know each other well so I expected they would be reluctant to ask questions early on, particularly if they were confused. This meant that I needed to provide them with very clear instructions before the class to make sure they had everything set up on their devices and we were able to get going with the experiments straight away. Given that nearly 85% of the students who were on the live class also joined the experiment without any queries, the provision of the instructions ahead of the session seemed to work well. Providing practical tips, like it would be best for them to use a phone or separate device for classex and their main screen to watch the online lecture, were small details that made a big difference to the smooth running of the session.
What took most time was thinking about how to integrate the experiments into the interactive live session. In a campus-based setting I would take time explaining the scenario that the experiment related to and give the students time to discuss with each other what the results that were emerging meant and what they implied for our understanding of the economics of strategic interaction. I wanted to replicate this structure as much as possible online. The live online session started with me reminding students of what they had read, ahead of the session, and I moved quickly to a warm-up Prisoner’s Dilemma game to get them used to the classex software. I set the game up to be in the context of two students being caught colluding on an assessment and having the option to confess or not; allowing for a side opportunity to discuss academic integrity with first-year students. I then ran a public good game, providing a specific context linked to flood defence, and ran a second one shortly after where there was scope for punishment.
As students could not turn to the person sitting next to them to discuss the experiment results, I ran polls after each experiment to check in with the student understanding. This allowed me to check their ability to connect the ‘game’ with the models and concepts that they should have pre-read. Where confusion on the basics, such as what type of game was being played, came up it was an opportunity to explain key elements of the course materials. I also used the polls to get their ideas on what might explain play that was not consistent with the model predictions. As students had read the textbook before the session, not necessarily the ideal way to run experiments, there was scope for their choices to be biased by knowing the predicted outcome. Their behaviour may also have been affected by reading materials beyond the standard case, for example consideration of preferences based on altruism or perceptions of fairness. Capturing the students’ thoughts through open poll questions provided ample material for me to discuss what they had learned in their prior reading, what we had learned from the experiments themselves, and how it would connect to what came later in the course. It also allowed me to discuss practical angles on policy design linked to nudges and how they might be affected by heterogeneous players in a game.
If you would like to know more about how the session went you can find a recording on the CORE teaching online resources. I will adopt a similar approach in the future, whether online or on campus. As the cohort for the course is always big (>350) it will be important to capture feedback through technology using online polls and to provide structure and context to the experiments, so the learning is evident, as well as the fun.
It was great to be able to engage many students in the experiments online. However, the downside to the large group was that there was a lot of noise in the outcomes of the experiments. Average contributions to a public good were easy to see and understand, as illustrated in Figure 1 below for the game with no punishment. However, as Figure 2 shows, there was a lot of noise behind this average value. It was clear that groups were playing the games in different ways. It would have been great to have had time to reflect more with the class on what types of behaviour might be driving different choice pathways, but it was hard to see from the immediate experiment outcomes. Maha, who gives her reflections below, and I have since spent time reviewing the underlying data and will produce a working paper soon examining how the students played. For next year I would like to do this type of analysis shortly after the live session and bring the results back to the students during the term. This would help their learning, particularly around the reasons why players don’t always make choices consistent with theory predictions, even when they know those predictions. This is one of my big takeaways for next year.
Figure 1: Average contributions per round in public goods game with no punishment
Figure 2: Contributions per round per group in public good game with no punishment
Reflections from Maha Khalid, BSc Economics student (2020-2023)
The Classex software was an extremely useful tool to aid my understanding of the different games and concepts explained in Units 4 and 5 of ‘The Economy.’ Playing the games was both enjoyable and insightful, as I was able to compare what I had learned from the textbook, with what I experienced in the games.
The software was very simple to use, even when extra elements like punishment were incorporated into the game. I found the end-of-round summaries very helpful, as I was able to see the individual contributions from each member of the group, as well as my resulting income, which affected my response in each round, and how inclined I was to punish others in the second game.
The public good game, which we played first, clearly demonstrated the concept of social dilemmas, which is covered in Unit 4. Over the course of the game, the average number of tokens contributed by my group fell, illustrating the issue of free riding which arises when public goods are being provided. Furthermore, when playing the game, it was clear to see that all individuals in my group would benefit more if we all decided to contribute a higher number of tokens, however due to the opportunity for free riding, there was no incentive to contribute. I often found myself reducing my contribution, as a result of other players’ low contributions, as a sort of punishment. Upon reflection, it amazes me how accurate the concepts discussed about public good games in Unit 4 are, as I can relate my behaviour in the experiments to that of a player punishing free riders, an example of the tragedy of the commons, which is discussed in Unit 4.7.
However, the number of tokens contributed by members of my group did not reach the expected equilibrium, which is 0 tokens contributed by each player. Reading Unit 4 of ‘The Economy’ before participating in the classroom experiments allowed me to analyse potential reasons as to why this equilibrium was not reached. For example, members of my group may have had strong altruistic beliefs or were perhaps playing randomly, as there were no real tokens at stake.
I personally decided to play the games based on how many tokens I wanted to contribute at the time, rather than following the expected equilibrium I had read about in Unit 4. Therefore, it was extremely interesting to see some of these expected results occur throughout the game. For example, during the public good with punishment game, as soon as I was punished for contributing less than my neighbours, I immediately increased my contribution out of guilt, and even began punishing others for low contributions. The threat of punishment forced players, including myself, to behave in a more cooperative manner, thereby avoiding the social dilemma which arose in the previous game. Comparing the market failure which can arise in the game with no punishment, when the public good is not provided, with the outcome I experienced in the game once punishment was introduced, helped me to solidify my understanding of, and experience first-hand, the impact rules of the game can have on the outcome.
As well as comparing the two games with each other, the Classex experiments also allowed me to think of how these games differ with other concepts I had read about in the textbook. As both games included repeated interactions, members of the group were forced to interact with each other after they had submitted their chosen level of contribution. In the first game (no punishment), this could have led to lower overall contributions. In the game with punishment, players punished others for contributing low amounts, or high levels of punishment, resulting in higher contributions. In comparison, during one-shot games, for example the prisoners’ dilemma, players only interact with each other once, so there is no opportunity to punish other players or react in any way to others’ behaviour. I learnt a lot about how the context of the game being played and its rules matter.
Centre for Teaching and Learning Economics 