Tuesday 25 June 2024

4:30pm-6pm BST//11.30am-1pm EDT

---

Chair: Swati Virmani (De Montfort University)

Speakers:

Enhancing Research Question Formulation in Economics Education: Evaluating the Impact of a GPT-based Exercise

Nazanin Khazra (University of Toronto)

This paper explores implementing a novel teaching strategy in a research-focused economics course, where students are evaluated based on their research papers, to address the challenge of guiding students in formulating quality research questions.

Students usually struggle to identify viable research questions and understand the challenges inherent in each question. To overcome this challenge, I designed a simple take-home exercise using Generative Pre-trained Transformer (GPT) technology to help students write better research questions.

The exercise required students to employ GPT to generate various economic research questions based on their dataset, subsequently engaging them in critically evaluating the pros and cons of each question. This process aimed to enhance students’ understanding of what constitutes a good research question and to familiarize them with the complexities and potential obstacles in different research topics.

The paper’s main contribution is to present a comparative analysis of the effectiveness of this GPT-based exercise by examining the quality of research questions formulated by students in this iteration of the course against those from previous years where I did not have this GPT exercise. The primary focus is on evaluating whether the integration of GPT technology improved students’ ability to develop well-defined, feasible, and significant research questions.

Initial findings suggest that the GPT-based exercise offered a significant advantage. Students demonstrated a deeper understanding of the key elements of a strong research question, as evidenced by the increased clarity, specificity, and originality in their formulations. Additionally, students showed improved awareness of the practical challenges and implications of their chosen research topics.

This paper contributes to the emerging discourse on integrating AI tools in educational settings, particularly in research-oriented courses. It provides insights into how AI technologies like GPT can be effectively harnessed to enhance students’ critical thinking and analytical skills, ultimately leading to a more profound and nuanced understanding of the research process in the field of economics. The study also calls for further research into the long-term impacts of such pedagogical innovations on student learning outcomes and research capabilities.

Custom chatbots to deepen student learning

Theodore Svoronos (Harvard Kennedy School), Sharad Goel (Harvard Kennedy School) and Dan Levy (Harvard Kennedy School)

The rise of widely accessible Generative AI (GAI) has caused concern among educators regarding its potential effects on student learning. In this intervention, we aimed to leverage GAI to potentially enhance student learning and the experience of students in our course. We created custom chatbots for our introductory statistics course that served as a one-on-one tutor for students. The primary chatbot was seeded with course documents and given instructions for how to interact with students related to tone, content, and willingness to provide direct answers vs. taking a Socratic approach. Separate bots took on roles such as generating practice problems to help students sharpen a set of key skills that they needed to succeed in the course, and acting as a practitioner who is misusing statistics to help students practice explaining ideas in accessible terms. We have since partnered with other courses around our University and beyond to help instructors create chatbots of their own.

Evaluating the quality of ChatGPT responses for student self-study

Christian Tode (Bonn-Rhein-Sieg University of Applied Sciences), Omar Serraj (Bonn-Rhein-Sieg University of Applied Sciences), Natalie Bröse (Bonn-Rhein-Sieg University of Applied Sciences), and Christian Spielmann (University of Bristol)

The discourse surrounding the impact of generative AI on education is heated; characterised by both great potential and associated risks. One of the most significant of these risks is the emergence of hallucinations – output that is objectively false but presented as accurate. Many of us have experienced amusing but erroneous responses from ChatGPT, highlighting the critical need for expertise as the primary countermeasure against such occurrences. While educators may be able to spot inaccurate answers, students may not have the same ability. We have no doubt that students are using the technology already. To make the best of this situation, educators play a critical role in guiding students on how to use technology appropriately and for which tasks.