This initiative will expand the utilization of an academic chatbot in core courses. The chatbot provides basic academic information, utilizes intelligent agents in the LMS to monitor student engagement and prompts as necessary, evaluates readiness for quizzes and reminds students about assignments and deadlines. GSU received a $7 million Post Secondary Student Success grant from the Department of Education to launch this technology in introductory English and Math courses in Fall 2024. Chatbot technology is already used in American Government, Macroeconomics and Chemistry.
Evaluation Plan and measures: Student academic outcomes in courses supported by the technology
KPIs: DFW rates
Baseline measure (for each KPI): Baseline will be DFW rates in Fall 2023
Current/most recent data (for each KPI): DFW rates in introductory math in Fall 2023 averaged around 30% on the Atlanta campus and between 32 and 50% at PC. DFW rates in introductory English in Fall 2023 averaged 19-25% on the Atlanta campus and between 32-34% at PC.
Goal or targets (for each KPI): Reductions of DFW rates
Random Control trials demonstrate chatbot is working well in American Government and Macroeconomics. The outcomes have been very encouraging. For example, in American Government, students with the chatbot were 8% more likely to earn a B or higher and 16% more likely to earn an A for a final grade than the control group. Students with the chatbot were 16% less likely to earn a W as a final grade than the control group and Pell students with the chatbot were 20% less likely to receive a DF or W than students in the control group. Similarly, the results in the Macroeconomics course were also positive. First-generation treatment group students were 23% more likely to earn a B or higher compared to control group first-generation students. Students in the treatment group were 38% less likely to drop the course than students in the control group.
Rolling technology out in additional new courses.
There are many instructors for these courses so set up and training will be especially important.
