Can we truly rely on AI to be a learning companion?

Written by Sai Gattupalli

Our latest research focusing on the intersection of large language models (LLMs) and their utility in education, has been accepted to the International Conference on Artificial Intelligence in Education 2023 (AIED). AIED is one of the core learning sciences conferences in the field of computer-assisted learning, and is ranked as A by the CORE. This year's conference adopted the hybrid attendance model and is being chaired by scientists from the Carnegie Mellon University, an institution known globally for its excellence and innovation in science, engineering, art and drama. Experts from all over the world will gather in Tokyo, Japan, in-person and online, from July 3-7, 2023, and present their latest research in the professional learning community.

Guided by Isaac Newton's wise words, "Stand on the shoulders of giants," our team, including Will Lee, Beverly Woolf, Ivon Arroyo, Danielle Allessio, and myself, embarked on this collaborative research journey - by treating the immense computing power of large language models present to the education field.

Here's a common scenario -

A student is grappling with a tough math problem late at night. What if AI could be their tutor?

We investigated this with LLMs such as GPT-4, and utilized prompt crafting techniques from a public GitHub repository focused on prompt engineering.

With a title "Exploring Pre-Service Teachers' Perceptions of Large Language Models-Generated Hints in Online Mathematics Learning," we sought to understand the potential utility and effectiveness of LLMs in an online educational setting. With a particular focus on online maths education, we studied pre-service teachers' reactions and interactions with AI-generated mathematical instructions and guidance. We found that while the human touch in the creation of mathematical content is still essential, AI-generated walk-throughs and guidance can be incredibly beneficial for tutoring in math problem-solving.

We employed a mixed methods analysis, where participants' hint preferences were plotted out via histograms. Next, we used a pre-trained BART language model supplied by Hugging Face API, in conjunction with k-means clustering, to:

• Compile and interpret written responses

• Categorize responses into themes

My colleague and co-author, Will Lee, led the number-crunching part, which produced several thematic analyses, which furthered our understanding of this nascent topic. The dataset, including the .ipynb files, have been made available via a Open Science Foundation public repo here - https://osf.io/t84v7/.

Net net, our research advocates for a more personalized learning experience for K12 and adult math learners, by envisioning LLMs as digital learning companions and addressing diverse learning styles in tandem with human teachers. Personally, I feel this study marks a stride in our collective understanding of personalized learning and the transformative potential of LLMs in education.

The Overleaf PDF will be made available to the public in the coming weeks.

Stay tuned as we prepare for AIED 2023.

#AIED2023 #LLMs #Education #MathEducation