Course-Specific AI Tutors

Summary

Academic Technology Initiatives (ATI) undertook a light assessment of AI tutors in response to a request from the Office of Undergraduate Education. The goal was to explore options that could enhance student learning and expand access to academic support. In addition, ATI sought to understand and test emerging use cases for AI chat tools that instructors at CU Boulder and other institutions are designing for their courses.

Process

Our evaluation of AI tutoring solutions followed a multi-step approach:

Literature Review

We examined recent research and emerging practices in AI tutoring. We focused on effectiveness, equity and scalability in order to establish a research-informed foundation for our assessment.

Tool Analysis

We researched available AI tutoring platforms and selected one tool for deeper analysis based on its relevance to our institutional context. In this assessment step, we examined integration requirements, user experience and potential challenges.

Peer Institution Review

We conducted a review of AI tutoring practices and adoption patterns at peer institutions. We interviewed institutions that had piloted or implemented the selected tool to gather insights on outcomes, challenges and lessons learned.

Vendor Meetings

We engaged with the vendor to understand technical and operational aspects, including integration with the learning management system, alignment with student and faculty workflows, support and training for users, and plans for future enhancements.

Results

Course-specific AI tutors are emerging as powerful tools to enhance personalized learning in higher education. These intelligent systems simulate one-on-one tutoring by providing instant feedback, hints and scaffolding. Increasingly, they leverage large language models combined with retrieval-augmented generation (RAG) to reduce hallucinations and ground responses sourced from course materials and learning objectives.

Types of course-specific AI tutors include:

• Domain-specific tutors designed for a single discipline
• Multi-subject tutors built for different subjects with adaptive learning paths
• Hybrid AI + human tutors that combine automated assistance and live help
• Interactive Socratic tutors that use guided questioning
• Learning assistants that convert course materials into personalized study aids

A preferred pedagogical approach for course-specific tutors is to use guided questioning, hints, and explanations. This Socratic method, along with refusal policies for inappropriate queries (i.e., "tell me the answer"), encourages reasoning rather than providing direct answers. While these tools offer unprecedented access to personalized instruction, their integration into the learning process can lead to cognitive offloading, reducing opportunities for students to develop crucial problem-solving or critical-thinking skills.¹ Likewise, faculty who rely heavily on AI to provide supplemental instruction and feedback may miss opportunities to build connections and engage students more deeply in their learning experiences.² Despite the promise of personalized instruction at scale, the use of AI tutors can widen existing equity gaps and may fail to address unresolved ethical issues such as bias, privacy and security risks.³

With these considerations in mind, we focused on one tutoring platform originally designed as a peer-to-peer learning app. This platform emphasizes student collaboration by providing a virtual space for group discussions, shared study resources, and remote interaction. An optional Socratic AI tutor can be trained on course syllabi and materials to provide additional support.

We met with institutions that had piloted the app to learn about their experiences. Because the AI tutoring feature is still new, we were unable to gather meaningful feedback on its effectiveness. However, these conversations provided valuable insights into adoption strategies for innovative learning tools.

Vendor meetings further clarified implementation requirements, student and faculty workflow, and planned future development. Given the early stage of the AI tutoring feature, we plan to follow up with peer institutions once they have had more time to evaluate the tool in practice.

The ATI group is continuing to monitor and explore developments in this area. For additional information, reach out to Jessica George (jessica.george-1@colorado.edu) or Ann Ruether (ann.ruether@colorado.edu).

OIT Technical & Business Operating Principles

Project Participants and Roles

References