Take the survey now, before the discussion, while everyone is still here. Tell us how the workshop went and whether you’d like to use coLearn-AI or take part in the research. The survey is the coLearn-AI Workshop Feedback activity in the CCSC:MW class on colearn-ai.com: log in with your student account to take it.
coLearn-AI is an active research project, in use in courses at several institutions. The hardest part to get right is the AI guidance layer: small wording changes in a feedback prompt produce very different student experiences. The platform is now instrumented for research. Every submission, AI decision, revision, turn change, and instructor intervention is reconstructed from the interaction trace into measures. Three questions guide the current analysis:
Does enforced turn-taking produce equitable participation in collaborative activities?
Measured by submit and turn balance, dominant-member share, and streaks, checked against live classroom observation.
When AI prevents progression because reasoning is insufficient, how do student responses change?
Measured by revision rates, edit distance, and code and output changes, with before/feedback/after pairs for qualitative coding.
Do students generally satisfy the AI's reasoning requirements through revision, or do they ultimately bypass them?
Each question ends in one outcome: AI-accepted, Continue after retries, instructor-advanced, or unresolved.
Instructors also get a live Observation view during class. It shows a timeline for each group, with submit outcomes, idle time, and one-tap behavior tags such as Talk, Watch, and Frust. These tags record what the trace alone cannot see. Exports use pseudonymous IDs and never include names or emails.
What do we want to talk about? Here are some starting points. We’ll pick a couple together; the rest are good questions to take home or to ask over coffee.
The epistemic trace captures a rich signal. Three sources of that signal can be fed back to improve the guidance that shapes future interactions:
Which questions caused the most nudges? Where did groups revise most? The trace makes this visible at scale.
During class, the instructor taps one of eight tags on a group’s card in View Groups: Talk, Code, Watch, Quiet, Help, Frust, Off, or ?. Each tag is saved with the moment and what the platform knew then (the question group, who held the turn, time since the last activity, the last AI decision), so what the instructor saw lines up with the trace. A cluster of Help or Frust tags on one question points to guidance that needs work.
LLM-powered analysis of the trace can surface where tighter or looser guidance is needed, and can suggest revised feedback prompts and sample responses.
These are not solved problems. They are genuine design tensions where empirical evidence from diverse classrooms is needed:
Tight guidance prevents bypass but can frustrate groups who are genuinely reasoning correctly but expressing it differently. Where is the right threshold?
Forced Spokesperson rotation distributes responsibility but can slow the group. Should rotation be mandatory, optional, or adaptive to pace?
When should the instructor step in versus letting the AI handle it? What signals should trigger instructor attention?
The epistemic trace is powerful for instructors and researchers, but what should students have access to? Their own trace? Their group's? Anonymized class-wide data?
The lowest-commitment entry point. Use one coLearn-AI activity in one class session. Share your feedback prompt, the trace, and your observations with the project team.
Use coLearn-AI for an entire unit (e.g., all Python loop activities in CS1). Provides richer trace data and a clearer picture of learning progression.
Formal or informal comparative study. If you have two sections, or can collect pre/post data, the project team can help design an IRB-friendly protocol.
Contribute activities to the shared activity library. Well-designed activities (especially for common CS1 and data structures topics) are the platform's most reusable asset.
Professor of Mathematics and Computer Science
Kenyon College · Gambier, Ohio
Karolinska Institutet · Stockholm, Sweden
All materials from this workshop are archived at:
jimskon.github.io/CoLearn-AI-Workhop2026
Including worked examples, authoring templates, and the activity JSON files from the hands-on session.