What is an intelligent tutoring system?

Imagine a tutor who watches you work through a problem, notices exactly where you went wrong, gives you a hint at that step rather than the whole answer, and remembers what you found hard last week. For most students, that kind of one-to-one help has always been rare and expensive. Intelligent tutoring systems are the long-running effort to make some of it available to everyone, using computers.

The problem tutoring solves

In 1984 the educational psychologist Benjamin Bloom published a paper with a memorable title: "The 2 Sigma Problem". His graduate students had compared three groups of learners: students in a normal class, students in a class using "mastery learning" (where learners keep working on a unit until they have mastered it), and students who each had a personal tutor. The tutored students did dramatically better. Their average result was about two standard deviations ("two sigma") above the normal class, which means the typical tutored student outperformed around 98% of the students taught in the ordinary way.

Bloom's challenge to researchers was simple: one-to-one tutoring works, but no society can afford a personal tutor for every learner. Can we find methods of group teaching, or tools, that come close?

Later research suggests the two-sigma gap was larger than what tutoring usually achieves in everyday settings, but the central point has held up well: good individual tutoring helps students learn a great deal more. Intelligent tutoring systems are one of the main answers to Bloom's challenge.

What makes a tutoring system "intelligent"

Computers have been used for teaching since the 1960s, but early programs mostly showed material and marked answers right or wrong. An intelligent tutoring system tries to do more of what a good human tutor does. Researchers usually describe four parts:

• A model of the subject. The system holds the knowledge and skills to be learned, often broken into small steps, so it can tell a correct step from a common mistake.
• A model of the learner. As you work, the system keeps an estimate of what you know and what you are still learning. A well-known technique called knowledge tracing updates the chance that you have mastered each skill every time you answer.
• A teaching strategy. Using both models, the system decides what to do next: give a hint, explain a mistake, offer an easier or harder problem, or move on.
• An interface. The screen where you work, ask for help and receive feedback.

One of the first systems built on these ideas was SCHOLAR, described by Jaime Carbonell in 1970, which held a conversation with students about South American geography. In the 1980s and 1990s, researchers at Carnegie Mellon University built Cognitive Tutors for algebra and geometry. These followed each step of a student's solution, recognised common errors and gave targeted hints. They went on to be used in many schools in the United States.

What the research says

Intelligent tutoring systems are among the most studied tools in education, and the overall picture is encouraging.

• In 2011 Kurt VanLehn reviewed studies comparing human tutoring, computer tutoring and no tutoring. He found that systems which give help at each step of a problem were nearly as effective as human tutors in those studies: an average effect of about 0.76 standard deviations for step-based tutoring systems, compared with about 0.79 for human tutors.
• A 2016 review by James Kulik and Dexter Fletcher looked at 50 controlled evaluations and found that students taught with intelligent tutoring systems usually outperformed students in conventional classes, often by a substantial margin.
• Other large reviews, including one by Wenting Ma and colleagues in 2014, also found positive effects compared with ordinary classroom teaching, while noting that results vary with the subject, the design of the system and how it is used.

These are averages across many studies, mostly in mathematics, science and computing, and mostly in North America and Europe. They do not promise that any particular tool will raise any particular student's grades. But they show that well-designed, adaptive practice with immediate feedback is a powerful way to learn.

The ideas that make them work

Several well-tested ideas from learning science sit behind effective tutoring systems:

• Immediate, specific feedback. Learning where you went wrong while the problem is still fresh helps you fix the misunderstanding.
• Hints before answers. Being nudged to the next step keeps you doing the thinking, which is where the learning happens.
• Mastery before moving on. Practising a skill until you have it, before building on it, prevents gaps from piling up.
• Retrieval practice. Testing yourself, rather than re-reading, strengthens memory. A widely cited 2006 study by Henry Roediger and Jeffrey Karpicke showed that students who practised recalling a text remembered far more a week later than students who studied it again.
• Spacing. Reviewing material over several days or weeks, rather than all at once, leads to longer-lasting learning.

Where AI tutoring fits in

Traditional intelligent tutoring systems took years to build, because experts had to write out the knowledge and the common mistakes for each subject by hand. That is one reason they have mostly covered a few well-defined subjects such as algebra.

Large language models change this. They can explain ideas in plain language, answer follow-up questions and create practice material on almost any topic. But they also bring new risks: they can state wrong things confidently, and they can make it too easy to copy an answer instead of learning. We look at what recent studies say about this in AI tutors: what works, and what can go wrong.

Studentlytica AI takes several ideas from intelligent tutoring research and applies them to your own course materials: an AI Tutor that explains using your notes and shows the source of each point, practice questions with feedback, flashcards you review on a spaced schedule, and a study plan for the weeks ahead. You can read how the AI Tutor works in How the Studentlytican AI Tutor answers from your own notes.

Sources

• Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13(6), 4–16. doi.org/10.3102/0013189X013006004
• Carbonell, J. R. (1970). AI in CAI: An artificial-intelligence approach to computer-assisted instruction. IEEE Transactions on Man-Machine Systems, 11(4), 190–202. doi.org/10.1109/TMMS.1970.299942
• Corbett, A. T., & Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253–278. doi.org/10.1007/BF01099821
• Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167–207. doi.org/10.1207/s15327809jls0402_2
• VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221. doi.org/10.1080/00461520.2011.611369
• Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of intelligent tutoring systems: A meta-analytic review. Review of Educational Research, 86(1), 42–78. doi.org/10.3102/0034654315581420
• Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Educational Psychology, 106(4), 901–918. doi.org/10.1037/a0037123
• Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255. doi.org/10.1111/j.1467-9280.2006.01693.x