Abstract: The Bayesian framework offers a number of techniques for inferring
an individual's knowledge state from evidence of mastery of concepts or skills.
A typical application where such a technique can be useful is Computer Adaptive
Testing (CAT). A Bayesian modeling scheme, POKS, is proposed and compared to
the traditional Item Response Theory (IRT), which has been the prevalent CAT
approach for the last three decades. POKS is based on the theory of knowledge
spaces and constructs item-to-item graph structures without hidden nodes. It
aims to offer an effective knowledge assessment method with an efficient
algorithm for learning the graph structure from data. We review the different
Bayesian approaches to modeling student ability assessment and discuss how POKS
relates to them. The performance of POKS is compared to the IRT two parameter
logistic model. Experimental results over a 34 item Unix test and a 160 item
French language test show that both approaches can classify examinees as master
or non-master effectively and efficiently, with relatively comparable
performance. However, more significant differences are found in favor of POKS
for a second task that consists in predicting individual question item outcome.
Implications of these results for adaptive testing and student modeling are
discussed, as well as the limitations and advantages of POKS, namely the issue
of integrating concepts into its structure.