A tour of advanced mathematical assessment in Numbas

Christian Lawson-Perfect
Newcastle University

Automated grading in mathematics & statistics: beyond the basics, Liverpool University, July 2025

Where are we starting from?

The framing of this event suggests "basic" means single-input, possibly randomised, questions automatically marked by comparison with an expected answer.

e.g. "Calculate $f (x)$ ", marked by comparing student's answer with the correct value of $f (x)$ .

More sophisticated assessment systems than this have existed for longer than I have!

My position on assessment

Formative assessment is useful.

Summative assessment is a necessary evil.

What we use Numbas for

Modules with Numbas assessments in the last two years:

Stage
1	2	3	4
Algebra Calculus and Differential Equations Dynamics Logic, Sets and Counting Multivariable Calculus Number Systems Probability, Statistics & R Problem Solving with Python Real Analysis	Bayesian methods Complex Analysis Computational Probability and Statistics with R Differential Equations Transforms and Waves Fluid Dynamics Groups and Discrete Mathematics Linear Algebra Scientific Computation with Python Vector Calculus	Coding Theory Electromagnetism Instabilities Mathematical Biology Methods for Differential Equations Metric Spaces and Topology Partial Differential Equations Relativity Stochastic Processes Topology	General Relativity Metric Spaces and Topology Statistical Foundations of Business Analytics

We cheat a bit

A lot of high-stakes assessments have the rote stuff assessed by Numbas, and harder stuff marked by hand.

How we write material for advanced modules

Simple question types go a long way!

We sometimes write an extension to add functions or new data types.

Basics

None of this is specific to advanced mathematics.

Parts

Break up a long question into parts which are marked independently of each other.

Steps

Give an option to break up a longer calculation into smaller pieces: offer scaffolding.

Parts and steps

Adaptive marking

Replace a question variable with the student's answer to a previous part.

Allows "error carried forward" marking.

Or, adventurously, allow the student to make up their own question. (more on that later)

Adaptive marking

Alternative answers

Mark against a few different expected answers / marking settings.

Give tailored feedback; catch common errors; generally give some more wiggle room.

Alternative answers

Custom marking algorithms

You can change how any part in Numbas is marked.

Feedback and score are built up through a series of notes.

Test the properties of the student's answer.
Do several things with it.
Give detailed feedback.
Branching decision trees.
Combine several answer inputs.

There's almost always more than one valid answer to a question.

Custom marking algorithms

Custom part types

Use different input methods / formats.

Permutation cycle notation input

Graph input

Explore mode

Allow more choice.

Assess the student's choice of method.

Let the student design their own question.

Follow the steps of an algorithm.

Explore mode

Thanks!

Website: numbas.org.uk
Email: numbas@ncl.ac.uk
Fediverse: @numbas@mathstodon.xyz
Source code: github.com/numbas

This slide intentionally left blank




	(%)