Numbas

Christopher Graham and Christian Lawson-Perfect, Newcastle University.

What is Numbas?

Numbas is an open-source system developed by the e-Learning Unit of Newcastle University's School of Maths and Stats, based on many years of use, experience and research into e-assessment.

It's aimed at numerate disciplines.

It creates SCORM-compliant exams which run entirely in the browser, compatible with VLEs such as Blackboard and Moodle.

History

Numbas follows the CALM model.

At Newcastle, we used the commercial system i-assess for six years before switching to Numbas.

Development began in 2011 with the aim of replacing i-assess.

Design goals

Scalable, reliable and accessible to a broad range of users.
Good-looking and easy for students to use.
Used by question authors who aren't experts, without support.

Why use Numbas?

Easy to obtain, use and tailor to your own requirements.
You have full control over the creation, use and dissemination of assessment.
Customisable everywhere, from appearance down to core functionality.
Browser-based so you can include all that the web can offer.
Sustainability built in - development is fully supported by Newcastle University.

Customisation

A large driver for Numbas was the lack of customisability in previous systems.

Interface and logic are completely separated in Numbas - custom themes can change the look of tests, or reimagine how they're run.

Extensions allow the addition of new functions, data types, and resources.

Integration with a VLE

Numbas can use the SCORM standard to integrate with compliant VLEs, such as Moodle and Blackboard.
Or there's a Basic LTI tool provider which works better than most SCORM implementations.
Or you can use it without a VLE.

Formative vs summative use

Computer-aided assessment is great for formative assessment.

Students can try randomised questions over and over until they're happy.

Summative assessment poses problems:

How to prevent cheating?
Can you ask the right sort of questions for a high-stakes summative exam?

Good formative assessment

How much more useful is adaptive feedback than a good worked solution?
Can you get students to test their own answer before getting feedback from the computer?
Apart from marking, what other resources can you give the student? E.g. interactive "toys" to experiment.

Current use

At Newcastle

Stage 1

Diagnostic test in week 1.
Practice mode for diagnostic test posted to incoming students on a USB key.
Bank of practice material in Blackbaord.
2 tests for each module, with questions picked from the practice bank.

Current use

At Newcastle

Stage 2

Core modules use CBAs for assessment.
Bank of practice material in Blackboard.
Tests open for one week each, with material picked from the practice bank.

Current use

At Newcastle

Other use

Some tests for stage 3 number theory, coding theory, and group thory modules.
Diagnostic and formative use in accounting, agriculture, biomedical sciences, business studies, engineering, psychology, physics, animal science (and growing!).
LTI tool introduced in 2016 - available to all throughout the university.
Revision material on Academic Skills Kit website.