Numbas

Numbas v5.0

Montage of screenshots of Numbas v5

It’s time for another major version of Numbas. This release includes a radically new way of presenting questions, as well as a few other helpful new features.

We’ve made a short video to demonstrate the new features:

We’ve also made a demo exam so you can try it out yourself.

Explore mode

“Explore mode” is a completely new style of question. Rather than presenting the student with a fixed list of parts that they must answer, in explore mode the student can choose their own path through a question, with the available routes depending on how they have performed on previous parts.

A question in explore mode. At the top is a tree showing the revealed parts of the question. In the middle is the current part, with input boxes and feedback. At the bottom is a breakdown of the score for the question.
An “explore mode” question

My initial mental model for explore mode was a “choose your own adventure” book: the student would be given a prompt and some activity, and then they could choose where to go next from a list of options.

Cover of the book "Your First Adventure: Little Owl Leaves the Nest"
Image from gamebooks.org.

I then wrote down a few use cases to guide the design:

  • Offer a selection of hints at varying degrees of helpfulness.
  • Walk the student through an iterative algorithm, giving feedback on each step.
  • Allow the student to choose the method they want to use to solve a problem.
  • Take free input from the student, such as measurements or an example of an object, then ask them questions about it.
  • Ask the student to define the criteria for a test, then assess their decisions based on those criteria.

Writing “Explore” questions is a bit more complicated than “Show all parts” questions – you have to set up how the parts link together, think about how variable replacements will work, and come up with a sensible marking scheme. I’ve tried to make the editing interface as intuitive and helpful as possible.

Screenshot of a panel titled "Is it factorised?", with fields "Label", "Locks this part?", "Availability", "Penalty to apply when visited", and "Variable replacements".
Defining a “next part” option in the editor.

This new mode enables a huge range of new activities in Numbas. I can’t wait to get started writing more questions in explore mode, and to see what others come up with.

See the documentation on explore mode.

Alternative answers

A long-standing difficulty of Numbas has been recognising specific incorrect answers and giving tailored feedback. Similarly, the standard marking for most parts assume there’s only one correct answer, up to some kind of equivalence.

Screenshot of a part with alternative answers. The prompt is "How many digits does 122588 have in base 3?" The student has answered "10". The feedback says "Did you calculate floor of log 3 of 122588?"
Using alternative answers to detect an expected error.

“Alternative answers” lets you set up copies of a part, with their own settings and expected answer, number of marks to award and a specific feedback message. When the student submits an answer, the version of the part that matches best is used.

Alternative answers are good for situations where you have a fixed number of cases that you want to check for.

"Add two numbers" title text, above "Alternative answers" heading, followed by three items, "Within 10", "Within 100" and "Within 1000$, followed by a button labelled "Add an alternative answer".
Alternative answers are neatly integrated into the editor.

Use cases for alternative answers:

  • Accept any of several correct answers, for example \(x\) or \(-x\), or a precise answer and one obtained by rounding in intermediate steps.
  • Award partial credit for an answer that is ‘nearly right’.
  • Give a relevant feedback message for an answer corresponding to an error you expect the student to make (commonly known as ‘mal-rules’). The paper Diagnosing student errors in e-assessment questions makes the case for catching these.

See the documentation on alternative answers.

Menu mode

Screenshot of an exam in menu mode. There is the exam's title, introduction text, then lists of question groups. Each question has a label and score details. Some questions are attempted and correct, with a green tick; some are partially correct with a grey tick. Each question group has a subtotal and feedback icon as well.
An exam in menu mode.

This is a small change in terms of functionality, but represents a change in how Numbas material can be presented.

There is now a “Question navigation mode” option for exams. The “Sequential” option is the classic presentation, with a list of numbered questions displayed in a sidebar. However, the “Choose from a menu” mode presents the student with a table of available questions, which they can pick and choose from. By default, the question’s name as it appears in the editor is used, but you can set a different name if you want.

The idea is that this mode is more appropriate for material which students access for revision or practice, to make it easier for them to pick just the subjects that they want to work on.

There’s no “End Exam” button in menu mode – menu mode is for purely formative use and each question should stand on its own.

See the documentation on menu mode.

Folders

Organising content in the Numbas editor has always been a challenge. We’ve made it a little bit easier to arrange content within projects with the addition of folders.

Rather than searching for an item by name, you can now navigate through a folder hierarchy.

Screenshot of the project browser, showing some items and a folder in a list.
Browsing a project’s content.

See the documentation on organising content inside a project.

Other changes

Accessibility

  • There is an accessibility statement for Numbas exams, and another one for the Numbas editor.
  • Part input warnings now have the “alert” role. (code)
  • The Numbas logo now has alt text. (code)
  • Input boxes have title attributes, and choices for multiple-choice parts are grouped in <fieldset> tags. (code)
  • The default theme and the Numbas editor use aria-live throughout to alert accessibility tools to changes in the interface. (code)
  • The default theme respects the browser’s prefers-reduced-motion setting, avoiding transition effects. (code)
  • The colours of some interface elements have changed to improve contrast, to meet WCAG AAA standards. (code)
  • The tick/cross feedback icons are hidden from screenreaders, instead there’s some screenreader-only text of the form “Your answer is correct”, etc. (code)
  • Input boxes have autocapitalization turned off for mobile keyboards.

Numbas runtime

  • filter accepts any list-like object. (code)
  • The new function prod computes the product of the elements of a list of numbers. (documentation, thanks to Grant Gollier)
  • The new function split_regex splits up a string at every occurrence of a substring matching a given regular expression. (documentation)
  • The new function substitute substitutes values for free variables in an expression. (documentation)
  • The new function diff differentiates an expression with respect to a given variable. (documentation)
  • The function latex can now take a sub-expression, which will be converted to a string of LaTeX code. (documentation)
  • The ‘relative tolerance’ and ‘absolute tolerance’ arguments for isclose are optional, with default values of \(10^{-15}\).
  • A new “Show results page” option for exams specifies when the results page is shown. The options are “never”, “when entering in review mode”, and “on completion”. (documentation)
  • In a gap-fill part with only one gap, no gap name header is shown in the feedback. (code)
  • Parts which use adaptive marking have their ‘expected answer’ updated when the parts referred to in adaptive marking are submitted. (code)
  • You can now apply a penalty when adaptive marking is used. (documentation)
  • The standard marking algorithm for “mathematical expression” parts no longer checks that the student uses exactly the same set of variables as the expected answer. (code)
  • SVGs embedded in content areas with <img> tags are replaced with <object> tag equivalents so that they scale nicely. (code)
  • There is an exam-wide “enable steps” option which you can use to turn off steps for all questions. (documentation, issue)
  • The initial value of the seed for the random number generator is saved in the SCORM suspend data. (code, thanks to Danil Polikarik)
  • The displayed percentage total for the exam is always rounded down, so that it doesn’t look like you’ve achieved a threshold when you haven’t quite. (code, issue)
  • There’s a flatFractions flag for displayed expressions, which renders fractions on one line instead of stacked. (documentation, thanks to tpilewicz)
  • The ‘mathematical expression’ part now automatically detects expressions in the form of a formula, and sets up the variable value generators accordingly. (code)
  • The “printed worksheet” theme displays the number of marks available for each part. (code)
  • Exams can now be created from a JSON description, without a display. This was already possible for questions and parts. Numbas exams can now run headlessly, for example for remarking an attempt after a bug is fixed. (code)
  • The “printed worksheet” theme now displays the exam’s introduction text before the questions.
  • There are some new signals under Numbas.signals: ‘exam ready’, ‘display ready’, and ‘Numbas initialised’. (code)
  • The operation e^x uses the built-in JavaScript function Math.exp(x) rather than Math.pow(Math.E,x), so it produces the same result as exp(x). (code)
  • Lists and dictionaries can have a trailing comma just before the closing bracket, e.g. [1,2,3,]. (code)
  • The “mathematical expression” part has some new settings to do with juxtaposition of names: “force single letter variables names?”, “allow unknown functions?”, and “use implicit function composition?”. (documentation)
  • There are some new exam-level options to do with feedback shown in review mode: “show score in review mode?”, “show part feedback messages in review mode?”, “show expected answers in review mode?” and “show question advice in review mode?” (documentation)
  • Variable names with different annotations are considered to be different. (documentation)
  • The ‘matrix entry’ part has minimum/maximum number of rows/columns settings. (documentation)

Numbas editor

  • There is now an “explore” page which shows public projects and recently published questions and exams. (See it on numbas.mathcentre.ac.uk)
  • There’s now a web-based editor for extensions, and documentation for extensions can be displayed within the editor. (documentation)
  • You can grant other users access to view or edit your extensions, similar to exams and questions. (documentation)
  • In the marking algorithm testing panel, the order of variables follows their order in the variables tab. (code)
  • The variable editor shows where in the question the current variable is used, such as content areas and part settings. (issue)
"Used by" above list items "Statement", "Part a prompt", "Part a minimum accepted value" and "Part a maximum accepted value"
You can now see everywhere that a variable is used.
  • Headers for editor tabs are styled differently when they’re “in use” – when any of the settings in that tab are different to the defaults. (issue)
  • You can now download and upload custom part type definitions. (code)
  • Content areas only become editable when clicked. This greatly improves the performance of the editor. (code)
  • The editor homepage displays a “tip of the day”. (issue)
"Tip: Check out the tutorials" above some text and a button labelled "Get started", in a box above the homepage activity timeline.
A tip on the homepage.
  • You can only view an exam if you can view all of its questions. (issue)
  • The “editing history” tab shows the last feedback label set by each person who has set a feedback label. (code)
Screenshot of a summary of feedback labels on an item. Users who are members of the item's project are shown separately to other users.
Feedback by different people on an item.
  • You can now choose to receive an email each time you get a notification on the editor, such as for feedback on an item.
  • When you open a notification about feedback on an item, you are taken straight to that item’s “editing history” tab. (code)
  • On the part marking algorithm tab’s marking test panel, the expected answer is shown above the input for the student’s answer. (issue)
Screenshot showing a field labelled "Expected answer" with a value of 13, above a field labelled "Student's answer" which is still empty.

Localisation

Since v4.0, Numbas has been translated into Hebrew, Turkish, Vietnamese and Chinese, bringing the total up to 18 languages.

As ever, there are new pieces of text in the Numbas interface that need to be localised. If you speak a language in addition to English, please go to our localisation page and write some translations.

Contributors

Thanks to the following people for contributing code to Numbas in the last year:

  • Christian Lawson-Perfect
  • Chris Graham
  • David Hadas
  • Grant Gollier
  • Danil Sergeev
  • tpilewicz

There are always plenty of opportunities for you to start contributing to Numbas. See our page on contributing to Numbas for more information. Our GitHub issues labelled “good first issue” are a good place to start.

Give it a go!

You can use all these new features right now on numbas.mathcentre.ac.uk. If you run your own instance of the Numbas editor, follow the upgrade instructions in the documentation.

If you have any questions about the new features, or spot a bug, you can post on the Numbas user group, file an issue on GitHub, or email us.


Documentation

Numbas features full documentation which is always in line with the most recent version.

Numbas Documentation

Numbas Blog

We regularly update our blog with articles about new and future features, as well as other useful information.

Numbas Blog