Numbas

The Numbas Blog

Recording of online Numbas training session

This morning we ran the second of our online Numbas training sessions organised in response to the coronavirus crisis.

Here’s the recording of the session:

There’s at least one more session to go, at 10:00 BST next Wednesday, April 8th. You can still register to attend. If there’s demand for it, we might schedule more sessions.

How Numbas can help during the COVID-19 crisis

Now that face-to-face teaching in many institutions has been cancelled in response to the coronavirus crisis, many lecturers are looking for ways to move their teaching and assessment online. 

We’ve put this page together as a reference to describe how Numbas could be used to help your students and to replace paper-based assessments. 

What Numbas is 

Numbas is a free, open-source e-assessment system for mathematical subjects. 

Key features include: 

There’s an editor at numbas.mathcentre.ac.uk where you can create your own questions or reuse existing questions. It’s quick to sign up and we have lots of documentation and tutorials to get you started. 

How it works 

  • Write questions using the Numbas editor. 
  • Collect questions into an exam. 
  • Share the exam with students. 
  • Students attempt the exam. They can pause or leave the exam at any point and return to it later.  
  • Scores are automatically sent back to your VLE’s grade book. 

Numbas exams run entirely on the student’s device, so the student’s experience is not affected by a spotty internet connection. Once the exam starts, the only communication with the server is to report scores back (even this is optional – if you don’t want to track students’ scores, Numbas can run entirely standalone). Other systems need constant communication back to the server to render mathematical notation or to handle marking and navigation. 

And if you’re using the Numbas LTI tool, instructors can: 

  • See statistics about the class’s performance. 
  • Review individual students’ attempts exactly as the students saw them. 
  • Download a full breakdown of students’ attempts in CSV format. 

What can I use it for?

Numbas was originally designed for use in our mathematics and statistics programmes.

However, Numbas has also been used for many other subjects, including engineering; chemistry; physics; business studies; psychology; sports science; accounting; and biomedical science.

Even non-mathematical subjects can benefit from Numbas by making use of its several multiple-choice question types.

Why should I use it? 

Automatic assessment gives students immediate feedback, allowing them to work at their own pace. 

Randomised questions are good both for practice and for exams where you don’t want students to be able to copy off each other. 

There’s a lot of free material available to use under a permissive Creative Commons licence. Use the Explore page or the search facility to find content. 

How do I get started? 

Follow the tutorials at docs.numbas.org.uk. They will show you how to browse the question database, write your question, and deliver an exam to your students. 

Once you’ve done that, there’s full documentation on every feature of the editor, as well as ‘how-to’ instructions for common tasks. A good next step is our document on effectively planning a Numbas question

Where can I get help? 

Post your questions to the numbas-users mailing list

We’re happy to answer questions emailed to numbas@ncl.ac.uk, but please understand we’re quite busy at the moment! 

Delivering Numbas tests to students 

There’s more detailed info in the Numbas documentation

For purely formative material, you can just share a link to a Numbas exam. Students can complete the test, but their scores won’t be saved anywhere. This is a good solution if you want to quickly provide a bank of revision material. 

To save scores, you need a virtual learning environment that supports either SCORM (Moodle and Blackboard do, among others) or LTI (pretty much everything). 

The table below shows which standards are supported by common VLEs. 

VLE SCORM LTI 
Blackboard Yes, but don’t rely on it for high-stakes Yes – admin must make connection 
Moodle Yes Yes – teacher can make connection 
Canvas No Yes – teacher can make connection 
Desire2learn Brightspace Yes Yes  – admin must make connection 
ILIAS Yes Yes, with a plugin 
Sakai Yes, with a plugin Yes – teacher can make connection 
Schoology Yes Yes – teacher can make connection 

For SCORM, no setup is required: you download the package from the Numbas editor and upload it to your VLE. 

For LTI, you might need your VLE administrator to set up a connection. The Numbas LTI software needs to run on your own server. There’s full documentation about what you need and how to set it up

Considerations for high-stakes assessment

If you’re thinking of using Numbas for a high-stakes assessment, consider the following:

Robustness

Do everything you can to avoid losing student data. If using your VLE’s built-in SCORM player, it is responsible for saving attempt data.

Blackboard’s SCORM player can not be relied upon. Consistent reports describe Blackboard failing to save data for around 10% of students’ attempts.

We have fixed several bugs in Moodle’s SCORM player, and are now quite confident that it is reliable.

The Numbas LTI tool has been heavily tested and has several features to ensure no data is lost:

  • Data is saved to the student’s local storage as well as the remote server. If the student’s device crashes, they can resume where they left off after restarting.
  • After finishing, the student is shown a large warning box until all data has been confirmed to be saved.
  • Students can be automatically emailed a receipt containing a code which the instructor can use to validate their score.

Additionally, you can instruct your students to click the “Print this results summary” button after they have finished the exam, and print to a PDF file. They could email this to you if there’s a discrepancy with the score saved in your VLE.

Security

Because Numbas exams run in the browser, it’s possible that students could use their browser’s developer tools to modify their scores. For high-stakes assessments, we have always recommended using a locked-down browser.

Safe Exam Browser is an open-source tool which provides a locked-down browser.

Validity

Think carefully about what questions you can ask. You might need to think creatively about how to assess questions which would normally be answered with long passages of text. These questions could be reconfigured as multiple-choice, or you might just need to ask different questions.

Training

We’re going to run online training sessions, open to all. These will be informal, with a flexible format.

At the moment, we’ve scheduled the following sessions:

  • 10:00 GMT, Wednesday March 25th.
  • 10:00 GMT, Wednesday April 1st.
  • 10:00 GMT, Wednesday April 8th.

We’ll begin by running through the Numbas

The sessions will run over Zoom. You’ll need to register in advance, using this link.

Accessibility statements for Numbas

A few people have asked us for an accessibility statement before they start using Numbas, so we’ve written one.

In fact, we’ve written two: one for the exam interface seen by students, and one for the editor.

The statements describe the accessibility requirements that we’ve designed Numbas around, some tips on how to use Numbas effectively with different assistive technologies, and details on the outstanding accessibility issues that we know about.

If you’ve got any feedback about the statements or accessibility in Numbas, please let us know.

Programme of talks released for international EAMS 2018 Conference 

EAMS logo

The programme for the E-Assessment in Mathematical Sciences (EAMS) conference has been released, with speakers from across the globe presenting the latest developments in the field and offering a unique opportunity to get hands-on with maths e-assessment systems, whether you are a user, a developer, or just interested! The conference takes place over 3 days at Newcastle University, between 28th and 30th August.

Introduction to Numbas workshops – June 2018

We’re running Numbas workshops this month in Durham and Huddersfield:

  • Monday 18th June, 10am-4pm, Durham University.
  • Thursday 21st June, 10am-4pm, Huddersfield University.

The workshops are a hands-on introduction to Numbas, including getting started on the Numbas mathcentre editor, selecting questions to make tests, and writing your own questions.

The workshops are free to attend and will include lunch. There are limited places available; if you would like to book a spot, please contact Chris Graham (christopher.graham@ncl.ac.uk) before June 13th.

Numbas Workshop

Out now: Numbas v3.0, the marking algorithms rewrite

Today we’ve released Numbas v3.0. It’s the thing I’m second-most proud of producing in the last year (my daughter was born last October).

The marking code at the heart of Numbas has been completely rewritten, to make it much easier for question authors to change how students’ answers are marked. This has also allowed the introduction of custom part types, to make it easier to use and reuse different marking algorithms.

Announcing the EAMS conference 2018

EAMS logo

The international conference on E-Assessment in Mathematical Sciences (EAMS) is a three-day academic conference organised by Newcastle University taking place 28th – 30th August 2018.

Building on the success of EAMS 2016, the conference aims to bring together researchers and practitioners with an interest in e-assessment for mathematics and the sciences, with an emphasis on enabling attendees to have a go at creating material, and getting an opportunity to share expertise directly. It will consist of a mix of presentations of new techniques, and pedagogic research, as well as live demos and workshops where you can get hands-on with leading e-assessment software.

Photo of EAMS 2016 attendees during a hands-on computer lab session

Photo of the audience during a talk at EAMS 2016

Over three days, EAMS 2018 will comprise a mix of talks and hands-on activities:

  • Developer updates from the people responsible for popular mathematical e-assessment systems, detailing the latest features.
  • Lightning talks on a variety of topics to do with e-assessment in mathematical disciplines.
  • Hands-on workshops led by experts in a variety of e-assessment systems.
  • Live demos led by experts in the field.
  • Code sprints with the aim of adding features to systems, writing documentation, or creating material on a particular topic.

Compared to EAMS 2016, the emphasis this time is much more on enabling attendees to have a go at creating material, and getting an opportunity to share expertise directly.

The call for talk and workshop proposals is currently open. If you have some research or an innovative technique related to mathematical e-assessment that you would like to present, EAMS 2018 is the perfect venue. The deadline for talk proposals is May 31st.

The conference fee is only £75 and includes a conference dinner. You can find out more about EAMS, as well as the forms to register for the conference and propose a talk, at the conference website, eams.ncl.ac.uk.

Documentation

Numbas features full documentation which is always in line with the most recent version.

Numbas Documentation

Numbas Blog

We regularly update our blog with articles about new and future features, as well as other useful information.

Numbas Blog