The Spring 2022 Numbas user meeting took place throughout the week beginning April 25th.
Timetable
Session abstracts
Monday 25th, 09:00-10:00 BST
Welcome
The Numbas team.
Welcome to the Numbas user meeting! We’ll go through the programme and show how to use the gather.town space.
Development update
Christian Lawson-Perfect, Newcastle University.
Christian, the lead developer of Numbas, will talk about what’s new in Numbas recently, and highlight some ongoing projects.
Monday 25th, 15:00-16:20 BST
Numbas at Newcastle
Chris Graham, Newcastle University.
Chris talks about how Numbas is used at Newcastle University.
The magic trick to increase attendance: Numbas formative e-assessment
Mario Orsi, UWE Bristol.
This talk reports on the impact of introducing Numbas in a compulsory science module of an undergraduate degree programme. We studied two paired student cohorts, from consecutive academic years, before and after replacing pen and paper formative assessment activities with digital alternatives based on the Numbas e-assessment platform. We found that introducing e-assessment correlates with a marked and statistically significant increase in student attendance. Module evaluation comments from students indicate an overall view of e-assessment as beneficial and engaging.
- Talk recording
- Slides (PDF)
From CALM to Numbas
Bill Foster and Cliff Beevers.
The talk will cover the development of Numbas from the CALM assessment system invented at Heriot-Watt University thirty years ago. The drivers behind this work will be discussed as they are still relevant.
Enhancing prospective students’ mathematics competences using online theory and exercise modules at KU Leuven
Yannick Van den Abbeel, Wim Obbels and Hanne Deprez, KU Leuven.
At KU Leuven, to support the transition to higher education, the faculty of economics and business, the faculty of science and the faculty of engineering technology cooperatively develop and implement an online programme to remedy and enhance prospective and incoming students’ mathethematics knowledge and skills. The programme has two main components: online modules based on modular building blocks which account for the theory (Ximera technology) and exercises and tests (Numbas technology). Exercises are used as formative assessment, i.e. to help (prospective) students to identify gaps in their knowledge/skillset and to assess their progress after studying the theory sections. The selected exercises mainly aim to ‘drill’ procedural math skills, i.e. to automate skills such as solving equations, calculating derivatives, etc. Numbas technology was selected because of the powerful mathematics grading engine and the randomisation possibilities.
The exercises are developed in Rumbas, an extension built on top of and using Numbas technology. Using Rumbas, Numbas exercises are generated based on text (yaml) input files, such that Git can be used for version control of, and collaboration on, an exercise and test database. The resulting Numbas tests are delivered via SCORM players or LTI links to students in KU Leuven’s learning management system Toledo (Blackboard).
Within the faculty of Engineering Technology, 18% of the total population students (N=633) reported having used the online modules to remedy their math competences. During the last trimester of 2021, students spend 7 to 8 hours per month on average studying the online modules. 75% of participating students reported the time spent on the online modules was well spent. In conclusion: if students participate in the online programme, they report benefits from the time invested. This is promising, since remedying programmes will become in the near future mandatory for students who fail a positioning test prior to subscription in the academic bachelors of the participating faculties. The teaching staff was enthusiastic about the grading engine and the randomisation possibilities, as this gives students maximal opportunity to exercise, with minimal staff involvement.
The individual faculties have supported the development of the online modules during the last two years, mainly by creating dedicated resources. With educational project funds, the equivalent of one full-time staff member will extend the existing programme with extra math topics and with reflection questions, to scaffold self-regulated learning. Furthermore, a learning dashboard, implemented as an extension of the Numbas dashboarding tool, will provide students with an overview of their math scores and the reflection questions’ results in order to enhance students’ self assessment, self regulation and reflection skills.
- Talk recording
- Slides (PDF)
Tuesday 26th, 09:00-10:00 BST
Cross-disciplinary Numbas initiative at the University of Glasgow
Shazia Ahmed, Niall Barr, Clare Brown, Frances Docherty, Anna Cartlidge, Ruth Douglas, William Finlay, Tess Lynn, Michael O’Connor, Beth Paschke and Elizabeth Petrie, University of Glasgow.
This talk will describe how staff and students at the University of Glasgow are using Numbas in a variety of contexts. It will briefly describe how early adopters in the School of Geography and Earth Science, and the School of Chemistry, have used it for traditional maths tests and online lab reporting. The main part of the presentation will describe our latest initiative, a cross-disciplinary partnership to provide basic maths support University-wide using Numbas.
Basic maths skills are essential to successful study in multiple disciplines, yet many students lack both confidence and ability in this area. Staff from five Schools/Services are undertaking a collaborative approach to curriculum design and delivery and are working with student interns to create a bank of cross-curricular, randomisable e-assessment questions using the well-established platform Numbas.
This student-centred and inclusive learning initiative will allow students to receive instant, formative feedback and the opportunity to practice infinitely many variations of problems to master key techniques and build maths confidence. By taking control of learning through this self-directed, flexible approach to learning, teaching and assessment, the platform reduces barriers to success in their subject areas and helps boost students’ sense of satisfaction relating to assessment and feedback.
Creating high quality e-assessment questions is a time consuming and specialist activity and having a bank of adaptable questions will save staff considerable effort. E-assessment, once set up, drastically reduces time spent in marking whilst enabling staff to provide meaningful, timely feedback and explanations for students. The platform also allows staff to gain a better understanding of students’ strengths and weaknesses in key areas, to identify gaps in knowledge and allows staff to focus on their own subject area and course learning objectives rather than having to spend valuable teaching time revising basics.
The content on the Numbas platform can be adapted by staff for their own purposes/areas and questions can be used to build self-tests, diagnostic tests, repeatable practice resources and summative assessments.
This talk will give more detail about the topics covered and will be of interest to anyone who thinks they could benefit from a resource of this type, or would like to learn more about Numbas and e-assessment.
- Talk recording
- Slides (PDF)
Videos, Hints and Misconceptions: Numbas as a Support Tool in SPIRIT Maths
Dr Catherine Palmer and Dr Clodagh Carroll, Munster Technological University, Cork.
SPIRIT Maths (Students’ Perceptions Informing and Redefining Innovative Teaching of Mathematics in Higher Education) is a project that was established in Munster Technological University (MTU) in order to investigate students’ attitudes towards mathematics and to explore the creation of student-centered mathematics resources. One of the key outputs of the project is the continued development of a collection of student-preferred digital materials to improve students’ confidence and proficiency in their mathematics modules. Following the responses to a survey disseminated to first-year students in MTU, three interlinked digital resources were developed – initially for two large student cohorts (one Business and one Engineering) with a view to expanding to other cohorts at a later date: (1) a series of interactive self-assessment questions, (2) corresponding videos showing worked solutions and (3) an associated bank of practice questions developed using Numbas to provide instant feedback.
This talk describes the development of the resources, with particular emphasis on the Numbas practice questions. We discuss some useful Numbas features that were incorporated into the practice questions that facilitate effective study, such as randomly generating questions to provide a variety of different but equivalent questions, providing sequential hints to support student learning depending on individual need and customising feedback to address misconceptions and common errors. We also discuss how the resources are made available to students in MTU and how we are hoping to measure student engagement with and the effectiveness of the resources.
Using Numbas for learning support
Ben Brawn, University of Newcastle, Australia.
As a learning adviser at a university, I mostly help students individually with numeracy or maths content that they are struggling with. Often after some explanation and examples, students really just need to put some time into practising the skill they are struggling with. This often means doing additional questions to those provided in their course. This is what I use Numbas for.
In this talk, I’ll show examples of questions from some of the following topics: arithmetic, algebra, coordinate geometry, functions, trigonometry, nursing calculations and finance calculations.
Tuesday 26th, 15:00-16:30 BST
Training sessions
Training sessions for beginner and advanced users of Numbas, run in parallel.
Followed by an opportunity to network with other Numbas users.
Wednesday 27th, 09:00-10:30 BST
Training sessions
Training sessions for beginner and advanced users of Numbas, run in parallel.
Followed by an opportunity to network with other Numbas users.
Wednesday 27th, 15:00-16:00 BST
The programming extension
George Stagg, Chris Graham and Aamir Khan, Newcastle University.
We teach several modules on computing for maths, stats and physics students. To support these modules, we created an extension for Numbas to automatically assess code written by the student in the languages R and Python.
- This talk failed to record. Sorry!
- Slides
Longer computational question setting and marking using Numbas
Chris Pearson, Nigel Penna, Stuart Edwards, Newcastle University.
The Numbas e-assessment system has been used since 2019 for summative assessments of students studying surveying. Numbas facilitated the successful replacement of a paper-based assessment and the development of numerous new assessments and remote exams during the COVID-19 pandemic.
The assessments consisted of a series of computation-based questions covering various topics within surveying. Using online assessments allowed students to be given individualised questions with a unique solution per student to reduce opportunities for plagiarism. Questions were automatically and instantly marked to provide accurate, quick and fair feedback to all students. Automated marking also allowed follow-through marks to be awarded when previous answers were incorrect.
However, follow through marks were awarded with a large penalty as a consequence of values being fundamentally incorrect. Applying this penalty resulted in more normally distributed marks. Comparisons to marks from previous years for the replaced assessment showed little change in the overall average mark.
Surveys of the students upon completion found over 75% found using Numbas straightforward and over 60% preferred the online assessment to paper-based assessment. However, a remaining and significant challenge is how to provide method marks via automated marking without providing too much guidance in the form of interim steps. Such steps would not normally be provided if the assessments were ‘hand-written’ and marked manually.
- This talk failed to record. Sorry!
- Slides (PDF)
Supporting students with a weak mathematical background during their first trimester of University study using Numbas.
Anthony Brown, University College Dublin.
This will be a study of how Numbas was used to support students taking two Level 0 courses at UCD (these are undergraduates but with poor results from their school leaving certificate). We consider the engagement with the Numbas exams which counted towards the final grade, and also the same exams which could be taken at any time by the students for revision purposes. We compare the grades of the final exam and of the weekly class tests with the grades of the Numbas exams.
- Talk recording
- Slides (PDF)
Thursday 28th, 09:00-10:00 BST
Numbas Open Resource Library / Q&A
The Numbas team.
Christian will introduce the Numbas Open Resource Library.
Followed by an opportunity to have a go at submitting material to the library, or ask general questions of the Numbas team.
Thursday 28th, 15:00-16:20 BST
Using Numbas for formative and summative assessment of Health students
Dr Thomas Cottrell, Department of Mathematical Sciences, University of Bath.
This academic year I taught mathematics to first-year students from the Health department here at the University of Bath. These students – on degree programmes in Sport, Health and Exercise Science – have a wide range of mathematical backgrounds: some have A Level maths, but many have not studied maths since GCSE. The student:maths staff ratio is also fairly high, with approximately 150 students to one lecturer and one tutor.
I chose to use Numbas for all the assessment – both formative and summative – on this maths course. The students were set repeatable weekly homework assignments on Numbas, and assessed with a 1-hour Numbas exam. In this talk, I will discuss my rationale for using Numbas, what types of activities the students were set and how they used these activities, and the various benefits, hazards, and lessons learned from this experience.
- Talk recording
- Slides (PDF)
Gaussian elimination calculator using Numbas exploration mode
Dr Julia Goedecke, Technische Universität Dresden.
(formerly University of Leicester, when this use of Numbas was developed)
The exploration mode for Numbas questions is a very flexible tool which can be used in many different ways. I will showcase two of my own uses: one particularly relies on the ability to overwrite variable names in between parts, which allowed me to make the calculation tool “Gaussian elimination calculator” for students. The underlying educational motivation is to help students practise which steps to choose in the Gauss algorithm, on a system of their choice, without having to do every single little calculation themselves. This followed the observation that many students did not understand the overall structure of the algorithm.
I will also show a question where use of exploration mode allows me to ask things in a way which would not be possible in standard mode.
Numbas for Chemistry – Adaptive Marking in Chemistry Practicals
Hanno Kossen, Nick Walker, Abbi Mullins, Matthew Sykes and Tom Smith, Newcastle University.
Chemistry students need to be able to apply numeric skills throughout their degree and beyond. To develop these skills, familiarity with the concepts and practice is required. Repeated practice boosts confidence with calculations, but its assessment can be time-consuming, prolonging the feedback cycle. Numbas has been introduced to Chemistry in 2018 to provide an e-assessment platform with highly flexible marking criteria.
In the laboratory setting (stage 1 general chemistry), students perform measurements and generate datasets unique to their experimentation. Historically, students’ results were assessed though a paper submission at the end of the lab session, where an answer was scored correct when it matched the model answer. This approach involved the coupling of two distinct learning outcomes: a) the appropriate performance of the laboratory technique and b) the correct handling of the data (e.g. converting units, deriving further quantities). E-assessment using Numbas allowed to decouple these distinct features by using adaptive marking. Here, a student answer on raw data was used to substitute a variable in subsequent calculations. This meant that separate scores were awarded for the accuracy of the measurement, and the correct method for data analysis. Feedback was deployed specifically to highlight which aspect students needed to improve on.
The use of Numbas in this setting has been successful for four years, including AY 2020, where this laboratory course had been replaced by using a variety of online resources. The talk will highlight key achievements in the use of adaptive marking in formative and summative assessment.
- Talk recording
- Slides (PDF)
Numbas ↔ Geogebra
William E. Haynes, Massachusetts Maritime Academy.
The GeoGebra extension provides a simple way for Numbas questions to include accurately scaled diagrams based on randomly generated variables without any programming. Each time the question is regenerated, the diagram changes accordingly.
This talk is an introductory ‘how to’ for authors who wish to incorporate GeoGebra diagrams into Numbas questions. It provides suggestions, tips and solutions to some common issues.
Friday 29th, 09:00-10:20 BST
Numbas in everyday work at Brighton
Timur Zaripov and Steven Kilgallon, University of Brighton.
In this talk, we will share our experience of introducing and leading the adoption of Numbas e-assessment system at University of Brighton. We will talk about learning Numbas as a user, the student experience, about involving and teaching colleagues, and getting support of the institution.
- Talk recording
- Slides (PDF)
Approaches taken in implementation of Numbas as the primary method of collecting candidate evidence for SQA mathematics units delivered in a Further Education College
John Dickson, Forth Valley College.
I will discuss the approach we took during the pandemic and how we are going to modify the assessments when remote learning is no longer required.
- Talk recording
- Slides (PDF)
Interactive Texts using Numbas
Don Shearman, Western Sydney University, Australia.
Following a presentation on the use of STACK to create a new version of the HELM (Helping Engineers Learn Maths) booklets as an interactive workbook, I have begun the process of translating these booklets into Numbas “exams”. This use of Numbas is somewhat different to the traditional form of a Numbas exam and has been adapted by some subject coordinators at Western Sydney University, Australia, to produce interactive notes for online courses.
In this talk we will look at the process of developing such interactive texts and discuss how the Numbas community might help to extend the conversion of the HELM booklets as a set of publicly accessible resources.
Combining LaTeX graphics with JSXGraph in Numbas
Jim Pettigrew and Don Shearman, Western Sydney University.
The LaTeX mathematical typesetting system can be used to generate elegant, high-precision figures with help from packages such as PSTricks and TikZ. In most applications these figures render as static vector graphics, combining mathematical type with an assortment of geometric or diagrammatic elements. Many of those who regularly use LaTeX will have accumulated a significant number of such figures that could be useful as illustrative or instructive tools in Numbas. By using a DVI-to-SVG conversion tool, it is possible to faithfully convert LaTeX-generated figures to vector graphics files that can be included as images in Numbas questions. But this can be taken further: the converted images can be embedded in JSXGraph objects within Numbas and hence animated using the dynamic geometric capabilities of this extension.
In this talk we will demonstrate a workflow that enables LaTeX-generated graphics to be ported to Numbas as SVG files, embedded in a JSXGraph object and made interactive using basic tools in the extension. A feature of the workflow will be the ‘syncing’ of the LaTeX and JSXGraph coordinate systems, allowing easy geometric placements and adjustments to be made in JSXGraph.
- Talk recording
- Slides (PDF)
- Supporting material:
Friday 29th, 15:00-16:00 BST
The future of Numbas
An open discussion about the future of Numbas:
- What direction should development take?
- What does the community need the most?
- Scope for cross-institution projects.
- How to widen participation in Numbas development?