Assessing programming in Numbas

Programming in MSP

Programming is embedded in our mathematics and physics programmes, with compulsory modules at stages 1 and 2.

• Stage 1:
  • Problem solving with Python
  • Introduction to probability and R
• Stage 2:
  • Numerical methods (Python)
  • Computational statistics (R)

Module structure

Teaching weeks are typically structured as:

• 1 hour lecture: covering theory and examples

• 2 hour practical: hands-on practical with embedded exercises

I loved the format; one theory lecture and a two hour practical really worked as I learnt the concepts and then was given contact time to make sure that I understood them

Student experience

Students have different backgrounds and experiences with programming and computers.

...from A-level computing experience to computer-phobic.

Answering queries

Our experience was that demonstrators often received "low-level" queries about exercises, often requiring no more than signposting.

How do I start this exercise?

I don't understand - what does this mean?!

Or... students gathered around one competent student working on problems in a very non-interactive way

Exercise feedback

Handouts contained written exercises to complete.

Students work through the material at very different speeds, so timing feedback is difficult:

• Solutions released during session, end of session, the next week...?
• Work through on big screens, but when?

Summative Assessment

Programming is difficult to assess.

Little variation between student answers - difficult to detect collusion.

Marking is very time-consuming and repetitive.

Why Numbas?

Many of the lucrative features of maths e-assessment also apply to basic programming: automatic marking, instant feedback, randomisation...

Numbas was ideal:

• familiar to our students
• tap into well-developed features: randomisation, steps, adaptive marking, explore mode, etc
• mix coding questions with other part types

Behind the scenes

Custom part type

Handout exercises

Exercises embedded in the handouts as Numbas questions:

• immediate marking and feedback
• alternative answers to catch common errors
• randomisation to try similar questions over and over
• full worked solutions

And can be scaffolded with steps, or multiple parts to build up solutions.

Embedded exercises

Exercises are embedded directly in handouts:

---

New experience of handout exercises

Students are able tackle exercises and learn at their own pace.

During Covid, modules ran successfully with asynchronous handout material.

Since returning to campus, the handout exercises have transformed the demonstrator-student dynamic.

Assessment

Formative "Test Yourself" question sets accompanying each week of content.

In-course assessments and final exams were hybrid, ~60% auto-marked: focus manual marking where it is most effective.

Student feedback

I like that I can work through the handout so that I'm learning in the best way for myself, at my own pace

Really efficient way off working, using the handouts and tests!!

The feedback from our assignments was detailed and personal to us and gave us information on what we did well and where we can improve.

Scaffolding with steps

Explore mode

Mark a plot (experimental)

R example