Augmented reality in neuroscience teaching

Visualising anatomy with AR

This project was undertaken to create an augmented reality (AR) smartphone and iPad app for teaching neuroscience at the ANU Medical School. This app uses the device camera to identify specific features of a scene e.g. part of an anatomical model or image, and then displays the scene in real time along with overlays comprising labels, text, images, or animations which allows the student to control the different graphical and text elements associated with that object.

Project snapshot

MChD Year 2 students; Neuroscience

Laboratory-based practical

Smartphone and iPad app

110 students

Can be used on student’s personal devices or on iPads provided during teaching.

Demonstrators indicated its potential to increase interactivity in class.

Students have control over what information they have while viewing the model.

Students found it easy to use and increased their motivation to learn more about the topic.

What was the issues?

Teaching staff needed a way to overcome the limitations of traditional plastic models.

Plastic models of the brain are an excellent resource to aid student study of the brain. However, to correctly identify each structure on the model, students need to refer to a small accompanying paper list that:

is text-dense with small font and often not in English
can easily be lost or misplaced and students are left with no reference
does not describe the functional relationships or clinical relevance of each structure
does not identify every structure visible on the model.

Therefore, studying complex plastic models can result in cognitive overload for students when using the detailed written lists to identify and understand important structures. Students can easily get distracted from the structures that are relevant to the learning outcomes and future clinical practice.

What was the solution?

Augmented Reality (AR) allows us to counter some of these challenges by overlaying virtual elements (labels, text, images) directly onto the plastic models.

In this project, an AR app for smartphones and iPad was created to facilitate the identification of structures, their functional connections and clinical relevance.

This app uses the device camera to identify specific features of objects e.g. part of an anatomical model or image and then projects them in real time with overlaying virtual elements (labels, text, images, animations). This way, students can interact with the virtual elements, such as dragging and dropping, clicking, moving, and toggling labels on and off.

A beta version of the app was launched for Year 2 practical sessions in 2019. During the sessions, students could use iPads preloaded with the app; if preferred, students could download the app to use on their personal devices.

The app in action…

An augmented reality app showing the anatomy of the eye using a tablet.

Students using the app in a lab teaching session.

Viewing sections of the brain.

Testimonials

The option to hide/show labels on the horizontal head - was good for self-testing Labelling of diagrams in terms of physiological functions is really good, especially without obstructing the diagram itself.

Having everything in one place and able to easily flick through to compare different tracts/anatomy.

In basal ganglia, how when drawing movement or disease only half the brain changed so you still had a reference point on the other half.

The Team

The project team consists of:

Corinne Carle (JCSMR and SMP)
Jessica Harrington (JCSMR)

Project funding:

CHM Dean’s Strategic Funds for Education and SMP Teaching Enhancement Grant