Our network meeting this week in Exeter was again fully booked; there does seem to be a growing interest in learning analytics in the UK. This was also a particularly informative meeting, I felt, and we were able to absorb a huge amount of expertise from our presenters.
Exeter University is ahead of much of the sector in this area, and has been preparing itself for learning analytics with some rigorous research activities as well as working on its data sources and technical infrastructure. We heard from three key members of the Exeter team.
Prof Wendy Robinson, Academic Dean for Students was first (Slides ppt 420KB). Wendy introduced Exeter’s learning analytics project, which is led by academics in partnership with students and aims to help students monitor and benchmark their own performance. Enhancing the information available to academic staff on their students, and ascertaining the factors affecting student success are also key aims.
Wendy discussed the University’s “My Grades” feature of their student app, which shows module results to date, and is proving popular with students. There are plans to add more sophisticated functionality.
Joanne Smith from Psychology was next (Slides 1 ppt 733KB | Slides 2 ppt 525KB). Joanne had led a systematic review of the effectiveness of learning analytics interventions. This was a comprehensive and impressive bit of work. She and a colleague assessed peer reviewed studies as “strong”, “moderate” or “weak” based on various aspects of the methodology. Out of 547 publications identified only 20 were retained for inclusion in the analysis.
Joanne concluded from the literature review that the three key factors in predicting student “success” are: social/demographic factors, academic history and engagement (usually in a VLE). She also points to the evidence being limited due to the small number of research studies to date, and issues around the quality of some of them.
Hywel Williams showing where Exeter’s data sources for learning analytics are coming from (green), aren’t coming from (red) or will do in the future (amber).
Joanne’s colleague, Hywel Williams (Slides pdf 2.5MB) then gave another fascinating insight into research carried out with colleagues Carmel Kent and Chris Boulton at Exeter surrounding the links between engagement and success. The data showed that use of the VLE at Exeter (and, he believed, by extension other bricks and mortar universities) was not a good predictor of student success.
They had identified many potential data sources for learning analytics. They ruled out the use of social media, private email and WiFi records as overly-intrusive. Sources that could be used included lecture recording data, access to past papers, use of the iExeter student app, VLE access, module feedback forms and careers data. Future data sources included attendance data, IT logins etc, use of the online helpdesk and assignment / assessment submissions.
Hywel and colleagues found that predictors varied across disciplines and also between high and low performers.
Shri Footring made an amateur but acceptable recording of the Exeter team’s presentations using Periscope.
Paul Bailey and Rob Wyn Jones then gave an update on activities in the Jisc Learning Analytics Project. (Slides pdf 2.8MB) | Recording The slides give a good summary of the current status of the project. Paul also demonstrated the Data Explorer tool, which has been developed by Jisc to enable institutions to provide quick and easy analyses of the data they hold in the learning records warehouse.
Our afternoon sessions were on the theme of the connection between learning analytics and learning gain. We began with Dr Camille Kandiko Howson, Academic Head of Student Engagement at Kings College London (Slides pdf 1.25MB).
Camille is an expert in learning gain and has been working across the 13 HEFCE-funded pilot projects in the area. She discussed the many different aspects of learning that can potentially be measured, such as intellectual skills, communication skills, interpersonal skills, vocational and employment preparedness and personal life quality enhancement. There are various ways to attempt to measure these, and different measures for students, subjects and institutions.
Dr Ian Scott from Oxford Brookes University was next (Slides pdf 5.8MB). He discussed the data sources in use at his university, and the ABC Learning Gains project, carried out with Surrey University and the Open University.
The project had had a number of ethical concerns, in particular with students opting in or out of the data collection. A literature review found that the concept of learning gain is mainly used to measure the effect of particular educational interventions. Research carried out by the project on Open University students showed that socio-demographic factors were the strongest predictors of variance in learning gain, in particular ethnicity and prior educational level. Meanwhile for Business students engagement with the VLE correlated with higher learning gains, while for Arts students it was not related.
There is also a recording of Camille’s and Ian’s slot.
Our final presentation was by Dr John Whitmer, Director for Analytics & Research at Blackboard, who came in by Skype from California. We heard from John, getting on for two years ago now, at our network event at Nottingham Trent. He’s been leading on some fascinating research since then, examining the data on large number of students and their use of their VLE, and exploring new ways of visualising the data (Slides to come). Recording
Our next session will be on 3rd May 2017, at the University of Strathclyde in Glasgow.