Welsh partnership working on AI system which could ensure safety of vulnerable people

Working with Justus Vermak of Cardiff-based Skystrym, the specialists at USW-based CEMET – the Centre of Excellence in Mobile and Emerging Technologies – have been developing technology which can anonymously monitor a person’s movements, learn their routines, and notify family or carers if there are any concerns.  

South Africa-born Mr Vermak, whose educational background is in econometrics - the mathematical side of economics – is an entrepreneur who initially focused on property development, and then went on to establish a retirement village and care home.

And it was the challenges involved in providing care for elderly or vulnerable people that gave him the idea for the Skystrym system.

“As with all care settings across the world, ensuring you have an adequate number of staff is the real pinch point and where the biggest challenges are – making sure there’s enough staff to monitor the number of people in care, and keeping them safe,” he said.

“This challenge got me thinking about ways to develop an innovation which could provide remote monitoring to keep an eye of an individual to ensure they have not come to any harm.”

The Skystrym system, which can be used in a care setting or an individual’s home, uses a number of cameras or infrared sensors which monitor an individual’s movements, uses AI to learn their routine, and then inform family or carers if there is a reason for concern.

“What the system can do is watch the person and get to know their movements, where they may be,  and their routines, and the build up a picture of what’s normal for them” Mr Vermak said.

“This can include things such as how the person walks, are they moving slower than usual, how much time are they spending in each room, when are they visiting the kitchen and, for example, it’ll know that, for example, a person lying on a bed is normal for them, but lying next to the bed is out of the ordinary.  

“It can also have a dementia-related element, perhaps highlighting when a person has forgotten to eat.”

Understanding the concerns of users around privacy, the data collected by the system is anonymised to ensure any individual cannot be identified.   

“The system will obviously monitor images of the individual, but then that data is ‘blackboxed’ so it becomes anonymised for end-users,” Mr Vermak said.

“It works by using a ‘stickman’ image of the person being monitored, and then uses what it has learned from their past movement to see if things are normal, or if the system needs to notify a carer or family member of any issues.

“And it can work in seconds. If a person falls or is not where they are expected to be, and perhaps can’t raise the alarm, the system’s algorithm will be able to respond to that within just a few seconds.”

While having the idea behind the system, Mr Vermak needed the help of CEMET to make it happen - and he’s been very impressed with the support on offer.

“The team at CEMET are five star. They are professional, on point, and focus driven,” he said.

“It was clear from the start what the outcomes from the partnership were going to be, and CEMET proved to be a very professional outfit.”

Gareth Evans, R&D Associate.at CEMET said: “We worked on a prototype application which will enable them to demonstrate how computer vision techniques such as frame differencing can be used to detect motion and generate activity logs using video data,” he said.

“With the correct implementation, this could potentially assist and improve the experience of both care home residents and staff members.

“CEMET aims to deliver the application in the form of a prototype which can be tested by Skystrm on real-world video data.

“In the long term, with a sufficient quantity of data, there is potential for implementation of advanced deep learning classification systems for activity monitoring and fall detection.”