REAL-TIME PERSONALIZED PHYSIOLOGICALLY BASED STRESS DETECTION

Abstract

Heart disease has now become a very common and impactful disease, which can actually be easily avoided if treatment 
is intervened at an early stage. Thus, daily monitoring of heart health has become increasingly important. Existing 
mobile heart monitoring systems are mainly based on seismocardiography (SCG) or photo plethysmography (PPG). 
However, these methods suffer from inconvenience and additional equipment requirements, preventing people from 
monitoring their hearts in any place at any time. Inspired by our observation of the correlation between pupil size and 
heart rate variability (HRV), we consider using the pupillary response when a user unlocks his/her phone using facial 
recognition to infer the user’s HRV during this time, thus enabling heart monitoring. To this end, we propose a 
computer vision-based mobile HRV monitoring framework-PupilHeart, designed with a mobile terminal and a server 
side. On the mobile terminal, PupilHeart collects pupil size change information from users when unlocking their 
phones through the front-facing camera. Then, the raw pupil size data is pre-processed on the server side. Specifically, 
PupilHeart uses a one-dimensional convolutional neural network (1D-CNN) to identify time series features associated 
with HRV. In addition, PupilHeart trains a recurrent neural network (RNN) with three hidden layers to model pupil 
and HRV. Employing this model, PupilHeart infers users’ HRV to obtain their heart condition each time they unlock 
their phones. We prototype PupilHeart and conduct both experiments and field studies to fully evaluate effectiveness 
of PupilHeart by recruiting 60 volunteers. The overall results show that PupilHeart can accurately predict the user’s 
HRV.