Keynote: Natural Language Processing Approaches to Text Credibility and their Implications for Information Security

Recent advancements in sensor technology have revolutionized the collection of biometric data, offering valuable input data to improve remote biometric identification and behavior analysis (e.g., facial expressions, keystroking, mouse dynamics, touch biometrics, etc.). This talk will discuss recent advances in biometrics and behavior analysis to enhance user-computer scenarios like remote education through trustworthy remote assessment and sensor-based improved user feedback. This research will be discussed in the framework of remote education, but most described methods and results are applicable to many other user-computer and user-smartphone scenarios like social networking, gaming, etc. As part of this research, the edBB platform was developed, designed to monitor students in remote learning environments using a variety of sensors. The platform enables continuous authentication and the detection of anomalous behaviors (like cheating and fraud). Additionally, key databases have been generated to train deep learning modules that exploit the diverse input signals, such as edBB, mEBAL, mEBAL2, and others. These databases gather essential information for evaluating student concentration and behavioral aspects in e-learning environments (and beyond). Among the features explored in our research, eye blink has proven to be a reliable indicator of attention and fatigue, two key student states which, if properly assessed, can help to improve remote education dramatically. In this talk we will also summarize our recent efforts in developing video-based multimodal Eye Blink and Attention Level (mEBAL) detection methods achieving state-of-the-art performance, even in uncontrolled environments. We will also discuss the integration of multiple input signals (e.g., heart rate, face pose, facial features, etc.) powered by machine learning techniques to significantly improve the accuracy in estimating the students’ cognitive load. Lastly, we will show how the mentioned signals and related models can help to generate learning and information forensics analytics useful for the remote learning environment, e.g. to detect cheating, misbehavior, and as a tool for the instructor to improve remote education.