Page 48 - THEi Student Applied Research Presentations 2025

WHISPERER – SMART WEARABLE FOR
SOUND DETECTING

Poster
CHAN James
BSc (Hons) in Information and Communications Technology
Department of Digital Innovation and Technology

OBJECTIVES
• Effectively simulate auditory perception RESEARCH BACKGROUND
through sophisticated directional
haptic feedback mechanisms Hearing loss affects hundreds of millions globally, with significantly higher
• Detect and classify critical prevalence among aging populations. Traditional hearing aids amplify sounds
environmental sounds (e.g., alarms, but lack crucial directional awareness and fail to prioritize critical environmental
car horns, sirens) using embedded AI alerts (e.g., emergency alarms, approaching vehicles), leaving users vulnerable
technologies. to potential hazards. This important gap necessitates innovative technological
• Provide precise real-time tactile alerts solutions that enhance situational safety without stigmatizing users. The
for potential hazards with accurate Whisperer project addresses these fundamental limitations by developing
sound-source directionality a discreet, intelligent wearable that converts auditory cues into intuitive
• Ensure an ergonomic, highly haptic feedback, leveraging sophisticated AI to bridge sensory deficits and
customizable design that seamlessly substantially improve real-time environmental awareness for individuals with
resembles everyday clothing for hearing impairments.
maximum comfort and social
acceptance METHODOLOGY
• Achieve reliable all-day battery
performance with waterproof durability The project employed comprehensive iterative hardware-software prototyping.
suitable for consistent daily use The hardware architecture integrated Raspberry Pi 5 (a minicomputer), an
• Deliver the technological solution in advanced 8-channel microphone array, and strategically positioned vibration
a dignified, non-stigmatizing manner motors. Software utilized Python programming language with TensorFlow Lite
that promotes user confidence and library for real-time sound classification (leveraging the YAMNet, a pretrained AI
social integration model). Haptic feedback systems mapped both sound location and classification
type to corresponding motor arrays for intuitive interpretation. A Flutter-based
mobile application enabled comprehensive user customization. Development
ABOUT THE INVESTIGATOR progressed through three distinct stages: basic sound-spectrum vibration
mapping, multi-sound AI recognition capabilities, and full 360° directional
detection, with continuous refinements focused on improving latency, accuracy,
I am an Information and Communications and overall user comfort.
Technology (ICT) student interested in
assistive technology and embedded AI FINDINGS
systems. Committed to designing inclusive
health-tech solutions that empower The finalized prototype successfully achieved comprehensive 360° sound
individuals with sensory impairments. localization capabilities, accurately classified potential environmental hazards,
and delivered directional haptic alerts through vibration motors. A mobile
I aspire to lead innovations in wearable application prototype is created to demonstrate how users will have extensive
technology, combining human-centered personalization options for vibration patterns and sensitivity adjustments
design with scalable hardware to address according to individual preferences. Notable challenges included power
accessibility gaps globally. My interests constraints that limited active usage duration and intermittent Bluetooth
include Internet of Things (IOT) integration connectivity issues that hindered seamless synchronization. Future iterations
and practical AI applications that improve will require flexible circuitry, lightweight high-capacity batteries, and further
optimization of AI algorithms to reduce processing latency in complex
quality of life. My FYP supervisor is Mr LAU environments. Despite these technical challenges, the device demonstrated
Ho Chuen Tommy. significant enhancement of spatial awareness and personal safety for users,
validating the fundamental concept. This research illustrates the transformative
potential of AI-integrated wearables to revolutionize assistive technology for
individuals with hearing impairments.

37 Student Applied Research Presentations 2025 Student Applied Research Presentations 2025