Task

To design, build, and test a prototype device that tracks speech activity and social interaction among aged care residents. Develop an ‘own-voice’ machine learning model and count the number of "conversation turns," which is a metric of social interaction and isolation.

Challanges

• Own-voice detection (conversation turn counting) must be robust in noisy, everyday environments, with false positives and negatives minimized.
• Recorded conversations are confidential, requiring strong encryption and a privacy-aware approach.

How we helped

• Devised, trained, and deployed a machine learning model capable of classifying ‘own voice’ versus ‘other voice’ with 70% accuracy, and extracted conversation turns and other metrics.
• Built 25 prototype devices that included assembling various components, programming the firmware, and 3D printing the enclosures.
• Guided the client in creating and annotating a dataset of conversations used to train the ML model.

Results

• Conversational turns were counted correctly 70% of the time in most situations.
• From scratch, we built 25 working units in preparation for a clinical study.
• Completed the proof of concept for the project, enabling the client to move forward with the clinical study.

More information about the project can be found here: CaT-Pin Project