Detecting Personalised Emotions at the BBC


The Smart Services CRC’s Personalised Emotion Detector project has been successfully trialed by the BBC for use on a major classification project.


Developed by a research team led by Dr Ligang Zhang at the Queensland University of Technology (QUT), the software uses face detection and emotional analysis algorithms to read individual facial expressions in video footage and still photographs.


“The software aims to help identify non-verbal participant reactions to the experience in a specific time or place,” says Gavin Kennedy, Head of the CRC’s Service Innovation Foundry.


The BBC licensed the personalised emotion detector for trialing in a major mood classification project, in which mood metadata is being extracted from thousands of files stored in the BBC archives. The software achieved a high rate of accuracy in classifying positive and negative emotions in over 5000 photographs.


The QUT team also worked collaboratively with the BBC to further adapt the software for more accurate classification of emotions in video footage. For video the software automatically detects the primary type of emotion, shows emotion intensity changes across video frames, and identifies segments with sentimental highlights that might be of higher interest to users.


The BBC collaboration has presented some new research challenges for the QUT team.


“Comedy, for example, is difficult for the software to detect accurately, as actors often deliver comedic lines while keeping a straight face,” says Kennedy.


“There’s also the challenge of capturing emotion when people are filmed at difficult angles, or when they move quickly across the screen.”


The personalised emotion detector was originally developed with the aim of increasing customer satisfaction by helping businesses, such as shopping centres, health facilities and airports, understand their customers’ personal service preferences. For example, a clothing store retail team might use the detector to study customer emotions in response to particular styles of service, and then tailor to meet the expectations of specific demographic groups.


But the BBC’s use of the software has shown its potential for application in the area of culture and multimedia, opening up new opportunities for its future development.


“The collaboration with the BBC is completed, but there is certainly scope for further work and we are in discussions with media organisations to see where we can take this technology next,” Kennedy says.


See the BBC Blog