Dr Li Zhang

Since emotion plays an important role in social interaction and decision making, I am especially interested in employing intelligent conversational agents to be involved in role-play situations and to perform affect sensing and interpretation in multimodal interaction in gaming context. My research focuses on affect detection from verbal (open-ended conversational dialogue & speech) and non-verbal (body language) communication, speech & gesture recognition in a gaming context, emotional speech synthesis, and autonomous learning and adaptation in affect inspired intelligent agents. I am also interested in how different cultures may influence the agent’s small talk behaviour.

Research topics that I am interested in include:

• Multimodal interaction between human and robots/agents
• Multimodal user interfaces for games
• Affect detection from speech, dialogue and body language
• Affect modelling & sensing in improvisational interaction
• Context-based emotion and personality modelling
• Speech recognition
• Emotion recognition in speech
• Mobile applications/games for educational/gaming purposes
• AI and adaptation in games (including autonomous learning, decision making and team coordination etc)

Selected Publications on Affective Computing and Intelligent User Interfaces:

• Zhang, L., Gillies, M., Dhaliwal, K., Gower, A., Robertson, D. & Crabtree, B. (2009). E-drama: Facilitating Online Role-play using an AI Actor and Emotionally Expressive Characters. International Journal of Artificial Intelligence in Education. Vol 19(1), pp.5-38.
• Zhang, L. (2010). Exploration of Metaphorical and Contextual Affect Sensing in a Virtual Improvisational Drama. Transactions on Edutainment IV. Pan, Zhigeng; Cheok, Adrian David; Müller, Wolfgang; Zhang, Xiaopeng; Wong, Kevin (Eds.). LNCS 6250, pp. 105–116.
• Zhang, L. & Barnden, J.A. (2010). Affect and Metaphor Sensing in Virtual Drama. International Journal of Computer Games Technology. Vol. 2010.
• Zhang, L. (2010). Exploitation on Contextual Affect Sensing and Dynamic Relationship Interpretation. In ACM Computers in Entertainment. Vol.8, Issue 3.
• Zhang, L. & Hossain, Alamgir. (2012). Contextual affect sensing and metaphor interpretation. Special Issue on Games and AI. Entertainment Computing. Elsevier.
• Zhang, L. (2011). Context-sensitive Affect Sensing and Metaphor Identification in Virtual Drama. In Proceedings of International Conference on Affective Computing and Intelligent Interaction. Oct 2011. Memphis, USA.
• Zhang, L. (2011). Exploration on Context-sensitive Affect Sensing in an Intelligent Agent. In Proceedings of the 11th International Conference on Intelligent Virtual Agents (IVA’11). 15-17 Sept, Iceland.
• Zhang, L. (2011). Affect Sensing in Metaphorical Phenomena and Dramatic Interaction Context. In Proceedings of the Twenty-second International Joint Conference on Artificial Intelligence (IJCAI). July 2011.Spain.

Full publication list:

Journal papers

• Zhang, L., Barnden, J.A., Hendley, R.J., Lee, M.G., Wallington, A.M. and Wen, Z. (2008). Affect Detection and Metaphor in E-drama. Int. J. Continuing Engineering Education and Life-Long Learning, Vol. 18, No. 2, pp.234–252.
• Zhang, L., Gillies, M., Dhaliwal, K., Gower, A., Robertson, D. & Crabtree, B. (2009). E-drama: Facilitating Online Role-play using an AI Actor and Emotionally Expressive Characters. International Journal of Artificial Intelligence in Education. Vol 19(1), pp.5-38.
• Zhang, L. (2010). Exploration of Metaphorical and Contextual Affect Sensing in a Virtual Improvisational Drama. Transactions on Edutainment IV. Pan, Zhigeng; Cheok, Adrian David; Müller, Wolfgang; Zhang, Xiaopeng; Wong, Kevin (Eds.). LNCS 6250, pp. 105–116.
• Zhang, L. & Barnden, J.A. (2010). Affect and Metaphor Sensing in Virtual Drama. International Journal of Computer Games Technology. Vol. 2010.
• Zhang, L. (2010). Exploitation on Contextual Affect Sensing and Dynamic Relationship Interpretation. In ACM Computers in Entertainment. Vol.8, Issue 3.
• Zhang, L. (2010). Affect Interpretation from Dramatic Improvisation using Character and Contextual Profiles. International Journal of Computational Linguistics Research. Vo1. 1, Issue: 3.
• Zhang, L. (2011). Affect Sensing and Contextual Affect Modeling from Improvisational Interaction. International Journal of Computational Linguistics. Vol. 1, Issue 4.
• Zhang, L. (2012). Exploitation in Context-sensitive Affect Sensing from Improvisational Interaction. Transactions on Edutainment. Pan, Zhigeng (Eds.).
• Zhang, L. & Hossain, Alamgir. (2012). Contextual affect sensing and metaphor interpretation. Special Issue on Games and AI. Entertainment Computing. Elsevier.

Conference papers

• Zhang, L. (2011). Context-sensitive Affect Sensing and Metaphor Identification in Virtual Drama. In Proceedings of International Conference on Affective Computing and Intelligent Interaction. Oct 2011. Memphis, USA.
• Zhang, L. (2011). Exploration on Context-sensitive Affect Sensing in an Intelligent Agent. In Proceedings of the 11th International Conference on Intelligent Virtual Agents (IVA’11). 15-17 Sept, Iceland.
• Zhang, L. (2011). Exploitation in Context-Sensitive Affect Sensing from Improvisational Interaction. In Proceedings of the 6th International Conference on E-learning and Games (Edutainment). Sept. 2011. Taiwan.
• Zhang, L. (2011). Developments in Metaphor Inference and Context-aware Affect Sensing. In Proceedings of IEEE International Conference on Computational Intelligence and Games (CIG). Sept. 2011. South Korea.
• Zhang, L. (2011). Affect Sensing in Metaphorical Phenomena and Dramatic Interaction Context. In Proceedings of the Twenty-second International Joint Conference on Artificial Intelligence (IJCAI). July 2011.Spain.
• Zhang, L. (2011). Contextual Affect Modeling and Detection from Open-ended Text-based Dramatic Interaction. In Proceedings of Annual Convention of the Society for the Study of Artificial Intelligence and Simulation of Behaviour (AISB’11), presented in the Symposium of Computing and Philosophy, York, UK.
• Zhang, L. (2010). Exploration on Affect Sensing from Improvisational Interaction. In Proceedings of the 10th International Conference on Intelligent Virtual Agents (IVA2010). Sept. 2010. Pennsylvania, USA.
• Zhang, L. (2010). Metaphor Interpretation and Context-based Affect Detection. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING 2010), Aug 2010. China.
• Zhang, L. (2010). Affect Sensing in an Affective Interactive e-Theatre for Autistic Children. In Proceedings of the 6th IEEE International Conference on Natural Language Processing and Knowledge Engineering (NLPKE), Aug 2010. China.
• Zhang, L. (2009). An Intelligent Agent with Affect Sensing from Metaphorical Language and Speech. In Proceedings of International Conference on Advances in Computer Entertainment Technology (ACE’09). Oct 2009, Greece.
• Zhang, L. (2009). Exploration of Affect Sensing from Speech and Metaphorical Text. In Proceedings of the 4th International Conference on E-Learning and Games (Edutainment). August 2009, Banff, Canada.
• Zhang, L. and Francisco, V. (2009). Affect Recognition from Speech. In Proceedings of the 14th International Conference on Artificial Intelligence in Education (AIED’09). 6th – 10th July, 2009. UK. pp.683-685.
• Zhang, L. (2008). Metaphorical Affect Sensing in an Intelligent Conversational Agent. In Proceedings of International Conference on Advances in Computer Entertainment Technology (ACE’08). 3-5th Dec. Japan. pp.100-106.
• Turunen, M., Hakulinen, J., Smith, C., Charlton, D., Zhang, L. and Cavazza, M. (2008). Physically Embodied Conversational Agents as Health and Fitness Companions. In Proceedings of Interspeech 2008. September 2008, Australia. pp.22-26.
• Smith, C., Charlton, D., Zhang, L., Cavazza, M., Hakulinen, J. and Turunen, M. (2008). An Embodied Conversational Agent as a Lifestyle Advisor (Demo Paper). In Proc. of 7th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2008), Padgham, Parkes, Müller and Parsons (eds.), May, 12-16., 2008, Estoril, Portugal, pp. 1713-1714.
• Zhang, L., Gillies, M. & Barnden, J.A. (2008). EMMA: an Automated Intelligent Actor in E-drama. In Proceedings of International Conference on Intelligent User Interfaces. 13th –16th Jan 2008. Canary Islands, Spain. pp. 409-412.

Master modules for Games Programming Pathway:

• Artificial Intelligence and Affective Computing
• Games Software Engineering
• Project and Dissertation

Undergraduate modules:

• Artificial Intelligence
• C++ Programming