HiParly is an AI-powered language learning app helping users practice real-life conversational skills in multiple languages. Their mission is to make language learning more engaging and practical.
Tasks of the IDP:
This IDP focuses on implementing real-time 2D avatar generation using both speech audio and text input. The project aims to create a dynamic and interactive avatar that can simulate real conversations by adjusting its expressions, lip movements, and gestures in response to both audio and textual cues.
1. Literature Review
- Conduct a literature review on existing methods and technologies related to audio-driven talking face generation, text-based avatar animation, and real-time synchronization.
- Use the findings to inform the project’s development, ensuring it builds upon and enhances existing techniques.
2. Avatar Animation and Synchronization
- Develop algorithms for processing speech audio and text input to extract features like pitch, intonation, phonemes, and contextual cues.
- Use extracted audio and text features to animate a 2D avatar in real-time, ensuring accurate lip-sync and dynamic expression changes that reflect conversational tone and content.
- Ensure the algorithms run efficiently in real-time, minimizing latency for smooth interaction.
Contact: As a TUM alumna, I’m excited to collaborate with TUM students, and HiParly is part of UnternehmerTUM and TUM VentureLabs! Apply via email: Send your CV and brief email to gunes(at)hiparly.com You can apply as a team or individual (and taico will assemble the team)
TUM ERI Supervisor: TBD
Proposed start date : ASAP