Project Mimicus

Imagine if you could turn a video of yourself into an animation, and then use this animation to make your favorite 3D character mimic you. Sounds amazing, doesn’t it?

ABOUT

Project Mimicus is an interdisciplinary team of students at the Silicon Valley campus of the Entertainment Technology Center. We are working with Google to use cutting edge Machine Learning research to convert videos of human movements into 3D character animations. Our goal is to document our learning process and design a game using this technology while exploring a variety of possible game design applications.

Our project is based on a machine learning research paper (Learning Acrobatics by Watching YouTube) which converts videos of human movements into animations. Our goal is to explore possibilities of using this technology in the game industry. We are coming up with a wide variety of game design applications for this technology, based on which we are creating a game that tries to makes the best use of this technology

Read the Research Paper

WHY WE ARE DOING THIS

To explore possibilities of using this technology in the game industry and answer the question “In its current state, can we make a game using this technology?”

WHAT WE ARE BUILDING

We are exploring a wide range of possible game design applications and documenting our learning process while trying to build a game which uses this technology.

OUR APPROACH

We are putting together a machine learning pipeline which goes from 2D video to animations in Unreal. We will take this pipeline and create an ideal game loop for it.

Our Team

Alexandra Gobeler

Alexandra has a background in mechanical engineering and came to the ETC to be in an artistic environment. She is seeking a creative role in engineering at a themed entertainment company, looking to use her technical skills in an imaginative way.

Artist | Designer

ETC PagePortfolio

Haran Kim

Haran received two MFA degrees from Indiana University majoring in Printmaking and Digital Art. She is a digital artist who has a great passion for the technical and creative aspects of the entertainment industry and is seeking opprtunities as an artist. 

Artist | Designer

ETC PagePortfolio

Siyu Ren

Siyu Ren earned his Bachelor’s degree in media art major at Hong Kong Baptist University. Having learned modeling and animation, he has kept on learning rendering and tool development in game engines and is striving for a career as a technical art.

Game Programmer

ETC PagePortfolio

Himanshu Telkikar

Himanshu is passionate about building software and games that tackle real world problems. He is an engineer who loves art, design and is interested in entrepreneurship.

Producer | Designer

ETC Page Portfolio

Eliza Zhang

Eliza is a software engineer who is interested in rendering and machine learning. Working at the intersection of technology and art is  exciting to her.

Machine Learning Programmer

ETC Page

Faculty Advisors

Melanie Lam

Carl Rosendahl

Latest Posts

Week 16

This was our final week of the semester. We gave our final presentation at EA on Wednesday (May 8, 2019). Here’s our final presentation: This was followed by a final[…]

Read more

Week 15

As we are one week away from our Finals Presentation, we are wrapping up our demo and working to polish our documentation for hand-off. PROGRAMMING We tried another approach for[…]

Read more

Week 14

PROGRAMMING AND ART This week, we focused on improving the quality of the animations which are recreated. The machine learning pipeline gives output as animation data (bvh) file, which is[…]

Read more

Contact

Email us: etc-sv-mimicus@lists.andrew.cmu.edu