Project Mimicus

Imagine if you could turn a video of yourself into an animation, and then use this animation to make your favorite 3D character mimic you. Sounds amazing, doesn’t it?

ABOUT

Project Mimicus is an interdisciplinary team of students at the Silicon Valley campus of the Entertainment Technology Center. We are working with Google to use cutting edge Machine Learning research to convert videos of human movements into 3D character animations. Our goal is to document our learning process and design a game using this technology while exploring a variety of possible game design applications.

Our project is based on a machine learning research paper (Learning Acrobatics by Watching YouTube) which converts videos of human movements into animations. Our goal is to explore possibilities of using this technology in the game industry. We are coming up with a wide variety of game design applications for this technology, based on which we are creating a game that tries to makes the best use of this technology

Read the Research Paper

WHY WE ARE DOING THIS

To explore possibilities of using this technology in the game industry and answer the question “In its current state, can we make a game using this technology?”

WHAT WE ARE BUILDING

We are exploring a wide range of possible game design applications and documenting our learning process while trying to build a game which uses this technology.

OUR APPROACH

We are putting together a machine learning pipeline which goes from 2D video to animations in Unreal. We will take this pipeline and create an ideal game loop for it.

Our Team

Alexandra Gobeler

Alexandra has a background in mechanical engineering and came to the ETC to be in an artistic environment. She is seeking a creative role in engineering at a themed entertainment company, looking to use her technical skills in an imaginative way.

Artist | Designer

ETC PagePortfolio

Haran Kim

Haran received two MFA degrees from Indiana University majoring in Printmaking and Digital Art. She is a digital artist who has a great passion for the technical and creative aspects of the entertainment industry and is seeking opprtunities as an artist. 

Artist | Designer

ETC PagePortfolio

Siyu Ren

Siyu Ren earned his Bachelor’s degree in media art major at Hong Kong Baptist University. Having learned modeling and animation, he has kept on learning rendering and tool development in game engines and is striving for a career as a technical art.

Game Programmer

ETC PagePortfolio

Himanshu Telkikar

Himanshu is passionate about building software and games that tackle real world problems. He is an engineer who loves art, design and is interested in entrepreneurship.

Producer | Designer

ETC Page Portfolio

Eliza Zhang

Eliza is a software engineer who is interested in rendering and machine learning. Working at the intersection of technology and art is  exciting to her.

Machine Learning Programmer

ETC Page

Latest Posts

Week 10

PROGRAMMING As previously stated we are using a Google Cloud Virtual Machine to speed up the machine learning computations, which is required to achieve an acceptable video processing time. This[…]

Read more

Week 9

This week most of our team was at Game Developers Conference in San Francisco. However we arrived at a decision about which game we want to build for our final[…]

Read more

Week 8

This week we we had our halves presentation. We spent most of our time preparing for halves and the rest of it building the prototype for our demo. We added[…]

Read more