Ideation Week 5:
After hearing the tech exploration report from programmers, we looked at technologies and features with three criteria: stability, feasability, and fun. We narrowed down the recommendations by programmers and iterated a modified game design concept that we could feasibly prototype within a 2-week sprint.
We felt that the lowest hanging fruit is a fixed-camera setup along a track that players can add pieces to, mapped to the length of a TV screen so it looks like the digital elements are interacting with the physical elements.
We also had a visit from Anthony Daniels, who absolutely loves LEGO and gave us some of his thoughts on the project! He suggested a hand sanitizer station located nearby and brought up a number of logistical considerations like clean-up and tear-down/reorganization. Additionally, he raised a great idea of keeping the various LEGO creations in a digital database, and letting souvenirs happen, since LEGO pieces will inevitably leave the space.
- Reference: Abstract: The Art of Design – Cas Holman
- Play as modular and constantly reinvented by kids
- Philosophy is on modular toys enabling an empowered child to reinvent their surroundings and inject fun into their own lives
- Interesting design prompt: designing for function (reinvent a new way to carry water vs. reinvent a jug)
- Reference: LEGO Hidden Side: Stunt Truck Set
- AR mobile component held a WOW moment when you scan the truck and the environment swirls/fades in and vehicle comes to life
- Game mechanic utilized a color wheel on the truck made out of spinning LEGO parts that you could use to input into your phone
- Seems to be subtracting a space using a mesh model of the truck
- Can scan a mini-fig into the scene as well also using the same technique
- Ghost ‘shooting’ part of the overall game experience is not LEGO-dependent and is basically a mini-game on your mobile phone using the position-tracked truck to set the environment’s position
- Rotating the truck rotates the environment on the phone view so that it is always aligned in the same orientation, and rolling it forward moves the phone’s perspective back
Art Progress
Our 2D artist has created some voxel-style concept art to communicate the current design iteration – we wanted to create a platformer style game that would correspond to three tracks of LEGO baseboards. One track would be regular terrain, one track would be the bouncy area and one track would be an anti-gravity area. The overall narrative driving the game would be that you play as Scotty and you’re trying to get some coffee for the student, who is slowly turning into a zombie from all the studying.
Since our client has emphasized the use of CMU-specific content, our 3D artist has made a 3D LEGO-style Scotty! The famous CMU Scottish Terrier. She has a rig and animations for running as well!
Tech Progress
Our programmers had thought of a solution to capture the LEGO board data by placing four Aruco markers on the corners of the board, and then calculating the homography to correct the perspective distortion on the captured data. Then we would divide the corrected image according to the size of the LEGO grid on the baseboard (for instance, one LEGO baseboard is 32×32 studs) in order to obtain a voxel-style height map and color map.
The video below shows the homography in action by moving the corner points. We would then obtain the position of the Aruco markers and provide that to this function in place of the corners we are manually moving right now.
Another thing we would then have to do, other than correcting the perspective distortion on the captured image, was to then correct the depth data based on the baseboard height. The reason for this is shown in the diagram below where we would have skew in the depth data: the Kinect would read the top depth as x and bottom depth as y, and something in between even if there was a LEGO piece on it could have a depth read of x as well.
What we would then do to account for this is to do bilinear interpolation on the whole LEGO board image using the depth values of the corners in order to know the depth of that point on the board, and then subtracting the read depth value to obtain the height of the LEGO piece on the board.
We also talked to Maria Montenegro, a visiting ETC alumni who has worked extensively with projection mapping and computer vision. We wanted to validate the initial ideas we had about implementation with someone more experienced to see if there were better ways we could accomplish something.
Talking to her allowed us to confirm that the Kinect was the better option compared to other cameras such as the Intel Realsense or the Zed Mini, because the time-of-flight method of capturing depth data would not be affected by the projected images. Additionally, she mentioned that using four Aruco markers on the corners seemed like overkill since we can get complete pose and scale data from just one Aruco marker.
This next video shows how we find the corners based on just one Aruco marker next to the board.
