Starscape

Matt (aka Stew) provide a bit more detail about our jpeg streaming that was discussed in our half-semester presentations:

Unity can not handle a stream of infinite length, and Anthony found the runtime class in Unity called WWW to facilitate our needs. He calls a function that returns an object, which can be applied as a texture to a plane. When we transition to the camera ‘feed’ in our experience, the script calls a URL that I have designated for him. The function that Anthony wrote refreshes this image as soon as it has downloaded the previous one, which simulates a live feed that runs at a few frames a second.

The camera is controlled via a set of computer gateway interface (CGI) commands provided by Panasonic. This set of instructions allows us to access the camera via a website, where the camera acts as the server. The camera has its local application files that translate the commands given by the website into the control language used to manipulate the pan, tilt, and zoom controls. My part of the workaround was to use the still camera functionality to capture an image of the designated resolution. 3-4 frames a second is not a video stream. But! We can get away with this because the camera will already be positioned on the child’s star before we transition to the live feed. The effect gives a ‘twinkle’ as the light subtly changes the RGB pixel values at this rate, furthering the imagined live stream.

Cassandra wrote up a summary of some progress the tech team made over the past few weeks:

We’ve made significant progress in connecting our network through the server and over the web. It was relatively simple to make the server pull data from the database and concatenate the data text into a URL that the camera would understand. There are many handy modules for Python to speak to MySQL database; in this case, we used PyMySQL. However, figuring out how to send that URL to the camera automatically was more challenging. Cassandra and Tom found the HTTP Protocol Client in the python library, which greatly simplified the process. One can send the values of pan, tilt, zoom, and focus by POSTing to the IP address and port of the camera. The function returns a number code to alert you as to the success of the action.

The server is also communicating with the iPad now, receiving strings of text. Anthony and Cassandra found C# scripting for socket client connections and, with a little tweaking, successfully sent strings – such as a 6 digit barcode – over the wireless router to the server. In the process, we’ve learned a lot about how servers and clients operate and how to derive a desired function from sockets. Looking forward, we’re establishing text codes to send for events on the iPad, such as click events or onscreen events. As we get these developed, we’ll be building the environmental elements for the world.

As part of our ongoing effort to highlight our technical achievements, Tom has some info on our system overview:

Guests using the Starscape application will be holding an iPad in their hands, but this is only one small part of the show.  Behind the scenes there are a number of systems and devices communicating and working together to make this experience happen.  Here is a brief description of the technology that makes Starscape run!

The iPad – Guests will explore and interact with the fairy realm using an Apple iPad 2.  The Starscape application uses built-in sensors such as the gyroscope and compass to determine where it should look in the virtual world.  When the guest interacts with an object or character inside the world, that generates an event that is sent out over a wireless network to the Server.

The Server – This computer works behind the scenes to keep everything running at the same time.  During a show, it listens for events from the iPad, and uses that information to determine what sounds to play and what lighting events to send to the Show Control system.  The Server stores the location of every star in the tower and when the guest scans their barcode at the entrance, that location is sent to the iPad and used to place the star in the virtual sky.  The Server also hosts a web-based interface to allow the staff at Give Kids to record the stars when they place them on the tower walls.

The Show Control System – This machine controls the lights and sounds inside and outside of the tower.  When the Server sends it a lighting cue during a show, the Show Control System will respond by sending signals to the equipment that controls the individual LED light fixtures in the tower.  This system will run events in the tower during times when the show is not running.  It also controls the sounds and effects for the outdoor show, where the fairy appears from the cupola on top of the tower each evening.

The Cameras – There are 9 networked cameras placed throughout the tower.  At the end of a show, the guest sees a live image of their star, sent over an internet connection by one of these cameras.   The Server uses its database to recall the location of the guest’s star, and sends a command to the appropriate camera to point and zoom onto that spot, then send the feed to the iPad.