Depthcast AR

Project

Depthcast AR

UI · UX · Unity Engineer

image

Overview

Red Bull E-Sports Studio initially contacted me to see how difficult adding augmented reality graphics to their broadcast using Unity and Vive tracking. Industry broadcast tracking solutions cost around $100k at the time, so that created the incentive to build an in-house product.

Approach

At the time Unity had no support for live broadcast workflows, so I needed to engineer solutions to multiple common problems:

  • Synchronizing framerates between Unity environment and the production camera shutter
  • Managing the offset between the tracker (Vive controller) and the broadcast camera
  • Outputting Unity as a chroma keyed layer that producers could mix live
  • Injecting the broadcast video feed back into Unity as a render texture to create reflections.
  • Mixing virtual lighting to match real-world lighting

I also eventually included functionality that made editing the scene easier during runtime. This included object manipulation inside VR space using controllers, as well as a desktop client that was synchronized using firebase + photon.

Media

Vive VR Tracking equipment connecting the broadcast camera to a virtual camera
Vive VR Tracking equipment connecting the broadcast camera to a virtual camera
The view from a helper app we developed to manage camera tracking
The view from a helper app we developed to manage camera tracking