UI · UX · Unity Engineer

Red Bull E-Sports and League of Legends contracted me to architect and then build out a method of injecting augmented reality LoL characters into their Twitch live streams with animations and and fx that could mirror the live play-by-play happening in-game.

After some experimentation with OpenCV tracking I realized the results just weren’t tight enough for what the clients needed. I eventually landing on ARKit with marker tracking as the most stable and reliable method of AR. Streamers were sent an iPad Pro and a unique marker.

Loaded on each iPad was a custom Unity ARKit app I designed and coded, with animated LoL characters that were listening to a Firebase database for gameplay events that would dictate what animations, characters, and FX would play for each streamer. The Firebase database was updated via simple react frontend that Red Bull producers had access during the stream.

League of Legends couldn’t provide 3D models, so I had to “unofficially” rip them from the game files, upscale the albedo textures, create custom normal / depth / metallic maps (as the game only had albedo), and generally clean up the models so they looked good for up-close broadcast shots.