Can dinosaurs open the door to AR for live audiences in NZ?

As the Coachellaverse launches and LiveNation and Snap partner to embed AR into a series of festivals and concerts across the USA, leading NZ digital creation agency Satellite demonstrate that ‘the magic is achievable” by setting six full-scale dinosaurs loose in Auckland Museum.

The big concerts and festivals in the US have been experimenting with augmented reality (AR). Coachella partnered with Unreal Engine, the Epic Games 3D software development tool, to create a livestream with massive psychedelic 3D images blending in with Flume’s stage design and set, visible only to the at-home viewers. For the festival attendees, a Coachellaverse app offered Coachella-themed AR effects and immersive Instagram filters were developed in collaboration with a number of Spark AR creators.

Snapchat and Live Nation also launched a series of AR experiences beginning with EDC 2022, the Electric Daisy Carnival, bringing EDC’s iconic owl to life in the sky and daisies sprouting around festivalgoers’ feet. Not necessarily adding anything to the music, but a promotor’s dream as audiences Snapped and shared. Lollapalooza in Chicago, Wireless Festival in London, Rolling Loud in Miami and The Governors Ball in New York all featured AR.

The question is; how doable is it in New Zealand?

Satellite, digital innovation, activation, event and content creation agency, recently partnered with Auckland Museum to create an AR experience to complement ‘Peter’, a twelve metre long, four metre high T. Rex skeleton currently on loan from the USA. Not only containing real bones, as opposed to the more common place replicas found in museums, this skeleton is a stunning black colour due to mineral deposits on the bones, one of only four ever discovered.

“We have a close working relationship with the Museum, collaborating on projects a couple of times a year. The Museum team knew what they wanted; six true to life AR dinosaurs located throughout the exhibition,” explains Bryn Irwin, Satellite’s Head of Production. “They were also adamant that there were as few barriers as possible; no apps to download or log ins.”

Satellite were in agreement on removing an app from the equation, having found on previous AR projects that no matter how clever the AR component, it has to be easily accessible, “We did our homework and offered the Museum two options. We researched an open-source solution which was cheaper but with less certainty around the end result, and we approached a third party company, 8th Wall, who offer a browser based AR platform, at discounted rates for a charitable trust.” The Museum went for the latter option and Satellite started work, partnering with 8th Wall.

“The 8th Wall platform hosts your AR objects and provides an existing set of libraries and code to support the placement of the physical objects. We sourced vector-based 3D models of the dinosaurs online and then used our 3D rendering software to add animation and ensure that they were factually accurate. We wanted them as near to life size as we could achieve whilst considering the practicalities of where they were to be located.”

The main challenges were the lighting and placement, “The lighting was complicated by the different light levels in the various locations across the exhibition; some very dim, some quite bright. We coded the lighting outside of our 3D rendering software to make it easier to tweak.”

“Placement came down to the size of each AR dinosaur. A huge brachiosaurus needs to be a few metres away otherwise you would only see his feet, whereas a smaller dinosaur could be cast nearer. To support users with positioning the models, we used a static photo including a wireframe of the dinosaur during loading.”

The Museum set up plinths with QR codes that could be scanned to reveal the dinosaurs, capitalising on the wide adoption of QR codes in recent times. “Anecdotally, it has been a resounding success. Everybody is delighted and enjoying photobombing the dinosaurs! It is the Museum’s first ever AR experience and has given them the confidence to experiment further.”

If it sounds straightforward, that’s because it was. “AR is pretty basic. It doesn’t take months and months of planning. Assets already exist and the barriers are getting less and less. Yes, there is some cost involved but that is coming down too. You just need the right creative and development skills. I knew which of our developers to choose right away. He has a background in the visual space and was passionate about AR. All the costs are upfront too bar 8th Wall’s monthly hosting fee, so extending the life of the project is simple.”

Bryn acknowledges that two of the potential risks of using AR at a larger live event, bandwidth and overloading the website, are mitigated by a museum environment, “We compressed the objects down to 10MB, but visitors can use museum Wi-Fi too. And the maximum number of concurrent users is relatively small. For larger audiences, 8th Wall would need to scale their service to meet demand and there would be a cost against that, but it is still doable. And with 5G potentially accessible at stadiums, it is just a matter of the target audience upgrading their phones.”

Doable is Bryn’s AR mantra. As well as the ‘EDC-esque’ AR objects flying in the sky over a concert, he cites examples such as AR treasure hunts at trade shows where you unlock five objects and receive a free gift, or placing AR objects in your own space. U2 sat on your settee! The Satellite team are confident that they have the capability here and in Australia, where Satellite already has strong relationships with Coca-Cola, Telstra, and Monash University, to pull off AR in a large-scale live entertainment environment, “It just comes downs to skills and creativity.”

For the Satellite team the barriers have already fallen or are falling by the wayside. No need to download an app, assets galore online, audience familiarity with QR codes, hosting platforms that can scale up for larger numbers, the ongoing roll out of 5G, and the increasing take up of 5G phones.

It is time to start planning.