Freakfort: Art, Music, and Augmented Reality

There are two words I’d use to describe late March in Downtown Boise—energy and discovery. And there are two reasons for those two words: one, the long-awaited sunshine bringing Boiseans out of hibernation and two, Treefort Music Festival. Treefort is the annual convergence of music, art, food, and technology—a Pacific North-by-Northwest of sorts. Over a five-day period over 400 bands and thousands of indie music lovers descend on the City of Trees.

Crowd watching a band play on a lit stage with guitar amps
Dinosaur Jr. playing at Treefort 2023. Photograph by Preston Valles.

In the spirit of Treefort, we wanted to build something that would inspire curiosity and discovery, and something that would literally bring an iconic staple of Downtown Boise to life. That iconic staple is Freak Alley, an open-air multi-artist mural gallery spanning a full city block in the heart of Downtown. 

Hands holding up a phone pointed at a mural in the background, the image on the phone shows digital banners with artist info

With the blessing of the bands and mural artists featured, we created Freakfort, a web-based augmented reality experience that introduces Treefort-goers to Boise-based bands by pairing them with animated murals from local artists. Users can simply visit Freak Alley, open the website and point their camera to discover animated murals. The murals come to life with animation matched up to a song from a local band, along with mural artist info and who/when/where the band is playing during Treefort. Plus, since the user doesn’t know which of the murals in Freak Alley are animated, they’re encouraged to explore the space, pointing their camera and engaging with the artwork in a way they may not have otherwise seen.

New Possibilities with Web-Based Augmented Reality

Augmented reality and spatial computing has been around for some time now, but until recently it’s been mostly limited to app-based experiences. This is great for things like social filters or spatial design tools, but the need to download an app presents a barrier to reaching wider audiences in-the-moment. Advances in web-based mobile technology have opened up new possibilities for creating quick and accessible AR experiences, no app required. Now, we can build for wider audiences and more ad hoc purposes, making AR available as a storytelling medium for more brands and organizations.

For this project, we explored three web-based AR platforms: AR.js, MindAR, and 8th Wall. These platforms all have a few things in common. First, they all give you the option to use the popular web VR framework A-Frame or the also-popular 3D framework Three.js to create the digital elements of the experience. They’re all web-based. And they all offer different options for how the AR functions, for example image tracking using a key visual, face tracking, and location-based tracking.

Ultimately, we chose 8th Wall with Three.js as our AR platform. We needed highly accurate tracking of complex images for our mural animation overlay effect to work, and through testing we discovered that 8th Wall seemed to be the strongest in that area. It also offers a solid cloud-based editing suite and emulator. Unlike AR.js and MindAR—which are both open source—8th Wall is a paid platform, which may be a barrier to consider as commercial licensing can get quite expensive. That said, the non-commercial pricing is reasonable and the platform provides a ton of added value in terms of guides and tools.

User interface of a code editor

How it Works

The AR experience has three main components. First, the animation itself, which is simply an MP4 video cropped to match the surrounding environment. The animated video is positioned directly over a target image, allowing the AR to be as seamless as possible against the real-world mural background. Aside from being the most optimal way to serve an animation of this type, using videos also gave us the benefit of automatically serving the audio, synced up perfectly.

A three quarter view mockup showing the AR element layers in relation to the image target and mural wall

As the camera position moves, the position of the elements in 3D space are recalculated in relation to the target image, keeping everything locked in place. 

To create the animations, we captured high-resolution images of the artwork and broke the images out into layers. We then conceptualized rhythmic, looping animation effects that pay homage to the original artwork while syncing up to the beat of the featured song.

Alongside the animated video overlays in 3D space are the band and artist info. For the bands, we tie in Treefort using the iconic banners that hang on behind the stages.

Photo by Christina Birkinbine

About The Artwork

For this initial proof of concept, we started with three murals and three local bands. Big thanks to the artists and bands that let us use their work for this project.

Dana Wagner + Amoeba Arena’s “Anthill

Animated by Gilberto Sauceda

Ashley Dreyfus + Sove the Second’s “Answer the Phonuuuh

Animated by Preston Valles

Julie Pegan + Moon Owl’s Mages “Pressure

Animated by Kelsey Rath

Try It For Yourself

The Freakfort AR Experience is free and open for anyone to use. If you’re not physically at Freak Alley, you can try it out by opening the website on your mobile device and pointing your camera at the mural images below:

Mural showing a Twinkie and a cockroach holding hands with a mushroom cloud in the background
Mural showing a cowgirl in pink and blue walking wrapped up in a large flowing number two pencil
Mural showing three blue humanoids sitting on a golden brick wall with flowers talking on cell phones

Let’s Build Something!

Do you have a story you’d like to tell through augmented reality? If so, we’d love to chat with you about it.

Up Next

Drake Cooper Welcomes New Employee Owners with Enthusiasm for the Year Ahead

mployee-owned advertising agency Drake Cooper is thrilled to announce ten new additions across creative, account management, media, and strategy and analytics disciplines, fueled by the acquisition of new clients and organic growth among current clients.

Drake Cooper
Drake Cooper
March 27, 20242 min read