A psychedelic music video for a legendary Russian rock band that not only scooped up awards but also harnessed the power of neural networks before the whole world even knew what those were!
Music Video
The Story
Back in 2017, the neural net was just a baby with big dreams, and I was playing the role of the proud parent. There were no Midjourney, Dall-e, or Runway Gen2's around to show how it's done, just a bunch of us tech cowboys riding the first wave of style transfer aka Tensor Flow—like pioneers panning for gold in the river of open-source codes. Apps like Artisto and Prisma were the first tools out of the gate, turning static snapshots into stylized prints.
I was after the bigger fish: crafting a video narrative that didn't stutter or blink too much across frames—a tall order for the tech at the time.
Armed with this drive, I took to New York’s streets, filming parkour runners as they flipped and flew against the skyline, all to catch a glimpse of what this new tech could do. The result? A snapshot turned into a work of art, thanks to a little neural network magic and some help from the Artisto folks. I took this prototype to the "Mumiy Troll" band, and they gave it a thumbs-up for a full-blown music video..
The quality of those early neural net images? Rough around the edges, sure, but they were a proof of concept that screamed potential. We were writing the rulebook for a game no one knew how to play yet.
Fast forward to production: "Waking Life" was our benchmark, only we were ditching the year-long rotoscoping marathon for the neural net sprint. I had five worlds to create, each with its own flavor, and I needed style transfer that didn't exist yet. So, I got a crew of code wizards on board to whip it up from scratch. When that plan hit a financial and expertise roadblock, it was time for a pivot.
Fast forward to production: "Waking Life" was our benchmark, only we were ditching the year-long rotoscoping marathon for the neural net sprint. I had five worlds to create, each with its own flavor, and I needed style transfer that didn't exist yet. So, I got a crew of code wizards on board to whip it up from scratch. When that plan hit a financial and expertise roadblock, it was time for a pivot.
Enter Oleg Illarionov, creator of the Vinci app, our tech Merlin, who handed me the keys to his castle: a server where video files went in and came out looking like Vinci had a field day with them. But here's the kicker: the tech was moody. Some styles turned drawings into eye candy but threw the color balance off a cliff. Others got the hue right but botched the lines.
Our workaround was a Frankenstein of filmmaking: a mix of shooting live action, creating 3D models, and stirring up a storm with particle effects. We'd composite, color correct, run the gauntlet through neural networks, and blend it all with the original footage—repeatedly, like seasoning to taste.
Looking back at the video stuff from 2023, it's kind of hilarious to think I might've gone overboard with the neural net redraws. There were moments when those 3D frames stood tall and proud on their own, no AI makeover needed. They actually looked a couple of notches cooler in their raw, un-AI'd glory than they did post-neural nip and tuck.
But back in the day, we were like kids with a new toy, giddy just to see those neural nets do their thing. It was like watching magic unfold in real-time. Every pixel shift and color swap was a little victory dance for technology—and we were throwing a dance party with every successful render.
Our workaround was a Frankenstein of filmmaking: a mix of shooting live action, creating 3D models, and stirring up a storm with particle effects. We'd composite, color correct, run the gauntlet through neural networks, and blend it all with the original footage—repeatedly, like seasoning to taste.
Looking back at the video stuff from 2023, it's kind of hilarious to think I might've gone overboard with the neural net redraws. There were moments when those 3D frames stood tall and proud on their own, no AI makeover needed. They actually looked a couple of notches cooler in their raw, un-AI'd glory than they did post-neural nip and tuck.
But back in the day, we were like kids with a new toy, giddy just to see those neural nets do their thing. It was like watching magic unfold in real-time. Every pixel shift and color swap was a little victory dance for technology—and we were throwing a dance party with every successful render.
It was a chess game, with each move more calculated than the last, building up to a checkmate of visual feast. In the end, it was a mix of the artistry from video games that influenced me and the untamed spirit of cinema that I admired, all brought to life by the ragtag tech of 2017—raw, unpolished, but bold and unapologetically futuristic
We pulled off this gig with pocket change, needing a solid eight and a half days to roll tape—that's epic for a music video. The shoot was a beast.
Picture this: most of the band, even after the slick animations and storyboards, didn't have a clue about the cinematic magic we were crafting. And talk about high-octane action—the script was peppered with stunts that would make a stuntman's heart race, demanding a battalion of body doubles and some serious prep work.
Then there was the curveball with Ilya Lagutenko, the band's lead. The guy was as elusive as a shadow—on tap for a day or two, tops. So, I had to whip up a Houdini act to make the story work without him hogging the spotlight.
This is why, in the video, the hero's mostly a mystery man. We see him from the back, through his eyes, or in broad strokes from the side or a bird's-eye view. We turned limitations into our storyboard, making sure the lead could be anyone, anytime.
Picture this: most of the band, even after the slick animations and storyboards, didn't have a clue about the cinematic magic we were crafting. And talk about high-octane action—the script was peppered with stunts that would make a stuntman's heart race, demanding a battalion of body doubles and some serious prep work.
Then there was the curveball with Ilya Lagutenko, the band's lead. The guy was as elusive as a shadow—on tap for a day or two, tops. So, I had to whip up a Houdini act to make the story work without him hogging the spotlight.
This is why, in the video, the hero's mostly a mystery man. We see him from the back, through his eyes, or in broad strokes from the side or a bird's-eye view. We turned limitations into our storyboard, making sure the lead could be anyone, anytime.
A few examples of raw scenes:
The beard-growing scene? Classic. To cut costs on the music video production, I ditched the whole computer-simulated hair growth thing. Instead, my crew and I rigged up a fake beard for Ilya and went old school with stop-motion. We trimmed it down, frame by frame, so when we ran the footage backward, it looked like the beard was growing in fast-forward. It turned out to be a blast: simple, budget-friendly, and visually killer.
The main villain of this trippy universe? A Chilean Degu squirrel from the pet store.
I had this wild idea to feature the little critter in our video, initially thinking of hiring an animal trainer. But here's the kicker: hiring a pro for a couple of hours cost more than buying two of the furballs outright.
So, this video turned out to be a game-changer for me. I didn't just jump on the neural net bandwagon before it was cool, I also ended up with two awesome pet pals who are still part of the fam.
And the reason behind casting that furry little Degu? Straight up, they bore an uncanny resemblance to the Moomins by Tove Jansson—those whimsical characters that inspired the rock band's name. It was like a little nod to the origins, a blend of nostalgia and rock vibes, all rolled into one fuzzy package.
The music video is a love letter to pop culture and vintage side-scrolling video games, chock-full of references for the eagle-eyed. Yeah, you’ve probably spotted those giant mushrooms straight out of Morrowind. Give the video a couple more watches, and you'll unearth a trove of Easter eggs—each viewing rewards you with a new find.
This was my first rodeo with heavy graphics in a music video. And, as they say, the first pancake is always a bit lumpy. I made a buffet of rookie mistakes that schooled me in the fine art of picking a solid team and lit a fire under me to dive deeper into the world of computer graphics myself.
If you reckon this music video is a visual effects smorgasbord that's off the charts... you're dead right. It's like a head trip through a tech-laced wonderland. Despite the tech imperfections and the now seemingly rough-hewn graphics, this project holds a special place in my heart.
This wasn't just any video; it was an experimental canvas, a testbed for new tech. And guess what? It nailed its mission, got the fanbase hyped, and even scored some festival cred. It screened at the Girona Film Festival, was a contender at the "Audience Awards" of the Music Shorts Film Festival in LA, sparkled at the Video Art Miden in open-air Greece, was featured in the official lineup at the "Caostica" festival, and even took home the gold at the 23rd AVANCA in Portugal.
Not too shabby for an experimental project aimed at testing new technologies.
