Being able to stream millions of songs from the cloud to your phone is a fantastic aspect of the modern era, but when you really get down to it, the music-listening experience is essentially the same as it’s always been: You press play on a song or album, toggle between different tunes, adjust the volume, etc. Beyond those options, there’s limited interaction with the music itself. But, if Moodelizer has its way, that could all change soon.
“Once you’ve experienced dynamic music it’s kind of boring to turn back to straight static music.”
A Swedish-born company, Moodelizer Music Technology has developed a new way to interact with music called Dynamic Playback. Accessed via the Moodelizer mobile or desktop app, Dynamic Playback allows you to adjust the style, intensity, variation, and tempo of a new kind of audio recording, in real-time.
With Moodelizer, toggling between which instruments play in the background of your selfie video can be as easy as dragging your finger across a touchscreen, and you can even shoot video and tinker with the music in real-time. Essentially, the interactive nature of Dynamic Playback puts creative control in your hands.
That’s all well and good for spicing up Instagram stories, but Moodelizer has grander ambitions for Dynamic Playback which could allow it to one day replace ubiquitous audio formats like MP3s. Soon, Dynamic Playback could allow users to switch between multiple versions of the same song as easily as adjusting the volume — and that’s only the beginning.
Digital Trends spoke to Moodelizer’s founder, Carl-Michael Herlöfsson, about how his company’s technology could revolutionize audio formats, its applications in augmented reality, and when we could start listening to music with Dynamic Playback.
Digital Trends: How was Dynamic Playback created?
Carl-Michael Herlöfsson: It was an idea of mine that has been milling around for quite some time. I experimented with a whole bunch of different methods trying to manipulate sound in real time. … But, there wasn’t any way to do [what I wanted] with the existing software that I had. I had every [type of] software imaginable. I was a sound engineer and music producer, originally. Then I hooked up with a really skillful coder who said that he and his friend had already written a sound engine and they were willing to try this whole idea out … . So, they wrote a prototype program and I dropped in some audio files … .
The first time I tried it out I realized that it was very cool. So, I rounded up some basic startup money and I got it started. The first two years were mainly spent on trying to figure out how to compose and write music for this format. So, I worked with a network of 30-40 composers, producers and musicians, and people that I had in my sort of network around me. … We wrote music and tried it out and we did some more coding and did some more music. It was sort of a collaborative process. Finally we reached the point where we thought, “Here is the dynamic playback format,” which we then went onto seek a patent for, which we got.
How is Dynamic Playback different from existing formats?
“Dynamic Playback opens up the music. It makes it almost three dimensional, in a way.It changes what’s playing and what you’re hearing.”
All the old kind of formats — if you want to call them that — like MP3, CD, and vinyl are just stereo files … You can’t do anything to it. You can turn it up, or turn it down, maybe add some bass. But, that’s it. Dynamic Playback opens up the music. It makes it almost three dimensional, in a way. It changes what’s playing, which instruments you’re hearing, which variation. Also, from the point of creation for musicians and the ones who are writing the music, they can open up the song where you can give the user access to sort of hidden gems within the song.
How does it work? Why would listeners use this format?
So, basically, if you have a song or a piece of music within the touchpad that is on the Moodelizer app, you can arrange the music, or decide which instruments you want to play. The interesting thing with Dynamic Playback technology is you do it in real time. Which means that you can [shoot a video] and you can adjust the music at the same time. It’s all instant, so you go from a single string line to a full orchestra — anywhere in between — seamlessly, with just the swipe of the finger. … swiping up and down changes intensity, and swiping left and right changes variety, or distinct versions of it.
But [it also works] diagonally. If you’re in the lower-left corner, you get the least amount of sound. If you’re in the upper right corner you get the most amount of music. You have thousands of different positions within that space where you can achieve different results.
If you’re crossfading different versions of the same song, it’s like having five songs playing at the same time.
Yeah. You can add different songs, as well. [But with the same song] you can have a techno version on one end, and a folk music version on the other end, and you can decide which one you want to listen to at any given time. Also, you can blend them together and get the techno-folk version …
The Moodelizer app is fun, but we get the feeling that there are other applications that could benefit from Dynamic Playback.
We’ve been experimenting heavily with different kinds of input modes. You can control the Dynamic Playback file, or a piece of Dynamic Playback music, with any kind of inputs like movement of the phone, face recognition — it recognizes your smile and plays happy music.
We have isolated a few functions that we are going to slowly implement into the application. The first one we’ll be launching within a month or a month and a half (we’re in beta testing right now), it’s the [augmented reality] function where you are placing musical pieces within a room. You can kind of score the room in advance. You mark a corner of the room or beach, or wherever you are.
You mark a place, and you say, “OK, this place is going to have this music,” and then you pan the camera, and you say, “OK, there’s my girlfriend, she’s going to have that piece of music [attached to her]. Oh, there’s my couch, It’s going to have this resting music.”
When you start to film, the music will automatically change depending on where you point the camera. So you’re scoring your environment in real-time. We’re actually in discussions with pretty much all of the major social media platforms, and also with manufacturers in China and the [United] States. They are very interested in what this can do for them and how to bring music forward in new ways.
You mentioned there could also be a geolocation component.
Definitely. We have experimented with a maps function where you use the compass of the phone and the GPS location of where you’re at.
“You mark a place, and you say, ‘OK, this place is going to have this music,’ and then you pan the camera, and you say, ‘OK, there’s my girlfriend, she’s going to have that piece of music [attached to her].'”
So, say you’re standing somewhere and you mark a destination on the map, and you are listening to the music; when you’re heading in the right direction, the music gives you feedback telling you you’re going the right way. When you get closer to where you’re going, the music will give you musical signals.
You pretty much can start it up, put the phone in your pocket, just listen to the music, and guide yourself to the locations on the map. This can be used in geo-based games like Pokemon Go. Or, if you have a store or restaurant and you want people to go there, or meeting with friends, you can have two people decide the meeting place and then both of them are listening to the music to steer them to the same spot.
That might be the most practical application. So many people lose themselves in their music and they don’t want to keep looking at their phones.
Running is another thing we’re working with right now, using wearables like watches for pulse input. If you’re running you can program running intervals to this location. It will both guide you with the music to where you’re going, but it will also drive you. When your pulse hits a certain rate, it will increase the music accordingly to create a sort of musical boost.
So, there are currently no notable artist or publisher using Dynamic Playback — the music in the Moodelizer app is all commissioned by your company. Are there any plans to change that?
This is a disruptive technology in a way that brings a whole new way of looking at something. Obviously, there is a lot of work trying to help people understand what it is, and how they can use it. The interest has been so great already, both from the music industry and from the tech industry.
There are so many possibilities for both parties with a format like this. So we’re trying to get someone really good both artistically and technically, or a good platform, to spearhead the way of using music to show people what you actually can do with it.
I’m sure that there are circumstances that you can start drawing from there. But ultimately the goal is, you log onto Spotify, and you can choose if you want this catalog of Dynamic Music. In terms of the timing, this is the year when we’re actually approaching people. Last year we released our own applications to get a proof of concept and to try out the technology. We have like half a million people using the existing app.
Could Dynamic Playback become the next MP3?
That’s what we think. That’s my mission. Once you have experienced dynamic music it’s kind of boring to turn back to straight, static music.
