Darwin: Okay. Today, I have the pleasure of speaking to Greg LoPiccolo. He has a hell of a resume that he's walking into the studio with. He was involved in the development of Guitar Hero and Rock Band, which I'm sure every single one of us has interacted with at some point. He's currently in the middle of a new venture, and we're going to learn a little bit more about that, but we don't need to hear me talk. We need to hear Greg talk. So with no more ado, let's speak with Greg. Hey, Greg. How's it going?
Greg: Hello, Darwin. Thank you for having me on.
Darwin: Yeah. I really appreciate you taking the time. I imagine your schedule is pretty full.
Greg: It is, but I'm a big fan of your podcast. I've been listening for years and feel privileged to be part of it.
Darwin: That's really great. Well, the privilege is extended to you as well. I'm really proud to say that you're willing to have a chat with us. So for people who might not be familiar or for people who are and just want to hear more, why don't you give a quick rundown of your body of work as well as filling us in a little bit on what you're currently working on?
Greg: So I have a weird arc where I started as a musician - as a rock musician - long time ago, and then fell sideways into the game development industry. So I worked on computer games in the '90s. I did some interactive scoring, and that evolved into creative direction-type roles. Then, as you mentioned, I ended up, for most of my career, at Harmonix Music Systems in Boston, where I led development on a whole host of games. The most famous or prominent of which were Guitar Hero and Rock Band.
In that process, I developed a set of ideas about the intersection of music and video games, and how there are some powerful commonalities that could be... that there's ways to exploit video game design methods, and actually cultural norms, to bring music within reach for a mass audience, which Guitar Hero and Rock Band did. That was the start, I think. Now, I have a startup called ToneStone, which is basically carrying that torch forward. So, basically, my personal obsession is bringing musical expression within reach for millions of people, which I think is doable.
Darwin: Right, now, I got a chance to take a peek, a little peek at what your ToneStone thing is, and actually, it's one of those things that just watching it in action, it actually touches on a lot of things that feel familiar. There's a grid orientation, which now has become such a normal part of music creation. But also, the way that it's laid out, it really speaks to the Guitar Hero and Rock Band thing. It has that look and feel also that people who are Beat Saber players are going to feel familiar with, and it just has this thing where it seems like it's easily addressable without a lot of thinking. I think that that's really smart, and I assume that that's what you're talking about when you say that you're bringing in some of the cultural norms from games into your development.
Greg: Well, so there's a couple of threads that work together. So I think the most important one... Oh, well, so just to pick up from what you said. Yeah. So like professional music software has been moving in a direction to make this more possible, right? It's things like Ableton Live, and frankly, even in Max/MSP. There's this trend toward modularity and abstracting a bunch of the housekeeping away. I can just drag clips into Ableton, and I don't have to worry about of timing or even key, right? I'm actually a lifelong Reason user. I've been using Reason for years. A lot of the same concepts, right?
Darwin: Mm-hmm (affirmative).
Greg: It's like they abstract away a lot of the stuff that you used to have to worry about. So I've had a foot in these two worlds for a long time. I'm a songwriter, and I've been in bands and continue to make music for my own enjoyment and so forth. So I'm reasonably proficient at those tools, but it always struck me that it's like, "Oh, it's way harder to do this." You have to get this discipline to shoulder your way past the technical fol-de-rol to get to your idea, and I'm sure everybody who's used these tools, it's a common complaint. It's like by the time I get my effects configured and so forth, I forgot the idea.
Darwin: Right.
Greg: A lot of us, I think, have these tactics. We're like, "Okay. Well, let's have these pre-programmed templates so that the housekeeping is taken care of ahead of time and so forth." But then, some of the actual musical gestures you end up with, it's like it's that awesome thing where you like to set up a whole arpeggiator with effects and multiple instruments, and you just hold out a key, and music pours out.
Darwin: Right.
Greg: You move the key around, and it's like the song writes itself. So part of the insight for me was like, "Well, this actually, once you get it all set up, it can be really easy to make very expressive music with a few simple inputs, but you have to know all this stuff. Well, what if we cleared all that way and all that stuff away, and packaged it up, and put it in front of non-expert users, and just gave them the fun? Then, over time, they can unpack the nuance and detail, and go as far as they want with it."
Then that smashed into this other deep insight that I just got out of being in games for 30 years, which is games are awesome at this. A lot of the most popular games, they're not easy. They're incredibly challenging, but they do this amazing job of starting you where you are at this low level of understanding and competency, giving you some confidence that you can master it, and then they just walk you up a skill curve over months and years. So the idea was like, "Well, why can't we do that with music? Why can't we package up musical concepts in a way that they're very accessible and rewarding to interact with for new users, but craft the system so it can have a very high ceiling?"
Darwin: Yeah. That's one of the things that I wanted to ask about because quite frankly, there have been a number of attempts to make highly accessible systems for music and music production. The problem is that you end up hitting your head on the ceiling very quickly. I mean, I know over the years, I've always been fascinated by whatever the new, groovy, fun thing is because I'm like, "Hey, sometimes I want music to just be fun." Right? I don't want to be grunting it out every time I want to make a track. Right? I'll get things, and all of a sudden, I'll find out like, "Oh my goodness. They're really locked in a tempo," or "They always have a drum track that's banging away," or there's always something that ends up tripping me up and making it while an easy experience, also an ultimately frustrating one. What are some of the things that you imagine allow for that high ceiling?
Greg: So there's a couple of threads. So one thing that's always been an obsession for me just from a user interface perspective is this tension between loop form factors and linear form factors, where loops are awesome, right, because, oh, you have four bars or two bars, and you can just tweak it as it goes by. It's a very comfortable... at least for me and I think for lots of people. I think that's part of the reason why loop-based music has that primacy is it does solve that near-term problem where you can just tweak your loop.
Darwin: Right.
Greg: Every four bars, oh, change it, change it. But for a lot of the software and hardware that I've encountered, you're in this prison. It's really hard to bust out of the loop and string the loops together into a linear form without losing track of it or having to hold a lot of state in your head or it's, "Oh, you can only see one at a time," kind of thing. So one of the innovations of ToneStone, which I'm actually really proud of, it works great, is you can be in this 3D track, have it march along linearly, and just bang. Hit "Return," and boom, you're in a four-bar loop. Tweak it, bang, you're back to the linear form so that you can move around in a linear form very effortlessly and just punch in on the place you want to work on and punch back out, and you can save them off as blocks, and then reinsert the blocks in other places and so forth. So that's one thing that's gotten a lot of attention because that's always something I was frustrated by.
Darwin: Right. Sure, sure.
Greg: Then, another one is just a thing that games do really well is one of the, I'd say, creative challenges of a full-on pro DAW is that you have so many options that it's paralyzing. Right? You have a hundred effects, and you have 50,000 samples or clips and so forth. So one of the things that games... and there's a whole genre of games that's basically card-based, right? So there's things like Magic: The Gathering, which is not even a computer game. Right? It's a card game, but there's like Hearthstone, and Gwent, and all these other very refined, crafted, very entertaining experiences where part of the design is that they constrain what you go into a session with. Right? So you craft your deck ahead of time. It's like, "Well, it's going to be this balance between offense and defense or whatever," whatever the game mechanics are.
Darwin: Yeah.
Greg: Then, when you're in the session, you're not thinking about... You have what you have, which is also like being in a band and writing a pop song. It's like, "Well, I got three minutes, and I've got a guitar, bass, and drums, and that's what I have. I'm not worried about the string section, necessarily. These are my tools." Imposing that constraint particularly on new users is super valuable. So that's a concept that ToneStone brings to the table, is you have a deck, so it has... I don't know. I think it's 64 clips, and so a bunch of them are just audio loops. So we have 12 drum loops, and 12 bass parts, and 24 instruments, and 12 vocals.
But then, that concept also extends to DSP effects, and harmonic progressions, and mix moves where it's like you have not only the core stuff, but all the modifiers that you can use to basically assemble a song, and you can layer them. I can put down a bass and 2 synths, and then drop a harmonic progression on top of that, and then save that as a chunk, and reuse it, and so forth. But just this idea that you can come into a session with... basically, deciding what's going to be in the session is a different creative process than being in the session. I'm not worried about it. It's like, "This is what I have. Let's do my..." Now, I'm doing my best to make music that I like with this constrained toolset.
Darwin: Wow. Okay. I have a billion things to ask about that, but before we get into it, one of the things that's a commonality in my podcast is I like hearing from people about what their actual background is, how they got to be both the musician that they are as well as the technologist that they are. In your case, you already gave us a hint, but I'd like [to] know a little bit more about what were the things, particularly, what were the things that influenced you both to become and keep working as a musician, but also that really made game development interesting to you, given that you were coming at it from a musician's mind. If you could fill me in on that.
Greg: Sure. It's a very unique and odd story. I was an indifferent musician as a teenager, whatever. I played trombone in the band. I wasn't very good. I wasn't very motivated, got to college, was a German major, but I was at Wesleyan University in Connecticut. They had an Electronic Music Lab, and I took an elective and was immediately transfixed. This was 1983, and they had a couple of ARP 2500s gathering dust, and they had the Apple IIe and eight channels of D-to-A hooked up with this. So you could basically write programs in... I think it was Forth that we used, almost this obscure programming language, and you could send voltages through the DACs to the 2500, and you could make music. You could write these little programs, and you could make music.
From an ideological perspective, they had taken all the keyboards out of the studio. It was like, "No." They weren't about tonal music, and that's all I cared about. Right? I was in to writing pop songs. So I had laboriously tuned up all the oscillators to actually be a normal pitch and key values, and that's what... I wrote just electronic prog rock on this system, but it was an epiphany. It was like, "Oh, you don't have to know how to play. You can tell the machine. You can instruct the machine, and the machine will help you." That was a big deal for me.
So, anyway. That was my start, and then I moved to Boston, and what was happening in Boston at that point was it was bands. It was the indie rock bands through the '80s. So I bought a bass, and learned how to play bass, and started a band. I did that for about 10 years. I was in a band called Tribe, which was a prominent Boston band in late '80s, early '90s. Because of that, some people at a game development studio called Looking Glass Technologies. Some of them were fans of the band, and I met them, and they were starting development on a game called System Shock, which is this seminal... It's one of the first 3D first-person games.
They asked me if I would be willing to write music. I was making very little money with the band. It was a paycheck, so I said, "Sure," and I had a Mac, and I learned... So the way that it was done... At that point, there was a... This was early '90s. So PCs had a sound blaster card, and it's a 2-op FM synthesis done on a little Yamaha chip that was on the Sound Blaster card. So I learned how to program this simple FM, and I wrote these little scores on that chip. I had the, actually, great fortune to fall in with a company at Looking Glass that was just some really smart people doing really interesting work. So I worked with them on building an interactive scoring system, which was one of the first, I think, hardcore attempts to do that well.
So we had this interactive scoring system that would track the parallel state of the player as he moved through his environments, and so we would have a parallel composed, like three different levels of... it was, I think, walking around, peril, and combat, and then at four-bar boundaries, it would transition between these different layers. So we did that for the Sound Blaster, and then we did a Roland Sound Canvas - General MIDI version. It was pretty successful. So, anyway. So that was my start in the games business, but it was really... I wasn't a gamer. It was basically a...
Darwin: Kind of a fluke. Yeah.
Greg: Yeah. It was an opportunity for employment, and then when I got into it, I was like, "Wow, this is really interesting. These people are doing really interesting things just in terms of..." It was my first exposure to virtual environments, and this is the whole landscape of player agency and how you could craft these deep worlds with emergent behaviors and have people enter them and find their own way. So I did that for five years or so, and then I got recruited by Harmonix, which was spinning up in the same town, and they had this vision.
So the two co-founders, Alex Rigopulos and Eran Egozy, had this vision that they expressed to me about like, "We want to use technology to make music accessible to a mass audience because we think that's important." So I was like, "Yes. That sounds awesome." So we got to work on that, and we did some games called Frequency and Amplitude on the PlayStation 2, which we're very proud of them. They didn't sell, but they got us on the map enough to survive long enough to get to Guitar Hero, which then was that was the first time where we cracked the code and came up with something that was actually accessible to a mass audience. So that's my weird story.
Darwin: It's really interesting. So you were involved with Harmonix, and you did these many things. What has got you staying in the game? I mean, in a way, it would probably... I'm curious with people who have a long career that has a look to it. Are you finding that just the culture of the game and the music-game combination is such that it remains fulfilling?
Greg: Well, it has been fulfilling. Right. I mean, I love working on a team. That's been fantastic, which is a... If you're in a band, you're working on a team, but that's been harnessing your abilities to other people who are better than you at the things you're not good at and having that coalesce in a thing that none of you could have done separately. Super satisfying for me, and that remains satisfying.
Darwin: Mm-hmm (affirmative). Yeah.
Greg: But really, the main thing is the original Harmonix mission of getting expression into the hands of a ton of people. I'm super proud of Guitar Hero and Rock Band. I love working on them, and they played their role, but we didn't really ever get there. Right? They were karaoke where you would pretend to play other people's music, and I think that also has a lot to do with why they're not a big deal anymore. I think people moved on, whereas a lot of other game genres, increasingly, they're about creativity like Minecraft, or Roblox, or whatever.
Darwin: Yeah. Right, right.
Greg: I'm still convinced it's possible and that when it happens, it's going to be a big deal. It's going to matter a lot to a lot of people.
Darwin: Do you feel like though that there was any actual musical... I mean, I have this conjecture, and I want you to tell me if I'm full of peas or not, but I have a son who really got into Beat Saber. But also, I saw people who, particularly with Guitar Hero, spent a lot of time with it, and one of the things I noticed is that over time, they actually developed really stunningly good sense of time, or timing, out of it.
Greg: Yep.
Darwin: Which I will tell you as somebody who also spends time... I spend a lot of time around the MPC communities. A lot of times, people, when they first get involved with it, they had to lean heavily into those quantizers because their innate sense of timing was off. I've been surprised the extent to which some of those things actually had a legitimate musical effect on people.
Greg: I mean, we got a ton of shade, right? Particularly for musicians like, "That's not music." To which our response is, "Yeah. Yes, it's not," because all of us working on it were musicians, and we could tell the difference. Right?
Darwin: Right.
Greg: But it did give you a set of relevant skills, and I think the aspect of those projects that I'm most proud of is that we heard... and I still to this day hear from tons of people said, "Yeah, I played Guitar Hero for two years, and then I bought a guitar. Now, I play guitar. It's part of my life." It was an on-ramp, but more to your point, it did a few things. It did develop this sense of rhythmic precision, right, because you had to play, and you had to play a lot of different kind of rhythms and so forth. It gave people a fair amount of domain knowledge about stuff, about like the difference between guitar and bass, and the difference between rhythm and lead.
Darwin: Sure.
Greg: Right? Because you would actually see the notes, and you immerse yourself in the part. People, I think, learned a fair amount about rock arrangement just by playing all these songs.
Darwin: Because it was deconstructed. Right.
Greg: Yeah. It's like if you stop playing, the guitar part would go away, or in Rock Band, then extend the bass. I guess the other thing I would say is for Rock Band Drums, if you get proficient at Rock Band Drums, you can play the drums because it represents kick and snare hat patterns.
Darwin: Right.
Greg: It does independent limb coordination. Your technique is terrible, but you can sit down in front of a kit and play, and play drums if you've mastered Rock Band Drums.
Darwin: Right. Sure. Now, one of the things that was really identifiable about the whole Guitar Hero and Rock Band thing is that they came with bespoke controllers - for good and for evil. Right? For good because it was affordable. You didn't have to go out and drop $500 on a guitar to play Guitar Hero, but it was a toy, and on purpose, but it was still a game controller/toy, but that was really vital to making that work. It wouldn't have worked, and in fact, I know of somebody who was involved in a process where they tried doing something that was very Guitar Hero-like, but they used a standard guitar as the input, and it was impossible for them. They struggled because what the controller did was it abstracted guitar into something that was accessible to people. Right? I'm curious about the work you're now doing with ToneStone, and as you're trying to develop this loop-based system, do you imagine there being a necessity for, again, a bespoke controller or bespoke hardware that really helps facilitate working with that system?
Greg: So the controls were a double-edged sword, right? I mean, I think you're totally right. I think they were necessary. One of the hard lessons we learned with the earlier games, Frequency and Amplitude, which were a little bit more like, in some ways, more like ToneStone is they were abstract, and people had difficulty understanding who they were and what they were supposed to do. Whereas Guitar Hero just solved that problem. It's like, "Oh, I play guitar."
Darwin: A Guitar Hero. "I am a Guitar Hero."
Greg: Right? Yeah. It's right on the box. It's not complicated, and that really helped. What I think is true, what I hope is true is that for a lot of the people we're trying to reach, mostly younger people, that focus has shifted. So now people like deadmau5, and Richie Hawtin, and so forth are... They are these cultural figures that... They're the new guitar heroes. Right?
Darwin: Right.
Greg: Everyone knows. Oh, they use computers, and DJ equipment, and so forth that I think... I think things have evolved in the last 15 years to the point where now, I think you can present software to an audience, particularly if it's dressed up in game clothing - like Beat Saber, right?
Darwin: Mm-hmm (affirmative).
Greg: It's totally abstract, but people aren't confused by it because it's been 20 years that people have been doing this. So I think it's less of a constraint than it was, and then the other thing about the hardware is you got to have the hardware manufactured, and it's expensive, and you got to bring it over from China on boats, and the boats are late. Just actually getting it into people's hands, oh my goodness, what a logistical nightmare. Being free of all that gives you more ability to focus on the software. My 10-year plan of the future, what I hope happens is we have MIDI drivers and people start hooking up Pushes, and the real serious practitioners will develop their own interfaces, and then those will start to take hold.
Darwin: Out of the box, is it going to support any controllers specifically?
Greg: Well, so we have a VR version in development...
Darwin: Okay.
Greg: So that just uses the Quest.
Darwin: Right. Paddles. Yeah.
Greg: Then, mouse and keyboard or touch.
Darwin: Got it.
Greg: So, right now, it's Mac, PC, and Quest in development, but then mobile/tablet is obviously a fast follow. We would do that pretty quickly.
Darwin: Do you imagine going to consoles too?
Greg: Yeah. There's a little bit of a boundary between that UI, but lots of other games have crossed that boundary, and we could as well. So, sure.
Darwin: Interesting, because I know one of the things that made Rock Band in particular really interesting for people is that no matter what system you imagined working on, Rock Band work there. Right?
Greg: Yes.
Darwin: The whole schism between Xbox people and PS people, and all that stuff just didn't interfere with people using that stuff. It's interesting. Now, I want to talk a little bit about something that you described, which I'm fascinated by, but also aligns with something I've been hearing a lot lately. Well, first of all, before I go there, I saw images and I saw some of your promo video stuff of how you have these grids that contain material, and the timeline spins through it and plays what's ever in these slots. Are those audio only, or is there MIDI pieces?
Greg: They're audio only. But the system is actually set up... It's basically set up to work with MIDI as well. We use a DSP, like a software package, we've licensed called Structure, which was built by a woman named Jennifer Hruska, who you should have.
Darwin: I need to, you're right?
Greg: She would be awesome. Anyway, it's like Reactor. There's a GUI, and it's a toolkit of oscillators, and filters, and enhanced audio, and MIDI, and so forth. What we're trying to do is abstract the distinction between audio and MIDI away from the user.
Like the clip, you're going to have a music clip. As a user, you don't know. If it's an audio clip, boom, you bring it across. If it's a MIDI clip, it's attached to a soft synth and maybe a little suite of effects. Whereas a new user, you don't have access to any of that. If you want a different sound, load a different clip.
Darwin: Do you imagine at some point that you'll be able to also route that to external MIDI, or is that just not in the purview of what you're shooting for?
Greg: At some point, we would love this to be a Swiss Army knife that can take inputs from anything and outputs to anything. That's full connectivity. That seems where the action is. We're a tiny team, and so we're basically very focused on just getting...
Darwin: Staying focus. True.
Greg: Getting the core interactions really compelling, and then ideally, we would love to output MIDI and be able to accept all kinds of different controllers and so forth.
Darwin: Got it. Yeah. Well, it's interesting though this idea of focus. I mean, one of the things that you're trying to do is you're also trying to innovate, and it could be so easy to get distracted by lots of details that you forget the innovative things that you're leaning on. One of the things that you talked about was this idea of almost a card deck-style creation system. That really rings a bell. That rings a bell for me, and I'll tell you why.
Lately, I've been interviewing a lot of people that do sound design for films, and one of the things that really surprised me is that... First of all, they all just took me through a real education of the difference between sound design, and effects design, and composer roles, and stuff like that. Right? Talking to them about the sound design process, it reminds me of this idea of building up a deck, and my son was a big Magic:The Gathering fan. I saw him, and I saw the way he strategized about it. He knew he was going to go into a certain kind of game, so he would adjust his deck to be heavy on defense and not needing a lot of, I don't know, whatever food-growing thing there is or whatever.
Greg: Yeah. Exactly. Yeah, yeah.
Darwin: But maybe stronger defense, but maybe not able to probe with an offense very much, but whatever. He would do these kind of things. When I talked to sound designers, especially, it's so often that: they're like, "Okay. We have location sound, and we gather that up, and we cut it up so that it's in usable pieces. We set up a folder system to help us hold all this stuff. But then, we also build up these things that are background sounds that maybe weren't there or maybe background sounds that weren't captured well. So we have to recreate a whole background. So we need all the pieces for that."
Then it's this process of pulling together little bits and pieces out of this collection that they've created that's all oriented towards getting the job done of supporting this film, but it's built on the things that were provided, but also things that they intuited they would need and things that they saw that was missing. It's really fascinating, and that "collectibles" kind of orientation really rang a bell for me then when you started talking about this deck-like selection thing.
So, first of all, congratulations on identifying or zeroing in on that as a process tweak. I like the idea of that a lot, but I'm wondering, to what extent then is this something that users are going to be able to collate/create themselves? I mean, I get that for beginners, there's going to be a lot of preset stuff. How difficult is it going to be for people to build up their own collections, their own card decks of stuff that they capture with a handheld recorder, or they rip off of YouTube videos, or whatever? How hard is that going to be for people to do?
Greg: So the goal is for that to be quite easy, and right now is have a recorder. You can just hook up your external mic or your laptop mic, and you can make your... You can sing or play into ToneStone to make your own clips and use those in the songs that you create. We've done an enormous amount of thinking and planning about how to handle rights and licensing because in the past... and this is one of the things that was a challenge for the Harmonix games. One of the things that made those difficult to turn those into creative platforms is they were... It was all major artist licenses, and you can't... Those artists are not interested in giving you access to their material.
Darwin: Giving away.
Greg: Well, to remix it, right? Even if you buy the license as opposed to you get something on Splice. Well, the whole point is that you're supposed to...
Darwin: Yeah. You're remixing. Yeah.
Greg: So what we envisioned is a scenario where... and this also very much is relevant to this emerging world of music NFTs and Web3, and so forth, which are now giving us the ability, hopefully, to track the providence and the rights of any specific module of music such that ownership can be retained or whatever. Just the fact that it's got a little smart contract attached to it that knows who owns it, or a franctional ownership or whatever. But I guess to cut to the chase, what we envision is a system where different content with different rights profiles can coexist. So you can have a library, and you could filter for Creative Commons or conventional license stuff that you bought in the ToneStone marketplace or NFTs.
Then, as you share with other people, like that it would automatically filters like, "Ooh, you're not allowed to share that because..." or, "Oh, this is all Creative Commons. You can do whatever you want with it" and so forth. But the first problem to solve is the rights problem so that people, particularly new users who don't... they're not ASCAP members or don't have a music attorney at their disposal, can just navigate the system and use it. Basically, use different kinds of contents and be able to do what they want with them, whether it's like push a soundtrack to Roblox, or write a pop song and push it to their SoundCloud account, or mint an NFT. All those things should be possible on the same platform.
Darwin: Got it. You actually took that in a place I wasn't even thinking about mainly because... This is maybe the myopic view of someone who just squirts out their own stuff: I don't even often think about all of the rights issues involved in most modern music creation. But yeah, I can see where that's an enormous issue, especially since after you pull together a card-like collection, you may have a broad variety of licensing issues to deal with if you wanted to release a piece.
Greg: Yes, and right now, it's simple because everything in ToneStone is stuff that we own that we buy out from artists that we negotiated with or stuff we just created in-house, and so you can do whatever you want with it. To answer your question more directly, yes, that's absolutely the plan is like - I load a deck, I sing some vocals, I print those, I post it to Discord, somebody else pulls it down, and remixes it, and so forth.
Darwin: Got it. Yeah. The thing I was wondering about though, and I will just say this from a standpoint of working with other systems. Primarily, going all the way back to when I first started playing around with a lot of looping using ACID, but then with Ableton and with some of the other software out there to actually, if you do your own recording, to actually get it to be a useful loop is actually a bit of a technical challenge, especially for someone who might not be used to doing a lot of audio editing. Right?
So not only do you have to find a way to actually properly cut it to appropriate length, but then if it's going to have to be warped, you have to figure out... I mean, if I take the Ableton model, right? If you're going to warp it, you either have to find the right warping algorithm and choose how to tweak that, or if you're going to allow for location-based warping, then you have to pick the right spots so that you get the timing right. It can be really quite involved, and I just think of something like all that's involved in Ableton and just having the tools necessary to make a viable clip. It's a hell of a challenge. How do you deal with that when your goal is to make it so people can feel free to just open a recorder, pop something in, and have a fun result?
Greg: So we see it in layers, and frankly, we're taking our cue from a lot of the other game-driven metaverses like Fortnite Creative and Roblox where you have these hierarchies of... You've got a bunch of 12-year-olds who are just playing. Right?
Darwin: Right.
Greg: Then, you've got the 16-year-olds who actually know enough JavaScript to construct their own modules with a toolset, and that's very much the way we think of it is like as a non-musician... Our biggest allegiance is to the non-musician who wants to express themselves. So that's getting most of the focus, right, is I come in, I know nothing, and I can have some fun in a half hour and start to learn. But over time, very much our ambition is this is a user-generated, content-driven system where either people are building more sophisticated stuff to sell in the marketplace or just like Minecraft because they're enthusiasts, they're into it, and they're making this higher quality content and making it available to other users. We have a recording module, and there are things... I think there's ways of simplifying that. Pitch correction is the obvious thing, right? I mean, obviously, that gets a lot of shade deservedly or not, but it is this great...
Darwin: But it's also the real world. It's what reality tells us is part of production right now.
Greg: Well, totally, and if you had these phone apps, like Voisey, where it's some compression, and some EQ, and some effects, and some pitch correction, and anybody can sing into it and be happy with the results. That's very much the vibe we're looking for. We're looking to lower the barriers so that just normal people can get some confidence in their own ideas and abilities, and then explore that as far as they want to take it.
Darwin: Yeah. Now, you talked about the different layers and stuff, and one of the things that I just tripped across in that Verve article that described the ToneStone thing is that the engine is written in JavaScript, which first of all, seems unimaginable, and second of all, seems like it's ultimately hackable. Is that the goal, or is that just... To what extent is that JavaScript engine available for someone who really wants to get multiple layers deep?
Greg: So since that article was written, we've shifted platforms. We are now in Unity. So, basically, we're re-concentering the entire thing in Unity, which has a number of benefits for us. But the goal is the same. The idea is that we would expose this layer. It doesn't exist yet, but the idea is that there would be this layer that would be infinitely hackable. Right?
Darwin: Mm-hmm (affirmative).
Greg: So, right now, the idea is the clip, "Oh, it's four bars of drums," but in the future, it would be amazing if that was like, "No, it's a musical behavior." It was a smart arpeggiator or a beat slap of the actual...
Darwin: Or a generative sound creator or whatever. Right. Yeah.
Greg: Exactly. Right, where you just drag that onto the grid. You already understand. It's time-bound, and track-bound, and so forth, but it can have all these behaviors that then you can touch. I mean, I keep my eye on the Max. I'm not an active practitioner, but Max/MSP is like... I think it's amazing, and watching people generate just ideas in that space; when we grow up, we'd like to be Max for dummies in a way. Right? It's like people could access those kinds of ideas without all the technical overhead necessary.
Darwin: Sure, sure. Yeah, I hear you. That sounds really interesting. Well, unfortunately, our time is up, but before we go, for people who are interested in learning more about this system, where would you like them to look?
Greg: You can go to tonestone.com and download... It's a technical proof of concept. It's a glorified prototype. It runs on Mac and PC. I really encourage people to go check it out and see what they think. It's not finished, but there's a fair amount there to check out to get a sense of our ideas, and look us up on Discord. We also have actually a Discord integration I'm really proud of. I think it's really cool. If you link your Discord account to your ToneStone account, you can take your multi-track song that you built, push it to Discord, it will render an MP3, and then if you go on the Discord server to listen to it and if you like it, you'll get the remixable file back into your client.
Darwin: Oh, wow. That's really cool.
Greg: Yeah. We have this little community of users who are sharing ideas, and throwing things back and forth, and so forth. It's pretty effortless and worth checking out if you're in to that kind of thing.
Darwin: That's fantastic. Well, Greg, I want to thank you so much for having this discussion. This is really fabulous.
Greg: I appreciate you taking the time, Darwin. As I said before, I'm a fan of the podcast. I've learned a lot from it and really happy to be a part of it.
Darwin: Right on. Okay. With that, we'll say goodbye. Bye now.
Greg: Thanks a lot.
Copyright 2022 by Darwin Grosse. All right reserved.