AUDIO FOR GAMES

Audio is one of the most important aspects of any videogame. It can be the key to a realistic game experience, and it can take center stage, quite literally, with the recent explosion of music-oriented games.

STAR WARS: THE FORCE UNLEASHED

The summer's latest blockbuster may not be coming to a theater near you; instead it may be coming to your home in the form of a videogame.

David Collins, lead audio/voice director of LucasArts (www.lucasarts .com), helped develop a new way of storytelling for Star Wars: The Force Unleashed. "It is a very ambitious project for LucasArts and LucasFilm in terms of what we are trying to do technically and with the story," he explains. "Haden Blackman and George Lucas created a story that takes place between the most recent Star Wars trilogy and the classic trilogy released in the '70s and '80s." This game takes place during the gap and introduces a new character, Darth Vader's secret apprentice. "It's the next chapter of the Star Wars saga," he reports.

On the game's workflow, Collins used an audio team of 15 to 20 people since it required more work than what is needed for a traditional film or TV series due to the many story variations involved. And he starts preparing for the audio early on, during preproduction and production. "Bringing in sound earlier in the process is a necessity in games, and part of it is education," he explains. "Someone who isn't an audio person wouldn't know to ask for and integrate the proper audio tools, like reverbs and mixers, into the code. It is different with software, you do have to be involved early on."

Unlike film, videogames have a seemingly endless amount of audio variations during game play. Some of that is derived from AI (artificial intelligence) from the game's engine, while most if it comes from the audio team. Collins notes that the amount of material needed to create one game far exceeds any film. "We actually record multiple variations of everything we do," he says. "At last count, we have 13,000 audio files, and that doesn't include voice or music — that's just sound effects."

He says this game isn't that big in terms of voice; it only has about 10,000 lines of dialogue. "The biggest [game] we've done is over 20,000 lines of dialogue. The reason for that is you want people to experience something cinematic rather than just talking to a machine. There is nothing worse than talking to someone who is repeating themselves, no matter how wonderful the actor or the script writing is. For stormtroopers, for example, we recorded 2,000 different reactions, which are then triggered by artificial intelligence."

In terms of the massive amount of work that goes into a game, Collins explains, "You watch a movie for roughly two hours, but you can play a game from anywhere between 20 and 40 hours. And re-playability is a huge thing, too. We need multiple variations for everything. Whenever I ask my guys to design something, they never design only one version. It's usually something like five to 10 variations of anything and it's never enough."

Variation and interactivity also applies to the game's music. Music fatigue is a real consideration for the team. "We want to have an interactive music system where if the player is doing very well, a victory flourish comes in," reports Collins. "But if the player is in danger, the music suddenly reacts, so when a big, bad character comes in the music gets really dark. When you defeat that character, there is a victory flourish. We can set a crossfade in the engine to make it all work together and make it feel like the orchestra is reacting to the picture, just like it would in a film score. That way, the player really feels like they are controlling what's going on. And the player expects it to sound like what they would hear from any media like television or film."

The voice recording for Star Wars: The Force Unleashed was done in a unique way, according to Collins. "We have a recording studio here at LucasArts and we set up a shotgun mic on a boom and recorded it just like you would production dialogue. They also set up a motion-caption camera and performed the dialogue in front of the camera. So literally, the dialogue you hear from these cinematics is the actors' actual performance. It's not ADR, it doesn't sound like it's got that proximity effect going on. It sounds like production dialogue. We shot it just like a film, but there were no [film] cameras set up, only the one motion-capture camera."

The well-known music and sound effects from the Star Wars films created certain expectations for the videogame's audio. "With Star Wars, certain things are so iconic that it's really about making sure those things are in place and they are sacred. When Darth Vader breaths, when R2D2 beeps, when Wookiees yell, and the sound of the light saber — all those things absolutely need to be in place with no real embellishment."

But, there are worlds, creatures and force powers that Collins and team created from scratch. "We had to match up anything we made with all these really famous sound effects and make it sound like it was all a cohesive soundscape. It all had to sound like it was Star Wars." And, he says, it's a wonderful challenge, and in some ways actually makes it harder. "You are constantly comparing yourself to this really groundbreaking stuff."

The gaming industry has grown by leaps and bounds since Collins first started eight years ago, and the trend shows no signs of stopping. Collins explains, "It has gone from very small teams to these very big, expensive, ambitious, wonderful-looking and sounding projects. The perception of games as the next generation of media is changing. It's becoming more mainstream. I think the best years are yet to come. Kind of like how the golden age of film was during the '40s and '50s — that is still to come for videogames."

Collins reveals another aspect of this growth: "As many post production schedules for TV and film get squeezed, we've kind of put a stick in the ground showing that we are not just post production. We need to be involved much, much sooner and work sort of along side the production as they complete things and be a part of the team."

He says competition plays a role as well. "People are a lot more willing to do great field recordings and do whatever it takes because everyone realizes how competitive it is. I think the industry is growing and trying to see what it can become. And because of that, audio is so important in selling the reality of games."

AMERICAN IDOL ENCORE

Music- and audio-driven games have carved out a new niche in the market, proving that a game does not need to be an "interactive movie," a la the latest Hollywood blockbuster, to become a mega-hit. Konami's Karaoke Revolution Presents: American Idol Encore and Dance Dance Revolution are perfect examples of how audio is the new centerpiece of gameplay.

Konami Digital Entertainment's (www. konami-digital-entertainment.com) Keith Matejka is associate producer of Karaoke Revolution Presents: American Idol Encore. "Music games are closer to the arcade experience than the 'interactive movie' experience that a story-based game would have," he explains. "In music games, the music takes center stage. Each song is like a level.

He says that music games are about performing and being in the spotlight. "There is something very attractive about this for people," he reports. "It's about feeling important and being bigger than life. Music games help fulfill that fantasy."

The workflow on Karaoke Revolution Presents: American Idol Encore starts once a song has been selected for the game. "After a song has been chosen for Karaoke Revolution, I notify the contracted recording studio of the selection," says Matejka. "Then, I layout the lyrics for both the recording studio and the game developer. This layout includes what version of the song to use, game play information like how the song should be broken into sections for scoring, and where the short version of the song should end. I also distribute an MP3 that includes any required edits. This becomes the song map for all parties working on the project. It's important that all personnel, especially when working on the audio on different continents, have a singular point of reference."

Using famous songs for gameplay involves licensing, rights and other legal issues. So Konami re-records the songs specifically for the game. Re-recording enables them to license only the songwriting and not the mechanical license for the original recording. But, there are limitations, says Matejka. "Some songwriters don't want their songs re-recorded, which can limit our selection. Another thing that can limit what songs are available — but also can help us — is requesting the ability to make minor edits and changes to the lyrics or composition for gameplay purposes and the desired ESRB rating. That allows us to extend song intros to allow players to get a sense of pitch before the vocal entrance or edit/change obscenities in a song's lyrics. What normally would be considered an unusable song can then become a usable song. On the other hand, some songwriters do not allow us to make any changes to their lyrics or composition, which would make a song unavailable."

Another challenge that comes from re-recording a hit song is how the song ends. "Many times, we add endings to the songs," says Matejka. "Many pop songs fade out, but with a fade it's hard to determine when to tell the user to stop singing or dancing. A definitive ending is required. It can be challenging to come up with something that is in the style of the band. We'll often look to live recordings to see how an artist ends a particular song."

The audio content drives not only a music game, but also many types of videogames. "Much of designing videogames is about creating a world or experience for the user to get immersed in," he explains. "Audio is a key factor in creating that experience. Game players have high expectations when it comes to audio, and it's the audio team's responsibility to meet and surpass those expectations. Audio in games is interactive just like the visuals. Music and sound effects change in realtime based on the input from the user. This creates an audio experience that is different for each user every time a game is played."

The music game genre has been around for a long time, and Konami has been a huge player in it since the beginning. "Success stories like Guitar Hero, Rock Band, Karaoke Revolution and Dance Dance Revolution create compelling musical experiences for users," says Matejka. "Often times, they expose players to music that they had not previously heard and allow users to experience that music in a new way."

GAMESHOW

GameShow from EA Sports (www.easports.com) is pioneering in many ways. It is an interactive game played online with multiple users centered around a live host asking trivia questions similar to live radio broadcast.

Roy Harvey, executive producer at EA Sports, says, "Music and audio are growing in a huge way. Sound, music and audio in general are taking a much more important, kind of first class role in the games themselves. Look at something like Rock Band from Electronic Arts. The entire game is built around music and sound at its core. And that's a huge trend. GameShow is completely sound focused. It's a trivia game, but ultimately, it's a radio station meets an interactive videogame."

GameShow's audio process is unique because it requires people on staff at a custom-made audio studio interact in realtime with the players online. "Audio is a central component of this game. We have brought together folks from the terrestrial radio world with the videogame industry. This has never been done before. Truly, pioneering. We've combined folks from Clear Channel and Sirius satellite radio along with traditional radio producers, program directors and live hosts with the folks who write, art direct and design games."

Some videogames are experiencing an evolution in delivery from traditional DVD media to delivering online. A DVD's linear experience is far too limited compared to the possibilities of online. "What's interesting is the generation of games that are played online has evolved even further," says Harvey. "For more and more games, the audio component has become more dynamic and adaptive over time. So, it's not the audio I heard when I bought the game. Now, the audio can change throughout the year.

"For GameShow, we do holiday themed music for Halloween, Thanksgiving, Christmas, Hanukkah, New Year's, etc. It is adaptive for the time of year and the situation," he explains. "We also have that dynamic element for Madden NFL 08 with weather feeds. You are playing at a photorealistic stadium, with 5.1 incredibly well produced sound, professional voice actors like John Madden, and it all sounds incredibly real. Now, we add this physical dimension. If you are playing in a stadium in Pittsburgh and at that time it happens to be raining, we have a feed from weather.com and it's raining in your stadium in the game. Air temperature and everything is done from data online. Because we are tied to the CPU, whether it's a PC or the game consoles, they all have broadband connectivity. That enables a part of you game that is considered dynamic."

Harvey is excited about the future of games and the opportunities of online games. "I can be doing it with my friends locally, I can be doing it globally across the Internet, and I can share those creations with other people. It's not just about competition, but it's about creation. Now, EA can deliver things for a mass audience and make entertainment out of it. And, we are just at the beginning. Between Rock Band, GameShow and The Sims On Stage, we are really changing the definition of interactive entertainment."

BATTLE OF THE BANDS

Some music games go beyond just singing or playing the guitar. Battle of the Bands offers musical combat with "weaponized" instruments. Laddie Ervin and Raymond Herrera, executive producers of 3volution Productions (www.3volutionproductions. com), worked on the music created for THQ's Battle of the Bands. Ervin oversaw the project and produced some of the tracks and recording sessions.

"This entailed hiring and managing the seven other producers on the project, all of whom were off-site, liaising with both our clients, the game's publisher, THQ, and developer Planet Moon Studios," reports Ervin. "We worked with 20-plus artists in 10 cities."

In terms of workflow, "Our task was to take 30 licensed songs and record each of them in five different genres, subdivided into subgenres where applicable, for a total of 150 tracks," he explains. "The game design dictated that the five genre tracks for every song had to be the same tempo, in the same key, have the same start and end points and match each other measure-for-measure from beginning to end.

"The next step was to send the original artist's recording of the song to one of our producers who then created a song breakdown and reference track for everyone else to use when creating their version of the track. The breakdown included the song's tempo, key and measure-to-measure chord changes, as well as the location of any tempo/key changes and anything else of note, along with the song lyrics. The reference track was a tempo and arrangement locked scratch track. These files were checked, and then sent to the other producers so they could begin work."

Each producer then created and submitted a rough mix of their tracks. "These were delivered to the client for feedback and approval," says Ervin. "Upon completion and final approval of all the tracks, the songs went through a final mix and mastering process."

Ervin shares his view on the rise in popularity of music-driven games. "Just as people can experience love, action, adventure and other things vicariously through movies, books and other mediums, they can live out interactive 'fantasies' in videogames, which would be too difficult, dangerous or antisocial to experience otherwise. Until recently home videogame technology was not advanced enough to deliver a music-game experience fun and rewarding enough to be adopted by a large audience. Now that the technology has evolved, more gamers are living out their dreams of musical stardom in their living rooms."

Music licensing, rights and other legal issues impact what songs are available for gameplay. Not surprisingly, not every song from every artist is available. Ervin sheds light on how this impacts not only music games, but aspects of videogames in general: "Players respond more favorably to games that allow them to 'perform' some of their favorite licensed songs in-game. If games like Battle of the Bands featured only original tracks by unknown artists their consumer appeal would be greatly diminished. This is not unique to music games. Licensed content gives the people what they want in almost all gaming genres. Racing games with real licensed cars sell better than their counterparts with generic cars. Sports games with real licensed teams, logos, players and stadiums do better than sports games without them."

Many videogames have made the audio portion of their games a higher priority, expanding their budgets and resources. Music games require the audio production to be the highest priority. "While it is true that some videogames have large music production budgets, these titles are still the exception rather than the rule," says Ervin. "More often than not the music budget for games is small. More and more music is being licensed for games at low-to-no cost from record labels and bands eager to promote their music to gamers. Often, developers and publishers have their own in-house audio guys and they either have very limited budgets, want to do everything themselves, or both. Every year game budgets get bigger for the most part, but this also means the number of titles being produced each year is getting smaller."

Any advanced music game, such as Battle of the Bands, would not have been possible even just a few years ago. "As gaming hardware gets more powerful, software gets more sophisticated and storage mediums get geometrically larger, it becomes possible to create more ambitious music-based games," explains Ervin. "The ability to play multiple individual tracks of recorded music together in a game on a home videogame console is a relatively new development and while, it was possible to do this on systems as early as the Sega Saturn and Sony PlayStation, it wasn't until the creation of custom guitar-shaped peripherals and the Nintendo Wii remote, that these games went mainstream."

He reports that on the audio production side, the advent of PC- and Mac-based digital audio workstations and audio processing have made it easier for producers and artists to create high-quality recordings more quickly and affordably. "The deeper penetration of affordable broadband Internet connections allows for much faster delivery of tracks for collaboration, submission, production and replication," concludes Ervin.

Some of the tools they used to create the music for Battle of the Bands included digital workstations such as Pro Tools, Cakewalk Sonar, Logic, Digital Performer, Sound Forge and Bias Peak Pro. A wide variety of VST plug-ins from UAD, Waves, Arturia and SSL, as well as musical instruments from, Korg, Moog, Fender, Gibson, Ibanez and Gretsch.