Adaptive music for increasing emotional impact of games (link to a paper I found)

I was recently looking for ways to express emotion in games through a dynamic soundtrack. I’m a long way from actually using this myself, but am doing research into many aspects of game creation with the intention of increasing player immersion. During my stumbling through the expanse of the interwebs, I stumbled across this…

The link to the original page that I found is:

This outlines the basic principles and summarises the paper that describes the experimental application. I found it to be very interesting reading, but could not find a free copy of the original paper that it refers to.

However… I have now tracked down a free download of a *.pdf of the original paper. It’s from a google search so is a bit of a long link…
http://www.google.co.uk/url?sa=t&source=web&cd=1&ved=0CBgQFjAA&url=http%3A%2F%2Fciteseerx.ist.psu.edu%2Fviewdoc%2Fdownload%3Fdoi%3D10.1.1.114.9497%26rep%3Drep1%26type%3Dpdf&rct=j&q=The%20soundtrack%20of%20your%20mind%3A%20mind%20music%20-%20adaptive%20audio%20for%20game%20characters&ei=WMxTTvfmGYKv8gPs0tnYBQ&usg=AFQjCNG3ogEQ2QkVH5YkG2d_RBB_Q-2dhg&cad=rja

I found this to be very interesting so thought I’d share. Game creators who want to develop a more dynamic situation-specific soundtrack for increased player immersion might find it interesting.

To quote a quote from the paper:

I find it surprising that the authors of the paper make no reference to the “Music Director” in Valve’s games Left4Dead and Left4Dead2. These games make use of leitmotifs for the spawning of certain ‘boss zombies’ and individually tailor the music to the player’s local environment so each of the four cooperative players experience a unique soundtrack that is relevant to their personal location and experiences.

Got any technical Tips concerning dynamic Mysic? I have started playing around with some Audaspace Functions (Thoughts like »first Ambient and subtle Melody, during Dialogue the Bass comes in (at Transition), as then a Battle starts, the Bass changes to a more staccatic Pattern, accompanied by some sinister Synth-Drums«, such Stuff), but I struggle a bit with how to get the Music to change at the Transitions or – I am honsest – to change at all in the first Place.^^
Would be great to know and I’d be very thankful, because I myself am very fond of makin’ Musik and therefore I see it as a very important Part for immersive Gameplay, but also to live out my artistic Crazyness. : D

Left 4 dead games are a example to me too(there are several other games which do this), to get that effect you would for example, check the number of enemies cornering the player, if there is a wall right behind, and if so(and the enemies are many), the game would add a new layer of instruments to the default music; thought some games just use a single music file with different parts instead of many music ‘layers’ playing. I still have not seen yet a game truly “invent” a unique music depending of the situation; but my range of games is quite narrow, so I have not seen that much.

I will do that in my game, but the music for that purpose must be quite specific; having a default theme and several variations of it; hard to find one music like that just searching, probably it will have to be made specifically for this purpose you spoke of.

I find the music of resident evil 3 to be quite nice too in the mood department, but it isn’t truly dynamic.

That is rather expectable, because even if you are a Freestyle-Jazz-Keyboarder improvising new Music, you will always navigate through Patterns that you have already learned and put them and their Elements together to new Variations. Now if you would want a Game Engine to »improvise«, so that it would make »new« Music, that Engine would ~need~ to navigate you already known, already learned Patterns and musical Rules, just as the musical Human Being would do that. Without that, you would only get random Notes like in an Oldschool-Horrorr-Movie. (That is, of Course, a neat possibility, and similar to the Monkey’s Word-Game towards the End of The Neverending Story, there would from Time to time some true Melodies be formed with just a random Note-Sequence – maybe even whole Patterns, maybe whole Tracks, whole Songs, whole Albums, …)

EDIT: How about scripting some Musical Rules: »After D play A, G or C for a Time of either 1/4, 1/8 or 2/4, after the Amount of Notes given in the Variable ‘timesignature’ have filled the required Space, save all played Notes and Loop them to make a Track Layer until the Situation is over!«
One Possibility to create »new Music« all the Time (and I know very well, it would NOT be truly new Music at all…) would be to mix the Techniques of »Layers« and »Patterns«. (One even could make a Games Music as one whole giant Track and depending on the Gameplay Situation adequate Patterns plus Layers would be chosen.)

@C.A.Ligari: As I said in my original post, I’m a long way from implementing any of this myself. I’ve not even started playing with any code at this point. Concerning a higher level interpretation of what should happen though, I have had some thoughts…

The biggest problem with this technique would be composing musical components that work well together. This includes different time signatures, transitions, instrument levels, etc. Rather than have the entire composition devised at runtime, I envision a suite of musical components that are developed before runtime that some kind of runtime engine could choose from based on parameters that are decided during the game in much the same way as felipearts suggested. There would then be a “music director” that mixes the components together dynamically. There could be various riffs, beats, fills, etc. that are available to the director and some kind of algorithm (serious hand-waving here and very little substance to my thoughts…) that mixes the predefined components into a coherent musical experience for the player.

I currently have no idea how I’d go about implementing such a feature either generally or within bge specifically. The technology that we are discussing here is very rarely applied in even AAA titles with a staff of 100s of people. However, the fact that such an obscure thread topic has resulted in a couple of well-thought-out responses in such a short time is encouraging.

(
I have to say here that I have serious doubts about the ability of AAA game developers to move games forwards. The huge financial investment in new gaming approaches and technologies discourages diversification from the accepted strategies.

Without doubt the visual and ai technologies are being pushed forwards by these companies, but more eyecandy is facing diminishing returns. Gamers are people and people have diverse desires and needs. Eyecandy can only go so far in fulfilling these desires.

Indy developers are coming up with some excellent concepts that are challenging the status quo. The recent Notgames Fest is testament to this growing movement within the interactive experience community.
)

Further thoughts about both the relevance and possible implementation strategies of dynamic music would be very welcome from the community. :slight_smile:

That actually is the easiest part. ; )
First of all: Scales. Of course, one musical Piece must/should consist of one specific Tone Scale. But that is no Problem, you even can make whole Albums around one Scale. TOOL, for Example, stick very strictly to A Minor and Variations of it.
Time Signatures are not as much of a Problem, because when you make a Piece of Music, you will, just like with Scales, stick with the one you already use, most probably. So, if you have begun makin’ a Track in 4/4 Patterns, you will definitely do the whole Piece this way, or 7/4 or 11/4. Of course you can vary with your Time Signatures within one Piece, but even then they do not necessarily need to match. (Yes, TOOL again…) And then there are Time Signatures that work very well together: You can play a 4/4 Pattern three Times accompanied by a 3/4 Pattern four Times and if the Pieces fit, it might sound great – eitherway they would fit together concerning the Time.
Transitions might be the only Thing that really are, indeed, hard to get done in a Game with dynamic Music. Akira Yamaoka once mourned about the limited Possibilities for proper Transitions that the Video Game Industry has, though I think of a good Example in particular: Gothic.

Nôôô, not the Genre, I mean that old German Roleplaying Game. The third Game had bad Transitions, because a real Orchestra was recorded – not so easy to cut, unfortunately. But the first and second Game, they actually LIVED quite a lot not only from their Music but also from the good Transitions the Music had and still has.
An Example:

This the »Old Camp« in Gothic 1. At 4:20, when the Player goes through the Gate, the Music get’s reduced to a Variation leading to the new Music. At 6:58 (when the Player goes back through the Gate), the Music easily switches back to the first Theme, even without a hearable Interludium/Intermezzo. I sometimes recognized in the Gothic Games that a Transition from one Piece to another would be done using rather empty Spaces with sometimes just a Cymbal crashing at the Patterns first not until new Piece would start.

EDIT: But to add more useful Notes as well: It is highly important which Kind of Tool one would use to compose the Musik! I don’t trust the common Tools where one would push block-shaped Samples around anymore, not enough Control! ModPlug Tracker is great for makin’ Video Game Musik, and not only Video Game Musik. There you also can export your Composition automatically in either Channel Mode (each Channel into one separate File) or Instrument Mode (each Instrument in one separate File), which is, of Course, very, very useful for using the already mentioned »Layer Technique«! : D

EDIT: I have just seen (http://openmpt.org/release_notes/), the newest Version of ModPlug Tracker seems to have great Functions useful for composing Video Game Musik! MUST… DOWNLOAD… INSTANTLY!!! Ô▄Ô

My main goal for posting this thread was to point out the importance of sound in games to those who were neglecting this part of the emotional construction and to develop my own understanding of the tools available to game developers.

You seem to be quite ahead of me in this. Those really are some excellent links and observations that you’ve posted. Thanks a lot for that. I’ll have to spend some time going through all of this, but I’m sure it will be time well spent. I’ll particularly look into the “Gothic” game. I’ve heard of it before, but never experienced it.

Edit:
Music is one major part of my knowledge that is not so good. I am intending to start the Open University course Technology of Music and hope that this will broaden my general knowledge of the subject. Also, I have been checking out the music of “Tool” on youtube and they are excellent. Thanks for bringing them to my attention. :slight_smile:

Edit2:
I actually spend most of my free time listening to music, but where I struggle is in the technical comprehension of it. My tastes tend towards electronica from electro-rock through trance to dub, although I appreciate a lot of classic rock, folk and some classical too. Tool appear to be electronic rock from what I’ve heard so far. :smiley: