The Faculty of Operation Catapult LXIII
Rose-Hulman Institute of Technology
Terre Haute, Indiana
Sean Cavanaugh North Central high School
Sahana Rajasekar Mallya Aditi International School
John Burton Parkway Christian Academy
June 30, 2009
Music has a major influence on America‟s culture. Within the last few years, a large
amount of the population carries around several days‟ worth of music that they can access and
use at almost any moment. What is the big fuss about? Why is music such a big deal? Music influences people‟s minds and stimulates different emotions. Music is incorporated into many facets of today‟s life for the purpose of enhancing the experience. For example movies incorporate music so that the viewer is set up emotionally and mentally for the scene. In most video games, music is used as a piece of the setting, intending to make the action in the game more exciting. Music is played in the background to set up the mood and even create tension in the player, for the purpose of creating an immersive interactive experience. What if that was backwards? What if the music being played was literally setting up the environment? What if the
games‟ actions were based on the music? From this line of thought sprung the idea to have the actions of our game determined by the music being played. We decided to try to design a computer game that would perform actions based on the different frequencies or tones of the music being played; the music would determine the actions of the computer, thus making a unique game play experience.
Introduction to Python
We had a basic plan and were eager to get started. Unfortunately, before we could start making any sort of game, we had to learn how to control and communicate with the computer so it would do what we wanted it to do. The first 4 days or so, we focused on learning the computer language Python. Python is similar to other high level programming languages (such as Java, C+, or C#) in that it is an Object Oriented Language (OOP), which allows for much better organization in the code. Object Oriented Programming also allows programming work between multiple people to be broken down much more easily than in a non-OOP Language. Ideally, one member of the group will not need to know how another member‟s code actually works, just
how to implement it. This also allows for objects to be related in a hierarchy with classes that are based on the main concepts of a “super class,” but are more specialized in function (called a
subclass). Python has far fewer syntax rules than other programming languages, which can make it much less tedious for someone other than the original creator to actually read and understand. In Python, the things we focused on learning to use were the „Zellegraphics‟ and „PyGame‟ modules. „Zellegraphics‟ provides a simple way to set up graphics creation and display within a window. It is highly limited in function, yet quite easy to understand for people completely unfamiliar with computer programming. „PyGame‟ has more advanced functions than „Zellegraphics‟, allowing for more advanced control of what actually happens within the graphics window, but at the cost of simplicity. „PyGame‟ also has a much greater variety of built
in functions that make it easier to create games, such as music playback, separate sprite and background animation functions (allowing the two to be edited separately from each other), simple rectangle objects with pre-programmed collision detection methods (which would be very difficult for us to program ourselves), and support for a variety of image file-types (which was very useful because it allowed us to use images from outside sources, edit them, and implement them in our game). We also used the „random‟ module, which allowed us to generate pseudo-
random numbers (for making simple decisions), and the „wave‟ module, which actually allowed us to extract data from our music files.
After we had a good grasp of Python we started to brainstorm how exactly we could make music affect our computer game. We came up with several proposals at first when we didn‟t know what exactly would work. We knew that we would have to have the computer read something from the music, we figured the frequencies or the amplitudes could be decoded. Based on this we hoped that the different readings would cause different antagonists to move, attack, or avoid the player. For example if the music played some high notes the enemy character would move to the right, then if the music played some low notes, maybe the character would move left. All of this would have the actual game play based on the music. It would be like the player had to defeat the song, not simply the programming of the computer. Different songs would cause different actions, so theoretically you could take all sorts of different music, implement it into the game and get a new challenge from each new song. So instead of playing different levels of the game you would play different songs with different attributes which could make the game easier or harder.
Reality vs. Idea
However, analyzing sound waves from a computer file is not very simple. We looked into multiple possible methods of breaking down a music file into useable data that could provide some sort of action within our game. We began by doing research on the different music file types to determine which one would allow us the best results. We chose the WAV file type, as it is the only pure one available. All other common sound files are compressed versions of the WAV type, because a 4 minute long song in WAV format takes up approximately 60 MB of data storage; 20 times more space than the same file as an MP3. WAV files are not true reconstructions of sound waves; they merely mimic them to such a degree that there is no distinguishable difference. This is accomplished by taking samples of the sound‟s amplitude at a rate of 44100 samples per second. This allows files to have a maximum frequency of 22050 Hz, which is approximately the highest sound a human can detect. Armed with this information, we looked for a way to find the frequency of our music files so we could implement that into our game. It quickly became apparent that the only way we could get the result we wanted was to use a Fast Fourier Transformation (FFT) to convert data from Time to Frequency. (Functions in the domain of time being the inverse of functions in the domain of frequency) The problem with using an FFT to get frequency data from the music is that we lose any information about the time that frequency occurs at, because it is not possible to convert a wave from time to frequency while still retaining time as a meter for the sound. In the end, we were forced to simply use the wave‟s amplitude data, which meant checking samples that had virtually no significance to us. While our game would technically be driven by sound, for most people, there would be no clear link between the music and the creation of anything within our game. It was only after we had finished the bulk of our project that we came across a method to implement frequency information alongside time based information. We realized that if we extracted a very small sample of the amplitude data, and calculated the frequency for that single part of the music, we could theoretically get an average frequency for that short interval and assume it to be the actual frequency. However, it was too late to try and test this method, because we would not have had the time to actually completely our game. Thus, we stuck with measuring the amplitude from the music files and comparing that to a sort of average value we thought up, to determine whether the game would create an object for the player to interact with (or avoid interaction with).
Once we had an idea how we would implement the music in our game we tried to come up with different concepts for our games. We decided to create a game where the player would have to get from one place to another and avoid all the obstacles along the way. These obstacles would be the things that were generated by the computer based on the background music. We also hoped that different readings taken by the computer from different songs would result in quicker and more numerous obstacles. We thought of many possibilities for our game. One option was to create a paintball game where the player has to keep from being hit by paintballs that would be generated by the computer. In another option, the player is a security guard who is guarding the president of the United States, and he or she must protect the president and take him to safety. Finally, we thought about creating the game where the player is a prisoner escaping from Alcatraz Federal Prison. All of these games where basically based on the same concept as that of the traditional arcade game in which the player‟s avatar is constantly under attack from a
particular direction. The player must dodge and avoid the oncoming enemy fire in order to make it to the goal.
We all agreed that the prisoner escaping prison was the most plausible and started to consider the details. We decided the game would have 3 levels, which would be the 3 main stages of the prisoner‟s escape. First, the prisoner must climb up a wall while he tries to avoid the
shots of the security guards below him.
After he makes it over the wall, he must swim through the San Francisco Bay in an attempt to reach the mainland. In the water he must skillfully avoid killer fish and spear gun fire, all of which come from different directions.
After the player makes it across the bay to the mainland, he must cross a highway full of crazy and reckless drivers in order to get to safety.
As previously mentioned the weapon fire, the fish, the spear gun, and the cars are all engineered to deploy at certain times based on the music‟s amplitude. As the music gets louder
the obstacles are deployed more often, and as the music gets quieter, these obstacles appear with much less frequency. However, the human ear creates some problems that may cause a person to misinterpret a sound as having lesser amplitude because the ear favors sounds within a certain frequency range.
One attribute that is used frequently and often taken for granted is image animation and motion. Image motion isn‟t as easy as it may seem with Python. With each animated character
there must be several images alternating in and out determining which one is being drawn on the screen. In most cases, the images you need aren‟t just available to use, so you have to make them.
This often is a long tedious process of editing to make the image the right size, make sure they match up with the program, and make sure they look good. That is where our extensive use of paint.net came in. Each image we used had to be either modified from an original file to fit our purposes or completely made the images from scratch. Python is picky about what images it accepts. With Python, it is best to use “portable network graphics” (png) for its image files. These types of images are already mapped out, making them perfect for use in Python. Most of our images inconveniently came as either jpeg or gif files, therefore had to be converted to the png file type before we could upload them. For our menu screen, we used Zellegraphics due to its easily implemented functions. Zellegraphics only accepts gif images, thus we also converted some images to the gif format. We made all the backgrounds, as well as the win and lose screens, from scratch using paint.net. For the player animation, we found two pictures of a person, saved them to Python, and set them up to swap between the two images.
This resulted in a simple form of animation, using only two highly repetitive frames. Several other pictures were edited and imported as images to be used as obstacles in the game. In
essence every picture had to take its course through paint.net before it could be implemented into the game.
So is our idea original, or has it been done before? There are several games that have been made in which the game play has been designed based on the music. For example: Guitar Hero and Dance Dance Revolution (DDR) are two popular games in which the game play and music are critically intertwined. But despite the connection between the game play and the music, the music does not control the game play, the programmers merely created game play that
matched the music. The game was created to simply reflect the music. Audio Surf is another computer game that implements music in a very interesting way. Audio Surf is a racing game in which the track turns, bends, and creates walls based on the audio path of the music that is programmed in. It is a very creative game and unlike Guitar Hero and DDR the music controls the game play. Pteranodon is one game we found which was somewhat similar to what we had in mind for our game. It is a top down shooting game in which the enemy aliens are deployed based on the background song‟s pitch and beat. The player moves up the screen as the enemy fires
down on them. The player must avoid the oncoming fire based on the music. This is very similar to our music based game concept, Alcatraz.
As a group, throughout the lengthy course of our project, we had to learn quite a few things in order to complete our rather sizable task. Our project required a basic understanding of the Python programming language for the purpose of setting up the characters and setting, the ability to manipulate images for display, a way to connect the information of audio waves to the Python console, simple interaction between the user and computer through the character‟s reaction to the commands given, and the ability for the computer to recognize its interaction with itself. All of these tasks where harder than they sound. In order to acquire the basic understanding of the Python language, we needed to sit through 4 days of classes in which we learned quite a bit. The hard part was in the middle of learning all this information we were also trying to figure out how we were going to implement the parts of it that we needed, into our computer game. We learned the all the basics of Python that would be necessary and useful for the production of our game. We learned a lot about all of these crucial areas and as a result were able to effectively make a game in which the music controls the action. But just simple knowledge about python wasn‟t all we learned. We also learned some good real life lessons, about teamwork and time management.
Since there were three of us working towards a common goal, teamwork was very important. We learned to split up the work so each member of the group always had something to do. One member would work on the coding, while another member worked on the graphics and images. It was necessary to assign each team member with the job in the area that they were most proficient at. But not every team member could work at the thing they were best at. Often we encountered problems when two members of the group tried to work on the same module at the same time. The conflict in the two versions caused quite a few problems. This helped us understand how important communication between group members was. Sometimes it was necessary to just sit back and let someone else take control of an area that you were more proficient with and simply trust your group member to do a good job with it, after all one person
can‟t do everything. So communication was necessary to keep things at a top-notch level so that
the less skilled people would be able to do the job sufficiently. As a result we soon started filling in the other members of the group every time we started working on something different. This way, everyone was always aware of what progress we were making and hence things were accomplished much faster. Teamwork prevailed!
When we first started the project 2 weeks seemed like more than enough time to complete our game. Unfortunately, before we knew it we were falling behind on our plan. We had to start cutting down on certain ideas and work more quickly and efficiently to achieve our goal. We had to make new plans to fill in the gaps of the old plans and consequently learned quite a bit about organization. There is a time when you can plan too much and overload yourself, but there is also the possibility of requiring too little and getting nothing accomplished. We
realized how important it was to have a good well organized plan and do whatever possible to stay on track.
While music still plays a key role in our game, it is not as significant as we had hoped we could make it. Due to both time constraints and a lack of previous knowledge about the various parts that had to be combined to make our game run successfully, we had to both modify and limit the control that music actually had over what happened in our game. Provided more time, we may have been able to vastly improve our game and implement our ideas more completely. This is very similar to what happens to real game designers, showing just how accurate our experience was (minus the budget problems that occur frequently in the industry). We planned to be able to make it so that the music was easily interchangeable. Though it is possible the replacement of the music is extremely complicated so it wasn‟t possible to make it so that the user could change it themselves. Our idea was very good, but with what resources we had we were extremely limited. But despite these limitations, we were able to come up with a game that is both challenging and ingenious, and one that offers a great game play experience.