Lesson 4: Pitch Shifting, Time-Stretching, and Mash-Ups

1. History of the Mash-up

A “Mash-up” can be loosely defined as the amalgamation of two or more song recordings to create an original work. While the term has gained substantial notoriety over the past few years, its history reaches all the way back to the 1950s.

In 1955 two producers had an idea. Bill Buchanan and Dickie Goodman’s idea was to use clips from the hit songs of the day and combine them into a mock newscast called “The Flying Saucer”. It was the first time that anyone had taken pieces of recorded music to create a new, original work. While “The Flying Saucer” doesn’t sound anything like what we’d consider a mash-up today, the idea was there. It was the first example of a “break-in” record, reached #3 on the Billboard sales chart, and had 17 music companies filing lawsuits, eventually settling for a portion of the royalties from the record.

Mashups entered the mainstream for the first time in 1968 when The Beatles released their sound collage “Revolution 9”, which was inspired by the work of the musique concreté artists like Karlheinz Stockhausen. It was released on their 1968 self-titled album, popularly known as “The White Album”. In the sound collage, you can hear 45 distinct sound sources, including recordings of works by Chopin and Sibelius, as well as Lennon and McCartney’s own vocalizations.

When sampling really took off with the rise of Hip-Hop and House music in the early 1980s, with artists like Grandmaster Flash taking the “breaks” from old soul albums and splicing them together, looping them, and reworking them to create new pieces of music, the groundwork for the modern mash-up was put into place. As new digital technology began to enter into the market, the barrier to entry for producing sample-based music was lowered. You no longer needed to learn how to take a blade to magnetic tape to create a sound collage, the MPC or the ASR-10 could do all of the heavy lifting.

In modern times, mash-ups of all kinds and mediums have become common place. DJs will use the technique of mashing together two songs as a transition tool in their sets and artists like Girl Talk will use samples from 21 different songs to create 1 new one. Outside of the musical realm, the proliferation of digital video editing software and the rise of YouTube has led to a significant groundswell of mash-up activity. Viral videos and even mainstream late-night skits increasingly feature pieces of mash-up and remix culture.

2. Rights Implications of the Mash-Up

For most of the history of the recorded music business, the legal implications of mashing-up two master recordings was a non-issue—the technology simply wasn’t there. Up until the advent of sampling machines like the MPC, mashing together two different songs would have been costly to the point of absurdity. You would have to book time in a studio, hire a skilled engineer, and get your hands on the original master tapes in order to create a mash-up. Even now, record companies aren’t too keen on handing out tapes from their vaults, let alone allowing them to be combined with other songs.

Originally, the mashing-up of two different songs flew mostly under the radar. It was well known that samples like the “Amen Break”, a 4 bar drum fill from the 1969 G. C. Coleman song “Amen, Brother”, were making their way into songs, but either the label legal teams chose to ignore it, or more likely, simply didn’t realize it was happening. But as hip-hop grew up, grew a fan base, and started to create a profit, they also started to attract litigation. By the mid 1980’s there was an explosion of cases involved in copyright infringement, including well known hit songs like MC Hammer’s “U Can’t Touch This” (sued by Rick James for sampling “Super Freak”), and Vanilla Ice’s “Ice Ice Baby” (never sued, but later paid Freddie Mercury and David Bowie royalties as co-writers).

There has been interesting and vigorous debate surrounding the rights implications of the mash-up in modern times. Much of the controversy has been around the producer Girl Talk, real name Gregg Gillis, and his label Illegal Art, whose releases feature the use of sampled work prominently. Interestingly, Illegal Art releases their albums under a “for donation” scheme, and claims that they fall under the protection of “fair use” (sect. 107) in the Copyright Act of 1976 which states:

“...the fair use of a copyrighted work, including such use by reproduction in copies or phonorecords or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright.”

Gillis’ argument latches on the idea of his music as a “commentary” on the music it contains. Also, he argues that his music is so unlike the original that they should be considered new pieces of music. This view is generally at odds with the rights holders, but makes an interesting argument. In copyright law there is a 4 step test that is considered by the court for something to be considered fair-use:

  1. the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes;
  2. the nature of the copyrighted work;
  3. the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and
  4. the effect of the use upon the potential market for or value of the copyrighted work.

These 4 categories were outlined in the landmark decision Folsom v. Marsh (1841) in which Justice Joseph Story writing the majority opinion weighed the infringement of a publisher against another’s copyright over the publishing of a certain selection of George Washington’s letter. After over 100 years of use as precedent in following cases, his arguments were officially codified into law by congress in the 1976 Copyright Act.

One important case which seems to give Gillis’ “fair-use” argument creedence was Campbell v. Acuff-Rose Music, involving the sampling of Roy Orbison’s “Oh Pretty Woman” by the 2 Live Crew in their song “Pretty Woman”. The United States Supreme Court found in favor of the 2 Live Crew, affirming that the song was covered under the fair-use clause as it could be considered a parody of the original. Further, and more pertinent to the Gillis argument, Justice Souter, in writing the court’s majority opinion, explained that the fact that it was a commercial release, or a release intended to generate revenue, was not the only mitigating factor to whether or not the work was protected under fair use. He makes the argument that you must take into account all of the criteria that congress has setup to define fair-use. In other words, just because a work is meant to be commercial doesn’t automatically exclude it from fair-use protection, and likewise, just because a work is educational or non-profit doesn’t automatically make it protected under fair-use. In a sense, the court set up a gray area, noting that the decision as to whether or not something can be protected under fair-use “has to work its way through the relevant factors, and be judged case by case, in light of the ends of the copyright law”. While Justice Souter was speaking specifically about parodies, we can assume the court’s reasoning can be applied across the spectrum of criteria congress has laid out in section 107.

An interesting provision included in copyright law is the compulsory mechanical license that pertains to the publishing side, or the actual written song, that allows you to record a cover of a song as long as you pay the original songwriters a royalty. The current rate for each copy of a cover sold is ¢9.1. Most mechanical royalties in the United States are administered by the Harry Fox Agency. Compulsory, or statutory, licenses arose from the common sense argument that you can’t stop someone from covering a song, so you may as well assure that the publisher gets paid. You could argue, as some have, that with the rise of digital technology you can no longer prevent anyone, like Girl Talk for example, from creating a derivative work, and therefore the license should be compulsory. In fact, there is some evidence that the master rights holders, essentially the record labels, are are beginning to bend on their chokehold over the industry. Some major labels have been seriously considering what to do with the stems from their artists songs in order to expose the work to a wider audience through remixes and other derivative works. There are already some business models that exploit this thinking. For example, YouTube will find and tag songs and give the rights holders a choice: Either they can request a takedown of the infringing video, or they can choose to serve an ad on the video and generate revenue.

Since the uproar over rights violations from rights holders, there has been a movement to fundamentally redefine “copyright”. Sometimes called “copyleft” by its proponents, the new way of thinking about copyright seeks to exploit the inherent gray areas in modern derivative works. No organization has done more to forward the thinking about modern copyright than Creative Commons, helmed in part by star legal scholar Lawrence Lessig, whose mission is to, “[develop], [support], and [steward] legal and technical infrastructure that maximizes digital creativity, sharing, and innovation.” At the root of the licenses, the idea is to grant certain rights to those who would use your work to create new works as long as they attribute the original piece to you. Then, there are 5 main variations on that license that you can choose to include. For example, the “ShareAlike” provision allows anyone to “remix, tweak, and build upon your original work even for commercial purposes”. The more restrictive “NoDerivs”, or no derivatives, license allows anyone to share your original work, but not to remix or tweak. Other groups like the Electronic Frontier Foundation are even more extreme in their defense of free culture, actively pursuing litigation on behalf of digital freedom.

Copyright’s future is anyones guess at this point, but what is clear is that common practice continues to be ahead of the law with each new technological advancement. At some point congress will seriously consider updating the 1976 law (putting aside the failed SOPA and PIPA bills of 2012), but until then it’s the responsibility of those who are actually creating today to shape what the future of copyright and culture might look like.

3. Time and Pitch-Shifting

One area in which digital production techniques have far outstripped the techniques of the past is in its ability to manipulate audio. If you take a record, place it on the turntable, and speed it up, the audio will speed up, but the pitch will also go up. Likewise, if you slow a record down, the pitch will also go down. In a modern DAW, however, you can raise the pitch without affecting speed, and you can change the tempo without affecting the pitch.

Time and Pitch Shifting is particularly useful when combining audio sources from different songs in different keys and at different tempos. For example, it would be almost impossible to combine the Lewis song “Hunter”, which is in the key of Bb Minor at 95 BPM with the Linkin Park song “A Light That Never Comes”, which is in G Minor at 115 BPM, without manipulating the audio from one song in some way. You could choose to raise the key of the Linkin Park elements 3 semi-tones (a semi-tone is another name for a half-step, or the smallest unit of tonal measurement in the western system of harmony, more about that later) and lower the tempo 20 BPM, or you could choose to do the inverse to “Hunter”. Things can get really interesting when you start to come up with clever ways to manipulate the different audio sources so that you can change some things, keep others as is, and fit everything together in a musical way. On a related note, when you take two or more songs and weave them together in a way where it’s clear which songs have been put together, you’ve created a mashup of the two songs.

Various DAWs have different ways of dealing with pitch and time. Robust functionality for manipulating pitch and time is built into the Surface Music Kit software.

TERMS: Time-Shifting, Pitch-Shifting, semi-tones

EXERCISE 1: Exploring Pitch Shift and Time Stretch Functions

When importing audio into SMK we can choose to import our sample as either a Stem or a One Shot, and this decision affects how it will play. Stems are a set volume and assigned a BPM and can be time stretched to match any project they are imported into. One Shots have no tempo and their volume is pressure sensitive. This means they will get louder and quieter as you tap it’s pad harder and softer.

Download the ‘Pitch and Time Samples’ folder and place it on the Desktop.

Open Surface Music Kit and create a new project and name it “Pitch and Time”. Go to the Sounds Collection page and tap the ‘Import File’ button beneath the Filter Bar. Navigate through your Surface’s file structure by tapping on folders and the ‘Go Up’ button to navigate to out of the folder currently displayed. Navigate to the Desktop, then into the ‘Pitch and Time Samples’ folder. There will be a folder containing Stems and a folder containing One Shots. Let’s import the Stems first. Tap the stems folder, select ‘Ex Guitar Loop’, and hit ‘Open’ to bring up the file in the Audio Editor.

The Audio Editor is used for both importing and editing audio files and its functionality is the same for both. The only difference is that when importing we are setting the default parameters for the file, while when editing from Sound Collection or Arrangement View we are editing the file only for that project.

The top of the page displays the project name. Beneath that is the waveform of the file and a Position Marker showing where in the file we are listening. Hit the Play/Pause button to listen to the file. The bar above it is the start and end points of SMK playback, so you can shorten the audio into smaller chunks if needed. Try moving the start and end points then playing back. When you’re finished, move the start and end points back to their original positions so the whole file will be played.

On the left side of the screen is the Clip Column. You can also tap the play button there to preview the file.

The center left column sets how SMK deals with this clips Pitch and Tempo. When you open a file to import SMK automatically scans it and attempts to determine if the file is pitched and is tempo specific. Though it is often correct, it is not always, so it is important to know the BPM and key information before starting the process. All the files we will be importing are 93 BPM and in the key of G Minor Pentatonic.

The ‘Harmonize to match project’ checkbox sets the clip as pitched or unpitched. If checked, the clip is pitched and will automatically be pitch shifted when being placed into a project to match the projects key. For all pitched clips, this is the best option. For unpitched clips like drums, it is best to leave this unchecked. Make sure that it is checked for the guitar loop.

The ‘Match project tempo’ box sets whether or not the clip will be time stretched to match a project’s tempo when being imported. Beneath this checkbox is the detected tempo, check to make sure that it is correct. Is the file tempo 93 BPM? If not, tap ‘Fix It’ beneath the tempo to set it manually.

SMK determines whether a clip is a Stem or a One Shot based on its tempo. If a clip has a tempo of zero, then it is treated as a One Shot. If it has a tempo greater than zero, it is considered a stem.

The right side of the Audio Editor is dedicated to how the clip responds to being played. Besides being pressure sensitive clips can be set to be cut off if continuously played (great for stutter fx and stems) and also be set to play only when the pad is being pressed. When this box is checked a clip will play all the way through if its pad is tapped, but, if held down, will only play for the time that it is held. This is useful if you want to only play part of a clip sometimes. This sets how quickly the clip will repeat if its pad is held down. If set to auto, it will repeat every when the clip finishes, if set to a time marking, it will repeat for that amount of time. For Stems, it is best for it to not be pressure sensitive, cut off when overlapped, and played only while pressing. Leave the Repeat rate a auto and import the clip.

Import the two other loops in the Stems folder, making sure that the drums are not harmonized and that both are set to 93 BPM.

Now import the One Shots, making sure that pitched clips are set to be harmonized and that all clips have a tempo of zero. For One Shots, enable the pressure control and play only while pressing options, but turn off the cut off feature.

Once you have imported the files, it’s time to build a kit out of them. First, let’s set the project tempo so that it matches our clips, 93 BPM.

When building a kit from scratch it is important to start with a pitched sample because SMK sets the project key based upon the first sample placed in the kit. Let start with the guitar loop, drag it onto track 2. Next drag the bass loop to track 6 and the drum loop to track 7. This leaves a lot of open space on the pads but is consistent with how other kits are built, making it easier to play and giving us space to fill in later if we desire.

Now lets drag the One Shots in. Place the pitched One Shots on tracks 9-12 and the drum One Shots on tracks 13-16. Now head to the Arrangement View.

SMK has automatically scanned our clips to see if they should be pitch shifted to match each other, but these clips were designed to work together without pitch shifting. We need to check that our tracks haven’t been pitch shifted out of tune by mistake. Tap a track to bring up the project bar, then hit the Edit button. Make sure that its ‘Adjust Pitch’ menu is set to zero tones. If it has been pitch shifted, uncheck the ‘Harmonize to match project’ box and set the pitch adjustment to zero. Repeat this process for all the tracks.

Now we can play our kit! Record a few bars using the whole kit, then loop those bars. Got a funky groove? Good, now lets go find it a lead part. Save your project, then go back to the Sound Collection. Tap the Kits filter in the Filter Bar and select ‘ElectricGrooveKit1’. Preview ‘EG Lead 1 Loop’ to listen to it, then drag it to Track 1. Switch back to the Arrangement View.

Try playing ‘EG Lead 1 Loop’ with the rest of your arrangement. Does it sound in tune? Let’s take a look at how it was changed to match the project. Open it in the Audio Editor. Notice the original tempo is 102 BPM, but it is now 93 BPM. The pitch should be adjusted -5 tones. If it looks good, save the clip and record it into your project.

Now let’s play with the tempo a bit. Pull open the project bar and adjust the tempo to 120 BPM. SMK will take a moment to stretch your clips. How does it sound? Should it be faster, slower? Take some time experimenting with different tempos. (Tip: when sped up a lot, the guitar becomes pretty surfy)

Now you know the basics of pitch shifting and time stretching in SMK! Go back into the Sound Collection and find some more clips to fill out your kit, or import duplicates of your clips and pitch shift them to create interesting melodic and harmonic changes. Happy mashing!

EXERCISE 2: Making a Mash-Up

At first, mashing something together with something else may seem like a daunting task, but in reality it’s more like solving a puzzle than magic. In fact, now that you’re more familiar with the Surface Music Kit, you should have no problem diving into the word of mash-ups.

Before you get started, there are a few questions you’ll have to address. First, you’ll have to choose the songs you want to work with. Second, you’ll have to decide which tempo and key you’ll want to use to homogenize the project. Generally you’ll want to choose one song as the fundamental song and build everything off of that. Finally, you’ll have to make some serious musical decisions like deciding which parts of each song will fit with one another without sounding out of tune and out of rhythm. There’s also the “x” factor of creating a catchy melody and harmony by weaving together parts from disparate songs.

Open up the Surface Music Kit and load up the session “Mash-up Tutorial”.

Take a listen to the pre-loaded material and get a feel for the layout. Pads 1-5 are vocal samples from 5 different songs, pads 6-9 are harmonic content, and pads 11-16 are drum samples. The main vocal, or the one that’s being treated as the “lead” in this case is the chorus male vocal from the Karmin song “Pulses”. Playing a supporting role are vocal samples from Lewis’ “Hunter”, Cyndi Lauper’s “Girls Just Want To Have Fun”, Rick Louie’s “Boulder”, and Linkin Park’s “A Light That Never Comes”.

What you should take away from the main overview is the breadth of material. This is the main challenge of a mash up—how do you take a seemingly arbitrary repertoire of 10 vastly different songs to create something new and musical? Well, you start at the beginning.

Clear the notes in the pre-loaded demo and get started with a drum pattern. Feel free to get creative, you don’t need to copy the demo. In fact, for this exercise it’s better if you try to work with the samples in your own way. In addition, you can change the samples on the pads from the song library.

Note in the demo, there are 2 kick drums with the same pattern and a clap and a snare with the same pattern. This is a useful technique known as layering. Often, one sample won’t have the impact you’re looking for on it’s own. This is especially true with kick drums. It’s a good idea to layer two elements to a kick that you can control independently: one short sample with a more mid-high frequency for the “punch”, and a longer bass frequency for the “depth”.

Next, play in the main vocal sample you’ll be working with on pad 1.

Now you have the main scaffolding done for your mash-up, now it’s time to fill in the holes. Generally, the hardest pieces of a mash-up to slot in are the vocal samples. This is where the audio editor comes in handy.

Specifically, the “adjust pitch” and “use pressure on pads to control volume” controls are very useful. While the Surface Music Kit does a good job automatically adjusting the pitch of a sample, occasionally, to get the vocal samples to work musically, you’ll have to change the pitch. Regarding volume, the easiest way to get good dynamics is with human feel. So, if you want to play in samples with different volumes, turn on the pressure sensitivity.

Play around with some of the vocal samples and when you’re comfortable start recording some ideas in.

One thing to notice about the demo’s arrangement of the vocal samples is the balance between the placement and dynamics of the vocal samples, and the lead vocal. It’s important that no element overpowers the main idea. In the demo, the choice was made to place the vocal samples in between one another, so that there would be a sort of call and response. For example, you can see visually the “Hunter”, “Girls Just Want To Have Fun”, and the “A Light That Never Comes” samples all work off each other. Another choice was made to have the “Boulder” sample as a constant, repeating drone, but at a low volume so that it doesn’t interfere with the main vocal sample.

Finally, once your vocals are meshing well together, it’s time to add some harmonic material. In the demo, there is a string hit from “Pulses”, a synth and bass sample from the SMK remix of Balloon Ride Fantasy’s “Okinawa”, and a guitar hit from the Earth Wind and Fire song “Shining Star”.

Similar to dealing with vocal samples from disparate sources, you’ll have to massage the pitch of the samples to get them to fit in a musical way. Now the balancing act gets more difficult. You have to take into account the arrangement of the vocal samples and find harmonic material that will work with the natural melodies that they create.

When you’re happy with how your mash-up is sounding, save your project and export it into an mp3, and come up with a DJ name—now you’re a mash-up expert.