What were the key points mentioned for those who don't have time to watch?
I took basic notes over a week ago and forgot to share:
Overall approach:
High-level "game-designer" standpoint - design the experience first, and the shape implementation around that.
Results obtained using a "signal flow" approach - understand the mechanics of what makes vehicle move (real or imagined), and how that affects the sound produced.
Do you target realism or "sci-fi"; what about a blend?
Mike prefers to make something "awesome" by offline (traditional media production) methods, then figure out how to implement it in real time. This is primarily for impressing producers, and being sure they get the green light to implement the features they want.
Apparently, there are five basic sensations for most vehicles:
- "What makes it sound like I'm hauling ass"
- "How does sound make me feel like I'm slowing down"
- "I'm changing direction"
- "I'm losing control"
- "Idle physics"
Basic components of the sound of a vehicle:
- Engine - the sound of power generation (given priority)
- Surface sounds / friction behaviour of vehicle
- Contact with surface (and / or air)
- "Violence layer" - giving the "go fast" experience some teeth. (He said "visceral")
Basic physics inputs for audio control:
- Throttle - "energy input" (?!)
- Braking
- Speed
- RPM
- Direction - steering
Main types of "implementation" (the code used to reproduce the sound)
- Loops - standard steady-state recordings processed into loops and played back blended with other loops.
- Granular - one long recording of an "RPM ramp" (up or down), playback consists of tiny "grains" (very short portions of the recording) sampled around the RPM point. Blend acceleration and deceleration ramps, or any other ramps as required.
- Synthetic - "the future"; recordings used for reference only. (No further details given.)
A special consideration is the Internal Combustion Engine, which has the following main sources of sound:
- Induction
- Valvetrain, engine bay (ancillaries)
- Exhaust
Mike's approach is to mix the resulting recordings like a drumkit, because engines are in a way analogous; there are lots of individual sources meant to sound like one "instrument".
(This is a great analogy - the combining of sources to make one cohesive sound is very important.)
Mike found that it was easier to take a modified recording and "dumb it down" than to "amp up" a stock recording; Forza stopped using stock recordings!!
Recording is better done on a "dyno", esp. "dynapack" (rollerless). Good idea to record on the track (as well). Must record close-up and from afar: just like a drum-kit.
(This is related to the colouration / spatialisation / acoustic transfer I'm always going on about).
Mike sometimes recorded cars at partial throttle for "better sound".
(Many of the irritatingly obviously looping sounds in GT are due to part-throttle recordings.)
Another advantage to the dyno is that some designs can "motor" the car, i.e. drive the engine via the rollers at a constant speed, for any throttle opening.
Other sound sources:
- Tyres
- Transmission
- Turbos / Superchargers
- Suspension / Chassis noise
- Propellers
- Flaps / Ailerons
- Rudder
- Wheel linkage (e.g. bogey)
- Suspension sfx
- Braking
Tyres get special consideration:
- Traction! It is a very important channel of player feedback.
- Scrub / Howl -> approaching peak slip
- Screech -> beyond peak slip
- Intensity of screech increases the more a tyre slides
- Load affects pitch: lower load = higher pitch.
Some tricks to try:
- Sustained vs. one-shot; e.g. engine loops plus thruster sound effects as "sweeteners"
- Use unpitched audio for "intensity"; e.g. filtering white noise
- Cycles per second not always pitch; e.g. one shots for train wheel sounds.
- Try Shepard tones for simple systems (constantly rising / falling, psychoacoustic trick). Or "endless gearbox", below.
Fakery! Audio-only physics fakes.
- Using spring mechanics to introduce instability (e.g. gear change "wobbles")
- RPM variance at redline to prevent "flatness" (I think this means rev-limiter modeling)
- Idle and Redline RPM offsets: introduces the concept of "audio rpm" as distinct from the physics rpm - most middleware uses 0 rpm as idle!! If recorded samples don't stretch to redline, offset what you have upwards so that you don't get stretching in the top end! (!!)
- Shift time offsets
- "Endless gearbox" - constantly changing gears, either in a pre-made loop, or via scripted controls. (For arcadey games, to prevent fatigue during full-throttle running).
Mixing concerns
- On vs. off throttle volume (also timbre, e.g. via blending of different samples)
- Camera perspectives: 2D vs 3D driven. Source placement on car and in environment; better to use a well defined 2D mix at times (e.g. interior)?
- Occlusion of external sources for cockpit view.
- Falloffs and Radii - different for replay and racing? Nearby opponents when racing, zoom shots in replays etc.
- Dynamic mixing: how many vehicles do you need to hear? What's more important to hear? Voice count (concurrent sounds being played back) an issue? Mix crowding, losing definition?
Mike's view of the near future:
Hybrid of granular and loops: the hard problems of granular systems need tackling, but Mike is sure that combining both traditional samples with ramps can alleviate the drawbacks of both
(I agree with this in principle, as long as it is actually possible, and have experimented myself - it is a hard problem).
Realistic instability in synthetic systems is hard to achieve, but doable.
(I agree with this as well - I've experimented with this as well, and there are several ways to tackle it - should be interesting).