Tracks
[Explore everything from technical advances to theoretical applications to sonic visualization.]
Audio and Music

What you hear can help create your reality. That’s why sound is so important to films, concert recordings, AR/VR experiences and more. We’ll explore everything from the technical advances to theoretical applications to sonic visualization, led by a lineup of cutting-edge scientists, professors, audio engineers and artists.

 

Topics Include:

  • Audio and the Brain – creating realistic immersive experiences

  • Sonic Visualization – what sound looks like and why it matters

  • 360-Degree Audio – experience this breakthrough in recording technology

  • Dolby Atmos – object-based audio mixing and compatibility for all playback devices

  • AI Composition – Can a computer ever truly be a composer?

  • The MAGIC of Music – film and gaming scores: recording, editing and more

Track Leaders:

Track Speakers
[Explore everything from technical advances to theoretical applications to sonic visualization.]
Audio and Music

Dave Sluberski

[RIT School of Film and Animation]

Mark F. Bocko

[University of Rochester]

Phil Hilmes

[Amazon Lab126]

Fuat Koro

[Bose]

Dr. Ravish Mehra

[Facebook Reality Labs]

Leslie Ann Jones

[Skywalker Sound]

Michael Heilemann

[University of Rochester]

Hans Gamper

[Microsoft Research]

Edgar Choueiri

[Princeton University]

Mark Watters

[Beal Institute for Film Music and Contemporary Media.]

Gary W. Elko

[mh acoustics]

Justin Mathew

[mh acoustics]
Track Schedule
[Hear inspiring talks in an intimate setting with the best minds in the field.]
MAGIC Spell Studios Facility, Rochester Institute of Technology

Wednesday, June 26

9:30 a.m.
Topic: TBD
Mark Watters ESM - Dir.Beal Institute: Mark Watters is a six-time Emmy Award winning composer, conductor and arranger. He was born in Irving, Texas and majored in Saxophone at the University of Southern California where he was a member of Tau Kappa Epsilon Fraternity. Mark served as the music director of the Centennial Olympics in 1996, and the Salt Lake City Olympics in 2002. Watters has served as guest conductor for ensembles such as The Los Angeles Philharmonic, The London Symphony, The Detroit Symphony, The Dallas Symphony and The Atlanta Symphony. In 2009, he co-conducted a nationwide tour of “Star Wars In Concert” with Dirk Brosse. In August 2012, he conducted the Japanese tour featuring the The Tokyo Symphony. He is a former president of The Society of Composers and Lyricists and a former member of the ATAS Board of Governors.
10:00 a.m.
Music and Story Telling
Leslie Ann Jones - Skywalker Sound: usic has a unique way of supporting action and emotion, whether it is used with visuals or not. Leslie will show several examples of this and discuss how the art of recording music can enhance that further.
11:00 a.m.
The Eigenmike Spherical microphone array for recording 3D spatial audio
Gary Elko - MH Acoustics: With surround sound playback becoming standardized for broadcast, film and AR/VR applications, the demand for surround sound recording solutions is rising. The two main solutions to recording spatial audio are object-based recording that uses multiple discrete microphones to record each source or scenebased recording methods the utilize higher-order Ambisonics (HOA) microphone arrays. The strength of HOA microphone arrays is their flexibility in terms of the number of output beams, beampattern design, and beam-steering direction from a single device. All processing is done electronically without the need for physical movement of the array itself. HOA microphones also allow for post-processing enabling the conversion to different surround sound standards and post editing of the spatial reception of the microphone. Spherical microphone arrays are particularly well suited for surround recordings since the underlying geometric symmetry records the field with no dependence on orientation. Another advantage resulting from the spherical geometry is the ability to realize low computational complexity sound field decomposition and beamforming: a first stage that spatially decomposes the soundfield into spherical harmonics base beampatterns (Eigenbeams or HOA) that are then used as building blocks to a second stage that forms the final desired output beam patterns or rendering processing. This talk will discuss the Eigenmike® spherical microphone array and its suitability for use in recording higher-order Ambisonics for spatial audio capture.
11:30 a.m.
New Dimensions in Sound Reproduction
Michael Heilemann - University of Rochester:Conventional methods for generating high-quality audio are not easily integrated with the thin, lightweight displays used in televisions, smartphones and tablets. As a result, audio quality is often sacrificed to meet the size and form-factor constraints of modern displays. Flat-panel loudspeakers, which radiate sound via the bending motion of a panel, offer a promising means of improving audio quality and satisfying form-factor requirements by employing the display screen itself as a loudspeaker. Though the concept has existed for nearly a century, a number of acoustical challenges have prevented widespread integration into consumer products. New approaches that combine signal-processing with structural control techniques have given flat-panel speakers acoustic properties that are comparable with conventional speakers. These approaches also allow for a number of intriguing applications in spatial audio such as wave field synthesis and audio/visual source pairing, as well as smart speaker technology.
12:00 p.m.
State-of-the-art Binaural Audio Tool
Edgar Choueiri - Princeton University: I will give a hands-on workshop on the latest tools for mixing and rendering binaural 3D audio for speakers and headphones for applications ranging from recording, mixing, and rendering to AR/VR. Using Theoretica’s industry-leading BACCH-dSP 8.0 application (https://www.theoretica.us/bacchdsp/), I will demonstrate the design and use of optimized crosstalk cancellation filters for rendering binaural audio in true 3D over a pair of the regular stereo of monitors, as well as binaural audio rendering through headphones with enhanced head externalization of sound and head tracking. I will also demonstrate 3D mixing and binaural content creation using simple and complex trajectories in 3D space and generic and individualized HRTFs, as well as up-to-fourth-order HOA-to-binaural rendering and 3D sound-field navigation using a 20-speaker full dodecahedron virtual speaker system, individualized HRTFs, and a 4th order 32-capsule HOA mic.

Thursday, June 27

9:30 a.m.
Audio with Alexa
Phil Hilmes - Amazon Lab 126: Smart speakers like Amazon’s Echo with the Alexa voice assistant have delivered a new level of customer convenience for content consumption. And the technology that goes into these devices and resides in the cloud also enable new experiences. Developments in microphone arrays, beamforming, echo cancellation, machine learning, and more are just some of the technologies that provide the ability to better understand what a customer is asking and provide a better listening experience for audio and music. Looking forward, leveraging multiple devices with further deep learning awareness will take this even farther.
10:15 a.m.
Creating the Audio First Approach to AR
Fuat Koro - Bose: Bose AR is a new audio augmented reality technology platform that superimposes sound on top of the real world. The platform works by using a Bose AR-powered app running on an iOS or Android device along with a Bose AR-equipped wearable. (e.g. eyewear, headphones or other form factors) The wearable includes an IMU sensor to detect head-motion and head-direction which, when combined with the GPS on the phone, helps deliver new and unique experiences to the user. Bose AR will enable a whole suite of new and exciting experiences in areas such as travel, learning, gaming, music, fitness, and more. By giving the user the ability to interact with this cloud content based on their physical location and with simple head-gestures, they can keep their phone in their pocket and be more engaged with the real world. Bose encourages 3rd party development of Bose AR apps and experiences by being the a comprehensive source of services and tools needed for Creators (including those with coding experience and those without (artists, businesses, and individuals) to incorporate Bose AR. Bose SDKs and Unity Plugin allow mobile applications to pair with the new generation of Bose wearables, including Bose Frames and Headphones. Using raw sensor data (accelerometer, gyroscope, & compass), developers can now create new experiences for users based on head movement and orientation.
11:00 a.m.
Topic: TBD
Ravish Mehra - Facebook Reality Labs: In this talk, I will introduce the concept of perceptual indistinguishability in the context of VR and AR audio and talk about what is required to generate virtual sounds that are perceptually indistinguishable from reality. I will give a broad overview of the different stages of sound generation and interactions in virtual environments, discuss the challenges in achieving perceptual indistinguishability for VR & AR audio and present recent research developments towards achieving this vision.
11:45 p.m.
Magic Leap
Justin Mathew - Magic Leap: An overall design objective of spatial audio computing systems is to maximize the plausibility of an artificial experience presented to a user via earphones or a head-mounted audio-visual display apparatus. Traditionally, the primary focus has been for designing virtual reality (VR) experiences, with an objective to suspend disbelief of the user by designing the system to fully synthesize the user experience. On the contrary, mixed reality (MR) audio systems aim to display digital objects with respect to user’s environment by allowing user’s to be fully aware of their space. A challenge of MR applications is therefore to prevent dissonance between the perception of computer-generated objects and the listener’s expectations. In this presentation, I will discuss the differences between these two objectives through the evolution of evaluating spatial audio computing technology and experiences.

Experienced Speakers

Hear inspiring talks in an intimate setting with the best minds in the field.

Demonstrators

Catch demos from world-class sound technology leaders and today’s newest audio entrepreneurs.

Track Venue

Eastman School of Music, Hatch Hall