Email Updates RSS Subscribe
Line

This blog is created and maintained by the technical team at Hook in an effort to preserve and share the insights and experience gained during the research and testing phases of our development process. Often, much of this information is lost or hidden once a project is completed. These articles aim to revisit, expand and/or review the concepts that seem worth exploring further. The site also serves as a platform for releasing tools developed internally to help streamline ad development.

Launch
Line

Hook is a digital production company that develops interactive content for industry leading agencies and their brands. For more information visit www.byhook.com.

Line

Flash 10 Audio Sequencer

Line
Posted on December 4th, 2009 by Chris
Line

Flash 10 saw the introduction of several new API methods that enabled much more control over audio data and playback. Flash 10 now includes the ability to access sound data directly as a ByteArray. This is done via the new event SampleDataEvent. This event is dispatched from a Sound when the player requests new audio data.

This provides lots of features that were previously unavailable in flash. You can access sound data, manipulate it at the byte level and write it back to the audio buffer. You can even create completely synthesized sounds by generating your own ByteArray and writing it back into the audio buffer.

This also enables multiple audio sounds to be precisely synced at the byte level. This is important if you have to ensure that several sounds all start at exactly the same time. So now you can build accurate sequencers and synthesizers in flash.

We are releasing version 1.0 of this application now. It is still in development and not feature complete yet.

Application Overview

Audio Sequencer is a system we are developing to take advantage of Flash 10’s new audio abilities. Audio Sequencer enables you to load in audio files as mp3 or Ogg (using Adobe’s OggVorbis importer ).

Audio Sequencer includes a TrackManager and a Sequencer that work together to allow you to organize SequencerObjects into Tracks. This process is similar to working with audio software like Garage Band. You create tracks, place SequenceObjects in them at specific times, click play and hear the audio.

The system allows for real time track composition. You can add/remove individual SequenceObjects or entire Tracks while the system is playing. The system supports an unlimited number of tracks and Flash 10 is so efficient at ByteArray operations that you can easily run 12+ tracks on most modern computers.

Volume and Pan are supported per Track and effects can be added to the final ByteArray after it is mixed down from the Tracks. We decided to support the effects Interface designed by http://www.anttikupila.com/ and have included some of his basic effects in this package.

Feature Summary – version 1.0

  • Real time track composition – add remove Tracks and SequenceObjects while audio is playing.
  • Audio is synchronized at the byte level.
  • Lots of Tracks
  • Per Track Pan and Volume
  • Audio Effects
  • Works with MP3 and Ogg
  • Includes MP3 and Ogg loaders and a SoundHelper so you can use your own loader.

Background – Introduction to Audio in Flash 10

Here is some additional information from http://www.kaourantin.net/

“Flash Player 10 code named Astro supports a new event on the Sound object: “samplesCallback”. It will be dispatched on regular interval requesting more audio data. In the event callback function you will have to fill a given ByteArray (Sound.samplesCallbackData) with a certain amount of sound data. The amount is variable, from 512 samples to 8192 samples per event. That is something you decide on and is a balance between performance and latency in your application. The less data you provide per event the more overhead is spent in the Flash Player. The more data you provide the longer the latency for your application will be. If you just play continious audio we suggest to use the maximum amount of data per event as the difference in overall performance can be quite large.”

“The sound format is fixed at a sample rate of 44100Hz, 2 channels (stereo) and using 32bit floating point normalized samples. This is currently the highest quality format possible within the Flash Player. We will be targeting a more flexible system in a future Flash Player. It was not possible to offer different samples rates and more or less channels in this version. If you need to resample you can either use pure ActionScript 3 or even Adobe Pixel Bender.”

Here is a simple example from Abobe that shows how sound can be generated via manipulation of the ByteArray.

var mySound:Sound = new Sound();
function sineWaveGenerator(event:SampleDataEvent):void {
    for ( var c:int=0; c<8192; c++ ) {
        event.data.writeFloat(Math.sin((Number(c+event.position)/Math.PI/2))*0.25);
        event.data.writeFloat(Math.sin((Number(c+event.position)/Math.PI/2))*0.25);
    }
}
mySound.addEventListener(Event.SAMPLE_DATA,sineWaveGenerator);
mySound.play();

Getting Started

We have created several examples that show how to set up the system. At a minimum you need an audio source (MP3/Ogg) either embedded or loaded in, then converted into a SoundSample. The loaders do this automatically or you can use SoundHelper and provide it a raw ByteArray.

import com.jac.sequencer.SoundHelper;
import com.jac.sequencer.SoundSample;
import com.jac.sequencer.tracks.SequenceObject;
import com.jac.sequencer.tracks.Sequencer;
import com.jac.sequencer.tracks.SequenceTrack;
import com.jac.sequencer.tracks.TrackManager;
 
[Embed(source='/../assets/ogg/BassDrum.ogg',mimeType='application/octet-stream')]
private static var BassDrum:Class;
 
//TrackManager is used to compile the song.
var trackManager:TrackManager = new TrackManager(122, 8, 44100, 0);
 
//Sequencer uses the TrackManager to play the audio. This is the audio player.
var sequencer = new Sequencer(trackManager);
 
//SoundSample is created using our SoundHelper from an ogg file. SoundSample holds the uncompressed byte data for the audio.
var soundSample:SoundSample = SoundHelper.decodeOgg(new BassDrum());
//SequenceTrack holds an unlimited number of SequenceObject objects.
var sequenceTrack:SequenceTrack = new SequenceTrack(122);
//SequenceObject contains positioning information for when the sound is played. by default, it plays it immediately.
var sequenceObject:SequenceObject = new SequenceObject(soundSample);
 
//add the sound to the track.
sequenceTrack.addObject(sequenceObject);
//add the track to the track manager.
trackManager.addTrack(sequenceTrack);
 
//play the mixed audio.
sequencer.play();

Example Showing the Ability to Add SequenceObjects to a Track

Get Adobe Flash player

Example Showing 4 Tracks and Track Controls

Get Adobe Flash player

Try it Out Yourself

Source, Documentation and Examples can be found here.

!!! Make sure to read the README.txt for how to set up the classes and your FLA so it compiles correctly.

This application is licensed under GPLv3. Some classes are derivative works of other open source projects and are subject to the original licenses of those copyright holders.

References:

http://www.anttikupila.com/

http://www.andre-michelle.com/

http://code.google.com/p/popforge/

Line
13 Responses to “Flash 10 Audio Sequencer”
  1. ssh says:

    Hi There

    Killer library, seriously…excatly what i was looking for. Although one thing i cannot get my head round. I am having massive performance issues. 3 Tracks bring my cpu close to 90% the fourth traks brings everything to a standstill up to the point of the plugin crashing. Has anybody experienced similar issues? can anyone point my in a direction?

  2. Pete Hutch says:

    Hi,

    Great library, thanks! you’ve saved me a load of time and it’s been a pleasure to work with such neat, well documented code.

    I’ve created a neat multi-track sequencer and I’d like to add a left and right level indicator to my UI to show the levels on each track during playback but I’m struggling to see a neat way to do it without adding a whole bunch of code to Sequencer’s sampleDataEventHandler method.

    So far, I’ve considered creating a buffer for each track in the sampleDataEventHandler method with a collection of these being dispatched to the UI (with tracks being id’s somehow). The UI would then determining some way of representing that data over the time period the buffer covers. Looks like a possible solution but doesn’t feel all that elegant so I’m curious to know if anyone may have a neater solution? …Perhaps on that doesn’t involve me butchering such nice code ;-)

    Thanks,
    Pete

  3. Prashanth says:

    Hi,

    I have been playing around with this library and I must say it is pretty awesome. In fact I have added ui and capability to open/save the audio samples designed around this framework. But one thing that is killing me right now is I can’t mix more than 2 tracks. If i mix more than 2 tracks there is huge stutter in the playback. Could you please help me with how to solve this problem. I guess the solution should be to write some optimizations inside sampleDataEvent code of the play event in Sequencer.as. But I cant think of any.

    Regards,
    Prashanth Nethi

  4. FlexiMusic says:

    I can easily understand the programmings and the demo you ve given here…………………………………………………..

  5. [...] very well done Ogg Vorbis decoder from Branden Hall at automatastudios.com. If you check out the sources for our Sequencer, you can see some examples of how to use the Decoder from Automata [...]

  6. Anders says:

    Thank you! This will help me a lot!

    Though I have a strange problem. If I set TrackManager to 193 bpm some of my short sound clips will only generate a loud distorted sound.
    If I set it to 196 bpm (a bit too fast) they all sound good, but the clips get a bit of beat.

  7. satya says:

    i have a small app which captures video through webcam and stores it in the form of a flv file. the task on hand is to figure out the sound within the flv file, the starting point and the duration of each audio part within the video and replace it with custom audio.

    does this engine anyway help

  8. Chris says:

    I don’t know of anything specific. Is it throwing an error you could post or is flash just locking up and crashing?

  9. jane says:

    can you think of a reason the 16th track always crashes the application ?

  10. Hugo says:

    Indeed, Flash 10 opens a whole new world of possibilities regarding dynamic audio.

    I’ve managed to code a basic Dolby Pro Logic II encoder class. Far from perfect, but works none the less.

    http://code.google.com/p/hso-classes-package/source/browse/trunk/sound/DPL2Sound.as

  11. quel baise says:

    I do agree with what you told us, keep updating your blog, that’s nice.

  12. What can I say. This is exactly what I’ve been looking for. Thank you for this and keep up the good work.

  13. [...] sound synthesis stuff, including a rudimentary Tracker which never saw the light of day. Also wish this had been around at the time, but alas. I had already studied some amount of linguistics long [...]


Leave a Reply to Ogg Vorbis Encoder for Flash: Alchemy Series Part 1 | Hook - Labs

Click here to cancel reply.

*

Line
Line
Pony