Microphone Access in Silverlight App: LIVE

Just a heads up that the technique described earlier (using Flash and Javascript for microphone support in a Silverlight app) is live now in the Tutor.com Classroom!

You'll need to create an account and connect to a tutor to see it in action (click the 'Connect to a Tutor' button in the classroom to get started)--

Microphone access in Silverlight via Flash and JavaScript

Silverlight is unquestionably a powerful tool for .NET application developers; using almost of all the same WPF skills,.NET developers can create rich internet applications (RIA) right out of the gate. However, Silverlight has no microphone/webcam support, and if you need this functionality in a cross-browser RIA, the common thinking is cross out Silverlight and pencil in Flash or Java.

While it's true that you'll need to dip into Flash or Java for microphone/webcam support (at least until Silverlight 3, and possibly not even then), there's no reason you can't take advantage of Silverlight as your base application host and selectively use Flash as necessary for Silverlight's missing features. Then, using a JavaScript bridge, you can get these browser technologies communicating with each other fairly painlessly. This integrated solution is what we're considering at Tutor.com when we add voice communication support to the Tutor.com Classroom, and this post will detail how to use Flash for microphone support inside a Silverlight application, where the button controlling the microphone on/off switch is in the Silverlight application.

Flash: Microphone management
First, we'll create a simple Flash movie (I'm using ActionScript 2) that manages and outputs microphone input. Thankfully, there's really very little code you need to write to do this; just note that your Flash movie must be at least 250x150 to allow room for the microphone Allow/Decline dialog, and if you don't want the movie to be visible, you need to set its background color to transparent.

As an aside, note that there's similarly very little code you need to write to connect to a Flash Media Server (FMS) for real-time communication, although the example here omits the FMS part. As another aside, note that the code for Flash webcam support is almost identical, except instead of a Microphone object you're working with a Camera object.

OK, so in the first frame of your Flash movie, write a function to access the microphone and monitor microphone activity:

var mic:Microphone = null;

function toggleVoice(isOn:Boolean)
{
  //setup the mic
  if (mic == null)
  {  
    //call Microphone.get() to access the microphone and prompt user with Allow/Decline dialog
    mic = Microphone.get();

    //Microphone.get() will return null if user declined access
    if (mic == null)
      return;

    //setup onActivity handler to get notification of mic activity
    mic.onActivity = function(active:Boolean)
    {   
      //call out to JavaScript bridge via ExternalInterface
      flash.external.ExternalInterface.call("IsTalking", active);
    };

    //create movie clip and attach mic to clip so we can hear output
    this.createEmptyMovieClip("sound_mc", this.getNextHighestDepth());
    sound_mc.attachAudio(active_mic);
  }

  //set the microphone gain as per the isOn input variable
  mic.setGain(isOn ? 50 : 0);
}

Below the function, add a callback handler that makes the function callable via JavaScript:

  flash.external.ExternalInterface.addCallback("ToggleVoice", this, toggleVoice);

Html: Application host page
When you add a Silverlight application to your solution in Visual Studio, the boilerplate html makes your application full-screen in the browser window (as in the Tutor.com Classroom); this is exactly what we ant. Next, we need to add our Flash html object tag. Silverlight is a bit picky about where you add this html tag since you need to preserve this 100% height style, so slot the Flash object tag immediately before the div tag, and style it with "position:absolute" so that it doesn't interfere in the html layout:

  <form id="form1" runat="server" style="height:100%;">
    <asp:ScriptManager ID="ScriptManager1" runat="server" />
            
    <object 
      id="voice" 
      style="position:absolute;"
      classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000"
      codebase="http://fpdownload.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=8,0,0,0" 
      width="250" 
      height="150" 
      align="middle"
      ...
    />

    <div style="height:100%;">
      <asp:Silverlight     
        ID="Xaml1" 
        runat="server"                 
        ...
      />                                
    </div>
  </form>

Javascript: Flash/Silverlight bridge
Next, we'll create the JavaScript function that our Silverlight app will call, which will in turn call into our Flash movie to set the Microphone input status. Since our Flash movie registered the ToggleVoice() callback function using ExternalInterface, Flash added a JavaScript entry point to function when the Flash movie loaded:

<script language="javascript" type="text/javascript">
function toggleVoice(on) 
{
    var flashVoiceObject = thisMovie("voice");
    if (!flashVoiceObject)
        return;

    flashVoiceObject.ToggleVoice(on);
}

function thisMovie(movieName) {
    if (navigator.appName.indexOf("Microsoft") != -1) {
        return document.getElementById(movieName);
    }
    else {
        return document[movieName];
    }
}
</script>

We can also very simply call from JavaScript into Silverlight, and we'll do this to alert our Silverlight application when Flash notifies us of microphone activity:

function IsTalking(active) 
{
    //access the Silverlight application
    var control = document.getElementById("Xaml1");

    control.Content.Page.ToggleIsTalking(active);
}

Silverlight – Microphone toggle button
Finally, in Page.xaml, add a toggle button with Checked and Unchecked handlers, and an indicator that we'll show when the microphone is active:

<ToggleButton Content="Toggle Microphone Input" Checked="ToggleVoice_CheckedToggled" Unchecked="ToggleVoice_CheckedToggled" />

<TextBlock Text="You are talking..." Visibility="Collapsed" x:Name="IsTalkingIndicator" />

Then in Page.xaml.cs, implement the handler and call into the JavaScript function:

private void ToggleVoice_CheckedToggle(object sender, RoutedEventArgs e)
{
    //call js layer to set microphone state
    ScriptObject voiceToggleScriptObject = (ScriptObject)HtmlPage.Window.GetProperty("toggleVoice");

    ToggleButton tb = sender as ToggleButton;
    voiceToggleScriptObject.InvokeSelf(tb.IsChecked.Value);
}

Lastly, implement the ToggleIsTalking() function that we'll call from JavaScript when the Flash movie responds to the microphone's onActivity() event. You just need to decorate this function with the [ScriptableMember] attribute so that Silverlight registers the function with the JavaScript runtime:

[ScriptableMember]
public void ToggleVoiceOn(bool On)
{
    IsTalkingIndicator.Visibility = (On) ? Visibility.Visible : Visibility.Collapsed;
}

And that's it! You now have a relatively straightforward Flash <=>JavaScript <=> Silverlight solution that affords you the best of all worlds.

beware of flash

sorry for the rant, but a couple of things came up today that wasted hours of time for me and are worth filing away in your flash development mental compartment.

adobe releases new flash builds frequently:

• there were 4 releases of flash 8 between august 2005 and september 2006 (plus 2 more “security enhancement” releases since then). release notes are here: http://www.adobe.com/support/documentation/en/flashplayer/8/releasenotes.html
• there have been 8 releases of flash 9 from august 2006 to december 2007. on the 3 computers i use frequently, i have 3 different versions of flash 9! release notes are here: http://www.adobe.com/support/documentation/en/flashplayer/9/releasenotes.html

you can see what version of flash you have installed here: http://kb.adobe.com/selfservice/viewContent.do?externalId=tn_15507

so what, you say? well, i mention this because i encountered a nasty bug with specific versions of flash 9 and ExternalCallback today. ExternalCallback is the interface that makes it easy to call from javascript into a flash movie. the bug is this: in IE, when a flash movie is inside a <form> tag, a cached flash movie is not properly re-loaded when it is pulled from the cache, and ExternalCallback just stops working. the bug was present or introduced at some point up to 9.0.44.0, but fixed by 9.0.115.0 (so i didn’t see it at first).

the fix (hack, really), is to append a unique querystring value to the .swf filename, so that IE never pulls the movie from the cache. this was acceptable in my case (an 8kb flash movie), but obviously wouldn’t have worked if the movie was larger.

another bug that was present in earlier versions of flash 9 caused IE to crash when a flash object was unloaded as the browser quit. the fix for that is just as nasty; get a load of this:

function cleanUpCounterObjects()
{
  var objects = document.getElementsByTagName("object");

  for (var i=0; i < objects.length; i++)
  {
    for (var x in objects[i])
    {
      if (typeof objects[i][x] == 'function')
        objects[i][x] = null;
    }
  }
}

beforeUnloadCounterObjects = function()
{
  //fix for memory bug with flash 9 and IE
  __flash_unloadHandler = function(){};
  __flash_savedUnloadHandler = function(){};

  window.attachEvent( "onunload", cleanUpCounterObjects );
}

if (window.attachEvent)
  window.attachEvent( "onbeforeunload", beforeUnloadCounterObjects);

i get it that cross-browser, cross-platform is hard to do, but doing flash development right is almost impossible with adobe’s versioning strategy. seems to me that side-by-side installations would solve all of these problems; developers could say “flash 9.0.45.0 only”, and when users update to 9.0.115.0, they actually just install it side-by-side with 9.0.45.0, so apps released for 9.0.45.0 continue to run in 9.0.45.0! or am i missing something?

again, just make sure you do as much testing as possible in different configurations. you can download older flash versions from adobe here:

http://kb.adobe.com/selfservice/viewContent.do?externalId=tn_14266&sliceId=1

and the flash uninstaller is here:

http://kb.adobe.com/selfservice/viewContent.do?externalId=tn_14157

good luck…

click to run an ActiveX control

here’s the lowdown from Microsoft on the ‘click to run an ActiveX control’ dialog (affects plugins like flash, shockwave, and quicktime) and a fix for it (you need to create the object with javascript and not use an object tag):

http://msdn.microsoft.com/library/default.asp?url=/workshop/author/dhtml/overview/activating_activex.asp

more info:

http://www.cssbeauty.com/skillshare/comments.php?DiscussionID=482