Thursday, January 22, 2009

Microphone access in Silverlight via Flash and JavaScript

Silverlight is unquestionably a powerful tool for .NET application developers; using almost of all the same WPF skills,.NET developers can create rich internet applications (RIA) right out of the gate. However, Silverlight has no microphone/webcam support, and if you need this functionality in a cross-browser RIA, the common thinking is cross out Silverlight and pencil in Flash or Java.

While it's true that you'll need to dip into Flash or Java for microphone/webcam support (at least until Silverlight 3, and possibly not even then), there's no reason you can't take advantage of Silverlight as your base application host and selectively use Flash as necessary for Silverlight's missing features. Then, using a JavaScript bridge, you can get these browser technologies communicating with each other fairly painlessly. This integrated solution is what we're considering at Tutor.com when we add voice communication support to the Tutor.com Classroom, and this post will detail how to use Flash for microphone support inside a Silverlight application, where the button controlling the microphone on/off switch is in the Silverlight application.

Flash: Microphone management
First, we'll create a simple Flash movie (I'm using ActionScript 2) that manages and outputs microphone input. Thankfully, there's really very little code you need to write to do this; just note that your Flash movie must be at least 250x150 to allow room for the microphone Allow/Decline dialog, and if you don't want the movie to be visible, you need to set its background color to transparent.

As an aside, note that there's similarly very little code you need to write to connect to a Flash Media Server (FMS) for real-time communication, although the example here omits the FMS part. As another aside, note that the code for Flash webcam support is almost identical, except instead of a Microphone object you're working with a Camera object.

OK, so in the first frame of your Flash movie, write a function to access the microphone and monitor microphone activity:

var mic:Microphone = null;

function toggleVoice(isOn:Boolean)
{
//setup the mic
if (mic == null)
{
//call Microphone.get() to access the microphone and prompt user with Allow/Decline dialog
mic = Microphone.get();

//Microphone.get() will return null if user declined access
if (mic == null)
return;

//setup onActivity handler to get notification of mic activity
mic.onActivity = function(active:Boolean)
{
//call out to JavaScript bridge via ExternalInterface
flash.external.ExternalInterface.call("IsTalking", active);
};

//create movie clip and attach mic to clip so we can hear output
this.createEmptyMovieClip("sound_mc", this.getNextHighestDepth());
sound_mc.attachAudio(active_mic);
}

//set the microphone gain as per the isOn input variable
mic.setGain(isOn ? 50 : 0);
}

Below the function, add a callback handler that makes the function callable via JavaScript:

flash.external.ExternalInterface.addCallback("ToggleVoice", this, toggleVoice);

Html: Application host page
When you add a Silverlight application to your solution in Visual Studio, the boilerplate html makes your application full-screen in the browser window (as in the Tutor.com Classroom); this is exactly what we ant. Next, we need to add our Flash html object tag. Silverlight is a bit picky about where you add this html tag since you need to preserve this 100% height style, so slot the Flash object tag immediately before the div tag, and style it with "position:absolute" so that it doesn't interfere in the html layout:

<form id="form1" runat="server" style="height:100%;">
<asp:ScriptManager ID="ScriptManager1" runat="server" />

<object
id="voice"
style="position:absolute;"
classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000"
codebase="http://fpdownload.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=8,0,0,0"
width="250"
height="150"
align="middle"
...
/>

<div style="height:100%;">
<asp:Silverlight
ID="Xaml1"
runat="server"
...
/>
</div>
</form>

Javascript: Flash/Silverlight bridge
Next, we'll create the JavaScript function that our Silverlight app will call, which will in turn call into our Flash movie to set the Microphone input status. Since our Flash movie registered the ToggleVoice() callback function using ExternalInterface, Flash added a JavaScript entry point to function when the Flash movie loaded:

<script language="javascript" type="text/javascript">
function toggleVoice(on)
{
var flashVoiceObject = thisMovie("voice");
if (!flashVoiceObject)
return;

flashVoiceObject.ToggleVoice(on);
}

function thisMovie(movieName) {
if (navigator.appName.indexOf("Microsoft") != -1) {
return document.getElementById(movieName);
}
else {
return document[movieName];
}
}
</script>

We can also very simply call from JavaScript into Silverlight, and we'll do this to alert our Silverlight application when Flash notifies us of microphone activity:

function IsTalking(active)
{
//access the Silverlight application
var control = document.getElementById("Xaml1");

control.Content.Page.ToggleIsTalking(active);
}

Silverlight – Microphone toggle button
Finally, in Page.xaml, add a toggle button with Checked and Unchecked handlers, and an indicator that we'll show when the microphone is active:

<ToggleButton Content="Toggle Microphone Input" Checked="ToggleVoice_CheckedToggled" Unchecked="ToggleVoice_CheckedToggled" />

<TextBlock Text="You are talking..." Visibility="Collapsed" x:Name="IsTalkingIndicator" />

Then in Page.xaml.cs, implement the handler and call into the JavaScript function:

private void ToggleVoice_CheckedToggle(object sender, RoutedEventArgs e)
{
//call js layer to set microphone state
ScriptObject voiceToggleScriptObject = (ScriptObject)HtmlPage.Window.GetProperty("toggleVoice");

ToggleButton tb = sender as ToggleButton;
voiceToggleScriptObject.InvokeSelf(tb.IsChecked.Value);
}

Lastly, implement the ToggleIsTalking() function that we'll call from JavaScript when the Flash movie responds to the microphone's onActivity() event. You just need to decorate this function with the [ScriptableMember] attribute so that Silverlight registers the function with the JavaScript runtime:

[ScriptableMember]
public void ToggleVoiceOn(bool On)
{
IsTalkingIndicator.Visibility = (On) ? Visibility.Visible : Visibility.Collapsed;
}

And that's it! You now have a relatively straightforward Flash <=>JavaScript <=> Silverlight solution that affords you the best of all worlds.

Tuesday, December 9, 2008

Tutor.com Classroom How-To #11: Build a Silverlight ToolbarButton user control

These How-To tips are taken from The Tutor.com Classroom: Architecture and techniques using Silverlight, WPF, and the Microsoft .NET Framework.

You can check out the Silverlight Tutor.com Classroom in "practice" mode, although the real experience is with a live tutor on the other side!

One of the controls missing out-of-the gate from Silverlight is the Toolbar. There is no magic to the WPF Toolbar (it’s just a control container that auto-docks itself and auto-styles its children), but for free you get the flat button look-and-feel that users expect at the top of an application.

In Silverlight, we have to manually create this look-and-feel. Since our application’s toolbar consists solely of buttons (both normal and toggle buttons), and since the buttons in our toolbar have the “icon followed by a text description” format, we can abstract this to build the ToolbarButton user control.

Begin by creating a new user control. Define the styles for the Hover and Pressed states in your user control’s Xaml resources:

<UserControl.Resources>
<LinearGradientBrush x:Key="HoverBrush" StartPoint="0,0" EndPoint="0,1">
<LinearGradientBrush.GradientStops>
<GradientStop Color="White" />
<GradientStop Color="#F8D28F" Offset=".3" />
</LinearGradientBrush.GradientStops>
</LinearGradientBrush>

<LinearGradientBrush x:Key="PressedBrush" StartPoint="0,0" EndPoint="0,1">
<LinearGradientBrush.GradientStops>
<GradientStop Color="#F8D28F"/>
<GradientStop Color="White" Offset=".3" />
</LinearGradientBrush.GradientStops>
</LinearGradientBrush>
</UserControl.Resources>

Then, use a Grid to hold two Border objects; one contains the image and the text label, and the other is a half-opaque, white filled Border that we use when the button is disabled.

<Grid>
<Border x:Name="MainBorder" BorderThickness="1" CornerRadius="1" MouseEnter="Border_MouseEnter" MouseLeave="Border_MouseLeave" MouseLeftButtonUp="Border_MouseLeftButtonUp" Cursor="Hand" Margin="1">
<StackPanel Orientation="Horizontal" Margin="5,1,5,1">
<Image x:Name="BorderImage" Width="16" />
<TextBlock x:Name="BorderTextBlock" Style="{StaticResource blackText}" Margin="5" />
</StackPanel>
</Border>
<Border Background="#B2FFFFFF" x:Name="DisabledBackground" BorderThickness="1" CornerRadius="2" BorderBrush="#B2FFFFFF" Visibility="Collapsed" />
</Grid>

Next, wire up the IsEnabled, ToolTip, ImageSource, and Caption properties, and implement the hover and click functionality. Note that these properties are backed by dependency properties on the objects we defined in our Xaml.

public partial class ToolbarButton : UserControl
{
bool m_IsEnabled = true;

public new bool IsEnabled
{
set
{
MainBorder.Background = null;
MainBorder.BorderBrush = null;
DisabledBackground.Visibility = (value) ? Visibility.Collapsed : Visibility.Visible;
}
}

public string ToolTip
{
get { return ToolTipService.GetToolTip(MainBorder).ToString();}
set { ToolTipService.SetToolTip(MainBorder, value);}
}

public ImageSource ImageSource
{
get { return BorderImage.Source;}
set { BorderImage.Source = value;}
}

public string Caption
{
get { return BorderTextBlock.Text;}
set { BorderTextBlock.Text = value;}
}

public bool IsToggleButton { get; set; }
public bool IsToggled { get; set; }

public event MouseButtonEventHandler Clicked;

public ToolbarButton()
{
InitializeComponent();
}

private void Border_MouseEnter(object sender, MouseEventArgs e)
{
//if we're a toggle button and we're on...
if (this.IsToggleButton && this.IsToggled)
return;

Border b = sender as Border;

b.Background = Resources["HoverBrush"] as Brush;
b.BorderBrush = new SolidColorBrush(ColorHelper.FromHexString("FFCA8103"));
}

private void Border_MouseLeave(object sender, MouseEventArgs e)
{
//if we're a toggle button and we're on...
if (this.IsToggleButton && this.IsToggled)
return;

Border b = sender as Border;

b.Background = null;
b.BorderBrush = null;
}

private void Border_MouseLeftButtonUp(object sender, MouseButtonEventArgs e)
{
if (!m_IsEnabled)
return;

if (this.IsToggleButton)
{
this.IsToggled = !this.IsToggled;

//set state
Border b = sender as Border;
b.Background = (this.IsToggled) ? Resources["PressedBrush"] as Brush : Resources["HoverBrush"] as Brush;
}

if (this.Clicked != null)
this.Clicked(sender, e);
}
}

Finally, in your container Xaml, use the ToolbarButton class inside a StackPanel with Orientation set to Horizontal:

<StackPanel Orientation="Horizontal">
<local:ToolbarButton ImageSource="/Resources/board.png" Caption="Add Whiteboard" Clicked="AddWhiteboard_Click" Margin="1" />
...
</StackPanel>

Friday, December 5, 2008

default DateTime parameters to an old date, never NULL

you want to use:

CREATE PROCEDURE …
@StartDate = ‘1/1/2000’

not

CREATE PROCEDURE …
@StartDate = NULL

the reason for this is that if you use NULL, you have to put WHERE (@StartDate IS NULL OR DateValue > @StartDate) in your where clause, which screws your query optimization. if you default to an old date, you can just put WHERE (DateValue > @StartDate) so the optimizer will be happy—

beware API overloads

on business object methods like Consumer.UpdateConsumer() we have several overloads that let you update only a few specific Consumer properties. for example, there’s one overload that takes a CustomerStatusId, another that takes a WhereHeard and a CustomerStatusId, etc. these overloads all call a common, private updateConsumer() function that takes care of the database update, and they pass in the object’s properties for the fields you are not updating, so that the original values are preserved.

however, this means that if you call an overload that takes multiple parameters, any value you pass in as a parameter will be used for the database update. so passing in “null” or empty string will update the database with “null” or empty string for that field, which may not be your intention--

Thursday, December 4, 2008

beware maxBufferSize, maxReceivedMessageSize, and maxStringCotentLength when adding service reference in VS2008

when you add a service reference in VS2008, you end up with a entry in your applications app.config file. this element contains binding elements for each service you add.

BEWARE: the default value for three critical attributes, maxBufferSize, maxReceivedMessageSize, and maxStringCotentLength, is set considerably low (64KB). we have hit this problem more than once now with deployed software, and unfortunately the only way to correct it is to adjust these values in app.config and re-distribute the software. so if you think there is even the slightest chance that your service will be returning more than 64KB in data, be sure to adjust these values after you add the service reference.

Friday, November 14, 2008

Tutor.com Classroom How-To #10: manually detecting and installing the Silverlight plug-in

These How-To tips are taken from The Tutor.com Classroom: Architecture and techniques using Silverlight, WPF, and the Microsoft .NET Framework.

You can check out the Silverlight Tutor.com Classroom in "practice" mode, although the real experience is with a live tutor on the other side!

Use the isInstalled() function provided by Silverlight.js:


<script language="javascript" type="text/javascript">

Silverlight.InstallAndCreateSilverlight = function(version)
{
var retryTimeout = 1000; //the interval at which instantiation is attempted(ms)
if (Silverlight.isInstalled(version))
{
...
}
else if (!Silverlight.isInstalled(version))
{
TimeoutDelegate = function()
{
Silverlight.InstallAndCreateSilverlight(version);
}

setTimeout(TimeoutDelegate, retryTimeout);
}
}

Silverlight.InstallAndCreateSilverlight('2.0.30923.0');

</script>

Thursday, November 13, 2008

Tutor.com Classroom How-To #9: creating a Path object from a Stroke

These How-To tips are taken from The Tutor.com Classroom: Architecture and techniques using Silverlight, WPF, and the Microsoft .NET Framework.

You can check out the Silverlight Tutor.com Classroom in "practice" mode, although the real experience is with a live tutor on the other side!

Simply transfer Stroke stylus points to a PolyLineSegment on a Path:


static Path getPathFromStroke(Stroke str)
{
//make a path from this stroke
Path path = new Path();
PathGeometry pg = new PathGeometry();

//start our figure from the first stylus point in the stroke
PathFigure fg = new PathFigure();
fg.Segments = new PathSegmentCollection();
fg.StartPoint = new Point(str.StylusPoints[0].X, str.StylusPoints[0].Y);

PolyLineSegment seg = new PolyLineSegment();

//add each additional stylus point to the line segment
for (int x = 1; x < str.StylusPoints.Count; x++)
StylusPoint sp = str.StylusPoints[x];

seg.Points.Add(new Point(sp.X, sp.Y));

fg.Segments.Add(seg);

pg.Figures = new PathFigureCollection();
pg.Figures.Add(fg);

path.Data = pg;

return path;
}