In the mid-90s, chat was one of the best products available on the web. Raise your hand if you were young and thought how cool it would be to develop your own chat application. One of their best features was their ability to capture microphone audio and/or video from a webcam, and send it over the Internet. To implement these features, developers have relied on plugins like Flash and Silverlight for a long time. However, Flash and Silverlight can be a problem if you don’t have the proper permissions or you’re not tech-savvy. Today, such plugins aren’t required anymore thanks to theWebRTC project and its related APIs. This article will introduce the getUserMedia API, one of the APIs derived from the WebRTC project.
What’s the getUserMedia API
The getUserMedia API provides access to multimedia streams (video, audio, or both) from local devices. There are several use cases for this API. The first one is obviously real-time communication, but we can also employ it to record tutorials or lessons for online courses. Another interesting use case is the surveillance of your home or workplace. On its own, this API is only capable of acquiring audio and video, not sending the data or storing it in a file. To have a complete working chat, for example, we need to send data across the Internet. This can be done using the RTCPeerConnection API. To store the data we can use the MediaStreamRecorder API.
The getUserMedia API is amazing for both developers and users. Developers can now access audio and video sources with a single function call, while users don’t need to install additional software. From the user perspective, this also means a decrease in the time to start using the feature, and also an increased use of the software by non tech-savvy people.
Although the getUserMedia API has been around for a while now, as of December, 30th 2013 it’s still a W3C Working Draft. So, the specifications may be susceptible to several changes. The API exposes only one method,
getUserMedia(), that belongs to the
window.navigator object. The method accepts as its parameters an object of constraints, a success callback, and a failure callback. The
constraintsparameter is an object having either one or both the properties
video. The value of these properties is a Boolean, where
true means request the stream (audio or video), and
falsedoes not request the stream. So, to request both audio and video, pass the following object.
Alternatively, the value can be a
Constraints object. This type of object allows us to have more control over the requested stream. In fact, we can choose to retrieve a video source at high resolution, for example 1280×720, or a low one, for example 320×180. Each
Constraints object contains two properties,
mandatory is an object that specifies
the set of Constraints that the UA must satisfy or else call the errorCallback.
optional, is an array of objects that specifies
the set of Constraints that the UA should try to satisfy but may ignore if they cannot be satisfied.
Let’s say that we want audio and video of the user, where the video must be at least at a high resolution and have a framerate of 30. In addition, if available, we want the video at a framerate of 60. To perform this task, we have to pass the following object.
You can find more information on the properties available in the specifications.
The other two arguments to
getUserMedia() are simply two callbacks invoked on success or failure, respectively. On success, the retrieved stream(s) are passed to the callback. The error callback is passed a
MediaError object containing information on the error that occurred.
The support for the getUserMedia API is decent on desktop but quite poor on mobile. Besides, the majority of the browsers that support it, still have the the vendor prefixed version. Currently, the desktop browsers that implement the API are Chrome 21+ (-webkit prefix), Firefox 17+ (-moz prefix), and Opera 12+ (unsupported from version 15 to 17) with some issues in older versions. On mobile browsers, only Chrome 21+ (-webkit prefix), and Opera 12+ (-webkit prefix from version 16) support the API. Also note that if a page containing the instructions to work with this API is opened through the
file:// protocol in Chrome, it won’t work.
The case of Opera is really interesting and deserves a note. This browser implemented the API but for an unknown (to me) reason, after the switch to the Blink rendering engine in version 15, they didn’t support it anymore. Finally, the API support was restored in version 18. As if it was not enough, Opera 18 is the first version to support the audio stream too.
That said, we can ignore the compatibility issues thanks to a shim called getUserMedia.js. The latter will test the browser and if the API isn’t implemented, it fallbacks to Flash.
In this section I’ll show you a basic demo so that you can see how the getUserMedia API works and concretely see its parameters. The goal of this demo is to create a “mirror”, in the sense that everything captured from the webcam and the microphone will be streamed via the screen and the audio speakers. We’ll ask the user for permission to access both multimedia streams, and then output them using the HTML5
video element. The markup is pretty simple. In addition to the
videoelement, we have two buttons: one to start execution and one to stop it.
Regarding the scripting part, we first test for browser support. If the API isn’t supported, we display the message “API not supported”, and disable the two buttons. If the browser supports the getUserMedia API, we attach a listener to the
click event of the buttons. If the “Play demo” button is clicked, we test if we’re dealing with an old version of Opera because of the issues described in the previous section. Then, we request the audio and video data from the user’s device. If the request is successful, we stream the data using the
video element; otherwise, we show the error that occurred on the console. The “Stop demo” button causes the video to be paused and the streams to be stopped.
A live demo of the code below is available here.
This article has introduced you to the WebRTC project, one of most exciting web projects in recent years. In particular, this article discussed the getUserMedia API. The possibility of creating a real-time communication system using the browser only and very few lines of code is terrific and opens a lot of new opportunities.
As we’ve seen, the getUserMedia API is simple yet very flexible. It exposes just one method, but its first parameter,
constraints, allows us to require the audio and video streams that better fit our application’s needs. The compatibility among browsers isn’t very wide, but it’s increasing, and this is good news! To better understand the concepts in this article, don’t forget to play with the provided demo. As a final note, I strongly encourage you to try to change the code to perform some task, for example applying a CSS filter to change how the video stream is shown.