A Heartfelt Call to Action for Web Developers!

10 min read

Deviation Actions

fartprincess's avatar
By
Published:
8.1K Views

If you are a developer, this should upset you!


This is a long-standing ticket for Chrome to implement one of the most important functionalities of the MediaStream API in WebRTC. I first heard about WebRTC last fall during the Austin Javascript/NodeJS MeetUp. Although the person giving the talk was largely uneducated as a developer and her awareness of what she was doing was uninspired, I saw a lot of potential.

WebRTC is partly a web API (It has a backend API in C++ geared at browser developers, but this is not my concern). It is meant to allow developers to offer a way of capturing and recording streamed media input, such as audio and video. Ah, the 21st century feels so modern and cool, right? Nope.

If you are familiar with YouTube (and most of us are; it's a great way to spend an afternoon watching dogs and cats act like morons), you may be aware of the fact that YouTube recently introduced a UI for recording video directly through the browser.

Screen Shot 2013-07-03 at 1.51.21 PM by fartprincess

Nice! Or is it?

Screen Shot 2013-07-03 at 1.53.02 PM by fartprincess

YouTube, the most renowned site for video hosting on the Internet, is using Flash for video recording on their desktop version of their site. To handle the recording session on mobile apps, YouTube has developed a separate recording app from their namesake app, dubbed YouTube Capture. Although we can argue relentlessly about the significance of native apps versus web apps for mobile use, it's clearly a sad state of affairs when the motivation for developing a native app for iOS and Android is solely based on limitations. The limitation here is that, despite the fact that Google now owns YouTube and it would be in their best interest to make progress on it, the most pivotal part of the WebRTC web API, MediaStream is not implemented  at all. 

Many developers I know are not familiar with this API, but you should be! It's officially part of the stable Chrome and Firefox build and you no longer have to enable it via flags. It's officially enabled by default in the standard distribution of both these browsers. For some reason it was supported in 2 versions of Opera and no longer is. It is also supported on the Blackberry mobile browser (I know, I know, haha).

So let me show you first-hand what a pain in the ass it is to use WebRTC for video recording right now.

First, every implementation of the getUserMedia API has its own separate vendor prefix. For the sake of simplifying code, it makes sense that we figure out which one we're working on and override the standard.

Screen Shot 2013-07-03 at 2.09.29 PM by fartprincess
If you've ever worked with CSS3, you are likely familiar with vendor-prefixes. Every browser has its own forked implementation of an attribute. This means if your UI designer has a big hard-on for fancy animations, transitions, and the like, you will end up with an incredibly cluttered bulk of CSS, with 4-5 forked prefixed attributes per style. That's pretty obnoxious! I mean, do you really care if someone using Opera sees that pretty box-shadow? Or do you think maybe they don't deserve to see it because they're using such a horrible browser? 

But unlike CSS3, this API can drive the core functionality of a web app and you can't afford to risk that!

To initialize the getUserMedia API, you'll need to tell it whether you want to record audio and/or video by passing it a constraints object. 

Screen Shot 2013-07-03 at 2.18.55 PM by fartprincess
Hey, that looks pretty good, right?  Turn on my video recording and my audio recording and I'm all set! The videoRecorder variable is a reference to a <video> tag in the DOM. Once you have invoked getUserMedia, it will return a stream to you and you can attach it to that <video> tag so that you can see your pretty face while you record! But unfortunately, the above code comes with a nasty caveat: the API does not support recording video and audio simultaneously! Well... crap. But for me, I don't care too much because the web app I am developing, a web app for learning sign language, is conveniently only concerned with video streaming, not audio. 

But see, the fun doesn't stop there.

Screen Shot 2013-07-03 at 2.24.57 PM by fartprincess
This stream is the same one created by getUserMedia on initialization. Say our web app has a button underneath the video player to allow us to start recording. And you decide to bind the click event on that button to this startRecording method. First of all, your user would be confused because the above method does nothing to notify them through the UI that they are recording, but that's hardly a problem when you discover that the above does not work at all anyway. Javascript will throw an error that record does not exist on the stream object. This, as we've discovered, is because no one has implemented the record API!

How do you get around this clearly broken thing, as a developer then?

Here's the sad reality, the best solution at the moment and why I am stressing so dearly that developers should have a vested interest in the ticket linked at the top of this article.

Screen Shot 2013-07-03 at 2.33.52 PM by fartprincess
That's right. I'm creating a <canvas> in my Javascript. At no point will this canvas be inserted into the DOM. It will be used solely for the purpose of data exchange and manipulation of the stream attached to our <video>. This is realistically what our record method will look like:

Screen Shot 2013-07-03 at 2.39.53 PM by fartprincess

When the user clicks the record button, we establish an array of frames. This array will store still frames from the video stream. We will recursively push frames to that array until the the stop button is pressed and cancelRequestAnimationFrame is called. The frame will be drawn onto the canvas and the canvas will be used to export a base64 PNG image, which will be stored in the frames array. I don't know about you, but I think that's kind of lame!

In order to produce a video from this, we will need to pass these images over to the server to be stitched together using ffmpeg, a command-line tool. How you choose to do this, using the language of your choice, is your decision. If you are using PHP, this will be done using exec(). Hopefully, you are not hosting your app on a shared hosting environment, because you likely will not have access to ffmpeg or privileges to install it :) 

There is also a popular 3rd party plugin called Whammy which will do a little bit of extra work for you and export everything into a single webm data blob after the fact. This does not get you out of using the aforementioned <canvas> with requestAnimationFrame!

While this isn't a lot of extra work in terms of code written (although as a minimalist at heart, I feel disgusted even suggesting that the above is "okay" in any way), it's executed on a shaky foundation and is a hacky workaround for something that has not received proper attention! Every moment that passes where this API does not get proper development is another moment where someone writes the above code which leads to costly tech debt later on. 

As of January 2013, the spec has been rewritten yet again and has gone nowhere. There are open-ended questions posed around the nature of its security/privacy and how efficiently it can handle processing this data, down to heavy-hanging questions like, "how do we deal with buffer overflow?" These are questions that people need to be voicing in on now so that it can continue its development and the Internet can grow in its warm embrace of rich media! 

If you care about clean code and advancing web technology to the fullest, please share this with your friends, coworkers, and peers, but more importantly, chime in your feedback, your expertise, your heart as a developer into this ticket! Thanks!



-Aimee

© 2013 - 2024 fartprincess
Comments4
Join the community to add your comment. Already a deviant? Log In
jekor's avatar
What about just implementing it? I'm not joking or trying to throw insults. It seems like you understand the problem well, and Chromium has a well-documented code contribution process.