HTML5 Video Accessibility

July 26, 2009

The most pressing Accessibility issue in HTML5 today? <video>

HTML5 Video Accessibility

Over a wonderfully authentic Thai dinner the other night with my friend, Opera Web Evangelist and HTML5 Doctor Bruce Lawson, conversation naturally turned to the outstanding accessibility issues that still need to be addressed in HTML5. While we both agree that finding a solution for <canvas> is going to be both vexing and messy, I suggested to Bruce that completing the <video> element to properly address accessibility before Last Call (currently scheduled for October, 2009) was likely even more pressing. Here’s why:

On June 26, 2009, Massachusetts Representative Edward Markey (D) introduced “The Twenty-first Century Communications and Video Accessibility Act of 2009” (H.R. 3101) [Download the PDF]. From 1987 to 2008, Rep. Markey served either as the Chairman or the Ranking Member of the House Energy and Commerce Committee’s Subcommittee on Telecommunications. (The proposed bill is also co-sponsored by California Reps. Linda Sanchez (D) and Barbara Lee (D)). If (when?) enacted, this comprehensive disabilities communications legislation will amend the United States Communications Act to ensure that new Internet-enabled telephone and television products and services are accessible to and usable by people with disabilities. It will also close existing disability gaps in telecommunications law. The bill in part proposes:

  • Requiring caption decoder circuitry or display capability in all video programming devices, including PDAs, computers, iPods, cell phones, DVD players, TiVo devices and battery-operated TVs
  • Extending closed captioning obligations to television-type video programming distributed over the Internet: covers web-based video services that offer television programs, movies, web clips, and live video streaming
  • Requires easy access to closed captions via remote control and on-screen menus

Before the chicken little crowd has a chance to start squealing, the bill would exempt user-generated content such as family videos and other personal videos on YouTube, etc. However for large content providers (including those that may still choose to use delivery platforms such as YouTube and iTunes) this will be a critical business issue. Finally, I believe that for developers of the tools that act as the user interface (i.e. Browsers), failing to have a solution (API) to this requirement could place them squarely in the sites of organizations such as the American Association of People with Disabilities (AAPD), the National Association of the Deaf (NAD) and/or other advocacy groups, or at the very least render the <video> element a developmental eunuch, given that current proprietary plug in solutions do offer this functionality.

Who Doesn’t Caption Online

  • A&E
  • ABC Family (owned by ABC)
  • AMC (Rainbow Media)
  • Animal Planet (Discovery networks)
  • BBC America
  • BET
  • Biography (A&E)
  • Blinkx (retransmitter)
  • Boomerang (Turner)
  • Bravo
  • Cartoon Network (Turner)
  • CBS
  • Cinemax (HBO)
  • CMT (MTV)
  • Comedy Central (MTV)
  • CNBC (NBC)
  • CNN (Turner)
  • CNN en Espanol (Turner)
  • CNN International (Turner)
  • Discovery (Discovery networks)
  • Discovery Health (Discovery networks)
  • Discovery Kids (Discovery networks)
  • Disney (ABC)
  • DIY
  • E!
  • ESPN
  • Fancast
  • FitTV (Discovery networks)
  • Food Network
  • Fox Movie Channel
  • Fox News Channel
  • FX
  • GSN
  • Hallmark
  • HBO
  • HD Theater (Discovery networks)
  • HGTV
  • History Channel (A&E)
  • HLN (Turner)
  • IFC (Rainbow Media)
  • IMDB.com (retransmitter)
  • Investigation Discovery (Discovery networks)
  • Joost (retransmitter)
  • Lifetime
  • Military Channel (Discovery networks)
  • MSNBC
  • MTV (also CMT)
  • National Geographic Channel
  • Nickelodeon
  • Online TV (retransmitter)
  • Oxygen
  • PBS
  • QVC
  • Science Channel (Discovery networks)
  • Sci Fi
  • Showtime
  • Speed
  • Spike
  • Starz
  • Style
  • Sundance (Rainbow Media)
  • TBS (Turner)
  • TCM (Turner)
  • The Weather Channel
  • Tidal TV (retransmitter)
  • TLC (Discovery networks)
  • TNT (Turner)
  • Travel Channel
  • TruTV (Turner)
  • Truveo
  • TV.com (retransmitter, owned by CBS.com)
  • TVLand
  • USA
  • Veoh (retransmitter)
  • VH1 (MTV)
  • Voom (Rainbow Media)
  • We (Rainbow Media)

Who Captions Online

  • ABC
  • CNET
  • Fox
  • Hulu (retransmitter)
  • NBC

Source: Caption Action 2

And that is the key point: current proprietary plug-ins can render this functionality, although each solution requires content expressly created for that player, and the way in which caption files are associated to the media differs from player to player (and in the case of captioning for the iPhone, requires arcane binary files burned into the media asset – providing on-screen captioning to the deaf but seeing users, but continuing to shut out deaf/blind users and making searchability problematic).

A Tangled Mess

I suspect that one of the reasons why this issue has not emerged more prominently is due to the current impasse surrounding a standardized codec that the <video> element should support natively, with two firmly entrenched front-runners (Ogg/Theora and H.264) and outside third-parties further arguing for support of any codec. With confusion and disharmony around simply how to implement a common media stream natively within the <video> element, there is little wonder that how to further support captioning has taken a back seat in the discussion. The problem is further compounded by questions surrounding time-stamping formats for the transcripts (DFXP, SRT, SCC, others), and in-band vs. out-band delivery of the transcript to the user-agent/user interface; in-band captioning ensures one single file that includes the captioning is ‘burned in’ (so that when media assets are re-distributed the captioning remains), whilst out-band more easily allows re-purposing the transcript file to alternative user-agents – for example to Braille output devices, or for fuller indexing by search engines, etc. (Google today can index an external caption file and use that data to improve SEO results of videos). However, externally referenced files can be separated from the media file quite accidentally, so there are issues with how to ensure that media and captioning remain ‘bound’.

In an June 24, 2009 posting to the HTML WG mailing list from Silvia Pfeiffer, she noted that: … it has been decided that the first version of HTML5 <video> (and <audio>) will not have an in-built solution for captions, audio annotations and the like, because it is possible to do such with javascript and external files. (Who actually did this deciding remains a problematic political issue, as surely the W3C WAI PFWG would not put forth such a reccomendation.)

While I’ve seen such experimental proofs of concept demonstrated, there remains interoperability issues: the linked example relies on one codec delivery (in this case ogg/theora), and sadly the example only works in one browser (Mozilla). Given that the whole point of the <video> element was to make it simple for content authors to embed video into their web pages, yet currently authors today need to supply multiple streams (theora/H.264) plus ‘roll-your-own’ javascripts (I am unaware of any libraries that facilitate extraction of caption files), one has to wonder why content creators wouldn’t simply continue to use existing proprietory solutions, especially since embeddable media players such as the Flash-based JW FLV Player deliver what might soon be legislated functionality.

No, if those working on the HTML5 specification truly want to see real-world commercial and institutional uptake of the <video> element become a reality, then solving the captioning issue prior to Last Call should be imperative; failing to do so will doom <video> to the ‘great ideas that didn’t catch on’ pile of the world wide web.

Read More:

Posted by John

I am a 16 year veteran of Web Accessibility, living and working in Austin, Texas. Currently Principal Accessibility Strategist at Deque Systems Inc., I have previously held accessibility related positions at JPMorgan Chase and Stanford University. I am also actively involved with the W3C - the international internet standards body - where I attempt to stir the pot, fight hard for accessibility on the web, and am currently co-chairing a subcommittee on the accessibility of media elements in HTML5.

View more posts from this author

Leave a Reply