Subtitling for an Inclusive Society Overcoming Audiovisual Barriers
We live in a world that is becoming more and more audiovisual. Audiovisual is the new universal language. We learn with videos instead of books, thanks to the proliferation of video tutorials on YouTube or e-learning platforms such as Udacity or Coursera. We communicate with gifs and share tons of silly videos on social media platforms. If we are not caught up with the latest Netflix or HBO hits, we feel completely clueless when hanging out with friends.
Deaf VS hearing population
Accessibility to audiovisual content is a human right
According to Article 30, persons with disabilities have the right to “enjoy access to television programs, films, theater, and other cultural activities, in accessible formats.” Thanks to this change and other regulations, many countries have specific legislation addressing the need for including subtitles in audiovisual content to ensure equal access to all citizens. In the US, the National Association for the Deaf (NAD) sued Netflix in 2010 claiming that the video on demand (VoD) platform was in violation of the Americans with Disabilities Act. The NAD won the lawsuit and set a precedent, and thanks to that, today we have subtitles in most (if not all) VoD platforms.
Who is using subtitles?
Content creators should be aware of the legal ramifications and provide subtitles for all content, in order to attract a wider audience and create a more inclusive society. However, we should not see subtitles as a necessary evil because legislation says so. Instead, they should be used as a way to engage our audience.
Viewers who consume subtitled content are increasing and evolving, and they are not only deaf and HoH or foreign language speakers anymore.
Ironically, in this audiovisual era that we are living in, content creators are going back to written text to catch viewers’ attention. Also, it seems like Gen Z is watching more and more subtitled content because it helps them to focus on the video while multitasking (for example, watching Netflix and checking Instagram on their smartphone). Language learners are also lucky because there are even browser extensions to watch multilingual subtitles on Netflix. This helps them make the most out of the learning experience while enjoying a good series.
Subtitles are really for everybody, and they are improving user experience, so investment in that area is a no-brainer. To cope with increasing demand, it makes sense to look for technologies that make the subtitling process more efficient. Thanks to innovation in subtitling technology, such as web-based subtitling platforms, automated speech recognition (ASR) software or integration with CAT tools, the industry is able to cope with the increasing demand (for more information on media localization technology, check the articles Part One and Part Two at Nimdzi).
Subtitling technology for improved accessibility
How can we use these technologies to make society more inclusive?
One of the main technologies to enhance accessibility in audiovisual products is ASR software which, together with speech alignment technology, can facilitate the automatic creation of intralingual subtitles (that is, subtitles in the same language that is being spoken). For example, YouTube includes a speech recognition module that generates automatic subtitles. Then, you can either post-edit those automatic subtitles or even translate them from the Transcriptions section in YouTube Studio. This is accessible from your YouTube channel, which comes automatically when you open an account.
This is an example of free captioning in one of the main video platforms worldwide. Of course, caption quality may vary as it depends on many factors (background noise, accents, etc.), but it is a way to streamline the creation of subtitles that can then be post-edited. We also need to consider that subtitles for the Deaf and HoH or closed captions include other information that is not always provided by automatic ASR, such as speaker identification or sound events.
For social media videos, there are free apps like AutoCap that can be used to add some automatic and colorful subtitles to short videos that are uploaded to social media. Recently on Twitter, actress and activist Jameela Jamil was promoting the use of this kind of app to engage people with hearing impairments in social media conversations.
Professional ASR software
Professional ASR software is out there too, and it is having an impact on the subtitling industry. For example, platforms such as Media Studio by Omniscien or Limecraft, or specific ASR software such as Trint or Apptek, offer professional ASR solutions.
Earlier this year, Mara Campbell from TrueSubtitles presented the results of a comparative study of ASR software in the media accessibility event Media4All (Stockholm). She argued that even the solution that offered the poorest results was still more efficient than manually transcribing the videos from scratch. ASR is one of the potential ways this industry can handle the demand and need for short turnarounds. And, of course, to increase accessibility.
But there are other scenarios where ASR solutions are not enough, for example, live events such as conferences, theater plays, or movie theater screenings where no captions are included. Still, innovative solutions are also appearing to tackle similar situations. The Rochester Institute of Technology enrolls around 1,500 deaf and HoH students annually, and they are using Microsoft Translate to generate automatic subtitles. Lessons are therefore more inclusive, and hearing participants are also using subtitles as class notes, so it is an advantage for everybody.
Webcaptioner is a free, web-based, real-time transcription app that can be used for subtitling live events, public speeches, conferences, etc. Its accuracy depends on the language, background noise, accent and speakers’ diction. Live Transcribe is another free app created by Android to enhance communication between deaf and hearing persons, although it can be used for transcription too.
Another modality that is gaining popularity for real-time subtitling live TV or events is respeaking. Respeaking is the process of repeating what is being said (including punctuation marks, sound events, etc.) in a live program or event into speech recognition software that then generates the subtitle text. Respeaking is usually intralingual, although interlingual respeaking is gaining ground too. The most common software used in this discipline is Dragon Naturally Speaking.
Augmented Reality and smart apps to open content to anyone, anywhere
The National Theatre of London went one step further toward customizing its user experience by launching the use of smart caption glasses. They are offering captions for all their plays thanks to augmented reality glasses and a speech-following software engine that creates live time codes. This process makes the subtitles appear in the glasses when the actors and actresses are actually speaking. Thanks to artificial intelligence technology, the pre-generated subtitles can be aligned with the play and deliver a nice experience to the deaf and HoH audience. This technology could also be used to create multilingual subtitles for any non-English speaking audience members.
Smartphone apps are also popping up to bring accessibility to movie theater goers. Screening in many countries do not include subtitles or audio description (AD) and, therefore, movies are not accessible to everybody. However, apps such as Actiview (US), Greta and Starks (Germany) or Audiomovie (Poland) are changing the game. These companies provide access services such as subtitles or AD available to viewers by downloading the app on their smartphone. They create access services and provide the technical infrastructure to sync the subtitles/AD with the movie in the cinema, making content accessible for everybody
We are living in the golden age of subtitling and accessibility. With new, exciting technology which localization professionals are able to leverage, we already see glimpses of a bright future without any audiovisual barriers.
Stay up to date as Nimdzi publishes new insights. We will keep you posted as each new report is published so that you are sure not to miss anything.