Quality assessment in interlingual live subtitling: The NTR Model

Authors

  • Pablo Romero-Fresco
  • Franz Pöchhacker University of Vienna

DOI:

https://doi.org/10.52034/lanstts.v16i0.438

Keywords:

intralingual live subtitling, respeaking, simultaneous interpreting, NTR model, NER model, quality assessment, accuracy, errors

Abstract

This article presents a model for assessing the quality of interlingual live subtitling, as carried out by way of respeaking and automatic speech recognition or other means. The first part reviews the requirements and formulas for quality assessment in intralingual live subtitling and presents the NER model, developed by the first author, which serves as a foundation for the new model. Drawing on insights from the literature on both prerecorded interlingual subtitling and simultaneous interpreting, the authors then introduce the NTR model, an NER-based formula for calculating the accuracy rate of (interlingual) live subtitles. The model features a set of categories for scoring the accuracy of content and form as well as a three-level grading scale for translation and recognition errors. The application of the NTR model is exemplified, highlighting the role of the accuracy rate as part of an overall qualitative assessment.

Author Biography

Franz Pöchhacker, University of Vienna

Franz Pöchhacker is Associate Professor of Interpreting Studies in the Center for Translation Studies at the University of Vienna. He holds Master’s degrees in Conference Interpreting (A: German, B: English, C: Spanish) from the University of Vienna and the Monterey Institute of International Studies, and a PhD from the University of Vienna. He has done empirical research on a range of topics, and lectured and published widely on various domains of interpreting, including conference, media and healthcare interpreting, as well as issues of interpreting studies as a discipline. In 2012 he was CETRA Chair professor at KU Leuven. He is the author of three monographs, including the textbook Introducing Interpreting Studies (Routledge, 2004/2016), co-editor of Interpreting. International Journal of Research and Practice in Interpreting (John Benjamins), and editor of the Routledge Encyclopedia of Interpreting Studies (Routledge, 2015).

References

Apone, T., Brooks, M., & O’Connell, T. (2010). Subtitle Accuracy Metrics Project. Subtitle viewer survey: Error ranking of real-time subtitles in live television news programmes. WGBH National Center for Accessible Media, Boston. http://ncam.wgbh.org/invent_build/analog/subtitle-accuracy-metrics (last accessed 1 July 2015).

Arumí Ribas, M., & P. Romero-Fresco (2008). A practical proposal for the training of respeakers. Journal of Specialised Translation, 10, 106–127. Available online: http://www.jostrans.org/issue10/art_arumi.php (last accessed 19 November 2016).

Barik, H. C. (1969). A study of simultaneous interpretation. Doctoral dissertation, Uni¬versity of North Carolina, Chapel Hill.

Barik, H. C. (1975/2002). Simultaneous interpretation: Qualitative and linguistic data. In F. Pöchhacker & M. Shlesinger (Eds.), The interpreter studies reader (pp. 79–91). London: Routledge.

Bühler, H. (1986). Linguistic (semantic) and extra-linguistic (pragmatic) criteria for the evaluation of conference interpretation and interpreters. Multilingua, 5(4), 231–235.

Carey, P. W. (1968). Delayed auditory feedback and the shadowing response. Doctoral dissertation, Harvard University.

Carroll, M., & Ivarsson, J. (1998). Code of good subtitling practice. Berlin: Euro¬pean Association for Studies in Screen Translation.

Chafe, W. (1985). Linguistic differences produced by differences between speaking and writing. In D. Olson, N. Torrance, & A. Hildyard (Eds.), Literacy, language, and learning: The nature and consequences of reading and writing (pp. 105–122). Cambridge: Cambridge University Press.

Chesterman, A. (1993). From ‘is’ to ‘ought’: Laws, norms and strategies in translation studies. Target, 5(1), 1‒20.

CSA. (2011). Charte relative à la qualité du sous-titrage à destination des personnes sourdes ou malentendantes, Paris: Conseil Supérieur de l’Audiovisuel. Available online: http://www.csa.fr/Espace-juridique/Chartes/Charte-relative-a-la-qualite-du-sous-titrage-a-destination-des-personnes-sourdes-ou-malentendantes-Decembre-2011 (last accessed 19 November 2016).

Dumouchel, P., Boulianne, G., & Brousseau, J. (2011). Measures for quality of closed captioning. In A. Şerban, A. Matamala, & J.-M. Lavaur (Eds.), Audiovisual translation in close-up: Practical and theoretical approaches (pp. 161–172). Bern: Peter Lang.

Eppler, E. D., & Ozón, G. (2013). English words and sentences: An introduction. Cambridge: Cambridge University Press.

Eugeni, C. (2006). Introduzione al rispeakeraggio televisivo. In C. Eugeni & G. Mack (Eds.), Intralinea, Special Issue on Respeaking. Available online: http://www.intralinea.org/specials/article/Introduzione_al_rispeak eraggio_televisivo (last accessed 19 November 2016).

Eugeni, C. (2008). Respeaking the TV for the Deaf: For a real special needs-oriented subtitling. Studies in English Language and Literature, 21, 37–47.

Eugeni, C. (2009). Respeaking the BBC News: A strategic analysis of respeaking on the BBC. The Sign Language Translator and Interpreter, 3(1), 29–68.

European Committee For Standardization (2006). European Standard EN 15038. Translation services – Service requirements. Brussels: European Committee for Standardization.

Falbo, C. (2015). Error analysis. In F. Pöchhacker (Ed.), Routledge encyclopedia of interpreting studies (pp. 143–144). London: Routledge.

Gerver, D. (1969/2002). The effects of source language presentation rate on the performance of simultaneous conference interpreters. In F. Pöchhacker & M. Shlesinger (Eds), The Interpreter Studies reader (pp. 53–66). London: Routledge.

Gerver, D., Longley, P. E., Long, J., & Lambert, S. (1989). Selection tests for trainee conference interpreters. Meta, 34(4), 724–735.

Gile, D. (2009). Basic concepts and models for interpreter and translator training (Rev. ed.). Amsterdam: John Benjamins.

Grbić, N. (2008). Constructing interpreting quality. Interpreting, 10(2), 232‒257.

House, J. (1981). A model for translation quality assessment. Tübingen: Gunter Narr.

International Telecommunication Union (ITU). (2015). Accessibility terms and definitions: Series F: Non-telephone telecommunication services. Audio¬visual services. Geneva: ITU.

Jakobson, R. (1959/2000). On linguistic aspects of translation. In L. Venuti (Ed.), The translation studies reader (pp. 113–118). London: Routledge.

Kalina, S. (2015). Compression. In F. Pöchhacker (Ed.), Routledge encyclopedia of interpreting studies (pp. 73–75). London: Routledge.

Kruger, J.-L. (2008). Subtitler training as part of a general training programme in the language professions. In J. Díaz Cintas (Ed.), The didactics of audiovisual translation (pp. 71–88). Amsterdam: John Benjamins.

Lambourne, A. (2006). Subtitle respeaking. In C. Eugeni & G. Mack (Eds.), Intralinea, Special Issue on Respeaking. Available online: http://www.intralinea.org/specials/article/Subtitle_respeaking (last accessed 19 November 2016).

Leuven-Zwart, K., van (1989). Translation and original: Similarities and dissimilarities. Target, 1(2), 151–181.

Liu, M. (2015). Assessment. In F. Pöchhacker (Ed.), Routledge encyclopedia of interpreting studies (pp. 20–22). London: Routledge.

Marsh, A. (2006). Respeaking for the BBC. In C. Eugeni & G. Mack (Eds), Intralinea, Special Issue on Respeaking. Available online: http://www.intralinea.org/specials/article/Respeaking_for_the_BBC (last accessed 19 November 2016).

Martí Ferriol, J. L. & de Higes Andino, I. (2015). Guía de corrección para subtitulación. Unpublished class material. Universitat Jaume I, Castelló, Spain.

Moser-Mercer, B. (1996). Quality in interpreting: Some methodological issues. The Interpreters’ Newsletter, 7, 43‒55.

Ofcom (2015). Measuring live subtitling quality: Results from the fourth sampling exercise. London: Office of Communications. Available online: https://www.ofcom.org.uk/research-and-data/tv-radio-and-on-demand/tv-research/live-subtitling/sampling_results_4 (last accessed 19 November 2016).

Pedersen, J. (2017). The FAR model: Assessing quality in interlingual subtitling. Journal of Specialised Translation, 28, 210–229.

Pöchhacker, F. (2001). Quality assessment in conference and community interpreting. Meta, 46(2), 410–425.

Rajendran, D. J., Duchowski, A. T., Orero, P., Martínez, J., & Romero-Fresco, P. (2012). Effects of text chunking on subtitling: A quantitative and qualitative examination. Perspectives: Studies in Translatology, 21(1), 5–21.

Remael, A., & van der Veer, B. (2006). Real-time subtitling in Flanders: Needs and teaching. In C. Eugeni & G. Mack (Eds.), Intralinea, Special Issue on Respeaking. Available online: http://www.intralinea.org/ specials/article/Real-Time_Subtitling_in_Flanders_Needs_and_Teaching (last accessed 19 November 2016).

Robert, I. S., & Remael, A. (forthcoming). Quality control in the subtitling industry: An exploratory survey study. Meta, 61(3).

Romero-Fresco, P. (2009). More haste less speed: Edited vs. verbatim respeaking. Vigo International Journal of Applied Linguistics, VI, 109–133.

Romero-Fresco, P. (2011). Subtitling through speech recognition: Respeaking. Manchester: Routledge.

Romero-Fresco, P. (2016). Accessing communication: The quality of live subtitles in the UK. Language & Communication, 49, 56–69.

Romero-Fresco, P. (forthcoming). Respeaking: Subtitling through speech recognition. In L. Pérez-González (Ed.), The Routledge handbook of audiovisual translation studies. London: Routledge.

Romero-Fresco, P., & Martínez, J. (2015). Accuracy rate in live subtitling: The NER model. In J. Díaz Cintas & R. Baños (Eds.), Audiovisual translation in a global context: Mapping an ever-changing landscape (pp. 28–50). Basingstoke: Palgrave Macmillan.

Russo, M. (2014). Testing aptitude for interpreting: The predictive value of oral paraphrasing, with synonyms and coherence as assessment parameters. In F. Pöchhacker & M. Liu (Eds), Aptitude for interpreting (pp. 129–145). Amsterdam: John Benjamins.

Soria, E. (2016). A proposal to assess accuracy for interlingual respeaking (EN>ES). Adapting the NER Model. MA dissertation, University of Roehampton.

Szczygielska, M., & Dutka, Ł. (2016). Live subtitling through automatic speech recognition vs. respeaking: Between technical possibilities and users’ satisfaction. Presentation at Language and the Media, 3 November 2016, Berlin.

Toury, G. (1995). Descriptive translation studies – and beyond. Amsterdam: John Benjamins.

Vinay, J.-P., & Darbelnet, J. (1958/2000). A methodology for translation. (Trans. J. C. Sager & M.-J. Hamel). In L. Venuti (Ed.), The translation studies reader (pp. 84–93). London/New York: Routledge.

Downloads

Published

29-01-2018

How to Cite

Romero-Fresco, P., & Pöchhacker, F. (2018). Quality assessment in interlingual live subtitling: The NTR Model. Linguistica Antverpiensia, New Series – Themes in Translation Studies, 16. https://doi.org/10.52034/lanstts.v16i0.438

Issue

Vol. 16 (2017): Translator Quality—Translation Quality: Empirical Approaches to Assessment and Evaluation

Section

Articles

License

Authors who publish with this journal agree to the following terms:

  • Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under the CC BY-NC 4.0 Deed that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal. The material cannot be used for commercial purposes.
  • Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  • Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).