Supporting Mouthing in Signed Languages: New innovations and a proposal for future corpus building

Rosalee Wolfe, John McDonald, Ronan Johnson, Ben Sturr, Syd Klinghoffer, Anthony Bonzani, Andrew Alexander, Nicole Barnekow


Abstract
A recurring concern, oft repeated, regarding the quality of signing avatars is the lack of proper facial movements, particularly in actions that involve mouthing. An analysis uncovered three challenges contributing to the problem. The first is a difficulty in devising an algorithmic strategy for generating mouthing due to the rich variety of mouthings in sign language. For example, part or all of a spoken word may be mouthed depending on the sign language, the syllabic structure of the mouthed word, as well as the register of address and discourse setting. The second challenge was technological. Previous efforts to create avatar mouthing have failed to model the timing present in mouthing or have failed to properly model the mouth’s appearance. The third challenge is one of usability. Previous editing systems, when they existed, were time-consuming to use. This paper describes efforts to improve avatar mouthing by addressing these challenges, resulting in a new approach for mouthing animation. The paper concludes by proposing an experiment in corpus building using the new approach.
Anthology ID:
2022.sltat-1.19
Volume:
Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Eleni Efthimiou, Stavroula-Evita Fotinea, Thomas Hanke, John C. McDonald, Dimitar Shterionov, Rosalee Wolfe
Venue:
SLTAT
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
125–130
Language:
URL:
https://aclanthology.org/2022.sltat-1.19
DOI:
Bibkey:
Cite (ACL):
Rosalee Wolfe, John McDonald, Ronan Johnson, Ben Sturr, Syd Klinghoffer, Anthony Bonzani, Andrew Alexander, and Nicole Barnekow. 2022. Supporting Mouthing in Signed Languages: New innovations and a proposal for future corpus building. In Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives, pages 125–130, Marseille, France. European Language Resources Association.
Cite (Informal):
Supporting Mouthing in Signed Languages: New innovations and a proposal for future corpus building (Wolfe et al., SLTAT 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.sltat-1.19.pdf