<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="fragkiadakis-van-der-putten-2021-sign">
    <titleInfo>
        <title>Sign and Search: Sign Search Functionality for Sign Language Lexica</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Manolis</namePart>
        <namePart type="family">Fragkiadakis</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Peter</namePart>
        <namePart type="family">van der Putten</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Dimitar</namePart>
            <namePart type="family">Shterionov</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Machine Translation in the Americas</publisher>
            <place>
                <placeTerm type="text">Virtual</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Sign language lexica are a useful resource for researchers and people learning sign languages. Current implementations allow a user to search a sign either by its gloss or by selecting its primary features such as handshape and location. This study focuses on exploring a reverse search functionality where a user can sign a query sign in front of a webcam and retrieve a set of matching signs. By extracting different body joints combinations (upper body, dominant hand’s arm and wrist) using the pose estimation framework OpenPose, we compare four techniques (PCA, UMAP, DTW and Euclidean distance) as distance metrics between 20 query signs, each performed by eight participants on a 1200 sign lexicon. The results show that UMAP and DTW can predict a matching sign with an 80% and 71% accuracy respectively at the top-20 retrieved signs using the movement of the dominant hand arm. Using DTW and adding more sign instances from other participants in the lexicon, the accuracy can be raised to 90% at the top-10 ranking. Our results suggest that our methodology can be used with no training in any sign language lexicon regardless of its size.</abstract>
    <identifier type="citekey">fragkiadakis-van-der-putten-2021-sign</identifier>
    <location>
        <url>https://aclanthology.org/2021.mtsummit-at4ssl.3</url>
    </location>
    <part>
        <date>2021-08</date>
        <extent unit="page">
            <start>23</start>
            <end>32</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Sign and Search: Sign Search Functionality for Sign Language Lexica
%A Fragkiadakis, Manolis
%A van der Putten, Peter
%Y Shterionov, Dimitar
%S Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL)
%D 2021
%8 August
%I Association for Machine Translation in the Americas
%C Virtual
%F fragkiadakis-van-der-putten-2021-sign
%X Sign language lexica are a useful resource for researchers and people learning sign languages. Current implementations allow a user to search a sign either by its gloss or by selecting its primary features such as handshape and location. This study focuses on exploring a reverse search functionality where a user can sign a query sign in front of a webcam and retrieve a set of matching signs. By extracting different body joints combinations (upper body, dominant hand’s arm and wrist) using the pose estimation framework OpenPose, we compare four techniques (PCA, UMAP, DTW and Euclidean distance) as distance metrics between 20 query signs, each performed by eight participants on a 1200 sign lexicon. The results show that UMAP and DTW can predict a matching sign with an 80% and 71% accuracy respectively at the top-20 retrieved signs using the movement of the dominant hand arm. Using DTW and adding more sign instances from other participants in the lexicon, the accuracy can be raised to 90% at the top-10 ranking. Our results suggest that our methodology can be used with no training in any sign language lexicon regardless of its size.
%U https://aclanthology.org/2021.mtsummit-at4ssl.3
%P 23-32

Markdown (Informal)

[Sign and Search: Sign Search Functionality for Sign Language Lexica](https://aclanthology.org/2021.mtsummit-at4ssl.3) (Fragkiadakis & van der Putten, MTSummit 2021)

• Sign and Search: Sign Search Functionality for Sign Language Lexica (Fragkiadakis & van der Putten, MTSummit 2021)

ACL

• Manolis Fragkiadakis and Peter van der Putten. 2021. Sign and Search: Sign Search Functionality for Sign Language Lexica. In Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL), pages 23–32, Virtual. Association for Machine Translation in the Americas.