<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="jin-schuler-2019-variance">
    <titleInfo>
        <title>Variance of Average Surprisal: A Better Predictor for Quality of Grammar from Unsupervised PCFG Induction</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Lifeng</namePart>
        <namePart type="family">Jin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">William</namePart>
        <namePart type="family">Schuler</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Korhonen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">David</namePart>
            <namePart type="family">Traum</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lluís</namePart>
            <namePart type="family">Màrquez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Florence, Italy</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In unsupervised grammar induction, data likelihood is known to be only weakly correlated with parsing accuracy, especially at convergence after multiple runs. In order to find a better indicator for quality of induced grammars, this paper correlates several linguistically- and psycholinguistically-motivated predictors to parsing accuracy on a large multilingual grammar induction evaluation data set. Results show that variance of average surprisal (VAS) better correlates with parsing accuracy than data likelihood and that using VAS instead of data likelihood for model selection provides a significant accuracy boost. Further evidence shows VAS to be a better candidate than data likelihood for predicting word order typology classification. Analyses show that VAS seems to separate content words from function words in natural language grammars, and to better arrange words with different frequencies into separate classes that are more consistent with linguistic theory.</abstract>
    <identifier type="citekey">jin-schuler-2019-variance</identifier>
    <identifier type="doi">10.18653/v1/P19-1235</identifier>
    <location>
        <url>https://aclanthology.org/P19-1235</url>
    </location>
    <part>
        <date>2019-07</date>
        <extent unit="page">
            <start>2453</start>
            <end>2463</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Variance of Average Surprisal: A Better Predictor for Quality of Grammar from Unsupervised PCFG Induction
%A Jin, Lifeng
%A Schuler, William
%Y Korhonen, Anna
%Y Traum, David
%Y Màrquez, Lluís
%S Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
%D 2019
%8 July
%I Association for Computational Linguistics
%C Florence, Italy
%F jin-schuler-2019-variance
%X In unsupervised grammar induction, data likelihood is known to be only weakly correlated with parsing accuracy, especially at convergence after multiple runs. In order to find a better indicator for quality of induced grammars, this paper correlates several linguistically- and psycholinguistically-motivated predictors to parsing accuracy on a large multilingual grammar induction evaluation data set. Results show that variance of average surprisal (VAS) better correlates with parsing accuracy than data likelihood and that using VAS instead of data likelihood for model selection provides a significant accuracy boost. Further evidence shows VAS to be a better candidate than data likelihood for predicting word order typology classification. Analyses show that VAS seems to separate content words from function words in natural language grammars, and to better arrange words with different frequencies into separate classes that are more consistent with linguistic theory.
%R 10.18653/v1/P19-1235
%U https://aclanthology.org/P19-1235
%U https://doi.org/10.18653/v1/P19-1235
%P 2453-2463

Markdown (Informal)

[Variance of Average Surprisal: A Better Predictor for Quality of Grammar from Unsupervised PCFG Induction](https://aclanthology.org/P19-1235) (Jin & Schuler, ACL 2019)

• Variance of Average Surprisal: A Better Predictor for Quality of Grammar from Unsupervised PCFG Induction (Jin & Schuler, ACL 2019)

ACL

• Lifeng Jin and William Schuler. 2019. Variance of Average Surprisal: A Better Predictor for Quality of Grammar from Unsupervised PCFG Induction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2453–2463, Florence, Italy. Association for Computational Linguistics.