Raphael Cohen


pdf bib
Shrinking Bigfoot: Reducing wav2vec 2.0 footprint
Zilun Peng | Akshay Budhkar | Ilana Tuil | Jason Levy | Parinaz Sobhani | Raphael Cohen | Jumana Nassour
Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing

Wav2vec 2.0 is a state-of-the-art speech recognition model which maps speech audio waveforms into latent representations. The largest version of wav2vec 2.0 contains 317 million parameters. Hence, the inference latency of wav2vec 2.0 will be a bottleneck in production, leading to high costs and a significant environmental footprint. To improve wav2vec’s applicability to a production setting, we explore multiple model compression methods borrowed from the domain of large language models. Using a teacher-student approach, we distilled the knowledge from the original wav2vec 2.0 model into a student model, which is 2 times faster, 4.8 times smaller than the original model. More importantly, the student model is 2 times more energy efficient than the original model in terms of CO2 emission. This increase in performance is accomplished with only a 7% degradation in word error rate (WER). Our quantized model is 3.6 times smaller than the original model, with only a 0.1% degradation in WER. To the best of our knowledge, this is the first work that compresses wav2vec 2.0.


pdf bib
Sentence Embedding Evaluation Using Pyramid Annotation
Tal Baumel | Raphael Cohen | Michael Elhadad
Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP


pdf bib
Query-Chain Focused Summarization
Tal Baumel | Raphael Cohen | Michael Elhadad
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)


pdf bib
Effect of Out Of Vocabulary Terms on Inferring Eligibility Criteria for a Retrospective Study in Hebrew EHR
Raphael Cohen | Michael Elhadad
Proceedings of the 2013 Workshop on Biomedical Natural Language Processing


pdf bib
Domain Adaptation of a Dependency Parser with a Class-Class Selectional Preference Model
Raphael Cohen | Yoav Goldberg | Michael Elhadad
Proceedings of ACL 2012 Student Research Workshop