LADIS: Language Disentanglement for 3D Shape Editing

Ian Huang, Panos Achlioptas, Tianyi Zhang, Sergei Tulyakov, Minhyuk Sung, Leonidas Guibas


Abstract
Natural language interaction is a promising direction for democratizing 3D shape design. However, existing methods for text-driven 3D shape editing face challenges in producing decoupled, local edits to 3D shapes. We address this problem by learning disentangled latent representations that ground language in 3D geometry. To this end, we propose a complementary tool set including a novel network architecture, a disentanglement loss, and a new editing procedure. Additionally, to measure edit locality, we define a new metric that we call part-wise edit precision. We show that our method outperforms existing SOTA methods by 20% in terms of edit locality, and up to 6.6% in terms of language reference resolution accuracy. Human evaluations additionally show that compared to the existing SOTA, our method produces shape edits that are more local, more semantically accurate, and more visually obvious. Our work suggests that by solely disentangling language representations, downstream 3D shape editing can become more local to relevant parts, even if the model was never given explicit part-based supervision.
Anthology ID:
2022.findings-emnlp.404
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2022
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5519–5532
Language:
URL:
https://aclanthology.org/2022.findings-emnlp.404
DOI:
10.18653/v1/2022.findings-emnlp.404
Bibkey:
Cite (ACL):
Ian Huang, Panos Achlioptas, Tianyi Zhang, Sergei Tulyakov, Minhyuk Sung, and Leonidas Guibas. 2022. LADIS: Language Disentanglement for 3D Shape Editing. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5519–5532, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
LADIS: Language Disentanglement for 3D Shape Editing (Huang et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-emnlp.404.pdf