Transformers Learn Transition Dynamics when Trained to Predict Markov Decision Processes

Yuxi Chen, Suwei Ma, Tony Dear, Xu Chen


Abstract
Language models have displayed a wide array of capabilities, but the reason for their performance remains a topic of heated debate and investigation. Do these models simply recite the observed training data, or are they able to abstract away surface statistics and learn the underlying processes from which the data was generated? To investigate this question, we explore the capabilities of a GPT model in the context of Markov Decision Processes (MDPs), where the underlying transition dynamics and policies are not directly observed. The model is trained to predict the next state or action without any initial knowledge of the MDPs or the players’ policies. Despite this, we present evidence that the model develops emergent representations of the underlying parameters governing the MDPs.
Anthology ID:
2024.blackboxnlp-1.13
Volume:
Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
Month:
November
Year:
2024
Address:
Miami, Florida, US
Editors:
Yonatan Belinkov, Najoung Kim, Jaap Jumelet, Hosein Mohebbi, Aaron Mueller, Hanjie Chen
Venue:
BlackboxNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
207–216
Language:
URL:
https://aclanthology.org/2024.blackboxnlp-1.13
DOI:
Bibkey:
Cite (ACL):
Yuxi Chen, Suwei Ma, Tony Dear, and Xu Chen. 2024. Transformers Learn Transition Dynamics when Trained to Predict Markov Decision Processes. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 207–216, Miami, Florida, US. Association for Computational Linguistics.
Cite (Informal):
Transformers Learn Transition Dynamics when Trained to Predict Markov Decision Processes (Chen et al., BlackboxNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.blackboxnlp-1.13.pdf