We explore the potential of large language models in resolving ambiguity in prepositional phrase attachments in grounded language. We find that when prompted in such a way that we can compute a probability of the respective attachment, models yield promising results. However, additional inputs from a measure of information structure may help improve prediction accuracy. We also investigate where we need more sophisticated tools, commonsense reasoning, world knowledge, and additional context to resolve ambiguity.