Visual Question Answering (VQA) is one of the most popular tasks in the fields of Computer Vision and Natural Language Processing. The task is to provide an answer to a question related to the contents of a given image. VQA has been expanded to several domains. One of these is document VQA, where the image is a page of a document and the question is related to the textual contents of the page. In this case, the system that is being evaluated has to understand textual contents that are written inside the image, differently from typical VQA where images are natural scenes. However, document VQA datasets are exclusively in English, hence the performance of the system is evaluated for the English language only. In light of this, we introduce MaDOVE (Multilingual DOcument VQA Evaluation), a novel dataset designed to evaluate document VQA in multiple languages. Moreover, to establish a strong benchmark, we conduct an extensive set of experiments by evaluating a diverse range of models on this task.