Feedback from field deployments of machine translation (MT) is instructive but hard to obtain, especially in the case of soldiers deployed in mobile and stressful environments. We first consider the process of acquiring feedback: the difficulty of getting and interpreting it, the kinds of information that have been used in place of or as predictors of direct feedback, and the validity and completeness of that information. We then look at how to better forecast the utility of MT in deployments so that feedback from the field is focused on aspects that can be fixed or enhanced rather than on overall failure or viability of the technology. We draw examples from document and speech translation.

“Embedded” machine translation (MT) refers to an end-to-end computational process of which MT is one of the components. Integrating these components and evaluating the whole has proved to be problematic. As an example of embedded MT, we describe a prototype system called Falcon, which permits paper documents to be scanned and translated into English. MT is thus embedded in the preprocessing of hardcopy pages and subject to its noise. Because Falcon is intended for use by people in the military who are trying to screen foreign documents, and not to understand them in detail, its application makes low demands on translation quality. We report on a series of user trials that speak to the utility of embedded MT in army tasks.