Large Language Models (LLMs) have demonstrated remarkable capabilities in processing long contexts and generating long-form texts. However, existing benchmarks evaluate these capabilities by focusing on either long-input comprehension with short outputs or short-input creative generation with lengthy outputs. This critical gap overlooks real-world Long-In-Long-Out Generation tasks, where models must simultaneously process long-context inputs and produce long-form outputs while adhering to fine-grained constraints. We introduce DocFollow, a comprehensive benchmark featuring 2,436 expert-validated instances across 5 major domains and 20 subdomains, designed to evaluate LLMs’ ability to follow hierarchical constraints in Document Following tasks. Our benchmark employs five types of constraints with increasing granularity. To ensure reliable evaluation, we develop a rule-based assessment protocol that decomposes overall evaluation uncertainty into atomic-level classification uncertainties, employing LLMs solely as information extractors for well-established NER and RE tasks rather than subjective scoring. Our evaluation of 13 state-of-the-art LLMs reveals a consistent pattern: while models maintain reasonable performance on coarse-grained constraints, they exhibit substantial degradation on fine-grained requirements, particularly entity-level constraints. This finding suggests fundamental architectural limitations in maintaining detailed structural and semantic fidelity throughout extended generation. Our findings highlight the critical need for advancing long-context generation capabilities beyond surface-level coherence to achieve precise fine-grained control. The implementation is publicly available. Source code is available at https://github.com/leonbig/DocFollow