Hard-negative preference optimization should improve support for ground-truth preferred responses while limiting drift from the reference model. In Japanese medical instruction tuning, we find that standard DPO and IPO improve the preferred–dispreferred gap but can leave preferred-response likelihood below the reference model and induce growing sequence-level KL divergence. Motivated by this failure-mode analysis, we propose a stabilized IPO objective that combines preferred-response anchoring with explicit KL control. Ablations show that positive anchoring improves preferred-response support, while KL control suppresses reference-model drift; their combination preserves anchoring gains while further reducing drift. These results suggest that reliable medical preference optimization benefits from both preferred-response anchoring and drift control.

Prior work synthesizes tool-use LLM datasets by first generating a user query, followed by complex tool-use annotations like DFS. This inherently leads to inevitable annotation failures and low efficiency in data generation. We introduce ToolGrad, an agentic framework that inverts this paradigm. ToolGrad first constructs valid tool-use chains through an iterative process guided by textual “gradients”, and then synthesizes corresponding user queries. This “answer-first” approach led to ToolGrad-500, a dataset generated with more complex tool use, lower cost, and almost 100% pass rate. Experiments show that models trained on ToolGrad-500 outperform those trained on expensive baseline datasets and proprietary LLMs.

Although automated image captioning methods have benefited considerably from the development of large language models (LLMs), generating humorous captions is still a challenging task. Humorous captions generated by humans are unique to the image and reflect the content of the image. However, captions generated using previous captioning models tend to be generic. Therefore, we propose incongruity-resolution chain-of-thought (IRCoT) as a novel prompting framework that creates content-specific resolutions from fine details extracted from an image. Furthermore, we integrate logit bias and negative sampling to suppress the output of generic resolutions. The results of experiments with GPT4-V demonstrate that our proposed framework effectively generated humorous captions tailored to the content of specific input images.