Sercan Karakaş
Author directoryAlso published as: Sercan Karakas
2026
From Lemmas to Dependencies: What Signals Drive Light Verbs Classification?
Sercan Karakaş | Yusuf Şimşek
Proceedings of the Second Workshop Natural Language Processing for Turkic Languages (SIGTURK 2026)
Sercan Karakaş | Yusuf Şimşek
Proceedings of the Second Workshop Natural Language Processing for Turkic Languages (SIGTURK 2026)
Light verb constructions (LVCs) are a challenging class of verbal multiword expressions, especially in Turkish,where rich morphology and productive complex predicates create minimal contrasts between idiomatic predicatemeanings and literal verb–argument uses. This paper asks what signals drive LVC classification bysystematically restricting model inputs. Using UD-derived supervision, we compare lemma-driven baselines(lemma TF–IDF + Logistic Regression; BERTurk trained on lemma sequences), a grammar-only Logistic Regressionover UD morphosyntax (UPOS/DEPREL/MORPH), and a full-input BERTurk baseline. We evaluate on a controlleddiagnostic set with Random negatives, lexical controls (NLVC), and LVC positives, reporting split-wiseperformance to expose decision-boundary behavior. Results show that coarse morphosyntax alone is insufficientfor robust LVC detection under controlled contrasts, while lexical identity supports LVC judgments but issensitive to calibration and normalization choices. Overall, our findings motivate targeted evaluation forTurkish MWEs and highlight that “lemma-only” is not a single representation but depends critically on hownormalization is instantiated.
Not All Evidence Is Equal: Source-Sensitive Calibration in Large Language Models
Sercan Karakaş | Yusuf Şimşek
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Sercan Karakaş | Yusuf Şimşek
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language models are increasingly used in settings where information must be evaluated not only for content, but also for reliability. This paper introduces a human-grounded evaluation[All data and code used in this paper will be made available with the camera-ready version.] of source-sensitive calibration in Turkish and English. Across hearer-side uptake tasks, we manipulate source reliability while holding propositional content constant, and test whether humans and LLMs adjust action-readiness judgments according to source trustworthiness. Turkish human participants show robust source sensitivity: they are substantially more willing to act on information attributed to high-trust sources than to low-trust sources, while Turkish evidential morphology itself does not independently affect uptake once the source is overtly specified. LLMs show weaker calibration. In Turkish, models partially register explicit source reliability, but the effect is much smaller than in humans and there is no reliable effect of the -DI/-mIş contrast. In English, models show small effects of both source reliability and the lexical marker apparently, but no robust interaction between them. These results suggest that current LLMs encode source cues only partially and often fail to convert them into human-like judgments about when information is reliable enough to guide action.
Transfer Learning for an Endangered Slavic Variety: Dependency Parsing in Pomak Across Contact-Shaped Dialects
Sercan Karakas
Proceedings of the First Workshop on Dialects in NLP — A Resource Perspective
Sercan Karakas
Proceedings of the First Workshop on Dialects in NLP — A Resource Perspective
This paper presents new resources and baselines for Dependency Parsing in Pomak, an endangered Eastern South Slavic language with substantial dialectal variation and no widely adopted standard. We focus on the variety spoken in Turkey (Uzunköprü) and ask how well a dependency parser trained on the existing Pomak Universal Dependencies treebank, which was built primarily from the variety that is spoken in Greece, transfers across dialects. We run two experimental phases. First, we train a parser on the Greek-variety UD data and evaluate zero-shot transfer to Turkish-variety Pomak, quantifying the impact of phonological and morphosyntactic differences. Second, we introduce a new manually annotated Turkish-variety Pomak corpus of 650 sentences and show that, despite its small size, targeted fine-tuning substantially improves accuracy; performance is further boosted by cross-variety transfer learning that combines the two dialects.
Benchmarking Source-Sensitive Reasoning in Turkish: Humans and LLMs under Evidential Trust Manipulation
Sercan Karakas | Yusuf Şimşek
Proceedings of the 15th Workshop on Cognitive Modeling and Computational Linguistics
Sercan Karakas | Yusuf Şimşek
Proceedings of the 15th Workshop on Cognitive Modeling and Computational Linguistics
Plausibility as Commonsense Reasoning: Humans Succeed, Large Language Models Do not
Sercan Karakas
Proceedings of the 15th Workshop on Cognitive Modeling and Computational Linguistics
Sercan Karakas
Proceedings of the 15th Workshop on Cognitive Modeling and Computational Linguistics
Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification
Sercan Karakaş | Yusuf Şimşek
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)
Sercan Karakaş | Yusuf Şimşek
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)
Turkish idiomatic light verb constructions (LVCs) are challenging for multiword expression processing because they often share the same surface form as fully literal verb–object combinations while functioning as a single, partially idiomatic predicate. We frame Turkish LVC detection as a binary classification task (literal meaning vs. idiomatic meaning) and evaluate on a manually created controlled set (N=147) with matched negatives: out-of-domain random sentences and in-domain literal controls (NLVC), alongside LVC positives. We compare a supervised Turkish encoder baseline (BERTurk with a classifier head) to three instruction-tuned LLMs from different families under zero-shot, one-shot, and few-shot prompting, and analyze how demonstrations shift error profiles. In zero-shot, LLMs perform well on negatives but show very low LVC recall. One-shot prompting sharply improves LVC detection but can induce strong, model-specific biases (over- vs. under-predicting LVC). A richer few-shot prompt improves calibration and yields robust overall performance for GPT-OSS-20B and Qwen 2.5-14B. Overall, the results highlight substantial prompt sensitivity in Turkish metalinguistic classification: the supervised baseline remains competitive, while prompted LLMs can match or exceed it on LVCs with carefully constructed demonstrations. We release code, prompts, and evaluation materials to support reproducibility.