Combining LLMs for Arabic punctuation restoration should help when their errors are complementary, but aggregation failure can reflect either missing coverage, where no predictor proposes the correct label, or failed selection, where the correct label is present but not selected. We investigate this distinction with vote-based, text-blind aggregators that use predictor labels, agreement, position, and neighbouring predictions, but not Arabic words. We evaluate three frontier LLMs and a fine-tuned AraBERT encoder on AraPunc-sub and ABC. The LLMs show strongly correlated errors: when all three are wrong, they choose the same incorrect label in 79.4% of cases on AraPunc-sub and 77.8% on ABC. Yet the four-predictor oracle achieves macro-F₁ scores of 0.857 and 0.778, surpassing the best standalone predictors (0.639 and 0.496). Leave-one-predictor-out analysis shows that AraBERT supplies the largest unique oracle coverage. Standard aggregation recovers only part of this headroom, and no tested aggregator outperforms standalone AraBERT on AraPunc-sub. A selectability probe confirms that some covered disagreements are learnable, but a deployable router offers only modest, corpus-dependent gains. Even a text-aware peer-deliberation step deepens rather than breaks the correlation, as weaker models conform to their peers. Hence, in the tested text-blind regime, the bottleneck is primarily selection rather than coverage, without ruling out text-aware selectors.

Various methods have already been proposed for learning entity embeddings from text descriptions. Such embeddings are commonly used for inferring properties of entities, for recommendation and entity-oriented search, and for injecting background knowledge into neural architectures, among others. Entity embeddings essentially serve as a compact encoding of a similarity relation, but similarity is an inherently multi-faceted notion. By representing entities as single vectors, existing methods leave it to downstream applications to identify these different facets, and to select the most relevant ones. In this paper, we propose a model that instead learns several vectors for each entity, each of which intuitively captures a different aspect of the considered domain. We use a mixture-of-experts formulation to jointly learn these facet-specific embeddings. The individual entity embeddings are learned using a variant of the GloVe model, which has the advantage that we can easily identify which properties are modelled well in which of the learned embeddings. This is exploited by an associated gating network, which uses pre-trained word vectors to encourage the properties that are modelled by a given embedding to be semantically coherent, i.e. to encourage each of the individual embeddings to capture a meaningful facet.

Conceptual spaces are geometric representations of meaning that were proposed by G ̈ardenfors (2000). They share many similarities with the vector space embeddings that are commonly used in natural language processing. However, rather than representing entities in a single vector space, conceptual spaces are usually decomposed into several facets, each of which is then modelled as a relatively low dimensional vector space. Unfortunately, the problem of learning such conceptual spaces has thus far only received limited attention. To address this gap, we analyze how, and to what extent, a given vector space embedding can be decomposed into meaningful facets in an unsupervised fashion. While this problem is highly challenging, we show that useful facets can be discovered by relying on word embeddings to group semantically related features.