Iordanis Fostiropoulos

A benchmark for evaluating LLMs' ability to understand users from their interaction histories in recommendation systems. We propose two metric families (Interest Groundedness and Interest Specificity) and evaluate eight open-weight LLMs (7B to 120B), revealing performance bottlenecks in counting and attributing engagement signals across heterogeneous interactions.

Personalized text-to-image generation centered on human subjects. We introduce a 20,000-prompt dataset, an evaluation framework with metrics aligned to human judgment, and a baseline that achieves SOTA without per-subject fine-tuning.

A multi-modal General Continual Learning benchmark that constructs long task sequences with controlled learning-gap between tasks. We introduce αMetaSup, a Transformer-based novelty detector trained on a dummy stream, achieving up to 10.5% AUC over prior baselines.

Previous work evaluates the reasoning of Language Models (LMs) using tests that can be inapplicable to a LM. We propose an unbiased evaluation setting using complex regular expressions composed of Quasi-Natural Language. We evaluate the inductive reasoning ability of an LM to generate Facts that abide by the Rules. When a Rule is injected in the training sequence, the LM learns to associate Facts with Rules implicitly; when probed with Inductive In-Context Learning, it generates probable Facts.

Ablation experiments require many trials per ML-model component. We propose ABLATOR, a stateful experiment-design framework that scales a single experiment to thousands of trials while remaining fault-tolerant. We performed the largest ablation experiment for tabular Transformers to date, evaluating 2,337 models, and open-source the framework.

Errors in the ablation setup can lead to incorrect explanations of which method components contribute to performance. We quantify the selection bias of HPO strategies and show that only random sampling produces reliable conclusions about the top and mean performance of a method under a limited compute budget.

We incrementally learn new tasks for a base model using multiple learners in a distributed fashion, consolidating their knowledge at large incremental steps. Larger consolidation steps reduce catastrophic forgetting compared to smaller steps. Simpler methods outperform state-of-the-art in our challenging Stream Benchmark.

A Shared Knowledge Lifelong Learning (SKILL) challenge: a decentralized population of LL agents each sequentially learn different tasks, then share and consolidate knowledge over a decentralized communication network so all agents end up mastering all tasks.

A Vector Quantization method for latent features that improves image reconstruction. Decomposing features in latent space and learning an auto-encoder end-to-end implicitly decouples features and improves reconstruction.

The quadratic complexity of self-attention limits Transformer deployment on low-resource devices. We propose multimodal Sparse Phased Transformer (SPT) to alleviate self-attention's complexity and memory footprint.

We learn representations of persistence diagrams on the Poincaré ball. By placing infinite-persistence features infinitesimally close to the boundary, their distance to non-essential features approaches infinity, preserving relative importance.