Isabella Luong

Dynamic multi-turn benchmark evaluating frontier models on animal welfare reasoning under escalating adversarial pressure. Targeting submission to Inspect Evals (UK AISI); outputs feed into model specifications and training interventions at frontier labs.

• Engineered an LLM-based scenario generation pipeline following ARENA dataset quality control standards, adapting ARENA's MCQ-oriented framework to open-ended adversarial welfare scenarios; implemented iterative rubric design for automated quality filtering, integrated manually curated few-shot exemplars to control output style and length distribution, and validated ~300 generated scenarios against benchmark contribution criteria
• Refactored the dataset generation pipeline for reproducibility and robustness, eliminating auto-generated few-shot prompting in favor of curated exemplars, structuring multi-file pipeline outputs for reproducibility, and distilling community feedback into concrete generation constraints that measurably reduced eval-aware and formulaic scenario outputs
• Identified and designed mitigation experiments for the comparability problem in dynamic multi-turn evaluation — where model-conditioned adversarial follow-ups cause non-equivalent pressure testing across models, confounding cross-model scoring; proposed and tested two solutions: a hybrid fixed-then-dynamic turn design providing a standardized baseline pressure before resuming adaptive generation, and controlled pressure injection ensuring welfare reasoning is consistently targeted regardless of Turn 1 model response

End-to-end detection system for cross-session malicious model misuse — targeting adversarial actors who decompose attack queries across multiple API sessions to evade per-session safety classifiers. Targeting publication at USENIX Security, NeurIPS D&B, and ICLR.

• Building cross-session monitoring infrastructure — session tracker and code embedding pipeline with vector DB to semantically link malicious fragments
• Constructing dependency graphs with a cross-session linker and subgraph extraction module, surfacing latent attack structures invisible to single-session classifiers
• Developing a GNN detection architecture with adversarial training loops (5–10 rounds) and explainability-focused outputs generating structured attack explanations
• Designing FragBench — a standardized benchmark suite with baselines and leaderboard for cross-session attack detection evaluation

Three interconnected empirical projects forming a systematic examination of benchmark and evaluation failure in frontier AI systems — with implications for scalable oversight and reward robustness.

• Characterizing trajectory-level behavioral drift under adversarial input pressure — identifying failure modes that standard single-pass evaluations systematically miss
• Empirically measuring how stylistic features (verbosity, hedging, formatting) distort LLM-as-judge scores independent of answer quality
• Investigating the absence of reliable ground truth in nonhuman welfare reasoning evaluations — proposing criteria for scalable oversight under partial verifiability