Lama Al-Shammari

Artificial Intelligence Specialization · Cohort 2026

When Should an LLM Listen to Another LLM? Predicting Beneficial Critique Acceptance in Multi-LLM Collaboration

This project investigates when one Large Language Model (LLM) should trust the critique of another LLM. Using GSM8K mathematical reasoning problems, we developed a Solver–Critic framework with Qwen2.5-3B-Instruct and Llama-3.2-3B-Instruct operating in both directions. The Critic first solves each problem independently, then evaluates the Solver’s final answer, and revision is triggered only when disagreement occurs. We extract only pre-revision features and train a lightweight logistic regression classifier to predict whether accepting a revision is likely to improve the answer. Our experiments show that critique is not always beneficial and that the Solver–Critic direction is a stronger and more consistent predictor of beneficial revision than the critique-content features examined. The project provides a reproducible framework for selective critique acceptance in multi-LLM collaboration.

Read the final report (PDF)