TPTU: Task Planning and Tool Usage of Large Language Model-Based AI Agents

Jingqing Ruan, Yihong Chen, Bin Zhang, Zhiwei Xu, Tianpeng Bao, Guoqing Du, Shiwei Shi, Hangyu Mao, Ziyue Li, Xingyu Zeng, Rui Zhao

NIPS 2023 Workshop Project

We propose a structured framework tailored for LLM-based AI Agents and discuss the crucial capabilities necessary for tackling intricate problems.

Benchmarking the Text-to-SQL Capability of Large Language Models: A Comprehensive Evaluation

Bin Zhang, Yuxiao Ye, Guoqing Du, Xiaoru Hu, Zhishuai Li, Sun Yang, Chi Harold Liu, Rui Zhao, Ziyue Li, Hangyu Mao

CASE 2026 Project

We evaluate LLMs through five Text-to-SQL related tasks, reveal performance differences and suggest task-specific optimization strategies.

TrustFlow: Adaptive Trust Calibration for Language Model Guided Reinforcement Learning

Hao Chen, Zhiwei Xu, Hu Fu, Yuelin Ma, Hangyu Mao, Yiqun Chen, Guoliang Fan, Bin Zhang.

NeurIPS 2026

We propose adaptive trust calibration for language-model-guided reinforcement learning, estimating when the agent should follow the language model’s guidance and when it should rely on its own policy.

Air-Branching: Learning to Branch via Adversarial Inverse Reinforcement Learning in Mixed-Integer Linear Programming

Ce Zhang, Bin Zhang, Guoliang Fan.

ESWA

We learn branching variable selection for mixed-integer linear programming with adversarial inverse reinforcement learning, using a two-stage warm-up plus joint-reward scheme that lowers the demand for expert demonstrations and surpasses both imitation and plain reinforcement learning baselines.

Subgoal Graph-Augmented Planning for LLM-Guided Open-World Reinforcement Learning

Shanwei Fan, Bin Zhang, Zhiwei Xu, Yingxuan Teng, Siqi Dai, Lin Cheng, Guoliang Fan.

ICONIP 2026

We propose Subgoal Graph-Augmented Actor-Critic-Refiner (SGA-ACR), which grounds LLM subgoal generation in an environment-specific subgoal graph and structured entity knowledge and splits planning across several LLM roles, closing the gap between plausible plans and feasible behaviour.

LeanRefiner: Agentic Global-to-Local Optimization of Lean Proofs

Tian Cui, Bin Zhang, Changwei Wang, Zhiwei Xu, Zeyang Liu.

AI for Math Workshop @ ICML 2026

We optimise already-verified Lean proofs with a verifier-guided agentic framework that alternates global theorem-level restructuring with local pattern-guided reduction, shortening proofs while the Lean verifier guarantees the same theorem is still proved.

Controlling Large Language Model-based Agents for Large-Scale Decision-Making: An Actor-Critic Approach

Bin Zhang, Hangyu Mao, Jingqing Ruan, et al.

CASE 2026

We develop LLaMAC, a modular actor-critic framework that controls LLM-based agents for large-scale decision-making, curbing hallucination and coordinating many agents while keeping token consumption tractable.

Beyond Local Views: Global State Inference with Diffusion Models for Cooperative MARL

Zhiwei Xu, Hangyu Mao, Nianmin Zhang, Shengtao Zhang, Xin Xin, Pengjie Ren, Dapeng Li, Bin Zhang, Guoliang Fan, Zhumin Chen, Changwei Wang, Jiangjin Yin

RLC 2026

We propose State Inference with Diffusion Models (SIDIFF), which reconstructs the global state from local observations alone so that agents can act on both the inferred global state and their own view under partial observability.

From Traits to Roles: Consensus-Guided Composition of Orthogonal Experts for Cooperative MARL

Yewei Zhou, Bin Zhang, Ying Zhou, Xuri Ge, Dapeng Li, Hangyu Mao, Pengjie Ren, Zhiwei Xu

IJCAI 2026

We propose DECOR, which models agent roles as dynamic compositions of orthogonal behavioural traits and uses an orthogonal mixture-of-experts to avoid destructive gradient interference in heterogeneous cooperative tasks.

QSIM: Mitigating Overestimation in Multi-Agent Reinforcement Learning via Action Similarity Weighted Q-Learning

Yuanjun Li, Bin Zhang, Hao Chen, Zhouyang Jiang, Dapeng Li, Zhiwei Xu

ICAPS 2026

We propose QSIM, which rebuilds the temporal-difference target with action-similarity weighting instead of the greedy joint action, mitigating the Q-value overestimation that the max operator causes in cooperative MARL.

Peak-Return Greedy Slicing: Subtrajectory Selection for Transformer-based Offline RL

Zhiwei Xu, Miduo Cui, Dapeng Li, Zhihao Liu, Haifeng Zhang, Hangyu Mao, Guoliang Fan, Bin Zhang

ICLR 2026

We propose Peak-Return Greedy Slicing, which estimates per-step returns with a maximum mean discrepancy estimator and greedily extracts high-return subtrajectories for Transformer training, with adaptive history truncation at inference to match the training scheme.

Towards Better Branching Policies: Leveraging the Sequential Nature of Branch-and-Bound Tree

Ce Zhang, Bin Zhang, Guoliang Fan

ICLR 2026

We propose Mamba-Branching, a branching policy that uses the Mamba state-space model to capture long sequential dependencies across branch-and-bound steps, with contrastive pre-training of variable embeddings, outperforming prior neural branching policies on real MILP instances.

Quality-Diversity for Multi-Agent Reinforcement Learning

Hao Chen, Pengyi Li, Bin Zhang, Hu Fu, Zhiwei Xu, Ce Zhang, Xinyue Lu, Guoliang Fan

AAMAS 2026

We propose MIQD, a mutual-information-enhanced quality-diversity framework that feeds fragment-based behavioural descriptors into the critic, replacing random exploration with informed diversity for more stable coverage in cooperative MARL.

Graph of Verification: Structured Verification of LLM Reasoning with Directed Acyclic Graphs

Jiwei Fang, Bin Zhang, Changwei Wang, Jin Wan, Zhiwei Xu

AAAI 2026

We propose the Graph of Verification (GoV), a node-block graph that adapts to different reasoning structures and supports multi-granular, localised verification of LLM reasoning, from formal proofs to informal narratives.

Reidentify: Context-Aware Identity Generation for Contextual Multi-Agent Reinforcement Learning

Zhiwei Xu, Kun Hu, Xin Xin, Weiliang Meng, Yiwei Shi, Hangyu Mao, Bin Zhang, Dapeng Li, Jiangjin Yin

ICML 2025

We propose Context-Aware Identity Generation (CAID), a causal Transformer decoder that produces agent identities aligning corresponding agents across problem variants, together with an action regulator that injects them into the action-value space for better policy reuse and generalisation.

Enhancing Branching Policy Generalization through Self-Supervised Adversarial Instance Augmentation

Ce Zhang, Bin Zhang, Guoliang Fan

IJCNN 2025

We introduce SATA, a self-supervised adversarial instance augmentation method that generates diverse training instances to improve the generalisation of learned branching variable selection policies in branch-and-bound.

QPO: Query-dependent Prompt Optimization via Multi-Loop Offline Reinforcement Learning

Yilun Kong, Hangyu Mao, Zhao Qi, Bin Zhang, Jingqing Ruan, Li Shen, Yongzhe Chang, Xueqian Wang, Rui Zhao, Dacheng Tao

TMLR

We introduce Query-dependent Prompt Optimization (QPO), which uses multi-loop offline reinforcement learning to fine-tune a small pretrained language model that generates prompts tailored to each input query, avoiding costly online interaction with the target LLM.

PET-SQL: A Prompt-enhanced Two-stage Text-to-SQL Framework with Cross-consistency

Zhishuai Li, Xiang Wang, Jingjing Zhao, Sun Yang, Guoqing Du, Xiaoru Hu, Bin Zhang, Yuxiao Ye, Ziyue Li, Rui Zhao, Hangyu Mao

DASFAA 2025

We present a two-stage text-to-SQL framework that prompts LLMs with a reference-enhanced representation of the schema and sampled cell values, then applies cross-consistency over the generated queries to cope with verbose databases and complex user intentions.

Unveiling Decision Intention for Cooperative Multi-Agent Reinforcement Learning

Zeren Zhang, Zhiwei Xu, Guangchong Zhou, Dapeng Li, Bin Zhang, Guoliang Fan

AAMAS 2025

We generate the decision intentions of agents with a diffusion model, first selecting task-crucial states through a polarisation index, so that humans can understand what a cooperative agent intends to do and agent strategies become more transparent.

Efficient Communication in Multi-Agent Reinforcement Learning with Implicit Consensus Generation

Dapeng Li, Na Lou, Zhiwei Xu, Bin Zhang, Guoliang Fan

AAAI 2025

We combine implicit consensus generation with efficient communication so that agents exchange only what is needed to reduce uncertainty about their teammates, significantly lowering communication overhead while preserving collaboration.

Stackelberg Decision Transformer for Asynchronous Action Coordination in Multi-Agent Systems

Bin Zhang, Hangyu Mao, Lijuan Li, Zhiwei Xu, Dapeng Li, Rui Zhao, and Guoliang Fan

ICML 2024

We propose the Stackelberg Decision Transformer (STEER), which handles hierarchical coordination among agents in both spatial and temporal contexts and converges to Stackelberg equilibrium solutions.

PTDE: Personalized Training with Distilled Execution for Multi-Agent Reinforcement Learning

Yiqun Chen, Hangyu Mao, Jiaxin Mao, Shiguang Wu, Tianle Zhang, Bin Zhang, Wei Yang, Hongxing Chang

IJCAI 2024

We propose Personalized Training with Distilled Execution (PTDE), which personalises global information for each agent and distils it into the agent’s local information, showing that one shared global signal is not enough for optimal performance.

Controlling Large Language Model-based Agents for Large-Scale Decision-Making: An Actor-Critic Approach

Bin Zhang, Hangyu Mao, Jingqing Ruan, et al.

ICLR 2024 Workshop on LLM Agents

We develop LLaMAC, a modular actor-critic framework that controls LLM-based agents for large-scale decision-making, curbing hallucination and coordinating many agents while keeping token consumption tractable.

Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems

Yilun Kong, Jingqing Ruan, Yihong Chen, Bin Zhang, et al.

EMNLP 2024 Industry Track

We introduce a comprehensive framework that strengthens the task planning and tool usage abilities of LLM-based agents in real-world systems, tackling the huge number of available APIs, complex multi-step tasks and prompt length limits.

SGCD: Subgroup Contribution Decomposition for Multi-Agent Reinforcement Learning

Hao Chen, Bin Zhang, Guoliang Fan.

IJCNN 2024

We propose SGCD, a subgroup contribution decomposition method that introduces subgroups and inner subgroups and distributes contributions with Shapley values, giving a more compact and flexible view of team structure.

PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning

Hangyu Mao, Rui Zhao, Ziyue Li, Zhiwei Xu, Hao Chen, Yiqun Chen, Bin Zhang, et al.

AAMAS 2024

We propose the Perception and Decision-making Interleaving Transformer (PDiT), which cascades a patch-level perceiving Transformer with a deciding Transformer conditioned on returns, perception output and past actions, and also yields explainable attention patterns.

From Explicit Communication to Tacit Cooperation: A Novel Paradigm for Cooperative MARL

Dapeng Li, Zhiwei Xu, Bin Zhang, and Guoliang Fan.

AAMAS Extended Abstract 2024

We propose a paradigm that gradually shifts from explicit communication during early training to tacit cooperation at execution, keeping the benefit of shared signals without the practicality cost of continuous communication.

Adaptive Parameter Sharing for Multi-Agent Reinforcement Learning

Dapeng Li, Na Lou, Bin Zhang, Zhiwei Xu, Guoliang Fan

ICASSP 2024

Inspired by functional specialisation in the brain, we propose a parameter sharing method that maps each type of agent to different regions of a shared network, producing differentiated subnetworks without losing scalability.

Decentralized Extension for Centralized Multi-Agent Reinforcement Learning via Online Distillation

Zeren Zhang, Bin Zhang, Guangchong Zhou, Dapeng Li, Zhiwei Xu and Guoliang Fan

ICONIP 2024

We introduce OLEN, an online distillation framework that transfers knowledge from a high-performing centralised MARL algorithm into fully decentralised policies during training, making it compatible with any centralised method.

GATE: Guided Contrastive State Space for Multi-Agent Reinforcement Learning

Hao Chen, Bin Zhang and Guoliang Fan

ICONIP 2024

We propose GATE, a guided contrastive state space method that uses contrastive learning to steer agents towards superior states and away from inferior ones, making better use of experience and accelerating convergence.

Dual Self-Awareness Value Decomposition Framework without Individual Global Max for Cooperative MARL

Zhiwei Xu, Bin Zhang, Dapeng Li, Guangchong Zhou, Zeren Zhang, Guoliang Fan

NeurIPS 2023

We propose a dual self-awareness value decomposition framework that fully rejects the individual-global-max premise, pairing an ego policy for action selection with an alter-ego value function for credit assignment.

Inducing Stackelberg Equilibrium through Spatio-Temporal Sequential Decision-Making in Multi-Agent Reinforcement Learning

Bin Zhang, Lijuan Li, Zhiwei Xu, Dapeng Li, Guoliang Fan

IJCAI 2023

We build a spatio-temporal sequential decision-making structure from the Markov game and derive an N-level policy model that induces Stackelberg equilibrium through asynchronous action coordination.

SEA: A Spatially Explicit Architecture for Multi-Agent Reinforcement Learning

Dapeng Li, Zhiwei Xu, Bin Zhang, Guoliang Fan

IJCNN 2023

We propose a spatially explicit architecture that lets agents share neighbourhood and global information through an encoder-decoder built on spatial positions, inspired by point-cloud processing, and it drops into existing MARL methods.

Consensus Learning for Cooperative Multi-Agent Reinforcement Learning

Zhiwei Xu, Bin Zhang, Dapeng Li, Zeren Zhang, Guangchong Zhou, Hao Chen, Guoliang Fan

AAAI 2023

Inspired by viewpoint invariance and contrastive learning, we propose consensus learning, in which agents infer the same discrete consensus from local observations, restoring a shared signal that is missing during decentralised execution.

HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism

Zhiwei Xu, Yunpeng Bai, Bin Zhang, Dapeng Li, and Guoliang Fan

AAAI 2023

We propose HAVEN, a hierarchical value decomposition framework with a dual coordination mechanism that regularises both inter-level and inter-agent policies to stabilise concurrent optimisation in fully cooperative tasks.

Mingling Foresight with Imagination: Model-Based Cooperative Multi-Agent Reinforcement Learning

Zhiwei Xu, Dapeng Li, Bin Zhang, Yuan Zhan, Yunpeng Bai, and Guoliang Fan

NeurIPS 2022

We propose an implicit model-based cooperative MARL method built on value decomposition, letting agents interact with a learned virtual environment to evaluate actions and easing the compounding model error that plagues multi-agent model learning.

Multi-Agent Hyper-Attention Policy Optimization

Bin Zhang, Zhiwei Xu, Yiqun Chen, Dapeng Li, Yunpeng Bai, Guoliang Fan, and Lijuan Li

ICONIP 2022

We propose a state-conditioned hyper-attention game abstraction that helps the centralised critic integrate the most relevant information from complex agent interactions, and add a point-probability-distance penalty to stabilise trust-region updates.

Efficient Policy Generation in Multi-Agent Systems via Hypergraph Neural Network

Bin Zhang, Yunpeng Bai, Zhiwei Xu, Dapeng Li, and Guoliang Fan

ICONIP 2022

We adaptively construct a hypergraph over agents to represent their interactions and extract neighbourhood-aware representations, presenting two actor-critic variants (HGAC and ATT-HGAC) that improve cooperation among many agents.

Cooperative Multi-agent Reinforcement Learning with Hypergraph Convolution

Yunpeng Bai, Chen Gong, Bin Zhang, Guoliang Fan, Xinwen Hou, Yu Liu

IJCNN 2022

We propose HGCN-MIX, which models agents and their relationships as a hypergraph and combines hypergraph convolution with value decomposition, alleviating the lazy-agent problem in cooperative tasks.

SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learning

Zhiwei Xu, Yunpeng Bai, Dapeng Li, Bin Zhang, and Guoliang Fan

AAMAS 2022

We propose SIDE, a value decomposition framework that removes the need for the global state by solving optimal control and state inference jointly, so training works from incomplete and noisy observations.

Learning to Coordinate via Multiple Graph Neural Networks

Zhiwei Xu, Bin Zhang, Yunpeng Bai, Dapeng Li, and Guoliang Fan

ICONIP 2022

We introduce MGAN, which combines graph convolutional networks with value decomposition and learns agent representations from multiple perspectives, allocating attention among agents for better credit assignment.