MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulation

Deploying Vision‑Language‑Action models on robots stalls because dense large language models consume too much compute, and early‑exit sparsifiers often discard the final layers that carry essential semantics. The authors set out to keep those semantics while still skipping unnecessary work. Their goal is a model that activates only the layers needed for the robot’s current situation.

They treat each LLM layer as an expert in a mixture‑of‑experts framework and build a Spatial‑Temporal Aware Router (STAR) that decides, on the fly, which layer blocks to run. STAR looks at the robot’s visual input and state history, then routes the signal through a subset of layers, mimicking brain pathways that separate cognition from causal reasoning. This dynamic selection replaces the static early‑exit approach used before.

Because skipping layers removes some of the model’s reasoning power, the authors add a Cognitive self‑Knowledge Distillation (CogKD) step. CogKD trains the active layers to reproduce the cognition features of the full model, effectively teaching the reduced network to understand task demands and generate appropriate action sequences. The distillation loss ties the output of the active subset to the full‑model representation, preserving semantic depth.

In RLBench simulations and real‑world tests across ten tasks, MoLe‑VLA raises the mean success rate by 9.7 % compared with the OpenVLA baseline. At the same time, inference speed improves by 36.8 %, showing that dynamic layer skipping can deliver both higher performance and lower latency.

TakeawayDynamic routing and distillation let large language models run faster on robots without losing task understanding.