FuseReg: Regularizing Layer Fusion Mitigates the
Representation autoencoders must pick which pretrained encoder layers feed both the pixel decoder and the generator, a choice that trades fine detail for generation quality. FuseReg trains on random layer subsets, yielding a decoder that handles full, sparse, and single‑layer fusions and improves PSNR, while decoder swapping cuts unguided gFID by 27% and joint regularization lowers it another 29% on diffusion models.