Author contact: yan.yang.research at proton.me This technical report presents a diagnostic study on differentiable synthesizer optimization. It consists of two closely related but distinct parts. Part I examines the rationale for using downstream-task feedback to optimize synthetic data genera- tors. We revisit the motivation originating from Klein et al. 17, 19, analyze its empirical limitations, and discuss supporting evidence from Meta-Sim and Task2Sim. We then introduce an EM-style framework that alternates between trainer and synthesizer optimization. Part II investigates backpropagation through reaction–diffusion systems, a concrete and challenging instantiation of the differentiable synthesizer that was explored during the development of the EM-style framework. We provide theoretical details of several approaches (truncated BPTT with intermediate supervision, implicit differentiation, adjoint method, and Krylov solver), present an empirical diagnosis of the loss landscape on a Gray-Scott system, and outline renewed solution directions. Code and experimental materials are available at https://github.com/Yan-Yang-bot/bp2renderer.
Yan Yang (Fri,) studied this question.