Modeling disruptions in tokamaks requires a robust coupling of runaway electron (RE) dynamics with magneto-hydrodynamic (MHD) and collisional radiative models in order to inform ideal mitigation scenarios. This multi-physics coupling of RE kinetics is traditionally prohibitive, while reduced-order fluid models are often fast but ignore key information. We introduce a novel deep learning approach that enables a multi-fidelity treatment of RE evolution within a single framework. This approach combines an adjoint formulation of the relativistic Fokker-Planck equation for RE evolution with a physics-informed neural network (PINN). The resulting framework yields rapid online inference at the cost of expensive offline training, allowing for orders-of-magnitude reduction in prediction time compared with traditional kinetic solvers while retaining a high-fidelity description of RE dynamics. This adjoint-deep learning framework can predict the density moment with full kinetic fidelity, followed by higher-order moments such as the RE current, average energy, and pressure anisotropy. While specific moments necessitate tailored neural network architectures, the use of an adjoint solution allows for predictions of RE moments for arbitrary initial conditions, while the deep learning treatment enables a single model to make predictions across a broad range of physical parameters. Further, the framework enables direct prediction of the energy distribution through a careful treatment of the adjoint solution. Ongoing work includes the development of a fully kinetic RE surrogate by reconstructing the RE momentum space distribution using a Green's function, enabling an autoregressive rollout of the RE distribution function. This capability provides an efficient means of capturing arbitrary time-varying, disruption-relevant plasma parameters where such a surrogate can be directly coupled with disruption codes at marginal cost. The resulting framework enables a trade-off between speed and physics fidelity, allowing the user to adapt the model to the targeted application.