After reviewing and updating the theory of energy-conserving Hamiltonian dynamics for optimization and sampling, I'll explain a new application of Energy Conserving Descent (ECD) optimization to precision scientific data analysis in which NN initialization variance has been a bottleneck. Specifically, we choose a particular ECD Hamiltonian whose measure on phase space concentrates the results in a controlled way, and describe a simple prescription for hyperparameter defaults. In a set of experiments on likelihood ratio estimation, using small simulated and real (Aleph) particle physics data sets, we find this reduces the error as predicted, performing better than Adam in this regard.
Time permitting, I'll discuss some separate ideas on machine learning theory, at an early stage of work in progress, including a novel architecture with a major reduction in parameter count based on the Abel-Jacobi map between a two-dimensional Riemann surface with many handles and its high-dimensional Jacobian torus.
