What is 2nd-degree-optimizer-fail-study?

This repository is an honest, openly labeled record of a failed experiment in machine learning research. The author was trying to build a new kind of optimizer, the part of a neural network's training process that decides how much to adjust the model's numbers after each mistake it makes. Most training setups use a fixed or gradually shrinking step size for these adjustments, but this project tried to make the step size adjust itself automatically based on the curvature of the problem, calculated using an advanced mathematical approximation. The README is refreshingly direct about the outcome: after building six different documented versions of this optimizer, and trying roughly five more that were abandoned as failures, the best version still could not beat a standard, well known optimizer called Adam paired with a common learning rate schedule. On the MNIST handwritten digit dataset, the new optimizer reached about ninety eight percent accuracy versus almost ninety nine percent for the standard approach, and on the more difficult CIFAR-10 image dataset, it reached under fifty eight percent accuracy compared to over sixty eight percent for the standard approach. The author offers their own theory for why it did not work: squeezing all of the curvature and gradient information for an entire model down into one single number was likely too much simplification, and suggests that a future attempt would need to calculate an adjustment per individual parameter instead of one shared value for the whole model. The repository is presented as archived and paused rather than actively maintained, and consists mainly of Jupyter notebooks recording the various test runs and results. This is aimed at machine learning researchers and students interested in reading through a transparently documented negative result, not at anyone looking for a working, ready to use optimizer.