The Precis for the book-in-progress, “Statistical Mechanics, Neural Networks, and Machine Learning,” covering the Seven Essential Machine Learning Equations.
When the Subtle Art of Mathematical Notation Defeats You (and How to Fight Back) A couple of years ago, I was teaching Time Series and Forecasting for the first time. I didn’t know the subject – at all – but that didn’t bother me. Hey, it was mathematics, right? Anything that’s mathematical, I can eat for lunch, and then want some dessert-equations afterwards. First week, introducing the subject. That went fine. Second week, Simple Exponential Smoothing (SES). That’s simple….
Making Machine Learning As Simple As Possible Albert Einstein is credited with saying, Everything should be made as simple as possible, but not simpler. Machine learning is not simple. In fact, once you get beyond the simple “building blocks” approach of stacking things higher and deeper (sometimes made all too easy with advanced deep learning packages), you are in the midst of some complex stuff. However, it does not need to be more complex than it has to be. …
Probabilities: Statistical Mechanics and Bayesian: Machine learning fuses several different lines of thought, including statistical mechanics, Bayesian probability theory, and neural networks. There are two different ways of thinking about probability in machine learning; one comes from statistical mechanics, and the other from Bayesian logic. Both are important. They are also very different. While these two different ways of thinking about probability are usually very separate, they come together in some of the more advanced machine learning topics, such…
Machine Learning Is Different Now: Actually, machine learning is a continuation of what it always has been, which is deeply rooted in statistical physics (statistical mechanics). It’s just that there’s a culmination of insights that are now a very substantive body of work, with more theoretical rigor behind them than most of us know. A Lesson from Mom: It takes a lot of time to learn a new discipline. This is something that I learned from my…
Essential Statistical Mechanics for Deep Learning If you’re self-studying machine learning, and feel that statistical mechanics is suddenly showing up more than it used to, you’re not alone. Within the past couple of years, statistical mechanics (statistical thermodynamics) has become a more integral topic, along with the Kullback-Leibler divergence measure and several inference methods for machine learning, including the expectation maximization (EM) algorithm along with variational Bayes. Statistical mechanics has always played a strong role in machine…