Browsed by
Category: Machine Learning

Deep Learning: The First Layer

Deep Learning: The First Layer

Comments 5 comments

It’s been something of a challenging week. We’ve kicked off our new PREDICT 490 Special Topics Course in Deep Learning at Northwestern University. I’ve got a full house; there’s been a waiting list since Thanksgiving, and everyone is acutely aware of the business climate surrounding deep learning. However (and I’m not terribly surprised here), most people who want to learn Deep Learning (DL) really don’t have a solid foundation in neural networks just yet. Thus, what we’re really doing is…

Read More Read More

Getting Started in Deep Learning

Getting Started in Deep Learning

Comments 5 comments

It’s been a lovely Christmas day. More social than any that I can remember, for a very long time. (Wonderful morning visit with my next-door neighbor. Great mid-day party. A sweet restorative nap afterwards.) And now, the thoughts that have been buzzing through and around my head for the past 48 hours — how to get started with deep learning. Of course there are all sorts of entry points. Historical, functional, mathematical… But what came to me, over these past…

Read More Read More

Approximate Bayesian Inference

Approximate Bayesian Inference

Comments 7 comments

Variational Free Energy I spent some time trying to figure out the derivation for the variational free energy, as expressed in some of Friston’s papers (see citations below). While I made an intuitive justification, I just found this derivation (Kokkinos; see the reference and link below): Other discussions about variational free energy: Whereas maximum a posteriori methods optimize a point estimate of the parameters, in ensemble learning an ensemble is optimized, so that it approximates the entire posterior probability distribution…

Read More Read More

Brain-Based Computing: Foundation for Deep Learning

Brain-Based Computing: Foundation for Deep Learning

Comments 3 comments

Three Key Brain Strategies Used in Deep Learning for Artificial Intelligence   References for Brain-Based Computing (Methodologies for Deep Learning and Artificial Intelligence) Maren, A.J. (2015) How the Brain Solves Tough Problems. In Making Sense: Extracting Meaning from Text by Matching Entities and Terms to Ontologies and Concepts, Chapter 2 Draft. (Dec. 31, 2015). pdf Maren, A.J. (2015). Brain-Based Computing. (PPT Slidedeck) PPT  

The Cluster Variation Method: A Primer for Neuroscientists

The Cluster Variation Method: A Primer for Neuroscientists

Comments 1 comment

Single-Parameter Analytic Solution for Modeling Local Pattern Distributions The cluster variation method (CVM) offers a means for the characterization of both 1-D and 2-D local pattern distributions. The paper referenced at the end of this post provides neuroscientists and BCI researchers with a CVM tutorial that will help them to understand how the CVM statistical thermodynamics formulation can model 1-D and 2-D pattern distributions expressing structural and functional dynamics in the brain. The equilibrium distribution of local patterns, or configuration…

Read More Read More

Novelty Detection in Text Corpora

Novelty Detection in Text Corpora

Comments 0 Comment

Detecting Novelty Using Text Analytics Detecting novel events – new words, meaning new events – is one of the most important text analytics tasks, and is an important step towards predictive analytics using text mining. On July 24, 2015, The New York Times (and many other news sources) published an article identifying potential inclusion of classified information in the emails which Hillary Clinton had sent via private email and stored on her private email server. How would we use text…

Read More Read More

The 1-D Cluster Variation Method (CVM) – Simple Application

The 1-D Cluster Variation Method (CVM) – Simple Application

Comments 2 comments

The 1-D Cluster Variation Method – Application to Text Mining and Data Mining There are three particularly good reasons for us to look at the Cluster Variation Method (CVM) as an alternative means of understanding the information in a system: The CVM captures local pattern distributions (for an equilibrium state), When the system is made up of equal numbers of units in each of two states, and the enthalpy for each state is the same (the simple unit activation energy…

Read More Read More

Big Data, Big Graphs, and Graph Theory: Tools and Methods

Big Data, Big Graphs, and Graph Theory: Tools and Methods

Comments 0 Comment

Big Graphs Need Specialized Data Storage and Computational Methods {A Working Blogpost – Notes for research & study} Processing large-scale graph data: A guide to current technology, by Sherif Sakr (ssakr@cse.unsw.edu.au), IBM Developer Works (10 June 2013). Note: Dr. Sherif Sakr is a senior research scientist in the Software Systems Group at National ICT Australia (NICTA), Sydney, Australia. He is also a conjoint senior lecturer in the School of Computer Science and Engineering at University of New South Wales. He…

Read More Read More

Modeling the Future: Tools from Complex Systems

Modeling the Future: Tools from Complex Systems

Comments 0 Comment

2012 and Beyond: Tools for Predicting the Next Sixty Years Is the world coming to an end on Dec. 21st, 2012, or not? Very likely, not. We’ll still wake up in the morning, in the same beds in which we went to sleep in the night before. We’ll still walk out to our cars, or get to our Metro stations, on time. And we’ll likely stop for the same “cup of joe” on the way to work. But will our…

Read More Read More

Gibbs Free Energy, Belief Propagation, and Markov Random Fields

Gibbs Free Energy, Belief Propagation, and Markov Random Fields

Comments 0 Comment

Correspondence Between Free Energy, Belief Propagation, and Markov Random Field Models As a slight digression from previous posts – re-reading the paper by Yedidia et al. on this morning on Understanding Belief Propagation and its Generalizations – which explains the close connection between Belief Propagation (BP) methods and the Bethe approximation (a more generalized version of the simple bistate Ising model that I’ve been using) in statistical thermodynamics. The important point that Yedidia et al. make is that their work…

Read More Read More