Why Shifting to the Object-Oriented Coding Approach REALLY IS Important: Well, I hate having to admit it. Mud on my face; all that. But I made a pretty significant Whoops! back this last winter when I posted a “Verification and Validation” document (hah!) to arXiv. The Sad Story of My Previous Ineptitude Well, there’s nothing like hearing about someone else’s screw-up in order to make us feel better about our own life, so here goes. I’d…
Showing How the Omega Equation is Obtained for the 1-D and 2-D CVM – Part 2: The cluster variation method (CVM) lets us characterize a system in terms of local patterns, and not just the numbers of units in on (A) and off (B) states. It works with a more complex entropy term. The natural question is: How do obtain this more complex entropy? This post continues a discussion begun in the last post, on how we actually get…
The 1-D CVM – A Single Zigzag Chain – Part 1: The cluster variation method (CVM) lets us characterize a system in terms of local patterns, and not just the numbers of units in on/off states. This is likely to be useful for machine learning and AI applications. Up until now, we’ve not told the story of how we actually compute the CVM entropy from the microstates. We’ll do that starting with this post; it will be a handy…
Looking around from a Node’s Point-of-View: As we move from procedural code to object-oriented Python for a Cluster Variation Method (CVM) grid, our perspective shifts. We now need to look at the world from a node’s point-of-view. It’s a lot like updating one’s relationship status in Facebook – except that after updating our own (node) status, we need to not only update the values for all of our own configuration variables, but then we need to travel around the…
The Cluster Variation Method – A Topographic Approach: Object-oriented programming is essential for working with the Cluster Variation Method (CVM), especially if we’re going to insert a CVM layer into a neural network. The reason is that approaching free energy minima via changing node states requires dealing with node, net, and grid topographies. If we’re going to be at all strategic in moving towards free energy minima, then we can’t just pick nodes at random. We need to know…
Working Through Salakhutdinov and Hinton’s “An Efficient Learning Procedure for Deep Boltzmann Machines” We can accomplish a lot, using multiple layers trained with backpropagation. However (as we all know), there are limits to how many layers that we can train at once, if we’re relying strictly on backpropagation (or any other gradient-descent learning rule). This is what stalled out the neural networks community, from the mid-1990’s to the mid-2000’s. The breakthrough came from Hinton and his group, with a…
Roy, A. Park, Y.J. and Pan, S.(2017, Sept. 21). Domain-SpecificWord Embeddings from Sparse Cybersecurity Texts, arXiv 1709.07470v1 [cs.CL]. pdf, accessed Apr. 25, 2018 by A.J.M. Abstract: Classic word embedding methods such as Word2Vec and GloVe work well when they are given a large text corpus. When the input texts are sparse as in many specialized domains (e.g., cybersecurity), these methods often fail to produce high-quality vectors. In this paper, we describe a novel method to train domain-specific word embeddings from…
A Principled Approach to AI: Representations and Transitions: In the last post, on “Moving Between Representation Levels: The Key to an AI System (Part 1),” I re-introduced one of the most important and fundamental AI topics: how we can effectively use multiple representation levels. If we’re going to build (or gauge the properties of) an AI system, we need a framework. The notion of representations, and of moving between representation levels, is as fundamental as we can get. In…
Representation Levels: The Key to Understanding AI “No computation without representation” Jerry Fodor (1975). The Language of Thought, p.34. online access. One of the key notions underlying artificial intelligence (AI) systems is not only that of knowledge representation, but that a good AI system will successively move disparate pieces of low-level, or signal-level information up the abstraction ladder. For example, an image understanding system will have a low-level component that extracts edges and regions from the image (or…
High-Precision Mind-**cking You already know the main storyline: Cambridge Analytica, Brietbart, Facebook, and possible other players. Trump’s win of the electoral vote by about 40,000 votes through carefully targeting not only certain swing states, but micro-elements within those states. The questions now are (for those of us techie folks): (1) Technically, just how did this happen? (We want more than the few words in the mainstream news), and (2) (That which really interests us:) What are the countermeasures? One…