The Cluster Variation Method – A Topographic Approach:

 

Object-oriented programming is essential for working with the Cluster Variation Method (CVM), especially if we’re going to insert a CVM layer into a neural network. The reason is that approaching free energy minima via changing node states requires dealing with node, net, and grid topographies.

If we’re going to be at all strategic in moving towards free energy minima, then we can’t just pick nodes at random. We need to know in advance if flipping any two nodes (one going from on to off, the other from off to on) will benefit us. This means that we need to know something about the topography of a candidate node. Is it an on node in the center of a “continent” of on nodes? Is it at the boundary? Is it an island unto itself, floating in a sea of off nodes?

Yes, we’ll still have a random element of pick-and-choose; a little bit of a stochastic process. At the same time, we need to be guided by strategy, otherwise we’ll spend too much time trying things that simply won’t work.

This means that we need to associate higher-level information with each node. In addition to representing individual nodes, we also need to build objects that represent the various blobs and globs in the topography; the nets. Further, we need describe the entire grid; we need to know whether or not we have continents, or a series of island archipelagos … we simply need to know what we’ve got. (See the next post in this series for more on nodes, nets, and grids.)

All of this means that we need object-oriented code; specifically object-oriented Python. (Because I refuse to learn any other language right now.)

 

 

Object-Oriented Programming for the Cluster Variation Method: Redux

 

Many, MANY years ago, I had a research grant to support a student doing code development over the summer. After several weeks, he said, “I need to rebuild the entire code, and go from procedural C to the object-oriented C++.”

And so, that’s what he did.

I spent the next three weeks twiddling my thumbs (and writing research proposals) while he rebuilt that code, step-by-step, function-by-function.

Over the past several months, I’ve realized that I need to do the same. The reason is that when we use the Cluster Variation Method (CVM) for anything, we essentially have to do topographic analysis.

 

 

The Simplest Possible 1-D CVM

 

Most of us are still wrapping our heads around the whole notion of a CVM grid. (See links to previous (and one future) blogpost at the end of this post, along with links to useful papers.)

Suffice to say, the following Figure 1 shows the simplest possible 1-D CVM grid. Key points:

1. We need minimally eight nodes; four on and four off, in order to create at least one instance of each of the six different kinds of triplets (two of which need two versions each, one where the pattern is read left-to-right, and the other read right-to-left), and
2. The 1-D CVM is developed as a single zig-zag chain; that’s why we need two rows.

I’ve used this figure to introduce the 1-D CVM in several locations. (See links to papers and previous blogs at the end of this post for more detailed explanations.)

All of my CVM grid illustrations have wrap-arounds. For the 1-D CVM (a single zig-zag chain), the wrap-around is simply in the horizontal dimension.

Notice: Figure 1 shows the wrap-around of the first nodes (the left-most ones) again on the right-hand-side; they are in mottled grey. That’s why you’re seeing ten nodes instead of the minimal eight; the two on the right are just there so that you can visualize the triplet patterns easier.

 

 

Tutorial Code: Expressing the 1-D CVM Using Object-Oriented Python

 

Today’s grand adventure was to do this simplest possible, tutorial-level instance of object-oriented CVM code. I’ve stayed with the 1-D CVM, and simply replicated the configuration found in Figure 1 (above) within the code. This is shown in Figure 2.

Actually, Figure 2 has three instances of the same base pattern; this gives us enough instances of each local configuration so that we can “swap” two nodes (one going from on to off, the other from off to on), and still have at least one instance of each of the z triplet configurations left. (This is necessary in order to have a computable entropy term.) Because we have three instances of the base pattern, and not just two, we can do several swaps before we run into trouble.

Here’s the GitHub link:

• GitHub for 1-D Object-Oriented Python Snippets

Update on Sunday, Sept. 23 – this is better code:

• 1D-CVM_OO_basic-config-vars_V-and-V_1-1_2018-09-23.py

The code so far just has the w (next-nearest-neighbor) configurations computed; these are in-line with the same row in which each node appears. The next round of code will contain the remaining y (nearest-neighbor) and z (triplet) configuration variables, as well as the thermodynamic variables for the entire grid. What this code gives us is a starter piece, showing how the code will work as we compute configuration variables using an OO basis.

There’s a PPT documentation in the same GitHub repository:

• 1D-CVM_Object-oriented-code_V-and-V_2018-10-10-18.pptx

 
Here’s what the code does:

• Creates the class “node” and instantiates it for two rows, each expressed as a list, where the activation values correspond to those shown in the figure (note that there are three instances of the base pattern, not just two),
• Computes the left- and right-next-nearest-neighbor values for each node (the w values), and stores them as attributes for each node,
• Prints the node values, both in column form (showing the attributes), and also as a 1-D CVM grid,
• Allows you, the user, to select two nodes to swap (one should be on, the other off),
• Determines if your selected nodes are an allowable swap, and if allowable, makes the swap, and
• Prints out the pattern of the new configuration.

This is very simple code; there are no linked lists, there’s no computation of the nearest-neighbors (those require looking at node activations on the row above / below a given node), nor is there computation of the z-values (triplets). There’s no thermodynamics. No entropy, no free energy minimization.

All that this does, at this stage, is just give us the basic node class, fill in some attributes, and display the nodes. Also, it gives us the chance to do a swap. It’s a step in the direction of object-oriented Python for the CVM.

 

 

Where We’ll Go with This

 

As we start doing anything interesting with CVMs – particularly flipping nodes in order to decrease the free energy of a system – we’ll soon start thinking in terms of topologies. That is, we’ll see islands of activation (on or “black” or A nodes) floating in seas of no-activation (off or “white” or B nodes).

We’ll want to stop just random node-flipping, and start being more strategic.

This means that we’ll have to think about the shapes of the islands – all sorts of things that are not currently on our agenda. They’re also not on the agenda of any deep learning systems, or of any energy-minimization strategies – because in the neural networks that we have now, we’re just adjusting connection weights between different layers. We’re not looking at a topology of activation patterns within a given layer.

That is, for a layer of “hidden” nodes, it currently doesn’t matter where a given active node is in terms of its spatial proximity to other (active and inactive) nodes.

When we start thinking about a CVM system, where the free energy is a function of the different pairs and triplets, then all of a sudden, the local spatial patterns count for a lot.

That’s why we need to start thinking about these more explicitly; we need to model each node and the neighbor-activations associated with each node.

More to come.

Obviously.

But as the Teacher said, “Sufficient unto the day is the evil thereof.” There’s enough in that little code to keep us amused for a little while.

 

 

Live free or die, my friend –

AJ Maren

Live free or die: Death is not the worst of evils.
Attr. to Gen. John Stark, American Revolutionary War

 

 

References & Resources

 

The Cluster Variation Method – Essential Papers

• Kikuchi, R. (1951). A theory of cooperative phenomena. Phys. Rev. 81, 988-1003, pdf, accessed 2018/09/17.
• Kikuchi, R., & Brush, S.G. (1967), “Improvement of the Cluster‐Variation Method,” J. Chem. Phys. 47, 195; online as: online – for purchase through American Inst. Physics. Costs $30.00 for non-members.
• Maren, A.J. (2016). The Cluster Variation Method: A Primer for Neuroscientists. Brain Sci. 6(4), 44, https://doi.org/10.3390/brainsci6040044; online access, pdf; accessed 2018/09/19.
• Maren, A.J. (June, 2014) The Cluster Variation Method I: 1-D Single Zigzag Chain: Basic Theory, Analytic Solution and Free Energy Variable Distributions at Midpoint (x1 = x2 = 0.5). THM TR2014-002(ajm). DOI: 10.13140/2.1.4415.6485

 

Previous Related Posts (and One That’s Forward-in-Time)

• Looking at Friends and Neighbors from a Node’s Point of View (Nodes, Nets, and Grids) (July 25, 2018)
• The 1-D Cluster Variation Method (CVM) – Simple Application (Nov. 3, 2014)
• Visualizing Configuration Variables with the 1-D Cluster Variation Method (July 9, 2014)
• Analytic Single-Point Solution for Cluster Variation Method Variables (at x1=x2=0.5) (Dec. 8, 2011)

 

The 1D CVM – Code, Documentation, and V&V Documents (Including Slidedecks)

• GitHub for 1-D Object-Oriented Python Snippets
• Code: 1D-CVM_OO_basic-config-vars_V-and-V_1-1_2018-09-23.py (Python 3.6; self-contained)
• Documentation / V&V: 1D-CVM_Object-oriented-code_V-and-V_2018-10-10-18.pptx