Correspondence Between Free Energy, Belief Propagation, and Markov Random Field Models

As a slight digression from previous posts – re-reading the paper by Yedidia et al. on this morning on Understanding Belief Propagation and its Generalizations – which explains the close connection between Belief Propagation (BP) methods and the Bethe approximation (a more generalized version of the simple bistate Ising model that I’ve been using) in statistical thermodynamics.

The important point that Yedidia et al. make is that their work “enables connections to be made with variational approaches to approximate inference.” They go on to explore the relationship between Generalized Belief Propagation (GBP) and the cluster variation method introduced in the 1950’s by Kikuchi.

The relevance to the Case Study example that I’ve been using is that it may, for some readers, be easier to visualize the relationship between financial institutions in terms of a Belief Propagation model. There is also a similarlity, developed in the Yedidia et al. article, between BP and Markov Random Fields (MRF). In particular, the MRF approach is comparable to an Ising spin glass model.

I’ve found some interesting references from some people using BP or MRF to model the recent financial meltdown; a few that are worth examining are:

Looks very relevant and interesting:

Paweł Sieczka and Janusz A. Hołyst, Collective firm bankruptcies and phase transition in rating dynamics, similar approach – will comment on this soon. They emphasize the straightforward phase transition, no mention of metastable states.

Sieczka and Holyst also note the relevant work of Eisdorfer & Hsu:

Assaf Eisdorfer and Po-Hsuan Hsu, Innovate to Survive: The Effect of Technology Competition on Corporate Bankruptcy
, where the comment by Sieczka & Holyst is:
“In the real economy, competition induces a higher default rate among
small and beginning firms and conserves a dominating position
of big and well-off companies. Since there is a larger
number of small firms compared to large firms, the default
rate is higher for higher competition level. For example,
patent competition in a software market is very painful for
smaller, especially starting, firms and is very convenient
for a large software corporation. A positive corelation between
competition level and a number of bankruptcies is
obvious in this case.”

Potentially relevant and interesting:

Li et al., Assessing the influence probability between objects: A random walker approach, which requires a fee – will try to find similar work that is freely available; the abstract indicates that the Lehman Bros. were used in the model set-up.

One that might be a bit of a reach, needs a look, is the notion of using semantic maps, in which the “propositions” connecting any two “nodes” in the concept hierarchy might prove to be belief vectors in a (sparse) graph:

Artificial Memory Semantic Wiki/Concept Hierarchy/Lehman Brothers

When time permits, look into any possible relationship between the statistical thermodynamics model (which does not describe rates) and the Hawkes Process, described with the Lehman Brothers as an application in a class exercise:

Simulating a Hawkes Prcess

Another to read (someday; not sure it will have relevance to this work):

An Entropic Approach to Analyze Investor Utility
Involving a Financial Structured Product

Worth a look, the notion of “contagion channels” may relate to a localized BP or vector in a MRF model, a more localized look at what I’ve been describing:

Financial contagion channels: Market microstructure
evidence from Lehman Brothers’ Bankruptcy

Worth a look, when time permits:

The international propagation of the financial crisis of 2008 and a comparison with 1931