Statistical Mechanics for Predictive Analysis and Modeling Ensembles of Neural Domains

Predictive Analysis and Information Content Estimation in Large Data Corpora

There are three big challenges in modeling not just the information content of large data corpora (big data), but also in modeling the changes in this information:

1. Data content distribution contains local patterns, and these patterns themselves are a source of information – showing the interconnectedness of distinct information units,
2. Data content can change in a nonlinear manner over time – it can undergo a phase transition, and
3. Data content can be organized hierarchically using ontology umbrellas – but the pattern distributions and changes over time need to be correlated across the hierarchical scales.

Clearly, existing predictive intelligence and information theory methods do not address these three issues.

The majority of predictive approaches use some form of linear forecasting.

Information theoretic methods are largely derived from the Shannon concept of information, leading to the Kullbeck-Leibler method.

These methods are all useful, yet they miss an important point: Shannon’s work on information theory – and all successor works – come from considering entropy, or an entropy-like term.

In nature, entropy does not stand alone.

Rather, the governing force is equilibrium, specifically equilibrium as the minimum of free energy, which is a combination of both an energy (enthalpy) term as well as entropy.

When we start using the full free energy equation, we get several benefits:

1. A more complete (and complex) entropy formation – in conjunction with the enthalpy parameter(s) – gives us insight into pattern distributions as well as the relative distribution of objects into different states,
2. Free energy allows us to model phase transitions – especially useful when there are rapid and nonlinear data content changes, and
3. We now have a theoretical model that will let us address pattern similarities across scales of data organization (via ontology umbrellas).

There are two other potential benefits, each of which can provide substantial value as we devise more formal descriptions for the information content large data corpora:

1. We can assess the information loss (via a measure-preserving function) as we create higher-order representations of information in data set, and
2. We can determine (using Tsallis nonadditive entropy) the information overlap – or the extent to which new information will be added – when we add a new data corpus to an existing one.

Ultimately, the greatest value here is not just with characterizing information present in a single corpus, but rather, in determining how much information we gain (or lose) as we perform large-scale operations on data sets.