Home-> Concepts->Waves

Mt Kailash, Tibet; photo:

Waves and Information

The internal structure of information is all around us. It's especially beautiful in the shape of trees and leaves. A binary tree cascades down in a pyramid form. It's this form that mathematicians have exploited to unveil the ubiquous, dynamic patterns in the universe. Binary trees as lists are use in structures on wave packet analysis. The binary tree represents information in its purest dynamical form.

Pyramids of Giza, Eygpt; photo:

Binary Tree and the Unfolding of Possibilities

Order in a sequence containing a finite number of elements can be intrinsically described by a binary tree. The unfolding of a path which splits in two enables the creation of new possibilities as you descend down a binary tree. This attribute of the binary tree describing its size is called the depth. The size a binary tree is equal to the size of a sequence because you can think of the binary tree as containing the ordered temporary structure of a sequence inside the mechanics of computation.

It's natural to visualize the size or length of a sequence when modelling Nature rather than a tree structure. But inside the mechanics of computation lies the structure of binary trees. The binary tree is the computational dimension of a sequence inside our wave packet model of a spike trains.

The Binary Tree

The wave packet transform, WPT, method of representating a signal lies in the decomposition of the complete binary tree in the figure above into parts or subsets of sequences as shown in the two figures below. The ordered subset is a connected or contiguous group of elements or paths of the complete binary tree [1, 2, 3].

The Branches of a Binary Trees

Figures (A) and (B) above contain parts of the complete binary tree sequence. The sequences down at the fourth level are (A) (ddda, dddd), and (B) (daaa, daad, dada, dadd).

The depth or size of the binary tree determines the resolution of the signal. If the sequence represents the signal, then the minimum length of the sequence containing the maximum amount of information is a compressed sequence in the sense of Kolmogorov complexity. The information holding capacity in a sequence is determined by the size of the binary tree in terms of the number of branches and leaves. In term of wave packet analysis, the minimum number of partitions in the sequence you make (in terms of decomposing of the complete binary tree) determines how well or accurately you can measure the signal or sequence. This comes full circle back to the idea of the limit in measuring the signal. The minimum number of descriptive "Fourier lexicons" or wave packet coefficients you use to describe the signal determines its resolution limit.

Wave packet analysis depends on decomposing a complete binary tree, also called a complete basis set, into branches of a binary tree or a basis subset. A wave packet represents a sequence which lasts for a limited time period.

Wave Packet

Entropy in Waves

In studying the neuron as an information processor, we want to know when the synapses occur as a sequence. More specifically, we want to know the duration of time between the occurrance of each synapse in the spike train. If we assume that the duration of time between each synapse is constant then we can considered this to be a Poisson process. Synapses as events occurring as a Poisson process are independent of each other; this independence is equivalent to making the temporal duration of each event relative to each other constant . If the duration between the synapses is a variable, then this is a Markov proces in which each synapse depends on other the synapses which occurred previously in time.

As a note, it is known from past studies in statistical mechanics that a Poisson process contains the maximum amount of entropy among the more general time ordered processes. A Poisson process, having the maximal entropy (uses the least amount of state varibles as a steady state ergodic system), and contains the lowest amount of information.

But we know that the neuron changes its synaptic firing rate in the spike train from listening to the chirps of individual neurons or their frequency modulated clicks. The firing rate of the neuron varies. So in general, from an information processing perspective, we say that the neuron's synapses is a temporally ordered Markov process.

A sequence contains a variable set of elements, S = {a, b, c, ... }. For example a sequence could be written s = (abcefd). In the case of a Poisson process, an example sequence would be s = (aaaaaa), and a sequence for a Markov process example would be s = (cdefab). In the case of a pure Poisson process, a binary tree structure representation of the synapse would not be needed. A binary tree provides the computational order in a sequence. A binary tree can represent the potential of possibilities in a temporal chain of events which is called a Markov process.

Time-Frequency Spectrum of Waves

The general description of wave forms are done using time-frequency plots [4] of wave packets. The internals of a wave packet maybe described using binary trees grown as Markov processes, but to get a real feel of this you can look at a program which displays a time-frequency graph [5].

Time-Frequency Graph of a 32-bit Sequence

The plot above is that of a simple binary sequence

S = (0 1 0 1 0 1 1 1 1 1 0 0 1 0 1 0 1 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0).

The plot on the top represents a time line in which the interval between the elements of this sequence can be set arbitrarily or scaled to any length. The plot below contain time-frequency rectangles, called "diagram of information" by Dennis Gabor. These rectangles have a simple intuitive meaning. For low oscillations the rectangles appears lower on the plot, and correspondingly for higher frequencies, the rectangles are on the top.

The sound of this very simple sequence played at 10K Hz sounds like a drop of water hitting the ground. This sequence is easy to inspect because it contains relatively little information. It contains about the amount of information in an 8 letter word like, "Hi_There".

The synapses of neurons may be represented by Markov processes in which the spike train sequences are constructed from binary trees. The wave packet formulation is a practical methodology because it combines the intrinsically intuitive wave model of energy propagation with a built in natural structure for information processing using the binary tree.

Temporal Parallel Computation

In computer software, in a language like Lisp for example, the idea of reduction as a method of divide and conquer is implemented using recursion. The software algorthm for traversing a binary tree can be simplified using recursive methods by calling the nested code for branching in a binary trees in the same nested code. This is a rather strange idea if you've not read computer software like Lisp, but it works really well.

In computer hardware, reduction machines [6] like wavefront array processors use asynchronous dataflow pipelining to implement a demand driven synchronous computational process.


1. The Wavelet Transform Beyond Fourier Transforms,
Mac A. Cody, Dr. Dobb's Journal, April 1992.

2. The Wavelet Packet Transform Extending the Wavelet Transform,
Mac A. Cody, Dr. Dobb's Journal, April 1994.

3. Entropy-Based Algorithms for Best Basis Selection,
Ronald Coifman and Victor Wickerhauser,
IEEE Transactions on Information Theory, 1992,

4. Theory of Communication, Dennis Gabor, 1946,
The Journal of the Institution Of Electrical Engineers, 93(3):429-457.

5. Fabian Brachere; Guimauve graphical software;
Gaussian decomposition of signals.

6. Introduction to Parallel Algorithms,
C. Xavier and S. S. Iyengar
Wiley-Interscience, 1998
(Chap. 1.3, Models for Parallel Computation, p 29)

next:   Thoughts