Concepts->Limits of Information
November 19, 2010
Notes on Programming Neural Systems
Om Namo Shiva Nataraja
The Light of my Mind
The Limits of Information
I've always naively or intuitively thought that information
had its basis in physical matter or energy; that it could not
exist without energy. Information is an embedded attribute in
physical energy. The ancient thinkers would call an exposition
like this metaphysics or religion. But thinking like this is
required to build a model.
As I've integrated my studies of entropy into my world view, I've
personified it with a Hindu kind of mysticism. Entropy, which
is the physical law of time ordered limitations, is also about
the unending potential unfolding in Nature.
Placing constraints or limits on physical systems for doing abstract
analysis helps us to learn about the system, but sometimes is
not appropriate. For example, I think that the concept of infinity 
or zero applied in certain instances to describe Nature may not be
correct all the time. In certain cases in statistical mechancial theory,
we cannot consider a system as a "close" system isolated from the rest
of the world because the law breaks down in the more general cases .
In building models of our world, we necessarily have to put limitations
on its behavior, but we need to try to understand these limitations
carefully. Just as using the concept of infinity is a limitation,
the process of breaking down a system by analysis is setting up a
series of limiting conditions.
The measure of static information is a bit. But in real physical systems,
no object remains at a constant level of information as function of
time. The molecules and atoms are changing through radioactive decay
for example. Nature is dynamic, and information is not static.
Imagine extracting all the energy out of localized physical system. The
movement in this system will approach zero. This system will get cold
and rigid, and contain almost no information, but the law of entropy tries
its hardest to prevents this extreme condition. Physical systems get very
cold, but not as in taking the mathematical limit to zero.
The measure of instantaneous dynamic information is relative depending
on the temporal resolution of the detector. The resolution of the
detector is limited by the wave fields of physical matter expressed
in the uncertainty relationship between energy and time. In this case
there's a fundamental finite limit to the "smallness" of matter of which
Planck's constant is an attribute. Unlike in theoretical mathematics
there's no physically uncountable or infinite fields as in Cantor's set
theory. The use of infinite limits in numerical equations are constructs
of mathematics, and it sometimes blurs the real constraints imposed
by Nature such as the limitations in processing information.
I think of the mind as an information generator. But it's capacity
to create information is limited by the physical law of entropy. The
brain's ability to create information is constrained by the dynamical
flow of order. A good analogy to use is to compared the brain to a
steam engine. The steam engine transfers the energy in coal to water
to produce steam. The brain converts the energy in the food we eat
and air we breath into synaptic waves of electro-chemical currents to
The fundamental limitations in measuring physical variables can be
channeled down into a formula which summarizes the uncertainty principle.
With regards to information, this formula lies in the wave packet
Fourier transformations used in describing waves.
There is a fifth dimension beyond that which is known to man.
It is a dimension as vast as space and as timeless as infinity.
It is the middle ground between light and shadow, between science
and superstition, and it lies between the pit of man's fears
and the summit of his knowledge. This is the dimension of imagination.
Rod Sterling's opening narration to season 1 of the TV series The Twilight Zone (1959).
I'm talking about the perspective on viewing the objects as a part
of a larger whole, a Bohmian idea. It's confusing a first,
but makes thinking of physical systems easier after awhile.