"Knowledge is a function of being." is a quote from Aldous Huxley's
The Perennial Philosophy. This work is a masterpiece. For months,
I've focused on this work, drawn to it by an obsessive kind of focus
that this work has created in me ... that this is how we humans think
reality is. Aldous Huxley was deeply aware of our limitations on how we
perceive our world.
Eknath Easwaran equates the perennial philosophy with sanatana dharma.
For much of this year, I've worked on writing software algorithms
for parsing language. As efficient as the algorithms are, there seems
to be no intelligence in it - not that of a kind a "sentient" being
can possess. The algorithms are just machines. The missing element that
gives "meaning", in the human sense, to intelligence is that the object of
the algorithmic processor has life. But, "What is life?" This vague notion
has been a reoccuring thought process of mine for months.
I think there's two reasons for my confusion about the notion of
intelligence. First, is that what I see as intelligence in Nature
is more than mechanistic. Yet, secondly, at a deeply foundational
level, I see the world as purely mechanistic. This is a really
interesting paradox. It's a real part of me.
Erwin Schrodinger wrote that "Consciousness is never experienced in
the plural, only in the singular." Kind of like a Hindu he was. This
statement has made people ponder whether an electron can possess a kind of
consciousness or intelligence. Can it be this extreme? Thinking about
this kind of reality, related to the paradox described above, is as if
you're walking on a very quiet path. In a sense, it is absurd, but yet
a wonderous walk in mystical stillness.
(Updated Dec 6, 2015)
Dec 31, 2014
uLisp - Another LIsp Interpreter
Language interpreters have a high degree of complexity. David Betz's
Xlisp-2.0 released in 1988 is beautifully written. It uses many of
the classic data structures in computer programming like dispatch
and hash tables. Xscheme-0.28, which was released in 1991, has an
even higher level of complexity than Xlisp-2.0 because Xscheme uses a
bytecode compiler, and moves towards a virtual machine style of
architecture used to process bytecodes. Xlisp-3.3, released in 2002,
is a very complex piece of software which is more accessible after
you have studied Xscheme-0.28. uLisp is my modified edition of Xlisp-3.3.
Computer language interpreters and compilers have basic similarities.
In studying computer languages like Lisp, I've learned the language on
a deeper level, and have gained a lot more appreciation for the
language and its features by trying to understand Lisp interpreters.
I spent the last month studying Bill Birch's beautiful code called
reflisp-2.67. You can really learn the
programming language well not only by reading the language manual,
but by studying an implementation of the language; either by reading
the implementation manuals or perusing the source code. You can see
how every programmer's mind is intrinsically different in their likes
and dislikes of a programming language.
The "rt" in rtLisp, stands for real-time. I'm studying this program
to learn how a real-time, multi-threaded or parallel lisp is implemented.
For me, the Lisp language is at the summit of the programming beauty and
elegance mountain.
Some Lisp implementations I've studied which have influenced
the code in ulisp are
xscheme-0.28, by David Betz, and
minischeme-0.85, by Atsushi Moriwaki.
Mind precedes matter. This is Vedantic theory.
Matter precedes mind. This is scientific theory.
--- Swami Sivananda
Mind - Its Mysteries
and Control
1935
"The study of the Vedas is really a study in the intricacies of the
human psyche."
That is a quote from Jeanine Miller, Vedic Myths and Vision,
http://jeaninemiller.org. She's was a remarkable scholar, and mystic.
In her book "The Blazing Dragon of Wisdom" she says,
One of the greatest gifts of the Rigveda to the world of thought is
its vision of harmony, wholeness, divine solidarity, of Cosmic Order
which subsequently became the fountain-head of the peculiarly Hindu
doctrines of dharma and karma, and of the law of vast cycles, ...
As the Rigveda falls within the esoteric tradition which concerns the
mystical insight of humanity, the present investigation will be taken
from the third level of interpretation which Indian exegesis labels
adhyatmika. The understanding of mysticism requires an intuitive insight
not developed in every one. For those who fail in this the ancient insights
will be meaningless and this book had better be left alone.
The Vedas, more specifically, the Upanishads are great works on the
mind or "pysche". They were written 3500 years ago. There's more
than ancient wisdom in the Upanishads. As we live in the age of "modern"
science, the ancient Hindu rishis or scribes who wrote the Vedas
lived in the age of a "spiritual" or a "pyschic" science. Their tool for
verification of truth was meditation. The Vedas reflects ancient science
as expressed in the studies of Carl Jung. In my opinion, we'll start
appreciating the value of the Rigvedas with respect to psychology as
modern day computational neuroscience models reveals how specific network
structures in our brain makes emergent our thoughts and behavior.
Sri Swami Sivananda who studied the mind in the Hindu tradition lived a
generation ago at a time when technology had not yet had machines like
the EEG imaging or MRI to see our thought patterns. In the generation
since Swami Sivananda's time, we created computer models of our physcial
world and the brain which has changed our perception of the mind.
In the quote above to the start of this log entry, we can see the
essence of the Upanishads reflected in Swami Sivananda's words.
Mind precedes matter. This is Vedantic theory.
Matter precedes mind. This is scientific theory.
We see that the great Hindu rishis of the past did not differentiate or
understand the significant difference between the brain and the mind as we
do today. We now appreciate the complexity and wonder of the biological
brain. But there's still much to learn about the mind from what the
ancient Hindus wrote in the Upanishads.
If you read the Upanishads, remember this: the "mind" in the Vedanta
refers to the human psyche ... not explicitly to the brain. In the
Vedanta, the mind is the spirit. In scientific theory, the mind emerges
from the neural activity in the brain. Understanding this, I've found
a wonderous union between what's in the Vedas and modern science.
Jan 27, 2014
Energy and the Mind
There is a Light that shines beyond all things on earth,
beyond us all, ...
This is the Light that shines in our heart.
--- Chandogya Upanishad, 3.13.7
The Upanishads,p 113
translated by Juan Mascaro)
This log entry is about our ability to perceive the physical energy
surrounding us. Our perception of the energy fields all around
us is masked because we don't usually experience energy directly
as a vibrating field especially as it propagates as waves.
Energy is propagated or transported as waves because of the spacial
medium or "field" we live in. For example, we hear sounds which
can propagate in the air, water, or solid earth. More specifically,
for example, this energy is transported on the surface of a pond
because of the compaction and rarefaction of the water medium by
an impulse like a rain drop.
This truth, that energy moves in a vibratory way, by compaction
and rarefaction, is a universal truth about our physical world.
If you study the mathematics of differential equations, you'll see
how essential harmonic or periodic sequences are in describing our world
of energy. I, intuitively or naively, did not sense the
all pervading significance of this truth years ago. We're immersed
in energy fields as fishes are immersed in water. It's taken me
years to get an appreciation of this simple and beautiful fact.
My "unawareness" of energy fields is analogous to how our cognizant
focus on breathing is masked by our nervous system because breathing
is programmed autonomically.
Nowadays, I feel or sense a vibratory field surrounding almost
everything. It's especially true when I look at plants like
ferns. The branches and leaves on ferns have an incredible symmetry.
Ferns, it seems to me, have a kind of primordial, fractal geometry.
When I walk in the topical forests where I live, I really sense,
in an indirect way, the vibratory electic fields in plants. By
"indirect way", I think I mean by my own mental constructs which
I've built in my own mind by training. These mental constructs were
built primarily from observing the structure of sound waves propagating
through water.
An ubiquitous property of energy fields like sound fields for example
is symmetry. Plants, in general, it seems to me, do not show a
great amount of symmetry directly. In physics, symmetry could be
hidden by in a multi-dimensional topology. So I wonder about the
chaotic, fractal nature in the appearance of plants. How do
we unravel this hidden geometry in plants? I don't know. I've been
thinking about the topology of plants since I was a teenager. Here,
for me, is where I start entering into the mysterious and great unknown.
When I was in Bali, I became seriously ill from fasting and too
much meditation in the Hindu tradition. I had lost too much weight.
My quest to explore my mind had almost killed me. I think I had a near
death experience. This is the first time that I think I experienced
a sense of "Prana", the life-force, as a kind of emotion of a mystical and
peaceful universe which was gently leaving at the bottom of my head. I had
searched for manifestations of Prana or Chi for many years without
finding any experiential clue to what this is. I really don't understand
any of this. That Nature keeps our mind so stable that we can only
experience these seemingly unique experiences only under the most
stressful condition on the body is truly amazing (and humbling). Maybe
one day I will understand what happened, then I can tell you more about
this experience.
After this experience, I became really absorbed by the paintings
of the Balinese artist Wayan Djumu. It seemed to me that Wayan Djumu
would look at his paintings sensing a kind of energy on the surface of
his canvas. He would run his hand over the canvas like he sensed an
intricate texture on it. At that time I didn't understand what he was
doing, but I think I do now. It's that primordial energy, the vibratory
field inside of nature that he sensed. If I ever get back to Bali, I'm
sure to ask him about this. Then again, it might be only me.
I can also sense this energy in the work of the Balinese artist Ketut Budiana,
the Japanese ceiling painting in the Zen Tenryu-ji temple, or the paintings
of the shaman Pablo Amaringo.
I think there's so much of our mind that's hard-wired. That is,
what we think is determined by our biology or DNA. Our hormones
determine much of how our brain wires itself. And our human evolution
over thousands of years has gotten the brain to be a remarkably stable
platform for thinking. The biochemistry of the brain is an exemplar
of control. There's so much more to learn about the mind, and maybe I
should really stop here in humble respect to the incredible order
in the universe. The Hindu mystics warned against speculating on
this edge of reality.
It seems to me, however, that there are circuits in the brain
that at some times can resonate with the vibratory nature of the
universe in a mystical way. Is it brain chemistry, or the structure
of Nature itself that's driving the mind here?
One more thing, the Tibetan mandalas have a symmetry really close to
what's seen in harmonic energy fields. It might be that the mind may
have hard-wired a kind of Fourier transform as a means to do pattern
recognition. This is because the Fourier transform so efficiently
reduces dimensionality using symmetry and information compression.
The Fermi-Pasta-Ulam, FPU, paradox interestingly reveals the structure
of nature, and energy propagation. Usually we think of energy dispersing
while propagating in a uniform media. However, if this media has
impurities in it, or is non-homogeneous structurely, then non-linear
resonance effects can cause a periodic type of propagation in
which the energy congeals instead of dispersing. It's as if time
reversed; switching the direction of entropy. But this is really
looks like a resonance phenomena enfolded in itself since time
can't go backwards. My point is that there are multi-dimensional
phenomena in physics (the real physical world) we, or maybe only
me, don't even understand yet. We're a long way off from putting too
much constraints on what we can really know about the mind.
As great as the infinite space beyond
is the space within the lotus of the heart. ...
Whether we know it in this world or know it not,
everything is contained in that inner space.
--- Chandogya Upanishads, 1.3
The Upanishads, p 142
translated by Eknath Easwaran
Jan 20, 2014
Time
O Kali, my mother full of bliss! ...
Thou art the Mover of all that move, ...
--- The Poet Kamalakanta
(from the gospel of Ramakrishna Paramahamsa)
Namaste,
With respect to my work, I spent most of last year on the software
project Kali. So I thought alot about temporal sequences, and
how to use the "duration in time" as coefficients in temporal sequences.
This log entry is about the study of time; philosophically, and
I think in a practical way which most philosophers do not experience.
That is, I studied time as a computer programmer.
In the early 1980s, we exhausted great effort trying to optimize processor
speed, working with machine registers and low level assembly language.
You had to program asynchronous software modules or procedures (daemons)
efficiently time splicing the software module you wanted executed.
I spent a whole bunch of hours trying to smoothly multi-task "real-time"
processes. You feel like you're holding chunks of "computing energy or
power" on your finger tips as you can determine which resources get
computing time.
From this kind of experience, I think you tend to think of time duraton
as a real quantity; not an illusion. It is a necessary physical dimension
existing independent of (or "orthogonal" to) the spacial dimensions.
The reason I say this is because many fine theoretical thinkers like
physicists and mathematicians go too far away from attributing to
time a real value, and placing too much emphasis on time as emerging
from a process. For example, I really admire the abstract studies on
time by Whitehead and Russell; but respect to operationally creating
information processing systems, they didn't get it.
A mechanical thinking machine requires time to reduce complexity. It's
in the nature of physical laws to consume energy "or time" to produce order;
and a mechanical thinking machine is an epitome of orderliness. I've seen
the enormous electrical power systems required to run the cooling systems for
computer rooms.
Last year, I spent time studying correlation coefficients in temporal
sequences. From this, and re-reading Satosi Watanabe's article "The
Symmetry of Physical Laws: Part 3, Prediction and Retrodiction" it
reinforced the notion in me of the entropy's role in creating order.
To me, it seems that order in the universe arises out of a divine like
rule in which the integral (or the organic like) system's evolutionary
process is limited by the amount of physical states it is allowed access to.
That is, systems evolves in time to a higher level of order because the
law of entropy paradoxically reduces the number of physical states
(or micro-states) it can transition to as a function of time. This
is a kind of indirect dimensionality reduction or increase in complexity.
The limiting form of the entropy function is logarithmic which really
suppresses the available states the system can transition to.
Professor Watanabe's insights into the role of entropy in physical
processes is incredible.
Born in a corner of the universe, where entropy is increasing in one
direction of time (and decreasing the the other), life is to survive
and expand. But, in which direction of time is it to live, grow
and age? The only possible direction is the one in which the future
is foreseeable and controllable, so that by adapting itself and
acting suitably it can satisfy its needs and desires. The foreseeable
and controllable direction is the one in which causality works.
Causality works in the entropy-increasing direction, but not in the
opposite direction.
--- Satosi Watanabe
Causality and Temporal Irreversibility (1977)
Japanese Studies in the Philosophy of Science
edited by Francis G. Nagasaka
Kluwer Academic Publishers
That's more than incredible. It's divine.
I was really glad to see Professor Watanabe's research papers on the
symmetry of physical laws posted on the Internet Archive web-site:
His research paper no.4 on "Intersymbol Correlation of Finite Range (1954)
which I have a hard-copy and used to edit a version of this paper for study
is on the same website at:
When I was 12 years old, I spent hours reading Spinoza's Ethics. Now, I
love the reading the Upanishads, although, I know it's crippled symbolically
in religious terms. It's a joyous celebration of human spirituality which
I could not have appreciated as deeply as a young boy.
Spinoza, the blessed one, had much in common with the Hindu rishis who
wrote the Vedas and the Upanishads. As flawed as Spinoza and the Upanishads
are philosophically, the singular concept that makes the ideas expressed in
this philosophy sanctified to me is the non-duality of the mind and body.
(Philosophically, as a discipline, the duality of the mind and body is a
fundamental stepstone in the path of a philosopher.) Our consciousness,
self-awareness, or in general thoughts are not outside or separate from
our body. Our brain generates our thoughts which emerge from harmonic
synapses. This, for me, is the message of Brahma taken symbolically.
It's easier to understand why Spinoza would not separate mind and body
because of his desire for the inclusivity of all "substances". As for
the Hindu yogis in meditation, the mind (the brain's energetic synapses)
permeates the body. The organic mind and body functions as a unit in a
seemingly uncountable number of processes. The Upanishads affirms this
unending chorus of divinity expressed in the union of all material and
spiritual (mental) energy in Brahma. All that is manifested comes from
a singular substance in Brahma.
For me, the realization that the mind and body are not separate is the
most significant concept in my philosophy. This thought gives all that
I can see a mystical, silent essence. I cannot describe it. This
realization shapes how I view universal order, my own identify, and my
quest for spirituality. If mind and body are not separate, then this
"I" will no longer exist after I die. "I" will
be as "I" was before "I" was born. However, this view does
not preclude the non-existence of "God". I like to think as Carl C. Jung
thought and said about "God", "I know God exists." The beauty in this
universe allows me no other alternative.
Oct 21, 2012
Time and the Temporal Automata
With respect to intuition, modern quantum mechanics and relativity spins
the classical concepts of physics on its head. The most profound conceptual
transformation of modern physics, in my opinion, has been in regards
to the notion of causality. Modern physics reveals an absolutely
astounding intrinsic order permeating the universe. But its unfamiliar
phenomenology of an unintuitive indeterminism in space and time makes me
have to think twice about casuality.
The study of neurons and thoughts fall at the edge of the "small" quantum
world. Most of the strange effects of the quantum world do not seem to
apply in our daily experience, and in all instances so far need not apply
to the way I've thought about neurons. The uncertainty principle which I
have applied to studying neurons can be considered a wave phenomenon
independent of quantum mechanical effects. But I don't known how much the
effects of the "small" quantum world affects neurons. My guess is that
I can sufficiently use classical mechanics. I've been intuitively working
on the premise or guess that neurons can be sufficiently modelled as a
classical system.
Philosophically, I like Kant's view of reality. I think Kant was honest
with respect to what we can or cannot know. What we know comes from
experience. We experience time. I also think we experience time as
a movement from seeing or hearing objects in our world being transformed.
Time is the ground of existence, and it's hard to conceptualize stuff
for sure beyond our existence in this mediate temporal instance.
I wanted to write about time because it's such an important variable in
the models we use to describe our world. This is especially true in
describing how we communicate in signal processing verses communicating
using language. In signal processing, time and its complimentary variable
frequency, are the essential variables of study. As we start using
symbolic language, the need for time recedes. With respect to modeling
neurons with signals, time dominates that world. The model we use to
describe communicating neurons is the model of temporal automata or
time machines. Encapsulated in the neuron's energy, which can also be
describe by frequency, is information. Information in signals has been
studied extensively in signal processing, particularily in using
wave-packets or wavelets to describe signals.
When using a model to describe our world, it's essential that we get
as much of an operational view of our world that this model can provide.
This includes, in my opinion, a broad view of the epistemological or
representational view of the model. For example, when I studied the
lisp language in the late 1970's, I spent lots of time on studying
the repesentation language KRL. KRL is a reflection on what your
belief system is, and how it pervades the network you're building.
With respect to temporal automata, time is essential in describing
computation. Computation is constrained by physical processes which
is determined by time. In fact, using time machines means considering
all processes as a computation, and all computing processes involves
time at the fundamental level. I also recognize that it is in my
belief system that I've made the existence of time the basis of
computation. Computation is process, and process is computation in
the temporal automata model.
But also, it's more than just thinking of the temporal automata model using
computation or process as sufficiently requiring time. Time is necessarily
required because time allows changes or transformation to occur in the
form of its duration between changes. Information is embedded in duration.
Oct 11, 2012
Temporal Automata Neurons
Alan Turing wrote about neurons as automata in the 1940s. In his state
diagram you can see his "connection-modifier" which makes it possible
to modulate the inter-neuronal signals much like transitors do.
Recently I came across an article by Thomas Wennecker's called
"Finite State Automata Resulting from Temporal Information Maximization."
For me, it's an astoundingly great article because I've been working
on trying to understand how neurons "minimize" or reduce informational
complexity through shortening their synaptic firing duration. In
essence the world would look clearer to a neuron if its sampling time
was small. This is consistent with Nyquist's rule for sampling
electronic signals. The spacial analogy would be looking at the
world through pin-hole glasses.
My route to thinking of neurons as masters of dimensionality reduction
went across paths trying to invent computer algorithms in pattern
recognition. I discontinued working on the matrix formulation of neural
networks using weight matrices in favor of using state automata algorithms
like the parallel Aho-Corasick algorithm. I think this is so because
the matrix black box approach does not intrinsically provide temporal
state information inside its matrices. In using the parallel automata
algorithm like Aho-Corasick, I (or you) could think of the neuron
as making decisions in time. And it seems to me that neurons that make
decisions in the shortest amount of time reduces dimensionality or
increases order or complexity the most.
On the theoretical side, I've been trying to study how neuronal circuits
can be wired so as to most efficiently use information in long temporal
sequences. The complexity of a sequence increases exponentially as a
function of its length, so I think groups of neurons must "architect"
themselves as some sort of temporal filter. To build this kind of filter,
I've been studying the correlation index of symbols in long time frames.
These long time frames are called ergodic in models using entropy. It
seems to me that if you could correlate the information in the symbols in a
sequence (like neural pulses or spike trains in a nerve circuit) to
each other, then you could build an efficient pattern matcher automaton.
In fact, I spend lot's of time on this problem in a software project.
There have been lots of research on correlating spike timing arrivals
between pairs of neurons and large groups of neurons, eg.,studying
correlation in interspike intervals, ISI.
I like the way Bruce Knight proposes how neurons synchronize themselves
in a population of neurons. He shows how a very simple model of a neuron
can perfectly replicate the input stimuli. For me, his 1972 article,
"Dynamics of Encoding Neuron Populations", is based more on his intuition
than on his effort to laid down an axiomatic foundation to base his
differential equations upon. But his proposal makes the most sense of
any other model of synapsing neurons because of its simplicity.
In 2000, he added more general mathematical features to his model saying
that the math would provide more efficient simulations. But I like
the simplicity of the 1972 article. The beauty is impressive.
A consequence of thinking that neurons replicate input stimulus efficiently
with minimal delay in time makes me comfortable with the idea that neurons
work like temporal automata. The temporal duration of spike trains is in
the order of milli-seconds, and neuronal circuits make decisions within this
time frame.
I've thought up a very simple way to show neural synchronization using
the idea of spacial clustering as constructed by Satosi Watanabe. Spacial
clustering is analogous to temporal synchronization if you use the
superposition principle of waves as a "force" to coalesce the coincidences
of the spike pulses. The waveform of similar pulses reinforce each other
while those that are not "coincident" become noise. Clustering related
synchronization reduces dimensionality. I'll write an article on this later.
I must add, many times I ignore whether I think my thinking going right or
wrong matters because if you follow your intuition, I sort of have faith
I'll endup in the right place. Besides, as they say, you can learn alot
from your mistakes. When I talk to others on subject matters out on the
edge, I can sense this too, and I always admire the person who knows
this also.
Lastly, for today, I like to write software using the scheme programming
language. I use Bigloo and Gauche. Both implementations are works of art.
Oct 7, 2012
The Turiya and Consciousness
Through the years I've worn out my copy of "The Upanishads" translated
by Juan Mascaro, @1965, published by Penguin Books. It's only 144 pages
long, and the chapter I think that is the most impressive is the Mandukya
which is only 1 and a half page long. The Mandukya Upanishad is about OM,
and the states of consciousness. I think it's the greatest text ever
written on the exploration of the mind and consciousness.
He is Atman, the Spirit himself, that cannot be seen or touched, that
is above all distinction, beyond thought and ineffable. In the union
with him is the supreme proof of his reality. He is the end of
evolution and non-duality. He is peace and love.
page 85, Mandukya Upanishad
The quote above is about the fourth state of consciouness: the turiya.
Some Hindus say the turiya is the seed of consciousness. The turiya
occurs at the cusp of time ... the efforescent concrescence between the
falling away of the non-existent past and the potentiality of the future.
The turiya is those primary or principal synapses that unfolds our
awareness into the future. The turiya are those synapses forming
the chain of spike trains in our principal neural circuits emanating
from the subthalamic nucleus and global pallidus at the base of the
basal ganglia.
This whole thing is a dynamic process; what David Bohm would call
movement, or the ground of all existence. In his introduction to
"Wholeness and the Implicate Order" David Bohm wrote:
To meet the challenge before us our notions of cosmology and of the
general nature of reality must have room in them to permit a
consistent account of consciousness. Vice versa, our notions of
consciousness must have room in them to understand what it means
for its content to be 'reality as a whole.' The two sets of
notions together should then be such as to allow for an
understanding of how reality and consciousness are related.
David Bohm, 1980, Wholeness and the Implicate Order
The quotation above is in essence about the spiritual quest of the seekers
who wrote the sacred Upanishads.
In his article "Time and the Probabilistic View of the World" compiled
in the textbook "The Voices of Time" edited by J.T.Frasier the
physicist Satosi Watanabe remarked that here should be a pragmatic fusion
of objective and subjective reality. The quantum wave function only
describes the possibilities of what we see.
Quantum physics is in perfect agreement with the viewpoint that
science should deal with a world-to-be-acted-on rather than
with a world-to-be-contemplated.
Satosi Watanabe, 1966
Like David Bohm, Satosi Watanabe reacted against the rigid positivism
of popular science. In his article "The Foundations of Cognitive
Relativity," 1991, he warned against a narrow focus on the reality
of particular objects, over reliance on reductionism and notions of
strict causality. Quantum physics is a field theory, and so can turn
the classical notions of order inside-out. The study of field theory
is by nature like studying an integrative organism; a whole body.
Studying quantum physics as an organism includes merging the relativistic
effects of space-time, and causality. That is, with respect to time,
time and space are interrelated, and the past and future are interrelated
as so far as a quantum can be considered a single body entity. I think
this is right, but maybe I'm speculating. You have to rely on experimental
proof for all of this. So the burden's on you as well as me.
Time plays a special role in expressing the unfolding of the quantum
field which is described in probabilistic terms. This is because
the Bayes theorem on conditional probabitily is essentially what
computer programmers call an "if-then" statement. This "if-then"
constriction is a causality constraint in time: "the" temporal dimension
imposing "the" primal constraint. From these concepts Satosi Watanabe
gives the quantum field an especially subjective reality.
... space provides room for being and time provides room for
becoming.
... it is taking the form which a thing has been
intended to assume. Becoming has intent, yet it has no plan.
Becoming is making of the yet-unmade.
... time is the vehicle of freedom and value.
The logician Alfred Whitehead who wrote "Process and Reality" in 1927
was also an amazing thinker. His views of time also seems reactionary
when compared with the classical philosophers like Aristotle and Socartes.
Alfred Whitehead wrote on creativity as a resonant superposition of our
thought processes emerging in a splice in time. The elements which
makes up Alfred Whitehead's "process" philosophy is what David Bohm and
Satosi Watanabe ideas seem to parallel.
That 'all things flow' is the first vague generalization which the
unsystematized, barely analysed, intuition of men has produced. It
is the theme of some of the best Hebrew poetry in the Psalms; it appears
as one of the first generalizations of Greek philosophy in the form
of the saying of Heraclitus; amid the later barbarism of Anglo-Saxon
thought it reappears in the story of the sparrow flitting through the
banqueting hall of the Northumbrian king; and in all stages of civilization
its recollection lends its pathos to poetry. Without doubt, if we are to
go back to that ultimate, integral experience, unwarped by the
sophistications of theory, that experience whose elucidation is
the final aim of philosophy, the flux of things is one ultimate
generalization around which we must weave our philosophical system.
Alfred Whitehead, 1927
Lately, I've been thinking about the fractal nature of the computational
abilities of ensembles of neurons. Neurons have to work around the
complexity of processing information. If the job of neurons is to
process information, then my opinion is that the neurons' primary
problem is reduction of dimensionality. Groups of neurons working
together can produce multiple gains in computational power, but this is
only a linear gain. So I've come to think that neurons must reduce
exponential information complexity through making decisions in a
time-sliced manner. Neurons make computational decisions in the shortest
amount of time possible. This allows the possibility of using
computational algorithms like Turing's machinery. A turing machine
is the simplest computational machine using a memory storage unit.
But there's a limit to how short a neuron can time-split information
because of the uncertainty principle. The signal-to-noise ratio, SNR,
for communicating neurons follows Nyquist rule, but neurons cannot drive
up their frequency of computation indefinitely because they'll overheat.
So I've come to assume operationally that individual neurons like
a mass essemble of neurons must essentially perform computations the in
same manner. Neurons compute the same across a fractal of dimensions.
Each computation of a neuron taking place as a synapse produces
a spike train. The information in this spike train is enfolded in
a pulse constrained by the uncertainty principle or wave packet.
When a synapse occurs I think we can think of the information as
contained inside the spike train and enfolded inside of a wave packet.
A synapse unfolds the potential in "our" universe. There's nothing
mystical about this is it. I'll write about this in detail within
a couple of days.
But somehow, this process of the unfolding of cognition or consciousness
always raises emotions in me. Maybe it's the ground of all existence.
He is Atman, the Spirit himself ...
Jan 7,2011
Dynamic movement as represented in an abstract sequence is always greatly
simplified. For the past 2 days, I've been studying Satosi's 1953 article
on ergodicity and sequence correlation index after I received the hardcopy
monograph I ordered which has handwritten formulas (which must be Satosi's)
in it. There're 14 pages for mathematical formulas Satosi uses to derive
his correlation index, W. There's more discussion about the reasons for using
concepts like ergodicity and redundancy in this report than in his 1960 article
on Multi-Variate Correlation.
The formula for W above work in principle for ergodic Markov chains. Satosi's
criterion was that the symbols in the sequence, S, had to be unique (which also
meant the sequence progression eventually made any conditions on the starting
symbols not matter).
Anyway, the interesting part of a dynamic movement in Nature seems to be at
the beginning of the movement ... entropy at work always increasing. I took
snap shots of a wave packet simulation. The wave packet bounded in a square
box hits a very thin vertical potential barrier in the middle of the box.
The snap shots are taken at 3 secs, 10 secs, 30 secs, 5 mins, 30 mins and
10 hrs.
In terms of dynamic movement, the wave packet settled into a rather constant mode
after 5 minutes into the simulation. According to ergodicty, it would not matter
what type of initial boundary barrier we had setup in the beginning of the
simulation. Eventually, after a long time, the waves in the box would look
like white-noise; just a noisy speckle pattern. In terms of the simulation,
this would be true primarily because of the floating point number roundoff
errors in the simulation equations.
Jan 4, 2011
Today I read some articles on spike train synchrony.
In articles 1 and 2 above, the authors rigorously study the relationship of
temporal spike trains. It's an admirable scientific study trying to pin
down the real grounds for what's inside a spike. Their principal analytical
object or tool is what they call the Inter-Spike Interval, ISI, which is
"parameter free and time-scale adaptive (Kreuz et al., 2007)". Their
construction of the "SPIKE" distance which is the temporal interval
between two spike is interesting. They actually reconstruct the differential
and integral cells (what I used to think of as cellular spaces in calculus)
for analysing spikes. These are two excellent articles. But it's also
interesting that in article 1, the authors say in the conclusion that
"First, it is obvious that no measure that results in a single number
quantifying the synchrony between two or more spike trains can be adequate
to deal with all kinds of potential coding schemes (e.g, time coding,
rate coding and pattern coding; ..."
Article 3 is a good study of neuronal synchronous temporal-dependent
plasticity, STDP, and the highly synchronized network response to stimuli.
My questions settled arount the equations used in this article. The
neuronal circuit equations contained interconnection weights, w_ij, so
it may not matter what boundary conditions you set in the beginning;
you could get periodic behavior in the end if you look for it like I do.
I'd just eliminate the word "non-periodic" in this article and use
"irregular" as they do in their introduction. Synchronization can occur
without resonance. This article, while confusing in places, was very
interesting. This article is a fine, extremely detailed study
of the simulation of neuronal topology and some experimental data. I
couldn't absorb most of it today.
The first 50 pages, 1st chapter, of a recently published book, August 2010,
edited by Christoph von der Malsburg, William A. Phillips and Wolf Singer
is online at
Dynamic Coordination in the Brain. The editors comment,
on page 17 on the section called "Temporal Structure and Synchrony" which
the authors of article 1 above carefully avoid implying in any manner, that
"One possibility is that spike rate and spike synchronization operate in
a complementary way such that salience can be enhanced by increasing either
or both."
The editors, in this paragraph on temporal synchrony, suggests that
synchronized rate codes, high frequency circuits, and neuronal circuit
inhibition, leads to a host of higher level cognitive functions which
I've always assumed to be true or could be true from a computer programming
perspective. In writing software you a free to create, and communicate
to others whatever you think are the most optimal structures. You can't
really do this when you're bounded by experimental, laboratory evidence.
I think spiking rate, and temporal coding are synonymous. I have described
both with the same mathematics.
Today, I re-read Bruce Knight's 2008 article,
The Faithful Copy Neuron. The simplicity of the idea of
an ensembly of neurons containing an innate propensity for synchronization
is very appealing. There are other concepts which add elegance to
the model such as "revised time" which seems to play the role of
phase shifting the spike trains. I don't understand this yet. I have not
thought about it enough. But I used to wonder about entropic correlation of
a set of sequences. The dynamic change of a set of sequences with respect
to its order in time usually occurs in the begin as it is constrained by
its initial "boundary" condition. The sequence as it progresses ergodically
appears like the beginning of the sequence. This is just the way it is
with most simple sequences in Nature.
Article 4 referenced above, covers essential technical analysis. The
authors state at the end of the article that
"To accomplish this type of decoding, neurons need not do
anything more sophisticated than be sensitive to the durations of
individual ISIs. This sensitivity can be embodied in a single
synapse and does not require averaging across stimulus repeats,
stretches of time that may be long compared with the time scale
of firing rate modulation, or a large population of neurons that
carry similar information."
This is quite an insightful conclusion. It also remainds me of the
significance of the prefix in sequences in pattern matching strings.
Dec 24, 2010
I really found what I was looking for today. I was skimming through
articles on neural coincidence detection for the pass few days. When I
read this article's summary "The firing rate of a population of neurons is
related to the firing rate of a single member in a subtle way.", I thought
if this article explains this even a little bit, it'll make my day. Most
articles on this subject go nowhere. Or maybe I wasn't tuned in and prepared
to resonate with this article before. The way an article is written, the
simplicity of expression of the math, etc., all determines how one absorbs
the material.
The next sentence in the introduction really impressed me.
"In a nervous system it is usual for extremely precise over-all results to
arise from the functioning of a collection of components which have very
modest precision in the individual construction and behavior."
Again I thought, wow, I hope this gets explained. Well, I tell you, I read
this single article for over 3 hours. I can't believe I never found this
article before. But then again, I wasn't tune in and ready for this before.
This article sort of bridges the gap for me between understanding how a
single neuron in a single neural circuit expresses itself as a larger
ensemble of neurons.
Bruce Knight,
the author of this article, published it in 1972. It's called "Dynamics
of Encoding in a Population of Neurons." It's perfect.
Dec 20, 2010
Around 10 years ago I downloaded Wade Lutgen's beautifully coded wave
packet scattering program. I put the program's simulation on video today
(it's 5.7 minutes long).
The Granularsynthesis.com website
features music artistry composed of "grains" which are sound atoms. The
following video called "decay" is made up of images of sound atoms which
are decomposed wave packets in frequency-time streams you see in signal
processing. These sound atoms are the building blocks of speech and
musical sound streams at the micro and milli second time scale. It's the
time scale at which we study the synapsing signatures of neurons. Pretty
amazing.
decay by Nikola Jeremic
Dec 18, 2010
This is a tribute to
Walter Freeman. His work
inspires me. He's done what many people, or at least I, would have
liked to have done for a few years, that is, to study the brain in a
clinical laboratory setting. Recently, he published some notes on
the theoretical foundations of his lifetime work on EEG called the
Hibert transform. The Hibert transform explains how we can
use data samples, the observation data taken in experiments, to use in
signal processing. In doing wave packet analysis, the Hilbert transform
makes working with the math more intuitive. I'll write an article
about this in the next few days.
When you want to model brain functions, you need experimental work. So
I've really tried to understand what Walter Freeman published. But that's
hard to do because of all the details. Today I was studying a publication
of his called "Application of Hilbert transform to scalp EEG containing
EMG",
Freeman, Burke, Holmes (2003) [PDF].
I know, as I read through these works, that I'm not really getting the
optimal feel for what's written because I wasn't there doing the work.
But Walter Freeman has been incredible in giving us his interpretation
of what the brain is really doing in the books he's written. Thanks
for helping us understand ourselves.
Dec 7, 2010
I watched the beautiful sunset the other day wondering about what to add to
this site. I'm studying again how Satosi derived his clustering formulas
to get the entropy equation. I think that's something I can understand.
But then I began to wonder if I should look at Renyi entropy. That's
really strange. The formula for Renyi entropy is:
where as q -> 1 Renyi entropy tends to Shannon entropy.
I can sort of feel what the natural log of the summation is like,
but how did Renyi get 1/(1-q). That's a singularity at q=1! What
a strange behavior for a limiting singularity to approach Shannon's
entropy. Renyi was a mathematician who worked with Erdos.
To get the full implications of this formula will take me a long time
so I've just left it alone. I know my limitations. I know that this quest
is bounded, so I have to smile.
December 3, 2010
I just put this page back online today. Ten years ago, most friends
who were not specialists in this field, said they could not
understand much of what I had written. But a few, rather small number,
seemed to appreciate the "strangeness." There are some great and
beautiful concepts that are simple and elegant enough to be in
wonderment of like entropy. David Bohm calls this awareness of a
streaming universal energy a "movement" which he saw all around him.
When I first read what he wrote, I had some doubts about what he
was saying, but now it's what I have incorporated into my worldview
completely. It's was strange at first, but not now.
October 14, 2010
Began updating this webpage with new contents about computing paths in
programming code. I naively started this web page in 1998, inspired
by the beautiful design of recurrent neural networks. However, I soon
realized that I was getting nowhere studying classical error minimizing
matrices in neural networks. But it's hard to overcome old notions.
I knew the neuron used the temporal code to talk to each other.
But in my developmental work I kept using the old data structures
and algorithms of the rigid networks. About 2001 I gave up on
really developing my ideas on waves and neurons. In 2007 thinking
that I might never find an interesting developmental path, I took
this website offline.
But about a year ago I just started feeling more confident about
the "wave model" of the neuron. This is because I could integrate
aspects of entropy or information into what I thought might be happening
in neural circuits.
"The study of the Vedas is really a study in the intricacies of the
human psyche."
That is a quote from Jeanine Miller, Vedic Myths and Vision,
http://jeaninemiller.org. She's was a remarkable scholar, and mystic.
In her book "The Blazing Dragon of Wisdom" she says,
Sri Swami Sivananda who studied the mind in the Hindu tradition lived a
generation ago at a time when technology had not yet had machines like
the EEG imaging or MRI to see our thought patterns. In the generation
since Swami Sivananda's time, we created computer models of our physcial
world and the brain which has changed our perception of the mind.
One more thing, the Tibetan mandalas have a symmetry really close to
what's seen in harmonic energy fields. It might be that the mind may
have hard-wired a kind of Fourier transform as a means to do pattern
recognition. This is because the Fourier transform so efficiently
reduces dimensionality using symmetry and information compression.
Temporal Automata Neurons
