Home -> Old-Log

Sep 7, 2015

Knowledge is a function of being.

"Knowledge is a function of being." is a quote from Aldous Huxley's The Perennial Philosophy. This work is a masterpiece. For months, I've focused on this work, drawn to it by an obsessive kind of focus that this work has created in me ... that this is how we humans think reality is. Aldous Huxley was deeply aware of our limitations on how we perceive our world.

Eknath Easwaran equates the perennial philosophy with sanatana dharma.

For much of this year, I've worked on writing software algorithms for parsing language. As efficient as the algorithms are, there seems to be no intelligence in it - not that of a kind a "sentient" being can possess. The algorithms are just machines. The missing element that gives "meaning", in the human sense, to intelligence is that the object of the algorithmic processor has life. But, "What is life?" This vague notion has been a reoccuring thought process of mine for months.

I think there's two reasons for my confusion about the notion of intelligence. First, is that what I see as intelligence in Nature is more than mechanistic. Yet, secondly, at a deeply foundational level, I see the world as purely mechanistic. This is a really interesting paradox. It's a real part of me.

Erwin Schrodinger wrote that "Consciousness is never experienced in the plural, only in the singular." Kind of like a Hindu he was. This statement has made people ponder whether an electron can possess a kind of consciousness or intelligence. Can it be this extreme? Thinking about this kind of reality, related to the paradox described above, is as if you're walking on a very quiet path. In a sense, it is absurd, but yet a wonderous walk in mystical stillness.

(Updated Dec 6, 2015)
Dec 31, 2014

uLisp - Another LIsp Interpreter

Language interpreters have a high degree of complexity. David Betz's Xlisp-2.0 released in 1988 is beautifully written. It uses many of the classic data structures in computer programming like dispatch and hash tables. Xscheme-0.28, which was released in 1991, has an even higher level of complexity than Xlisp-2.0 because Xscheme uses a bytecode compiler, and moves towards a virtual machine style of architecture used to process bytecodes. Xlisp-3.3, released in 2002, is a very complex piece of software which is more accessible after you have studied Xscheme-0.28. uLisp is my modified edition of Xlisp-3.3.

Computer language interpreters and compilers have basic similarities. In studying computer languages like Lisp, I've learned the language on a deeper level, and have gained a lot more appreciation for the language and its features by trying to understand Lisp interpreters.

You can download uLisp here.

(Updated Dec 6, 2015)
May 22, 2014

rtLisp Interpreter

I spent the last month studying Bill Birch's beautiful code called reflisp-2.67. You can really learn the programming language well not only by reading the language manual, but by studying an implementation of the language; either by reading the implementation manuals or perusing the source code. You can see how every programmer's mind is intrinsically different in their likes and dislikes of a programming language.

The "rt" in rtLisp, stands for real-time. I'm studying this program to learn how a real-time, multi-threaded or parallel lisp is implemented. For me, the Lisp language is at the summit of the programming beauty and elegance mountain.

Some Lisp implementations I've studied which have influenced the code in ulisp are xscheme-0.28, by David Betz, and minischeme-0.85, by Atsushi Moriwaki.

You can download rtLisp here.

Feb 9, 2014

The Upanishads and the Mind

Mind precedes matter. This is Vedantic theory.
Matter precedes mind. This is scientific theory.

--- Swami Sivananda
Mind - Its Mysteries
and Control

"The study of the Vedas is really a study in the intricacies of the human psyche." That is a quote from Jeanine Miller, Vedic Myths and Vision, http://jeaninemiller.org. She's was a remarkable scholar, and mystic. In her book "The Blazing Dragon of Wisdom" she says,

One of the greatest gifts of the Rigveda to the world of thought is its vision of harmony, wholeness, divine solidarity, of Cosmic Order which subsequently became the fountain-head of the peculiarly Hindu doctrines of dharma and karma, and of the law of vast cycles, ...

As the Rigveda falls within the esoteric tradition which concerns the mystical insight of humanity, the present investigation will be taken from the third level of interpretation which Indian exegesis labels adhyatmika. The understanding of mysticism requires an intuitive insight not developed in every one. For those who fail in this the ancient insights will be meaningless and this book had better be left alone.

The Vedas, more specifically, the Upanishads are great works on the mind or "pysche". They were written 3500 years ago. There's more than ancient wisdom in the Upanishads. As we live in the age of "modern" science, the ancient Hindu rishis or scribes who wrote the Vedas lived in the age of a "spiritual" or a "pyschic" science. Their tool for verification of truth was meditation. The Vedas reflects ancient science as expressed in the studies of Carl Jung. In my opinion, we'll start appreciating the value of the Rigvedas with respect to psychology as modern day computational neuroscience models reveals how specific network structures in our brain makes emergent our thoughts and behavior.

Sri Swami Sivananda who studied the mind in the Hindu tradition lived a generation ago at a time when technology had not yet had machines like the EEG imaging or MRI to see our thought patterns. In the generation since Swami Sivananda's time, we created computer models of our physcial world and the brain which has changed our perception of the mind.

In the quote above to the start of this log entry, we can see the essence of the Upanishads reflected in Swami Sivananda's words.

Mind precedes matter. This is Vedantic theory.
Matter precedes mind. This is scientific theory.

We see that the great Hindu rishis of the past did not differentiate or understand the significant difference between the brain and the mind as we do today. We now appreciate the complexity and wonder of the biological brain. But there's still much to learn about the mind from what the ancient Hindus wrote in the Upanishads.

If you read the Upanishads, remember this: the "mind" in the Vedanta refers to the human psyche ... not explicitly to the brain. In the Vedanta, the mind is the spirit. In scientific theory, the mind emerges from the neural activity in the brain. Understanding this, I've found a wonderous union between what's in the Vedas and modern science.

Jan 27, 2014

Energy and the Mind

There is a Light that shines beyond all things on earth,
beyond us all, ...
This is the Light that shines in our heart.

--- Chandogya Upanishad, 3.13.7
The Upanishads,p 113
translated by Juan Mascaro)

This log entry is about our ability to perceive the physical energy surrounding us. Our perception of the energy fields all around us is masked because we don't usually experience energy directly as a vibrating field especially as it propagates as waves.

Energy is propagated or transported as waves because of the spacial medium or "field" we live in. For example, we hear sounds which can propagate in the air, water, or solid earth. More specifically, for example, this energy is transported on the surface of a pond because of the compaction and rarefaction of the water medium by an impulse like a rain drop.

This truth, that energy moves in a vibratory way, by compaction and rarefaction, is a universal truth about our physical world. If you study the mathematics of differential equations, you'll see how essential harmonic or periodic sequences are in describing our world of energy. I, intuitively or naively, did not sense the all pervading significance of this truth years ago. We're immersed in energy fields as fishes are immersed in water. It's taken me years to get an appreciation of this simple and beautiful fact. My "unawareness" of energy fields is analogous to how our cognizant focus on breathing is masked by our nervous system because breathing is programmed autonomically.

Nowadays, I feel or sense a vibratory field surrounding almost everything. It's especially true when I look at plants like ferns. The branches and leaves on ferns have an incredible symmetry. Ferns, it seems to me, have a kind of primordial, fractal geometry. When I walk in the topical forests where I live, I really sense, in an indirect way, the vibratory electic fields in plants. By "indirect way", I think I mean by my own mental constructs which I've built in my own mind by training. These mental constructs were built primarily from observing the structure of sound waves propagating through water.

An ubiquitous property of energy fields like sound fields for example is symmetry. Plants, in general, it seems to me, do not show a great amount of symmetry directly. In physics, symmetry could be hidden by in a multi-dimensional topology. So I wonder about the chaotic, fractal nature in the appearance of plants. How do we unravel this hidden geometry in plants? I don't know. I've been thinking about the topology of plants since I was a teenager. Here, for me, is where I start entering into the mysterious and great unknown.

When I was in Bali, I became seriously ill from fasting and too much meditation in the Hindu tradition. I had lost too much weight. My quest to explore my mind had almost killed me. I think I had a near death experience. This is the first time that I think I experienced a sense of "Prana", the life-force, as a kind of emotion of a mystical and peaceful universe which was gently leaving at the bottom of my head. I had searched for manifestations of Prana or Chi for many years without finding any experiential clue to what this is. I really don't understand any of this. That Nature keeps our mind so stable that we can only experience these seemingly unique experiences only under the most stressful condition on the body is truly amazing (and humbling). Maybe one day I will understand what happened, then I can tell you more about this experience.

After this experience, I became really absorbed by the paintings of the Balinese artist Wayan Djumu. It seemed to me that Wayan Djumu would look at his paintings sensing a kind of energy on the surface of his canvas. He would run his hand over the canvas like he sensed an intricate texture on it. At that time I didn't understand what he was doing, but I think I do now. It's that primordial energy, the vibratory field inside of nature that he sensed. If I ever get back to Bali, I'm sure to ask him about this. Then again, it might be only me.

I can also sense this energy in the work of the Balinese artist Ketut Budiana, the Japanese ceiling painting in the Zen Tenryu-ji temple, or the paintings of the shaman Pablo Amaringo.

I think there's so much of our mind that's hard-wired. That is, what we think is determined by our biology or DNA. Our hormones determine much of how our brain wires itself. And our human evolution over thousands of years has gotten the brain to be a remarkably stable platform for thinking. The biochemistry of the brain is an exemplar of control. There's so much more to learn about the mind, and maybe I should really stop here in humble respect to the incredible order in the universe. The Hindu mystics warned against speculating on this edge of reality.

It seems to me, however, that there are circuits in the brain that at some times can resonate with the vibratory nature of the universe in a mystical way. Is it brain chemistry, or the structure of Nature itself that's driving the mind here?

One more thing, the Tibetan mandalas have a symmetry really close to what's seen in harmonic energy fields. It might be that the mind may have hard-wired a kind of Fourier transform as a means to do pattern recognition. This is because the Fourier transform so efficiently reduces dimensionality using symmetry and information compression.

The Fermi-Pasta-Ulam, FPU, paradox interestingly reveals the structure of nature, and energy propagation. Usually we think of energy dispersing while propagating in a uniform media. However, if this media has impurities in it, or is non-homogeneous structurely, then non-linear resonance effects can cause a periodic type of propagation in which the energy congeals instead of dispersing. It's as if time reversed; switching the direction of entropy. But this is really looks like a resonance phenomena enfolded in itself since time can't go backwards. My point is that there are multi-dimensional phenomena in physics (the real physical world) we, or maybe only me, don't even understand yet. We're a long way off from putting too much constraints on what we can really know about the mind.

As great as the infinite space beyond
is the space within the lotus of the heart. ...
Whether we know it in this world or know it not,
everything is contained in that inner space.

--- Chandogya Upanishads, 1.3
The Upanishads, p 142
translated by Eknath Easwaran

Jan 20, 2014


O Kali, my mother full of bliss! ...
Thou art the Mover of all that move, ...

--- The Poet Kamalakanta
(from the gospel of Ramakrishna Paramahamsa)


With respect to my work, I spent most of last year on the software project Kali. So I thought alot about temporal sequences, and how to use the "duration in time" as coefficients in temporal sequences. This log entry is about the study of time; philosophically, and I think in a practical way which most philosophers do not experience. That is, I studied time as a computer programmer.

In the early 1980s, we exhausted great effort trying to optimize processor speed, working with machine registers and low level assembly language. You had to program asynchronous software modules or procedures (daemons) efficiently time splicing the software module you wanted executed. I spent a whole bunch of hours trying to smoothly multi-task "real-time" processes. You feel like you're holding chunks of "computing energy or power" on your finger tips as you can determine which resources get computing time.

From this kind of experience, I think you tend to think of time duraton as a real quantity; not an illusion. It is a necessary physical dimension existing independent of (or "orthogonal" to) the spacial dimensions. The reason I say this is because many fine theoretical thinkers like physicists and mathematicians go too far away from attributing to time a real value, and placing too much emphasis on time as emerging from a process. For example, I really admire the abstract studies on time by Whitehead and Russell; but respect to operationally creating information processing systems, they didn't get it.

A mechanical thinking machine requires time to reduce complexity. It's in the nature of physical laws to consume energy "or time" to produce order; and a mechanical thinking machine is an epitome of orderliness. I've seen the enormous electrical power systems required to run the cooling systems for computer rooms.

Last year, I spent time studying correlation coefficients in temporal sequences. From this, and re-reading Satosi Watanabe's article "The Symmetry of Physical Laws: Part 3, Prediction and Retrodiction" it reinforced the notion in me of the entropy's role in creating order. To me, it seems that order in the universe arises out of a divine like rule in which the integral (or the organic like) system's evolutionary process is limited by the amount of physical states it is allowed access to. That is, systems evolves in time to a higher level of order because the law of entropy paradoxically reduces the number of physical states (or micro-states) it can transition to as a function of time. This is a kind of indirect dimensionality or complexity reduction. The limiting form of the entropy function is logarithmic which really suppresses the available states the system can transition to.

Professor Watanabe's insights into the role of entropy in physical processes is incredible.

Born in a corner of the universe, where entropy is increasing in one direction of time (and decreasing the the other), life is to survive and expand. But, in which direction of time is it to live, grow and age? The only possible direction is the one in which the future is foreseeable and controllable, so that by adapting itself and acting suitably it can satisfy its needs and desires. The foreseeable and controllable direction is the one in which causality works. Causality works in the entropy-increasing direction, but not in the opposite direction.

--- Satosi Watanabe
Causality and Temporal Irreversibility (1977)
Japanese Studies in the Philosophy of Science
edited by Francis G. Nagasaka
Kluwer Academic Publishers

That's more than incredible. It's divine.

I was really glad to see Professor Watanabe's research papers on the symmetry of physical laws posted on the Internet Archive web-site:

Symmetry of physical laws. Part 3 : Prediction and retrodiction. (1955)

His research paper no.4 on "Intersymbol Correlation of Finite Range (1954) which I have a hard-copy and used to edit a version of this paper for study is on the same website at:

A Study of Ergodicity and Redundancy Based on Inter-Symbol Correlation of Finite Range (1954).

He always had a smile for me.

OM Namo ...

Nov 3, 2012

Turing's Circuits

Oct 28,2012

Mystical Notes on Spinoza, and the Upanishads

When I was 12 years old, I spent hours reading Spinoza's Ethics. Now, I love the reading the Upanishads, although, I know it's crippled symbolically in religious terms. It's a joyous celebration of human spirituality which I could not have appreciated as deeply as a young boy.

Spinoza, the blessed one, had much in common with the Hindu rishis who wrote the Vedas and the Upanishads. As flawed as Spinoza and the Upanishads are philosophically, the singular concept that makes the ideas expressed in this philosophy sanctified to me is the non-duality of the mind and body. (Philosophically, as a discipline, the duality of the mind and body is a fundamental stepstone in the path of a philosopher.) Our consciousness, self-awareness, or in general thoughts are not outside or separate from our body. Our brain generates our thoughts which emerge from harmonic synapses. This, for me, is the message of Brahma taken symbolically.

It's easier to understand why Spinoza would not separate mind and body because of his desire for the inclusivity of all "substances". As for the Hindu yogis in meditation, the mind (the brain's energetic synapses) permeates the body. The organic mind and body functions as a unit in a seemingly uncountable number of processes. The Upanishads affirms this unending chorus of divinity expressed in the union of all material and spiritual (mental) energy in Brahma. All that is manifested comes from a singular substance in Brahma.

For me, the realization that the mind and body are not separate is the most significant concept in my philosophy. This thought gives all that I can see a mystical, silent essence. I cannot describe it. This realization shapes how I view universal order, my own identify, and my quest for spirituality. If mind and body are not separate, then this "I" will no longer exist after I die. "I" will be as "I" was before "I" was born. However, this view does not preclude the non-existence of "God". I like to think as Carl C. Jung thought and said about "God", "I know God exists." The beauty in this universe allows me no other alternative.

Oct 21, 2012

Time and the Temporal Automata

With respect to intuition, modern quantum mechanics and relativity spins the classical concepts of physics on its head. The most profound conceptual transformation of modern physics, in my opinion, has been in regards to the notion of causality. Modern physics reveals an absolutely astounding intrinsic order permeating the universe. But its unfamiliar phenomenology of an unintuitive indeterminism in space and time makes me have to think twice about casuality.

The study of neurons and thoughts fall at the edge of the "small" quantum world. Most of the strange effects of the quantum world do not seem to apply in our daily experience, and in all instances so far need not apply to the way I've thought about neurons. The uncertainty principle which I have applied to studying neurons can be considered a wave phenomenon independent of quantum mechanical effects. But I don't known how much the effects of the "small" quantum world affects neurons. My guess is that I can sufficiently use classical mechanics. I've been intuitively working on the premise or guess that neurons can be sufficiently modelled as a classical system.

Philosophically, I like Kant's view of reality. I think Kant was honest with respect to what we can or cannot know. What we know comes from experience. We experience time. I also think we experience time as a movement from seeing or hearing objects in our world being transformed. Time is the ground of existence, and it's hard to conceptualize stuff for sure beyond our existence in this mediate temporal instance.

I wanted to write about time because it's such an important variable in the models we use to describe our world. This is especially true in describing how we communicate in signal processing verses communicating using language. In signal processing, time and its complimentary variable frequency, are the essential variables of study. As we start using symbolic language, the need for time recedes. With respect to modeling neurons with signals, time dominates that world. The model we use to describe communicating neurons is the model of temporal automata or time machines. Encapsulated in the neuron's energy, which can also be describe by frequency, is information. Information in signals has been studied extensively in signal processing, particularily in using wave-packets or wavelets to describe signals.

When using a model to describe our world, it's essential that we get as much of an operational view of our world that this model can provide. This includes, in my opinion, a broad view of the epistemological or representational view of the model. For example, when I studied the lisp language in the late 1970's, I spent lots of time on studying the repesentation language KRL. KRL is a reflection on what your belief system is, and how it pervades the network you're building. With respect to temporal automata, time is essential in describing computation. Computation is constrained by physical processes which is determined by time. In fact, using time machines means considering all processes as a computation, and all computing processes involves time at the fundamental level. I also recognize that it is in my belief system that I've made the existence of time the basis of computation. Computation is process, and process is computation in the temporal automata model.

But also, it's more than just thinking of the temporal automata model using computation or process as sufficiently requiring time. Time is necessarily required because time allows changes or transformation to occur in the form of its duration between changes. Information is embedded in duration.

Oct 11, 2012

Temporal Automata Neurons

Alan Turing wrote about neurons as automata in the 1940s. In his state diagram you can see his "connection-modifier" which makes it possible to modulate the inter-neuronal signals much like transitors do.

Recently I came across an article by Thomas Wennecker's called "Finite State Automata Resulting from Temporal Information Maximization." For me, it's an astoundingly great article because I've been working on trying to understand how neurons "minimize" or reduce informational complexity through shortening their synaptic firing duration. In essence the world would look clearer to a neuron if its sampling time was small. This is consistent with Nyquist's rule for sampling electronic signals. The spacial analogy would be looking at the world through pin-hole glasses.

My route to thinking of neurons as masters of dimensionality reduction went across paths trying to invent computer algorithms in pattern recognition. I discontinued working on the matrix formulation of neural networks using weight matrices in favor of using state automata algorithms like the parallel Aho-Corasick algorithm. I think this is so because the matrix black box approach does not intrinsically provide temporal state information inside its matrices. In using the parallel automata algorithm like Aho-Corasick, I (or you) could think of the neuron as making decisions in time. And it seems to me that neurons that make decisions in the shortest amount of time reduces dimensionality or complexity the most.

On the theoretical side, I've been trying to study how neuronal circuits can be wired so as to most efficiently use information in long temporal sequences. The complexity of a sequence increases exponentially as a function of its length, so I think groups of neurons must "architect" themselves as some sort of temporal filter. To build this kind of filter, I've been studying the correlation index of symbols in long time frames. These long time frames are called ergodic in models using entropy. It seems to me that if you could correlate the information in the symbols in a sequence (like neural pulses or spike trains in a nerve circuit) to each other, then you could build an efficient pattern matcher automaton. In fact, I spend lot's of time on this problem in a software project.

There have been lots of research on correlating spike timing arrivals between pairs of neurons and large groups of neurons, eg.,studying correlation in interspike intervals, ISI.

I like the way Bruce Knight proposes how neurons synchronize themselves in a population of neurons. He shows how a very simple model of a neuron can perfectly replicate the input stimuli. For me, his 1972 article, "Dynamics of Encoding Neuron Populations", is based more on his intuition than on his effort to laid down an axiomatic foundation to base his differential equations upon. But his proposal makes the most sense of any other model of synapsing neurons because of its simplicity. In 2000, he added more general mathematical features to his model saying that the math would provide more efficient simulations. But I like the simplicity of the 1972 article. The beauty is impressive.

A consequence of thinking that neurons replicate input stimulus efficiently with minimal delay in time makes me comfortable with the idea that neurons work like temporal automata. The temporal duration of spike trains is in the order of milli-seconds, and neuronal circuits make decisions within this time frame.

I've thought up a very simple way to show neural synchronization using the idea of spacial clustering as constructed by Satosi Watanabe. Spacial clustering is analogous to temporal synchronization if you use the superposition principle of waves as a "force" to coalesce the coincidences of the spike pulses. The waveform of similar pulses reinforce each other while those that are not "coincident" become noise. Clustering related synchronization reduces dimensionality. I'll write an article on this later.

Lastly, for today, I like to write software using the scheme programming language. I use Bigloo and Gauche. Both implementations are works of art.

Oct 7, 2012

The Turiya and Consciousness

Through the years I've worn out my copy of "The Upanishads" translated by Juan Mascaro, @1965, published by Penguin Books. It's only 144 pages long, and the chapter I think that is the most impressive is the Mandukya which is only 1 and a half page long. The Mandukya Upanishad is about OM, and the states of consciousness. I think it's the greatest text ever written on the exploration of the mind and consciousness.

He is Atman, the Spirit himself, that cannot be seen or touched, that is above all distinction, beyond thought and ineffable. In the union with him is the supreme proof of his reality. He is the end of evolution and non-duality. He is peace and love.

page 85, Mandukya Upanishad

The quote above is about the fourth state of consciouness: the turiya. Some Hindus say the turiya is the seed of consciousness. The turiya occurs at the cusp of time ... the efforescent concrescence between the falling away of the non-existent past and the potentiality of the future. The turiya is those primary or principal synapses that unfolds our awareness into the future. The turiya are those synapses forming the chain of spike trains in our principal neural circuits emanating from the subthalamic nucleus and global pallidus at the base of the basal ganglia.

NIH/NLM/Journal of Anatomy: Synaptic organisation of the basal ganglia
Synaptic organisation of the basal ganglia [PDF]

This whole thing is a dynamic process; what David Bohm would call movement, or the ground of all existence. In his introduction to "Wholeness and the Implicate Order" David Bohm wrote:

To meet the challenge before us our notions of cosmology and of the general nature of reality must have room in them to permit a consistent account of consciousness. Vice versa, our notions of consciousness must have room in them to understand what it means for its content to be 'reality as a whole.' The two sets of notions together should then be such as to allow for an understanding of how reality and consciousness are related.

David Bohm, 1980, Wholeness and the Implicate Order

The quotation above is in essence about the spiritual quest of the seekers who wrote the sacred Upanishads.

In his article "Time and the Probabilistic View of the World" compiled in the textbook "The Voices of Time" edited by J.T.Frasier the physicist Satosi Watanabe remarked that here should be a pragmatic fusion of objective and subjective reality. The quantum wave function only describes the possibilities of what we see.

Quantum physics is in perfect agreement with the viewpoint that science should deal with a world-to-be-acted-on rather than with a world-to-be-contemplated.

Satosi Watanabe, 1966

Like David Bohm, Satosi Watanabe reacted against the rigid positivism of popular science. In his article "The Foundations of Cognitive Relativity," 1991, he warned against a narrow focus on the reality of particular objects, over reliance on reductionism and notions of strict causality. Quantum physics is a field theory, and so can turn the classical notions of order inside-out. The study of field theory is by nature like studying an integrative organism; a whole body.

Studying quantum physics as an organism includes merging the relativistic effects of space-time, and causality. That is, with respect to time, time and space are interrelated, and the past and future are interrelated as so far as a quantum can be considered a single body entity. I think this is right, but maybe I'm speculating. You have to rely on experimental proof for all of this. So the burden's on you as well as me.

Time plays a special role in expressing the unfolding of the quantum field which is described in probabilistic terms. This is because the Bayes theorem on conditional probabitily is essentially what computer programmers call an "if-then" statement. This "if-then" constriction is a causality constraint in time: "the" temporal dimension imposing "the" primal constraint. From these concepts Satosi Watanabe gives the quantum field an especially subjective reality.

... space provides room for being and time provides room for becoming.
... it is taking the form which a thing has been intended to assume. Becoming has intent, yet it has no plan. Becoming is making of the yet-unmade.
... time is the vehicle of freedom and value.

The logician Alfred Whitehead who wrote "Process and Reality" in 1927 was also an amazing thinker. His views of time also seems reactionary when compared with the classical philosophers like Aristotle and Socartes. Alfred Whitehead wrote on creativity as a resonant superposition of our thought processes emerging in a splice in time. The elements which makes up Alfred Whitehead's "process" philosophy is what David Bohm and Satosi Watanabe ideas seem to parallel.

That 'all things flow' is the first vague generalization which the unsystematized, barely analysed, intuition of men has produced. It is the theme of some of the best Hebrew poetry in the Psalms; it appears as one of the first generalizations of Greek philosophy in the form of the saying of Heraclitus; amid the later barbarism of Anglo-Saxon thought it reappears in the story of the sparrow flitting through the banqueting hall of the Northumbrian king; and in all stages of civilization its recollection lends its pathos to poetry. Without doubt, if we are to go back to that ultimate, integral experience, unwarped by the sophistications of theory, that experience whose elucidation is the final aim of philosophy, the flux of things is one ultimate generalization around which we must weave our philosophical system.

Alfred Whitehead, 1927

Lately, I've been thinking about the fractal nature of the computational abilities of ensembles of neurons. Neurons have to work around the complexity of processing information. If the job of neurons is to process information, then my opinion is that the neurons' primary problem is reduction of dimensionality. Groups of neurons working together can produce multiple gains in computational power, but this is only a linear gain. So I've come to think that neurons must reduce exponential information complexity through making decisions in a time-sliced manner. Neurons make computational decisions in the shortest amount of time possible. This allows the possibility of using computational algorithms like Turing's machinery. A turing machine is the simplest computational machine using a memory storage unit.

But there's a limit to how short a neuron can time-split information because of the uncertainty principle. The signal-to-noise ratio, SNR, for communicating neurons follows Nyquist rule, but neurons cannot drive up their frequency of computation indefinitely because they'll overheat.

So I've come to assume operationally that individual neurons like a mass essemble of neurons must essentially perform computations the in same manner. Neurons compute the same across a fractal of dimensions.

Each computation of a neuron taking place as a synapse produces a spike train. The information in this spike train is enfolded in a pulse constrained by the uncertainty principle or wave packet. When a synapse occurs I think we can think of the information as contained inside the spike train and enfolded inside of a wave packet. A synapse unfolds the potential in "our" universe. There's nothing mystical about this is it. I'll write about this in detail within a couple of days.

But somehow, this process of the unfolding of cognition or consciousness always raises emotions in me. Maybe it's the ground of all existence.

He is Atman, the Spirit himself ...

Jan 7,2011

Dynamic movement as represented in an abstract sequence is always greatly simplified. For the past 2 days, I've been studying Satosi's 1953 article on ergodicity and sequence correlation index after I received the hardcopy monograph I ordered which has handwritten formulas (which must be Satosi's) in it. There're 14 pages for mathematical formulas Satosi uses to derive his correlation index, W. There's more discussion about the reasons for using concepts like ergodicity and redundancy in this report than in his 1960 article on Multi-Variate Correlation.

The formula for W above work in principle for ergodic Markov chains. Satosi's criterion was that the symbols in the sequence, S, had to be unique (which also meant the sequence progression eventually made any conditions on the starting symbols not matter).

Anyway, the interesting part of a dynamic movement in Nature seems to be at the beginning of the movement ... entropy at work always increasing. I took snap shots of a wave packet simulation. The wave packet bounded in a square box hits a very thin vertical potential barrier in the middle of the box. The snap shots are taken at 3 secs, 10 secs, 30 secs, 5 mins, 30 mins and 10 hrs.

In terms of dynamic movement, the wave packet settled into a rather constant mode after 5 minutes into the simulation. According to ergodicty, it would not matter what type of initial boundary barrier we had setup in the beginning of the simulation. Eventually, after a long time, the waves in the box would look like white-noise; just a noisy speckle pattern. In terms of the simulation, this would be true primarily because of the floating point number roundoff errors in the simulation equations.

Jan 4, 2011

Today I read some articles on spike train synchrony.

1. Measuring multiple spike train synchrony
2. Time-resolved and time-scale adaptive measures of spike train synchrony
3. Nonperiodic Synchronization in Heterogeneous Networks of Spiking Neurons
4. Interspike intervals, receptive fields, and information encoding in primary visual cortex.

In articles 1 and 2 above, the authors rigorously study the relationship of temporal spike trains. It's an admirable scientific study trying to pin down the real grounds for what's inside a spike. Their principal analytical object or tool is what they call the Inter-Spike Interval, ISI, which is "parameter free and time-scale adaptive (Kreuz et al., 2007)". Their construction of the "SPIKE" distance which is the temporal interval between two spike is interesting. They actually reconstruct the differential and integral cells (what I used to think of as cellular spaces in calculus) for analysing spikes. These are two excellent articles. But it's also interesting that in article 1, the authors say in the conclusion that

"First, it is obvious that no measure that results in a single number quantifying the synchrony between two or more spike trains can be adequate to deal with all kinds of potential coding schemes (e.g, time coding, rate coding and pattern coding; ..."

Article 3 is a good study of neuronal synchronous temporal-dependent plasticity, STDP, and the highly synchronized network response to stimuli. My questions settled arount the equations used in this article. The neuronal circuit equations contained interconnection weights, w_ij, so it may not matter what boundary conditions you set in the beginning; you could get periodic behavior in the end if you look for it like I do. I'd just eliminate the word "non-periodic" in this article and use "irregular" as they do in their introduction. Synchronization can occur without resonance. This article, while confusing in places, was very interesting. This article is a fine, extremely detailed study of the simulation of neuronal topology and some experimental data. I couldn't absorb most of it today.

The first 50 pages, 1st chapter, of a recently published book, August 2010, edited by Christoph von der Malsburg, William A. Phillips and Wolf Singer is online at Dynamic Coordination in the Brain. The editors comment, on page 17 on the section called "Temporal Structure and Synchrony" which the authors of article 1 above carefully avoid implying in any manner, that

"One possibility is that spike rate and spike synchronization operate in a complementary way such that salience can be enhanced by increasing either or both."

The editors, in this paragraph on temporal synchrony, suggests that synchronized rate codes, high frequency circuits, and neuronal circuit inhibition, leads to a host of higher level cognitive functions which I've always assumed to be true or could be true from a computer programming perspective. In writing software you a free to create, and communicate to others whatever you think are the most optimal structures. You can't really do this when you're bounded by experimental, laboratory evidence.

I think spiking rate, and temporal coding are synonymous. I have described both with the same mathematics.

Today, I re-read Bruce Knight's 2008 article, The Faithful Copy Neuron. The simplicity of the idea of an ensembly of neurons containing an innate propensity for synchronization is very appealing. There are other concepts which add elegance to the model such as "revised time" which seems to play the role of phase shifting the spike trains. I don't understand this yet. I have not thought about it enough. But I used to wonder about entropic correlation of a set of sequences. The dynamic change of a set of sequences with respect to its order in time usually occurs in the begin as it is constrained by its initial "boundary" condition. The sequence as it progresses ergodically appears like the beginning of the sequence. This is just the way it is with most simple sequences in Nature.

Article 4 referenced above, covers essential technical analysis. The authors state at the end of the article that

"To accomplish this type of decoding, neurons need not do anything more sophisticated than be sensitive to the durations of individual ISIs. This sensitivity can be embodied in a single synapse and does not require averaging across stimulus repeats, stretches of time that may be long compared with the time scale of firing rate modulation, or a large population of neurons that carry similar information."

This is quite an insightful conclusion. It also remainds me of the significance of the prefix in sequences in pattern matching strings.

Dec 24, 2010

I really found what I was looking for today. I was skimming through articles on neural coincidence detection for the pass few days. When I read this article's summary "The firing rate of a population of neurons is related to the firing rate of a single member in a subtle way.", I thought if this article explains this even a little bit, it'll make my day. Most articles on this subject go nowhere. Or maybe I wasn't tuned in and prepared to resonate with this article before. The way an article is written, the simplicity of expression of the math, etc., all determines how one absorbs the material.

The next sentence in the introduction really impressed me.

"In a nervous system it is usual for extremely precise over-all results to arise from the functioning of a collection of components which have very modest precision in the individual construction and behavior."
Again I thought, wow, I hope this gets explained. Well, I tell you, I read this single article for over 3 hours. I can't believe I never found this article before. But then again, I wasn't tune in and ready for this before.

This article sort of bridges the gap for me between understanding how a single neuron in a single neural circuit expresses itself as a larger ensemble of neurons. Bruce Knight, the author of this article, published it in 1972. It's called "Dynamics of Encoding in a Population of Neurons." It's perfect.

Dec 20, 2010

Around 10 years ago I downloaded Wade Lutgen's beautifully coded wave packet scattering program. I put the program's simulation on video today (it's 5.7 minutes long).


Wave Packet

The Granularsynthesis.com website features music artistry composed of "grains" which are sound atoms. The following video called "decay" is made up of images of sound atoms which are decomposed wave packets in frequency-time streams you see in signal processing. These sound atoms are the building blocks of speech and musical sound streams at the micro and milli second time scale. It's the time scale at which we study the synapsing signatures of neurons. Pretty amazing.

decay by Nikola Jeremic

Dec 18, 2010

This is a tribute to Walter Freeman. His work inspires me. He's done what many people, or at least I, would have liked to have done for a few years, that is, to study the brain in a clinical laboratory setting. Recently, he published some notes on the theoretical foundations of his lifetime work on EEG called the Hibert transform. The Hibert transform explains how we can use data samples, the observation data taken in experiments, to use in signal processing. In doing wave packet analysis, the Hilbert transform makes working with the math more intuitive. I'll write an article about this in the next few days.

When you want to model brain functions, you need experimental work. So I've really tried to understand what Walter Freeman published. But that's hard to do because of all the details. Today I was studying a publication of his called "Application of Hilbert transform to scalp EEG containing EMG", Freeman, Burke, Holmes (2003) [PDF].

I know, as I read through these works, that I'm not really getting the optimal feel for what's written because I wasn't there doing the work. But Walter Freeman has been incredible in giving us his interpretation of what the brain is really doing in the books he's written. Thanks for helping us understand ourselves.

Dec 7, 2010

I watched the beautiful sunset the other day wondering about what to add to this site. I'm studying again how Satosi derived his clustering formulas to get the entropy equation. I think that's something I can understand. But then I began to wonder if I should look at Renyi entropy. That's really strange. The formula for Renyi entropy is:

where as q -> 1 Renyi entropy tends to Shannon entropy.

I can sort of feel what the natural log of the summation is like, but how did Renyi get 1/(1-q). That's a singularity at q=1! What a strange behavior for a limiting singularity to approach Shannon's entropy. Renyi was a mathematician who worked with Erdos. To get the full implications of this formula will take me a long time so I've just left it alone. I know my limitations. I know that this quest is bounded, so I have to smile.

December 3, 2010

I just put this page back online today. Ten years ago, most friends who were not specialists in this field, said they could not understand much of what I had written. But a few, rather small number, seemed to appreciate the "strangeness." There are some great and beautiful concepts that are simple and elegant enough to be in wonderment of like entropy. David Bohm calls this awareness of a streaming universal energy a "movement" which he saw all around him. When I first read what he wrote, I had some doubts about what he was saying, but now it's what I have incorporated into my worldview completely. It's was strange at first, but not now.

October 14, 2010

Began updating this webpage with new contents about computing paths in programming code. I naively started this web page in 1998, inspired by the beautiful design of recurrent neural networks. However, I soon realized that I was getting nowhere studying classical error minimizing matrices in neural networks. But it's hard to overcome old notions. I knew the neuron used the temporal code to talk to each other. But in my developmental work I kept using the old data structures and algorithms of the rigid networks. About 2001 I gave up on really developing my ideas on waves and neurons. In 2007 thinking that I might never find an interesting developmental path, I took this website offline.

But about a year ago I just started feeling more confident about the "wave model" of the neuron. This is because I could integrate aspects of entropy or information into what I thought might be happening in neural circuits.