|
|
Great Wave, Hokusai: theculturetrip.com
Structure of the Machine:
Dimensionality Reduction Using Prefix Trees
In wondering how to build a neural machine, the principal obstacle
I tried to remove was redundancy in computer code. Redundancy
can be also be removed by simplicity and efficency. Simplicity
in computer code can be obtained by using a binary tree structure.
For me, it's the fundamental structure I use to build my machines.
A special case of the binary tree is the prefix tree, also called
a trie. The prefix tree or trie is a favorite structure of mine
because it is easy to make an analog of a simple neural circuit from
a trie. When you propagate, or traverse, through a trie, the information
at the start of the propagation is more important than what is down
the tree. This is especially true when you couple tries together.
Our model of the neural machine gets its value from the fact that
coupling tries together exponentially increases information compression.
It's as if you produced a stock waves of compressed information
at the start of a propagation down a tree. It's especially important
to get this kind of behavior when modelling a neural information
circuit when you are using a temporal or time-dependent model of
information flow.
In 2012, 7 years ago, I started my project on using tries as
the epitomes of the neuron. I called the project Kali in honor
of the Hindu goddess of time. Kali is taking a lot longer than
what I expected it to. Remainds me of what Bernard Widrow wrote
about Adaline in his 1960 IRE article in which he thought the
problems of modelling brains would be done in a few years. It's
been 60 years, and we're still working on the fundamental problems.
Kali has been on hold as I worked on building a grammar to run
the heart of Kali. [1]
[1] This is another project called mxLisp. I hope this is NOT
a recursive loop of projects.
|
|