Wednesday, September 07, 2005
The cellular automata (CA) people are particularly pathetic and impractical when it comes to producing something useful. They tend to just sit there, looking at pretty pictures, then philosophize about the globality and profundity of what CA could do -- "it's alive!". The most practical applications of CAs have been snowflakes and fire for computer graphics. That's about it.
But the sad truth is that no one is even trying to compute something using CA. By compute I don't mean make a turing machine out of CA -- the game of Life is universal, so is rule 110. I really mean using CAs to solve problems that cannot be solved otherwise.
The neural network guys are far ahead of the CA guys in this respect; at least from the very beginning, they were trying to do something practical. As early as 1959, buckets of cold water were poured over perceptrons; this lead to progress. But CA borrows from the mystical reputation of fractals to avoid any real criticism. For one, the problem with CA's is that one cell constantly polls its surroundings in order to change its state; Neural networks use more precise node-to-node communication, where every weight can be reduced to zero if necessary. No such flexibility with CAs. Next, the most pathetic of CAs are one-dimensional; we won't even consider those. 2D CAs generally use the nearest 8 neighbors to update a cell's state -- not a lot. Finally, each node in a CA generally limits itself to 2 or 3 states not a lot, considering that a good neuron has at least 4 floats.
Spiking neural networks with non-linear nodes are really a generalization of CAs. By choosing an appropriate tolopology and edge delays (e.g. a 2D grid with all edge delays = 1), you can reproduce any of the existing CA patterns. You also get tons of degres of freedom that you don't get with CAs.
But the sad truth is that no one is even trying to compute something using CA. By compute I don't mean make a turing machine out of CA -- the game of Life is universal, so is rule 110. I really mean using CAs to solve problems that cannot be solved otherwise.
The neural network guys are far ahead of the CA guys in this respect; at least from the very beginning, they were trying to do something practical. As early as 1959, buckets of cold water were poured over perceptrons; this lead to progress. But CA borrows from the mystical reputation of fractals to avoid any real criticism. For one, the problem with CA's is that one cell constantly polls its surroundings in order to change its state; Neural networks use more precise node-to-node communication, where every weight can be reduced to zero if necessary. No such flexibility with CAs. Next, the most pathetic of CAs are one-dimensional; we won't even consider those. 2D CAs generally use the nearest 8 neighbors to update a cell's state -- not a lot. Finally, each node in a CA generally limits itself to 2 or 3 states not a lot, considering that a good neuron has at least 4 floats.
Spiking neural networks with non-linear nodes are really a generalization of CAs. By choosing an appropriate tolopology and edge delays (e.g. a 2D grid with all edge delays = 1), you can reproduce any of the existing CA patterns. You also get tons of degres of freedom that you don't get with CAs.
# posted by Notetaker @ 3:11 PM 
