Wednesday, September 07, 2005
Artificial Development - News
"Artificial Development is building the largest neural network to date to model a state-of-the-art simulation of the human brain. With 20 billion neurons and 20 trillion connections"
Why bother to sell something you clearly don't have? This company joins Numenta in the list of fake software companies to try to simulate human intelligence before it's clear to anyone how to do even something so simple as associate "red" and "apple" to produce "red apple" in a neural network, spiking or not. This is known as the combinatorial problem. Nobody knows how to teach the spiking networks anything except simple auto-association (which is why everyone is excited about them in the first place). Also, nobody knows the algorithm that wires the human brain during growth. Finally, simulating 40 million neurons on one computer (something the company is claiming to be doing) is simply out of reach even on dual-CPU dual-core opterons (unless we're talking about 1-bit neurons) so we can safely say this press release is full of shit.
How can I judge? Experience. I've been able to run ~15,000 x 100 neurons on one CPU in real-time (that's 1ms per step); I'm using non-linear neurons, so of course this slows me down a bit; But in reality, the number of synapses and the delay of each synapse, not the number of neurons, is the killer -- the total number of spikes traveling in a system is Sum(synapse[i].delay), and unless you're careful this number will creep into billions. The maximum I can comfortably run in under 1 gig of ram on one node is 200,000 neurons with 100 synapses each. That's 20,000,000 synapses up to length 10; the maximal spike capacity of this network is 200,000,000 spikes. When the number of spikes is small (2 million spikes/ms), this beast runs at 5 fps. To get real-time, you need 1,000 fps assuming your time resolution is 1ms.
"Artificial Development is building the largest neural network to date to model a state-of-the-art simulation of the human brain. With 20 billion neurons and 20 trillion connections"
Why bother to sell something you clearly don't have? This company joins Numenta in the list of fake software companies to try to simulate human intelligence before it's clear to anyone how to do even something so simple as associate "red" and "apple" to produce "red apple" in a neural network, spiking or not. This is known as the combinatorial problem. Nobody knows how to teach the spiking networks anything except simple auto-association (which is why everyone is excited about them in the first place). Also, nobody knows the algorithm that wires the human brain during growth. Finally, simulating 40 million neurons on one computer (something the company is claiming to be doing) is simply out of reach even on dual-CPU dual-core opterons (unless we're talking about 1-bit neurons) so we can safely say this press release is full of shit.
How can I judge? Experience. I've been able to run ~15,000 x 100 neurons on one CPU in real-time (that's 1ms per step); I'm using non-linear neurons, so of course this slows me down a bit; But in reality, the number of synapses and the delay of each synapse, not the number of neurons, is the killer -- the total number of spikes traveling in a system is Sum(synapse[i].delay), and unless you're careful this number will creep into billions. The maximum I can comfortably run in under 1 gig of ram on one node is 200,000 neurons with 100 synapses each. That's 20,000,000 synapses up to length 10; the maximal spike capacity of this network is 200,000,000 spikes. When the number of spikes is small (2 million spikes/ms), this beast runs at 5 fps. To get real-time, you need 1,000 fps assuming your time resolution is 1ms.
# posted by Notetaker @ 4:03 PM 
