Are Deep Neural Networks our first true breakthrough at reverse-engineering the human intelligence?
(part of a series of a slightly philosophical stories by an AI dilettante)
Deep neural networks seem to have revolutionized AI. For now, I would avoid delving too much in ChatGPT — as it merits its own story — but rather at the general adoption of DNNs — which seem to have become an ubiquitous miracle cure in AI. So have we finally started gaining any real insights in how our brain works? Are DNNs our first real success at reverse-engineering ourselves?
Before continuing, let me tell you a personal story. 5 years ago I discovered electric monowheels. I live in downtown Paris and commuting has always been a huge problem — so I instantly fell in love with the concept of the monowheel. The only problem was that I had to learn to ride it. I was in my late 30s, had never skateboarded or rollerskated during my teens, and I haven’t been skiing since my early childhood. I knew this one was going to be difficult. I bought and old second-hand wheel and decided to simply give it a try on my own. The very first time I spent an hour without ever doing more than 50 cm. The next evening, after lots of failing at about 1 m, I was suddenly able to keep myself standing on it for 20 m. Half an hour later, I was riding it without falling too much. On the third day, I used it to get to work. I was stunned, because, initially, it seemed like an absolutely impossible task — than in a few hours it became a breeze. Just like that. Baffled by the experience, my next stop was Wikipedia where I started searching for motor-skills and muscle memory. It is an absolutely fascinating subject. You should probably try learning about it too. Maybe you remember how you learned to ride a bike or drive a car. In the beginning, it was a very tedious conscious process, where you were thinking hard about every step. Then — you simply stopped thinking about it and started doing it. It became embedded in the hardware. Often, just like for DNNs, it is a very sudden process.
So, to answer that question in the beginning, definitely yes, we do have started reverse engineering parts of our brain. It is just not the really important part — the neocortex. We have started cracking the “peripherals”: visual recognition, motor skills, language skills and even gaming strategy — which are mostly intuitive. Now, language skills and gaming strategy are particularly interesting since they have both a “hardware” first-order intuitive part — as well as a higher-order “software” part — and the truth is that this second part is still somewhat lacking in the current generation AI.
When 20 years ago, Deep Blue won its first victory against Kasparov, many noted that humans were still unequaled at extreme complexity — and the one example they frequently gave was Go — a game with a much higher order of complexity compared to chess — in which humans were still impossible to beat. Then there was StarCraft 2 and DOTA 2. Both video games were supposedly fine examples of solution-spaces that were impossible to explore using classical branch-and-bound exhaustive approaches. Only “human creativity”, and human capacity of inventing strategies could secure a win. Personally, for having played — competitively — both of these — even if I never came out of the bottom of the ranking ladders — I consider both of them to be very bad examples. True — these games are not accessible to classical algorithms that simply explore the solutions space. Predicting even 2 or 3 seconds in the future would require considering gazillions of possibilities — and one needs to think minutes in advance to have any chance of winning.
DNNs are however extremely good at precisely this kind of tasks. In fact, this is probably the type of game where a DNN is the absolutely best possible player. They simply have to learn various — not so complex — strategies and to apply them perfectly in what is a very fast-paced game where no human is ever perfect — and where one of the very characteristic differences between the entry-level amateur and the top-level professional player is his APM — actions per minute. Obviously, in these games, DNNs easily gained the upper hand without requiring the huge R&D investment that IBM made for Deep Blue.
I even think that if there was a large corporation with deep pockets and a project to create an entirely mechanic top soccer player, its most difficult problem would probably be the power source — as power storage is a technology in which we are still far behind our own bodies. And of course the gargantuan task of having to analyze thousands of hours of soccer matches that are available only as analog video. DOTA 2 has a Linux API — which meant that the AI creators could focus entirely on the AI itself.
In fact, we have started understanding — and successfully copying — all those basic skills that we are able to learn to perform without thinking. We are still decades away from understanding the higher functions of the brain. Heck, it might even be centuries away. Or maybe there are no other mechanisms in the brain besides what we already know. Maybe it is simply a matter of adding even more neurons. Which is easier said than done, because ChatGPT has been trained on a — very expensive — supercomputer. From what we know, the human brain is not similar to a CPU — but rather to an FPGA. Its most distinctive advantage over DNNs is that it is fully dynamic, capable of adding neurons to existing structures and interconnecting stages on the go. But the hard truth is that we simply do not even know what there is to know for the next step.
Because let’s be honest. We, the computer engineers, didn’t invent anything. Neural networks, reinforcement learning, genetic algorithms — we didn’t come up with those. Evolution produced them. Then they were discovered by the scientists. Only then, we started adapting them, filling the voids — and even — improving them — since nature has never been a perfect designer. So for there to be a significant AI breakthrough — there must be a significant neurology breakthrough first. This is not coming out of a computer lab for sure — it simply has to come from medical science.
