As you may know, Google is quite interested in AI. That being said, they are working on some pretty amazing things.
They have as it seems, managed to create artificial intelligence that can then create its own artificial intelligence. Yes, their AI can pretty much make baby AI.
The company made a post on one of their blogs about this interesting feat months ago and I am completely amazed and surprised I didn’t see it till now. Their Automated Machine Learning system is given a controller AI and then it proposes the designs for the; as they called it ‘child’ AI. From this point, the child AI is given a task and feedback for this task is sent to the controller AI, AKA the parent.
According to their post, this approach can design models that achieve accuracies on par with ‘state-of-art’ models designed by machine learning experts. The parent is able to improve how the child is and can design a second or more. The parent could end up with thousands of children.
The machine-chosen architecture does share some common features with the human design, such as using addition to combine input and previous hidden states. However, there are some notable new elements — for example, the machine-chosen architecture incorporates a multiplicative combination (the left-most blue node on the right diagram labeled “elem_mult”). This type of combination is not common for recurrent networks, perhaps because researchers see no obvious benefit for having it. Interestingly, a simpler form of this approach was recently suggested by human designers, who also argued that this multiplicative combination can actually alleviate gradient vanishing/exploding issues, suggesting that the machine-chosen architecture was able to discover a useful new neural net architecture.
This approach may also teach us something about why certain types of neural nets work so well. The architecture on the right here has many channels so that the gradient can flow backwards, which may help explain why LSTM RNNs work better than standard RNNs.
These AI that were created by AI work quite well and are without a doubt more amazing that I would have ever imagined for them to be. They can make things more efficiently than some human engineers even. More recently the Automated Machine Learning System has been brought up, it seems there is much more to it than we thought.