20. Future
Smaller Data
And
Less Compute
Image Source: https://twitter.com/andrewyng/status/1045399898537873408?lang=en
“Deep Learning is getting really good on Big
Data/millions of images. But Small Data is
important too.” – Andrew Ng
Editor's Notes
Hi! I am Tyler Folkman – the head of AI at Branded Entertainment Network. And in this talk I would like to attempt to take you through the history and future of AI in 5 minutes. So hold on! This is going to be intense!
How can we start anywhere other than with Alan Turing and his Seminal paper, “COMPUTING MACHINERY AND INTELLIGENCE.” It was with this paper that he posed the question “Can Machines Think?” And put forth the imitation game - the test of whether a human could distinguish a human vs a machine in a typed interrogation.
Not long after, John McCarthy coined the term “Artificial Intelligence” in his proposal for the Dartmouth Conference – the first AI conference. He sought to explore ways in which machines could reason like humans.
About 10 years later at the MIT AI Lab, Joseph Weizenbaum created Eliza. Eliza implemented pattern matching technology that allowed it to simulate conversations without any real understanding or contextualization. This was one of the first chatbots and programs capable of attempting the Imitation Game.
Later, at Stanford, the ”Stanford Cart” was developed. It used stereo vision with its mounted television camera that allowed it to detect objects and compute a navigation path. It took 10-15 mins to travel 1 meter and 5 hours to navigate a chair cluttered room.
The term "AI winter" was coined by researchers who became concerned that enthusiasm for AI had spiraled out of control and that disappointment would certainly follow. Their fears came true with funding decreasing substantially in the late 1980s and 1990s.
This time was not completely devoid of progress, though. In 1986, Geoffrey Hinton co-authored, “Learning Representations by back-propagating errors” which set forth “a new learning procedure for networks of neurone-like units.” And is the basis for how current deep learning networks learn.
In 1997, IBM rocked the world when its machine “Depp Blue” defeated Garry Kasparov – a world chess champion. People saw this as a sign that AI was starting to catch up to human level intelligence, but was more an example of the power of brute force.
In 2009, the ImageNet paper revolutionized the world of neural networks and deep learning by providing a then challenging dataset of 1M images of 1k classes. This large dataset supercharged research into deep learning and now has results which surpass human level performance.
In 2011, IBM’s Watson defeated the legendary Jeopardy champions Ken Jennings and Brad Rutter. The impressive part of Watson was its ability to process natural language in a question and answer setting even with limited context and clues.
In the same year, Siri launched as a part of Apple’s ecosystem. This was the first mainstream example of a virtual assistant leveraging AI to process speech and then actually perform an action or respond back to you.
In 2014, Generative Adversarial Networks were invented by Ian Goodfellow. And called the most interested idea in the last 10 years of ML by Yann LeCun. Who knew then that these algorithms would spawn new celebrity faces and deep fakes.
Just a year later, Google’s AlphaGo defeated Lee Sedol in the game of Go and now could be considered the strongest Go player in history. Go has 1 million trillion trillion trillion trillion more configurations than chess making any sort of brute force solution impossible and a truly remarkable feat of AI.
Finally, earlier this year, OpenAI announced GPT-2 that was so good at creating synthetic text given an arbitrary input that they wouldn’t release the best model due to fears of “malicious applications of the technology”.
So where are we headed. I work at a company where we use AI to place brands in the content – often on social media such as Instagram. There is a rising trend of bots on IG, so I think a lot about fraud and AI. I foresee a future where AI is even more critical to combatting (and even creating) fraudulent content such as deep fakes, and false followers.
It is only natural then that I also believe regulation and safety of AI will be critical to the future. The more we rely on AI, the more we need to be thoughtful on how to use it appropriately. The “Future of Life Institure” raised over $2 million towards this cause in 2018. Elon Musk was a large doner.
One of the people who received funding was Allan Dafoe of Yale University. In his abstract he said he believes AGI may be developed this century. AGI could be defined as a “machine that has the capacity to understand or learn any intellectual task that a human being can.” I am sure we will see continued research in these areas by groups such as OpenAI.
Related to safety, is Fairness and Bias. We went and built a lot of exciting technology and only recently started analyzing how they might be biased. For example, a program used to estimate how likely defendants are to re-offend found that “black offenders were seen almost twice as likely as white offenders to be labeled a higher risk but not actually re-offend” This is not okay and dangerous. Work in this area has exploded and needs to continue in the future.
Perhaps you have heard of Elon Musk’s investment in Neuralink? A company developing implantable brain-machine interfaces. I don’t think the idea of augmenting human cognition is going away anytime soon and we will see continued research in this futuristic area.
Lastly, I would like to end with a more practical future application: small data and less compute. Our algorithms are becoming more and more data hungry, but there are many important problems with small datasets. At BEN, we are actively working on doing more in marketing with less data and I am excited that people like Andrew Ng also see the value of small data. I am sure this will continue to be researched as more and more industries (some with limited data) adopt AI. So – there you have it. Some thought it couldn’t be done. But together we have seen the history and future of AI in 5 minutes!