Artificial intelligence (AI) continues to rapidly expand and make a big impact on the workplace. Yann LeCun is one of the original creators of AI technology, as he is now a highly respected professor at New York University and is also the Chief AI Scientist at Facebook. LeCun also won the Association for Computing Machinery A.M. Turing Award, which is often considered the “Nobel Prize of Computing.” Recently, LeCun was interviewed about the history of artificial intelligence, as he played a critical role in working with others to develop this innovative technology.
Yann is most well-known for his work in computer vision at Bell Labs by using convolutional neural networks to perform a wide range of activities. For example, this technology allows banks to automatically read checks, provides facial recognition to unlock phones, uses speech recognition to perform searches, and it can even detect tumors through medical imagery. Yann believes that artificial intelligence is specialized to perform a variety of actions to make the world a better place.
The Origin of AI
Understanding what is artificial intelligence is important for anyone interested in the history of this technology. Initially, the concept of AI technology began in the 1940s, as Alan Turing set the foundation for traditional computer science with his innovative research. The first artificial neuron was also proposed by Water Pitts and Warren McCulloch, as they believed that the actions of neurons could be viewed as a computation, which allowed the circuits of neurons to perform logical reasoning.
During this time, researchers also worked on cybernetics, which is essentially the science of how each part of a system communicates with each other. This idea created the term “autopoiesis,” which is a system capable of maintaining and reproducing itself while also learning, regulating, and adapting to its environment. All of this research created a significant amount of interest in this field.
Marvin Minsky (co-founder of the Massachusetts Institute of Technology’s AI laboratory) and John McCarthy (co-founder of the Stanford AI Laboratory) organized a conference at Dartmouth College with the help of a couple of scientists at IBM in 1956. This conference gave an opportunity for Allen Newell and Herbert A. Simon to debut Logic Theorist, which was a computer program engineered to perform automated reasoning. Ultimately, the term “artificial intelligence” was born, as this conference played a pivotal role in artificial intelligence history and future.
1960s-1980s: Understanding the Limitations of AI
The academic community was eventually split into two categories for the next twenty years, as one focused on biology and the human brain while others drew more inspiration from mathematics. Initially, no one was working on what we consider machine learning in the early 1980s, which was when Yann began his career. However, the field gained new interest in 1986 with a paper titled “Learning Representations by Back-Propagating Errors,” which was published in the Nature Journal. This paper highlighted the potential success of neural network applications and the learning algorithm for backpropagation. However, this excitement was eventually lessened as research found that only a few applications could be solved due to the extensive amount of data that needed to be trained properly. Data was much more expensive during that time, as it couldn’t be easily obtained from open-source data sets or internet archives like today.
1990s-2010s: Lack of Development
The history of artificial intelligence during the 1990s through 2010s was known as the “black period” due to the lack of development. Neural nets were ignored and often mocked by other professionals. Yann eventually stopped working on neural nets in 1996 and didn’t return until after 2002. Anyone that clung to neural nets was often viewed as marginal crazy people due to the lack of understanding in the field.
However, Yann believed that neural networks would eventually be vindicated over time. The limitation of traditional methods for computational was often due to the reliance on front-end hand engineering, while deep learning on convolutional neural nets was much more efficient. In other words, hand-engineered features are designed by very smart people, while a backpropagation algorithm uses data to create the “feature extractors” to fit each task as efficiently as possible.
This realization of the benefits of artificial intelligence took researchers in the community over twenty years to understand, as Yann believes that the adoption of this technology took longer due to satisfaction of pitting your wits against a problem without the assistance of a machine. However, relegating yourself to a “machine coach” is often a humbling yet very sold decision.
Another way to consider the long-term trends of AI technology is that it initially began with intelligent people focusing on specific tasks that proved successful within the abstract world of which they were created. However, AI technology uses intelligence shaped by empirical evidence that’s fully formed with a specific set of axioms. One of the most interesting challenges of AI is the ability to match “common sense” and the learning ability of a toddler instead of creating synthetic versions of a chess grandmaster.
2012: AI Becomes Mainstream
Geoffrey Hinton and Li Deng introduced the use of deep feedforward, non-recurrent networks to recognize speech in late 2009. Neural networks eventually came back in the academic spotlight in October 2012 due to the impressive benchmarks from AlexNet and other submissions to the ImageNet Large Scale Visual Recognition Challenge at the European Conference on Computer Vision and the PASCAL Visual Object Classes Challenge. Jeff Dean and Andrew Ng also programmed a computer cluster in 2012 to train itself to recognize images. Later in the fall of 2012, the New York Times cited Dr. Richard F. Rashid’s presentation with Microsoft that highlighted the potential of deep learning with the translation of Mandarin.
Yann also notes that the build-up of neural networks being embraced goes several years back due to the innovative work of several pioneers in the field. Artificial intelligence examples include Yoshua Bengio’s text prediction in the early 2000s and Pascal Vicent’s work with using denoising autoencoders. Jason Weston and Ronan Collobert also played a key role in their work at the NEC Research Institute in Princeton with their 2011 paper that focused on the “Natural Language Processing (Almost) from Scratch.”
2013-2017: Significant Advancements in AI Technology
The year 2012 was pivotal in the development of AI technology. All of these recent advancements began to snowball and made a significant impact on artificial intelligence and machine learning. For example, Tomas Mikolov, with the help of his colleagues at Google, developed Word2vec, which is an innovative method for learning a representation of words that do not require labeled data while also allowing NLP systems to overlook spelling and focus more on semantics. This AI technology also made it much easier to train multilingual systems, which is especially helpful for speakers of “low-resource” languages to access volumes of information that others take for granted.
Ilya Sutskever published the “Sequence to Sequence Learning with Neural Networks” in 2014, as it described a method of creating multilayered Long Short-Term Memory (LSTM) system that’s suited for tasks such as summarization and automated translation. Dzmitry Bahdanau also published the “Neural Machine Translation by Jointly Learning to Align and Translate” in 2015. A few months later, Google, Microsoft, Facebook all had translation systems that were based on recurrent neural networks. Researchers at Google also proposed a more simple network architecture in 2017 that was solely based on attention mechanisms with their paper “Attention is All You Need.” The use of funny titles is always popular in AI research, as many recent papers include names for Sesame Street characters.
While it took decades for gifted researchers to explore very different approaches, which were often focused on narrow application areas or even on abstract problems, we have finally reached a point that uses a data-driven approach to highlight the value in all types of practical domains. The use of AI theorists will still be needed, but using real-world data has been a critical ingredient in getting all of the answers to apply to real problems.
Latest Developments in Artificial Intelligence
Neural networks will continue to be refined and used in a variety of new and exciting ways. Yann believes that the work of Francois Charton and Guillaume Lample in a recent paper shows how each system is surprisingly adept at mathematics, as these problems used to be a particular area of weakness for neural networks. One of the most important advancements in using AI technology isn’t in the research by itself, but it’s how the research is conducted. Now it is perfectly normal for industrial research labs to publish papers that include the code and data to produce the results, instead of being listed internally as a trade secret. This openness in the AI field makes it much easier for other artificial intelligence companies to work together.
Ultimately, Yann believes that this openness with other AI companies will accelerate the development of this technology. For example, Google may publish a new technique, and within a few months, Facebook will make an improvement on it. Similarly, Google may make another improvement, as all of these advancements add up and create a culture of providing “reproducible research” to help spread advancement. This new cultural norm is also spreading in other sciences, as this will benefit everyone in helping to develop new technology.
Self-supervised learning is another exciting development for AI companies. Yann believes that AI technology can transform the health care industry and improve the entire world by making it easier to meet the needs of patients. Artificial intelligence and machine learning will continue to play a key role in the development of new technology while also improving upon existing structures. The world of artificial intelligence will only continue to expand as more companies work with each other in the development of this tech.
Closing Thoughts
Understanding artificial intelligence history and the latest developments are essential for anyone interested in how this technology will continue to shape the world. The use of AI technology has rapidly evolved compared to only a few years ago. AI companies will continue to research and find new ways to take advantage of this technology. The rise of AI technology will impact all types of industries, whether it’s in the healthcare field, manufacturing, education, transportation, media, banking, and much more.
The one constant in the world of AI technology is change, as the biggest tech companies look at new ways to utilize this technology and improve their business operations. Today’s work environment is extremely competitive, as utilizing AI technology is essential in gaining an edge over other businesses while also better meeting the needs of customers.
AI technology will continue to become even more widespread, whether it’s autonomous cars transporting people from place to place or AI-powered robots working alongside humans to manufacture products. The use of AI technology can also assist nurses in monitoring patients while also identifying any potential signs of disease. The world of education will also continue to be impacted by AI, as more textbooks become digitized with the use of AI technology, and virtual tutors can help assist human instructors.
All of these exciting changes will make a big impact on day-to-day life now and in the near future in a wide range of sectors. Learning about the impact of AI technology is a great way to stay up to date with these evolving trends. AI companies will continue to find ways to improve upon this existing technology, as the future looks bright with the ever-increasing advancement of AI technology.
