This unit introduces deep learning approaches in AI. Deep learning is
a subfield of machine learning that’s characterized by the use of
multilayer neural networks with several layers (i.e., “deep” neural
networks). Deep learning techniques have seen success in a variety of
real-world tasks, and they power most of the modern technologies that
people think about when they hear the term “AI” (e.g., chatbots, image
generators, video generators, etc).
Module Learning Outcomes
After successful completion of this module, you should be able to do
the following:
Explain the advantages of deep-learning-based AI (CLO 1)
Why deep neural networks can learn more complex concepts than
shallow ones?
In deep neural networks, where does non-linearity come from?
Explain how convolutional neural networks classify images (CLO 3)
Why is an MLP not good at computer vision tasks?
What is convolution?
How do CNNs use convolution to classify images?
Characterize the concept of word embeddings (CLO 4)
What is a word embedding?
What are the advantages of word embeddings over one-hot
representation?
What are the two common ways to train a word embedding?
What linguistic properties do word embeddings have?
What are the limitations of (static) word embeddings?
Characterize the training methods of large language models (CLO 5)
What is the basic idea of an RNN?
How do you use RNN to translate from one language to another?
Why do we need attention along with RNN for translation?
What are the key ideas in Transformer that replaced RNN?
What are the two influential pretraining paradigms based on
Transformer, and which one eventually led to ChatGPT?