Interactive Syllabus

Learning and Natural Language

Info

EXAM TOPICS:

Natural language (pre)processing techniques (the "classical" NLP pipeline)
Classes of machine learning algorithms, examples of models representing the classes
Typical machine learning pipeline
Typical applications of machine learning - a selection of examples, detailed description of one approach
Bag of words representation of text - pros and cons
Distributional hypothesis - historical context, linguistic motivations and practical implementations
Distributional vs. formal semantics
Word embeddings - the basic principles, example of a specific technique, pros and cons
Latent semantic analysis - basic principles, pros and cons
Document classification and text clustering
Perceptron - motivation and basic principles
Deep learning - description of the approach, and how does it differ from other machine learning techniques
Gradient descent and back-propagation - motivation and basic principles
Feed-forward neural networks - basic principles, pros and cons
Convolutional neural networks - basic principles, pros and cons
Recurrent neural networks - basic principles, pros and cons
Vanishing/exploding gradients and how to deal with them - description of a selected approach
Encoder-decoder architecture - basic principles
LSTM architecture - motivation and basic principles
Attention mechanism - motivation and basic principles
Transformer architecture - motivation and basic principles
Basic principles of language models (both traditional and neural ones)
Training language models - typical approaches and architecture of the models
Evaluation of standard machine learning models - description of the process and an example of an evaluation metric
Evaluation of language models - description of the process and an example of an evaluation metric
Typical applications of language models - a selection of examples, detailed description of one approach
Sentiment analysis - the problem addressed, justification of its practical relevance, general description of typical approaches
Detailed overview of a selected lexicon-based approach to sentiment analysis
Detailed overview of a selected classical machine learning approach to sentiment analysis
Detailed overview of a selected deep learning approach to sentiment analysis
Comparison of lexicon-based, classical machine learning and deep learning approaches to sentiment analysis
Basic principles of knowledge representation
Ontologies vs. knowledge graphs - pros and cons of each approach to knowledge representation
The stack of typical tasks in ontology learning
Main challenges and open problems of ontology learning
Techniques used for term extraction, synonym discovery and concept formation
Techniques used for taxonomy extraction
Techniques used for relation, rule and axiom extraction
Overview of a selected deep learning approach to knowledge extraction