Interactive Syllabus

EXAM TOPICS:

• Natural language (pre)processing techniques (the "classical" NLP pipeline)

• Classes of machine learning algorithms, examples of models representing the classes

• Typical machine learning pipeline

• Typical applications of machine learning - a selection of examples, detailed description of one approach

• Bag of words representation of text - pros and cons

• Distributional hypothesis - historical context, linguistic motivations and practical implementations

• Distributional vs. formal semantics

• Word embeddings - the basic principles, example of a specific technique, pros and cons

• Latent semantic analysis - basic principles, pros and cons

• Document classification and text clustering

• Perceptron - motivation and basic principles

• Deep learning - description of the approach, and how does it differ from other machine learning techniques

• Gradient descent and back-propagation - motivation and basic principles

• Feed-forward neural networks - basic principles, pros and cons

• Convolutional neural networks - basic principles, pros and cons

• Recurrent neural networks - basic principles, pros and cons

• Vanishing/exploding gradients and how to deal with them - description of a selected approach

• Encoder-decoder architecture - basic principles

• LSTM architecture - motivation and basic principles

• Attention mechanism - motivation and basic principles

• Transformer architecture - motivation and basic principles

• Basic principles of language models (both traditional and neural ones)

• Training language models - typical approaches and architecture of the models
  

• Evaluation of standard machine learning models - description of the process and an example of an evaluation metric

• Evaluation of language models - description of the process and an example of an evaluation metric

• Typical applications of language models - a selection of examples, detailed description of one approach

• Sentiment analysis - the problem addressed, justification of its practical relevance, general description of typical approaches
  

• Detailed overview of a selected lexicon-based approach to sentiment analysis

• Detailed overview of a selected classical machine learning approach to sentiment analysis

• Detailed overview of a selected deep learning approach to sentiment analysis

• Comparison of lexicon-based, classical machine learning and deep learning approaches to sentiment analysis

• Basic principles of knowledge representation

• Ontologies vs. knowledge graphs - pros and cons of each approach to knowledge representation

• The stack of typical tasks in ontology learning

• Main challenges and open problems of ontology learning

• Techniques used for term extraction, synonym discovery and concept formation

• Techniques used for taxonomy extraction

• Techniques used for relation, rule and axiom extraction

• Overview of a selected deep learning approach to knowledge extraction