PA164 Machine learning and natural language processing

Faculty of Informatics
Autumn 2023
Extent and Intensity
2/1/0. 3 credit(s) (plus extra credits for completion). Recommended Type of Completion: zk (examination). Other types of completion: z (credit).
Teacher(s)
doc. Mgr. Bc. Vít Nováček, PhD (lecturer)
Guaranteed by
doc. Mgr. Bc. Vít Nováček, PhD
Department of Machine Learning and Data Processing – Faculty of Informatics
Supplier department: Department of Machine Learning and Data Processing – Faculty of Informatics
Timetable
Mon 14:00–15:50 A318
  • Timetable of Seminar Groups:
PA164/01: Wed 27. 9. to Wed 6. 12. each odd Wednesday 18:00–19:50 C525, V. Nováček
Prerequisites
The basics of machine learning (e.g. IB031), computational linguistics (e.g. PA153) and neural networks (e.g. PV021), is assumed. The course is given in English (or in Czech depending on the audience). Task solutions can be in English, Czech or Slovak (exceptionally in another language).
Course Enrolment Limitations
The course is also offered to the students of the fields other than those the course is directly associated with.
fields of study / plans the course is directly associated with
Course objectives
Students will obtain knowledge about methods and tools for text mining and natural language learning. At the end of the course students should be able to create systems for text analysis by machine learning methods. Students are able to understand, explain and exploit contents of scientific papers from this area.
Learning outcomes
A student will be able
- to pre-process text data for text mining;
- to build a system for analysis of text by means of machine learning;
- to understand research papers from this area;
- to write a technical report.
Syllabus
  • Course overview, a sample text (pre)processing pipeline
  • Quick and dirty intro to ML
  • Distributional semantics, LSA, word embeddings
  • Deep neural networks for NLP
  • Language models and their applications
  • AutoML for NLP
  • Student poster session(s), including extensive feedback during the students' work and its presentation
  • Application example: sentiment analysis
  • Application example: knowledge extraction from text
  • Guest lecture(s) from international experts on various ML applications in the NLP field
  • Final project presentations
Literature
    recommended literature
  • Chang, Yupeng, et al. "A survey on evaluation of large language models." ACM Transactions on Intelligent Systems and Technology 15.3 (2024): 1-45.
  • Zhao, Wayne Xin, et al. "A survey of large language models." arXiv preprint arXiv:2303.18223 (2023).
  • Charu C. Aggarwal, Machine Learning for Text. Springer 2018
  • Mining text data. Edited by Charu C. Aggarwal - ChengXiang Zhai. New York: Springer Science+Business Media, 2012, xi, 522. ISBN 9781461432227. info
  • MANNING, Christopher D. and Hinrich SCHÜTZE. Foundations of statistical natural language processing. Cambridge: MIT Press, 1999, xxxvii, 68. ISBN 0-262-13360-1. info
Teaching methods
a lecture combined with independent work on and demonstrations of selected techniques in the labs, work on a project
Assessment methods
Oral examination with written preps (optional). Project presentations are a part of the examination.
Language of instruction
English
Further Comments
Study Materials
The course is taught annually.
The course is also listed under the following terms Autumn 2003, Autumn 2004, Autumn 2005, Autumn 2006, Autumn 2007, Autumn 2008, Autumn 2009, Autumn 2010, Autumn 2011, Autumn 2012, Autumn 2013, Autumn 2014, Autumn 2015, Autumn 2016, Autumn 2017, Autumn 2018, Autumn 2020, Autumn 2021, Autumn 2022, Autumn 2024.
  • Enrolment Statistics (Autumn 2023, recent)
  • Permalink: https://is.muni.cz/course/fi/autumn2023/PA164