Interactive Syllabus
News
- The course is a [i]regular [re]search seminar, mentored in a "family" manner. In this term, the seminar [sub]topic is Thinking Slow. The enrolled student must give a presentation on an agreed-upon topic (of interest, or her thesis, or he will talk about a research paper or area) once during the term. Topics of presentations focus primarily (but not necessarily) on those related to machine learning, representation learning, and scientific visualization or about our subtopic (tackling complexity by LLM agents, proving P=NP, etc. ;-).
- There is a discussion group with official course information and a communication channel in addition to the
course outline below: watch both frequently!
Topics and Course Outline
Readings: Motivating video: DEK's advice to young students.
Week 1– 20. 2. Introduction, research strategy, evaluation, and course schedule planning
Join us at A502, Faculty of Informatics MU, on February 20th at 10 AM (CET) [or on Zoom, on-demand only].
- Why? Put readers in your place! Specifics of CS research and doctoral studies and their evaluation at FI MU: CS conference rankings
- How? to write
- What (and where)? It is important to "sell the ideas and work," pick the right topics and questions, research "big issues," and pick the proper publication forums (in CS and NLP). An h-index as a measure of impact. The danger of Tyranny of metric.
Week 2 – 27. 2. Petr Sojka, et al., Thinking Slow in our/your projects
Join us at A502, Faculty of Informatics MU, on Oct 3rd at 9:50 AM (catering preparation) and 10 AM (Invitation of newcomers, questions on readings, a summary of Week 1, and then brainstorming on Thinking Slow). To join via Zoom, ask for a password in advance.
Thinking slow (Kahnemann) is the key to expanding and searching for knowledge. While in thinking fast (retrieving facts, basically) current large language models beat humans, in reasoning tasks, the exponential growth of possible "trains of thoughts" (today called CoT - chain of thoughts) makes it very hard for LLMs. In the brainstorming session, we will discuss how to tackle the "curse of reasoning" in the grant proposal under preparation.
Week 3 – 6. 3. Marek Kadlčík, Michal Spiegel, Thinking Fast and Slow
Join us at A502, FI MU, on March 6th at 9:50 AM (catering preparation) and 10 AM (lecture). To join via Zoom, ask for a password in advance.
Marek: Jazykové modely se těší velké popularitě pro svoji použitelnost na celé řadě problémů v NLP. Přesto často vyplouvají na povrch až překvapivé základní slabiny, mezi které patří numerická gramotnost, která je klíčová v mnoha oblastech, jako např. ve vědě, finanční analýze nebo hospodářství.
V této přednášce se podíváme na způsoby, jakými můžeme reprezentovat čísla v jazykových modelech. Těšit se můžete na diskuzi o návrhu architektury neuronových sítí a jejich experimentální validaci.
Michal: Can transformers learn to perform algorithmic tasks reliably across previously unseen input/output domains? While pre-trained language models show solid accuracy on benchmarks incorporating algorithmic reasoning, assessing the reliability of these results necessitates an ability to cleanse models' functional capabilities from memorization. In this paper, we propose an algorithmic benchmark comprising six tasks of infinite input domains where we can also disentangle and trace the correct, robust algorithm necessary for the task. This allows us to assess (i) models' ability to extrapolate to unseen types of inputs, including new lengths, value ranges or input domains, but also (ii) to assess the robustness of the functional mechanism in recent models through the lens of their attention maps. We make the implementation of all our tasks and interoperability methods publicly available at https://github.com/michalspiegel/AttentionSpan .
Week 4 – 13. 3. Marek Kadlčík (cont.), Barbora Danková: TBA
Join us at A502, FI MU, on March 13th at 9:50 AM (catering preparation) and 10 AM (lectures). To join via Zoom, ask for a password _in advance_.
Week 5 – 20. 3. Ota Mikušek: TBA
Join us at A502, FI MU, on March 20th at 9:50 AM (catering preparation) and 10 AM (lectures). To join via Zoom, ask for a password _in advance_.
Week 6 – 27. 3. Katarína Hudcovicová: TBA
Join us at A502, FI MU, on March 27th at 9:50 AM (catering preparation) and 10 AM (lectures). To join via Zoom, ask for a password _in advance_.
Week 7 – 03. 4. Jan Franěk: TBA
Join us at A502, FI MU, on April 3rd at 9:50 AM (catering preparation) and 10 AM (lectures).
Week 8 – 10. 4. Martin Kňažovič: TBA
Join us at A502, Faculty of Informatics MU, on April 7th at 9:50 AM (catering preparation) and 10 AM (lectures).
Week 9 – 17. 4. Jakub Hartmann: TBA
Join us at A502, Faculty of Informatics MU, on April 17th at 10 AM (CET) [and on Zoom].
Week 10 – 24. 4. Zuzana Moravčíková: TBA
Join us at A502, Faculty of Informatics MU, on April 24th at 10 AM (CET) [and on Zoom].
Week 11 – 1. 5. Canceled (state holiday)
Week 12 – 8. 5. Canceled (state holiday)
Week 13 – 15. 5. Martin Beňa
Join us at A502, Faculty of Informatics MU, on May 15th at 10 AM (CET) or [or on Zoom].
Week 14 – 19. 5. TBA
Join us at A502, FI MU, on Monday, May 19th at 10 AM (CET) or [or on Zoom].
Tips for readings, discussions, and presentation preparations:
- Top2Vec towardsdatascience.com/top2vec-new-way-of-topic-modelling
- How to Speak by Patrick Winston (YouTube video)
Žákovi, který se hrozil chyb, Mistr řekl: "Ti, kdo nedělají chyby, chybují nejvíc ze všech – nepokoušejí se o nic nového." Anthony de Mello: O cestě
To a student in danger, the Master said: "Those who do not make mistakes most of all – they do not try anything new." Anthony de Mello