Taggers
PA154 Language Modeling (6.2)
Pavel Rychlý
pary@fi.muni.cz
Pavel Rychlý ·Taggers · 1 / 9
Statistical Tagger
using Viterbi algorithm to find the most probable sequence of tags
sometimes even greedy search works
the hard part is to find probabilities
Pavel Rychlý ·Taggers · 2 / 9
TreeTagger
Helmut Schmid, Stuttgart 1994
originally developed and evaluated on English, later also German
disambiguation of proper nouns (named entities) and regular words
smoothing with Equivalence Classes
words with the same set of possible tags
tag is atomic, no attributes or categories
probabilities: decision trees
Vitterbi algorithm
Pavel Rychlý ·Taggers · 3 / 9
TreeTagger - decision tree
house in “The big house” is
NN with probability 0.7
ADJ with probability 0.1
## TreeTagger - suffix lexicon
Pavel Rychlý ·Taggers · 4 / 9
TreeTagger – results
Pavel Rychlý ·Taggers · 5 / 9
TreeTagger – results
Pavel Rychlý ·Taggers · 6 / 9
RFTagger
Helmut Schmid, Florian Laws, Stuttgart 2008
non-atomic tags
Pavel Rychlý ·Taggers · 7 / 9
RFTagger – decision tree
Pavel Rychlý ·Taggers · 8 / 9
RFTagger – results
Pavel Rychlý ·Taggers · 9 / 9