Parsing with CCG - Lecture 6Syntactic formalisms for natural language parsing FI MU autumn 2011 2 Categorial Grammar is : a lexicalized theory of grammar along with other theories of grammar such as HPSG, TAG, LFG, . . . : linguistically and computationally attractive language invariant combination rules, high efficient parsing 3 Outline 1. A-B categorial system 2. Lambek calculus 3. Extended Categorial Grammar ● Variation based on Lambek calculus – Abstract Categorial Grammar, Categorial Type Logic ● Variation based on Combinatory Logic – Combinatory Categorial Grammar (CCG) – Multi-modal Combinatory Categorial Grammar 4 Main idea in CG and application operation ● All natural language consists of operators and of operands. ● Operator (functor) and operand (argument) ● Application: (operator(operand)) ● Categorial type: typed operator and operand 5 1. A-B categorial system The product of the directional adaptation by Bar-Hillel (1953) of Ajdukiewicz’s calculus of syntactic connection (Ajdukiewicz, 1935) Definition 1 (AB categories). Given A, a finite set of atomic categories, the set of categories C is the smallest set such that: • A C⊆ • (X\Y), (X/Y) C if X,Y C∈ ∈ 6 ● Categories (type): primitive categories and derivative categories – Primitive: S for sentence, N for nominal phrase – Derivative: S/N, N/N, (S\N)/N, NN/N, S/S... ● Forward(>) and backward (<) functional application a. X/Y Y X⇒ (>) b. Y X\Y X⇒ (<) 7 ● Calculus on types in CG are analogue to arithmetic subtraction x/y x → y ≈ 2/4 * 2 = 4 8 defeated Germany Brazil operator operand operand @ defeated (Germany) @ ((defeated(Germany))Brazil) Applicative tree of Brazil defeated Germany 9 Limitation of AB system 1. Relative construction a. teami that ti defeated Germany b. teami that Brazil defeated ti a'. that (n\n)/(s\n) b'. that (n\n)/(s/n) team [that](n\n)/(s\n) [defeated Germany]s\n team [that](n\n)/(s/n) [Brazil defeated]s/n team that Brazil defeated (n\n)/(s/n) n (s\n)/n (?) 10 2. Agrammatical sentence considered as well-formed structure *a man good n/n n n\n n : ((good)man) n : (a((good)man)) a good man n/n n\n n n : ((good)man) n : (a((good)man)) 3. Many others complex phenomena – Coordination – Object extraction, unbounded dependencies,... 4. AB's generative power is too weak. 11 2. Lambek calculus (Lambek, 1958, 1961) - on the calculus of syntactic types The axioms of Lambek calculus are the following: 1 . x → x 2 . (xy)z → x(yz) →(xy)z (the axioms 1, 2 with inference rules, 3, 4, 5) 3 . If xy → z then x → z/y, if xy → z then y → x\z ; 4 . If x → z/y then xy → z, if y → x\z then xy → z ; 5 . If x → y and y → z then x → z. 12 The rules obtained from the previous axioms are the following: 1 . Hypothesis: if x and y are types, then x/y and y\x are types. 2 . Application rules : (x/y)y → x, y (y\x) → x ex: Poor John works. 3 . Associativity rule : (x\y)/z ↔ x\(y/z) ex: John likes Jane. 4. Composition rules : (x/y)(y/z) → x/z, (x\y)(y\z) → x\z ex: He likes him. s/(n\s) n\s/n 5. Type-raising rules : x → y/(x/y), x → (y/x)\y 13 3. Combinatory Categorial Grammar ➢ Developed originally by M. Steedman (1988, 1990, 2000, ...) ➢ Combinatory Categorial Grammar (CCG) is a grammar formalism equivalent to Tree Adjoining Grammar, i.e. ✗ it is lexicalized ✗ it is parsable in polynomial time (See Vijay-Shanker and Weir, 1990) ✗ it can capture cross-serial dependencies ➢ Just like TAG, CCG is used for grammar writing ➢ CCG is especially suitable for statistical parsing 14 ● several of the combinators which Curry and Feys (1958) use to define the λ-calculus and applicative systems in general are of considerable syntactic interest (Steedman, 1988) ● The relationships of these combinators to terms of the λ-calculus are defined by the following equivalences (Steedman, 2000b): a. Bfg ≡ λx.f(g x) b. Tx ≡ λf.f x c. Sfg ≡ λx.fx(g x) 15 CCG categories ● Atomic categories: S, N, NP, PP, TV. . . ● Complex categories are built recursively from atomic categories and slashes ● Example complex categories for verbs: – intransitive verb: S\NP walked – transitive verb: (S\NP)/NP respected – ditransitive verb: ((S\NP)/NP)/NP gave 16 Lexical categories in CCG ● An elementary syntactic structure – a lexical category – is assigned to each word in a sentence, eg: walked: S\NP ‘give me an NP to my left and I return a sentence’ ● Think of the lexical category for a verb as a function: NP is the argument, S the result, and the slash indicates the direction of the argument 17 The typed lexicon item ● The CCG lexicon assigns categories to words, i.e. it specifies which categories a word can have. ● Furthermore, the lexicon specifies the semantic counterpart of the syntactic rules, e.g.: love (S\NP)/NP λxλy.loves′xy ● Combinatory rules determine what happens with the category and the semantics on combination 18 ● Attribution of types to lexical items: examples Predicate ex: is as an identificator of nominal as an operator of predication from a nominal (S\NP)/NP from an adjective (S\NP)/(N/N) from an adverb (S\NP)/(S\NP)\(S\NP) from a preposition (S\NP)/((S\NP)\(S\NP)/NP) ex: verbs unary (S\NP) binary (S\NP)/NP ternary (S\NP)/NP/NP 19 Adverbs Adverb of verb (S\NP)/(S\NP) (S\NP)/NP/(S\NP)/NP Adverb of adverb (S\NP)/(S\NP)/(S\NP)/(S\NP) (S\NP)/NP/(S\NP)/NP/(S\NP)/NP/(S\NP)/NP Adverb of adjective (N/N)/(N/N) (N\N)/(N\N) Adverb of proposition S/S Adverb: operator of determination of type (X/X) 20 Preposition Prep. 1: constructor of adverbial phrase (S\NP)\(S\NP)/NP (S/S)/NP (S/S)/N Prep. 2: constructor of adjectival phrase (N\N)/NP (N\N)/N Preposition: constructor of determination of type (X/X) 21 Dictionary of typed words Syntactic categories Syntactic types Lexical entries Nom. N Olivia, apple... Completed nom. NP an apple, the school Pron. NP She, he... Adj. (N/N), (N\N) pretty woman,... Adv. (N/N)/(N/N), (S\NP)\(S\NP)... very delicious,... Vb (S\NP), (S\NP)/NP... run, give... Prep. (S\NP)\(S\NP)/NP (NP\NP)/NP... run in the park, book of John,... Relative (S\NP)/S... I believe that... 22 Combinatorial categorial rules ● Functional application (>,<) ● Functional composition (>B, T) ● Distribution (S) ● Coordination (<Φ, >Φ) 23 Functional application (FA) X/Y:f Y:a ⇒ X:fa (forward functional application, >) Y:a X\Y:f ⇒ X:fa (backward functional application, <) ● Combine a function with its argument: NP S\NP S Mary sleeps → (sleeps (Mary)) NP (S\NP)/NP NP S\NP → (likes (Mary)) S John likes Mary → ((likes (Mary))John) ● Direction of the slash indicates position of the argument with respect to the function 24 Derivation in CCG ● The combinatorial rule used in each derivation step is usually indicated on the right of the derivation line ● Note especially what happens with the semantic information 25 Function composition (FC) X/Y:f Y/Z:g ⇒B X/Z:λx.f(gx) (>B) ● Functional composition composes two complex categories (two functions): (S\NP)/PP (PP/NP) ⇒Β (S\NP)/NP S/(S\NP) (S\NP)/NP ⇒Β S/NP Generalized forwardcomposition(>Bn) 26 Generalizedbackwardcomposition(T) ● Type-raising turns an argument into a function (e.g. for case assignment) NP ⇒ S/(S\NP) (nominative) ● This must be used e.g. in the case of WH-movement Forward type-raising (>T) 28 Example of functional composition (>B) and type-raising (T) 29 X:a ⇒ T\(T/X):λf.fa (S) (X/Y)/Z Y/Z ⇒S X/Z • Application to parasitic gap such as the following: a. team that I persuaded every detractor of to support 32 Substitution (S) Backward crossed substitution ( X ➢ The Principle of Directional Inheritance: If the category that results from the application of a combinatory rule is a function category, then the slash defining directionality for a given argument in that category will be the same as the one defining directionality for the corresponding arguments in the input functions. ex: X/Y Y/Z ≠> X\Z. 34 Semantic in CCG ● CCG offers a syntax-semantics interface. ● The lexical categories are augmented with an explicit identification of their semantic interpretation and the rules of functional application are accordingly expanded with an explicit semantics. ● Every syntactic category and rule has a semantic counterpart. ● The lexicon is used to pair words with syntactic categories and semantic interpretations: love (S\NP)/NP ⇒ λxλy.loves′xy 35 ● The semantic interpretation of all combinatory rules is fully determined by the Principle of Type Transparency: – Categories: All syntactic categories reflect the semantic type of the associated logical form. – Rules: All syntactic combinatory rules are type-transparent versions of one of a small number of semantic operations over functions including application, composition, and type-raising. 36 proved := (S\NP3s )/NP : λxλy.prove'xy ● the semantic type of the reduction is the same as its syntactic type, here functional application. 37 CCG with semantics : Mary will copy and file without reading these articles 38 Parsing a sentence in CCG Step 1: tokenization Step 2: tagging the concatenated lexicon Step 3: calculate on types attributed to the concatenated lexicons by applying the adequate combinatorial rules Step 4: eliminate the applied combinators (we will see how to do on next week) Step 5: finding the parsing results presented in the form of an operator/operand structure (predicate -argument structure) 39 Example: I requested and would prefer musicals STEP 1 : tokenization/lemmatization → ex) POS Tagger, tokenizer, lemmatizer a. I-requested-and-would-prefer-musicals b. I-request-ed-and-would-prefer-musical-s STEP 2 : tagging the concatenated expressions → ex) Supertagger, Inventory of typed words I NP Requested (S\NP)/NP And CONJ Would (S\NP)/VP Prefer VP/NP musicals NP 40 STEP 3 : categorial calculus a. apply the type-raising rules b. apply the functional composition rules c. apply the coordination rules I- requested- and- would- prefer- musicals 1/ NP (S\NP)/NP CONJ (S\NP)/VP VP/NP NP 2/ S/(S\NP) (S\NP)/NP CONJ (S\NP)/VP VP/NP NP (>T) 3/ S/(S\NP) (S\NP)/NP CONJ (S\NP)/NP NP (>B) 4/ S/(S\NP) (S\NP)/NP NP (>Ф) 5/ S/(S\NP) (S\NP)/NP NP (>B) 6/ S/NP NP (>) 7/ S 41 STEP 4 : semantic representation (predicate-argument structure) I requested and would prefer musicals 1/ :i' :request' :and' : will' :prefer' : musicals' 2/ :λf.f I' 3/ : λx.λy.will'(prefer'x)y 4/ : λtvλxλy.and'(will'(prefer'x)y))(tv xy) 5/ : λxλy.and'(will'(prefer'x)y)(request'xy) 6/ :λy.and'(would'(prefer' musicals')y)(request' musicals' y) 7/S: and'(will'(prefer' musicals') i')(request' musicals' i') 42 Variation of CCG : Multi-modal CCG (Baldridge, 2002) ● Modalized CCG system ● Combination of Categorial Type Logic (CTL, Morrill, 1994; Moortgat, 1997) into the CCG (Steedman, 2000) ● Rules restrictions by introducing the modalities: *, x, •, ◊ ● Modalized functional composition rules ● Invite you to read the paper “Multi-Modal CCG” of (Baldridge and M.Kruijff, 2003 ) 43 The positions of several formalisms on the Chomsky hierarchy 44 Classwork Exercise of taggings and of categorial calculus See the given paper!!