Dialogue systems Dialogue systems Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies Luděk Bártek Laboratory of searching and dialogue, Fakulty of Informatics, Masarykovy Univerzity, Brno spring 2023 Basic Terms Dialogue systems Luděk Bártek Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies Dialogue - communication of two or more participants (flow of utterances). Utterance - continuous communication from one participants to the other. Turn - Utterance of one participant and response of the other. Dialogue strategy ■ Function that assigns to subsequent utterance to the given one. ■ It utilises a knowledge of a dialogue state: ■ required and entered informations ■ dialogue participants abilities It is property of each dialogue participant. Basic Concepts Dialogue Communication Dialogue systems Luděk Bártek Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies Evaluation function: ■ function assigning a real number to every dialogu ■ Used notation is E(/_), where L is a dialogue. Dialogue communication - An ordered quaternion M = (S1,S2,E1,E2) ■ S/, / G {1,2} - dialogue strategy of a participant. ■ E/, / G {1,2} - evaluation function of a participant. Basic Concepts Cooperation in a Dialogue Dialogue systems Luděk Bártek Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies Dialogue communication M = (Si, S2, £1, £2) is called: ■ Cooperative o Ei = E2. Both participants has the same objective and try to cooperate. ■ Non-cooperative o Ei ^ E2. The objectives of both participant differs. ■ zero sum communication 44> Ei = —E2. The objectives are contradictory. The evaluation is based on the game theory: ■ dialogue can be viewed as a game of two participants. Cooperative Conversation Rules Dialogue systems Luděk Bártek Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies A dialogue system should try to keep the dialogue to be cooperative. Author Herbert Paul Grice - English linguist. Aspect of informativeness: Q Be reasonably informative - no less no more than needy. Aspect of quality: Q Don't tell false informations. Q Don't tell the informations you're not able to prove by adequate evidence. Aspect of manner: Information should be direct and straightforward. Avoid ambiguity or obscurity. Strive for brevity. Be disciplined, keep the dialogue orderly. Cooperative Communication Rules cont. Dialogue systems ■ Aspect of civility, empathy and ethics: Luděk Bártek □ Minimize the demands on the communication partner and Dialogue Flow maximize the benefits for him. Control Q Minimize the shortcomings of the communication partner a Basic Terms Cooperative Conversation Rules maximize his strengths. Dialogue Initiative □ Maximize the partner's agreement and minimize his and Feedback Next Aspects of disagreement. Dialogue Strategies □ Maximize the empathy towards your partner. ■ Aspect of asymmetry: Q Inform a user of all important characteristics that deviate from the expected dialogue flow and that he should consider to ensure cooperation. Q Ensure short but still adequate informations about the system possibilities and limits. Q Give the user enough and understandable informations about the system interaction. Cooperative Communication Rules Human — Computer Communication Dialogue systems Luděk Bártek Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies Aspect of knowledges and abilities: Consider the user's relevant knowledge. Consider the user's possible wrong analogies. Differentiate between novice and experienced system user. Consider the user's legitimate perceptions of the system's knowledge and capabilities. Aspect of clarification and elimination of errors: Initiate the meta-communication that leads either to elimination or to clarification of a communication error in the case of communication failure. Ensure a meta-communication explaining either non-consistent or ambiguous user input data. Aspects of a Cooperative Dialogue System Dialogue systems Luděk Bártek Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies The dialogue system should take into account the following aspects when communicating with the user: ■ aspect of informativeness ■ aspect of persuasiveness ■ aspect of manner ■ aspects of civility, empathy and ethics ■ aspect of asymmetry ■ aspect of a user knowledges and abilities ■ aspect of error clarification and elimination Dialogue Initiative Dialogue systems Luděk Bártek Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies The dialogue next step is determined by a dialogue strategy of one of communicating participants. ■ One participant usually asks the questions and the other responds. We can distinguished in the case of human — computer communication: ■ user initiative dialogue ■ system initiative dialogue ■ mixed initiative dialogue The systems in a real life utilizes: ■ mixed initiative dialogues ■ system initiative dialogues. Dialogue Initiative Examples Dialogue systems Ludek Bartek Dialogue Flow ■ System i initiative dialogue: Control Basic Terms Cooperative System Your user name please Conversation Rules Dialogue Initiative a nrl FppH narl User: xyz dllU rtCUUdL^ Next Aspects of Dialogue Strategies System Say the sentence: " My voice is my passport. Verify me!" to verify you. User: ■ ■ ■ My voice is my passport. Verify me! □ rfp 1 1 ► < » ► Dialogue Initiative Examples Dialogue systems Luděk Bártek Dialogue Flow Control Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies Mixed initiative dialogue: User: System: User: System: I'd like to go by train from Adamov to Kerkyra today. Would you like to find a direct connection or a connection with transfers? I'd like a direct one. I'm sorry. There is no direct Adamov - Kerkyra connection. □ rS1 Dialogue System Feedback Dialogue systems Luděk Bártek ■ The system should verify the information prior to process them: Dialogue Flow ■ speech recognition errors correction Control Basic Terms Cooperative ■ user errors corrections Conversation Rules Dialogue Initiative ■ ... and Feedback Next Aspects of ■ Acquired Data Verification manners: Dialogue Strategies ■ Summarizing feedback - system repeats the user input and allows their correction when needed. ■ " Echo" feed back - the system repeats every input right after it has been entered and offers user to correct it. ■ Implicit feedback - the last data entered by user are part of the query on the the next data. ■ Explicit feedback - system validates the input data using the explicit queries on their values. Dialogue System Feedback Examples Dialogue systems Luděk Bártek Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies Summarizing feedback: User: I'd like to go by train from Adamov to Kerkyra. System: Searching the connection by train. Departure Adamov, final stop Kerkyra. Echo feedback: System: What kind of transport do you prefer? Train. You wan to go by train. Where do you want to go from? Adamov. You want to go from Adamov. Where do you want to go? To Kerkyra. User: System: User: System: Uživatel: Dialogue System Feedback Examples Dialogue systems Luděk Bártek Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies Implicit feedback: System: Student name. User: Jan Novak. System: Where was Jan Novak born? ■ ■ ■ Explicit feedback: System: Student name. User: Jan Novak. System: Student name is Jan Novak. Correct? Correction dialogue example Dialogue systems Luděk Bártek Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies User: System: User: System: User: System: My name is Jan Horák and I was born on August 8th 1988 in ABC. User entered name Jan Novak, birth date August 8th 1988 and birth place ABC. No. My name is Jan Horák. Your name is Jan Horňák. Birth date August 8th 1988 and birth place ABC. No, My name is Jan Novák. Your name is Jan Novák, born on August 8th 1988 in ABC. Other Aspects of Dialogue Strategies Dialogue systems Luděk Bártek Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies Possibility to interrupt system. Possibility to start correction and meta communication by user: User: Would you be so kind and repeat the last question? Possibility to request a help: User: Could you specify the required data? User knowledges and abilities detection: ■ Partially can be derived from a previous dialogue: ■ number of user errors ■ number of help requests ■ ... ■ Dialogue system initiative adoption. Dialogue system initiative adoption - for example user in a hurry, stressed user, ... Next Aspects of Dialogue Strategies Dialogue systems Luděk Bártek Basic Terms Cooperative Conversation Rules Dialogue Initiative and Feedback Next Aspects of Dialogue Strategies User emotions detection: ■ recognition based on non-verbal voice characteristics required support in a speech recognition module ■ using another sensors - EEG, EKG, ... multil ingual ity multimodality ■ the dialogue is conducted by avatar (talking head) -suitable for the hearing impaired users for example ■ alternative inputs: ■ keyboard ■ face detection courtesy prosody learning from mistakes.