Dialogue systems Digital Signal Processing Luděk Bártek Laboratory of Searching and dialogue, Fakulty of Informatics, Masaryk University, Brno spring 2023 Digital Signal Processing Introduction Dialogue systems Luděk Bártek ■ The sound characteristics do not change on short time intervals - short term analysis. Digital Signal Processing Time-domain Signal ■ The time interval is called micro segment - length 10 — Processing Frequency Domain 40 ms. Analysis ■ Short term analysis methods: ■ Time domain methods - the values of samples are processed directly. ■ Frequency domain methods - the sample values are transformed into the frequency characteristics that are processed. ■ Corti's apparatus modelling - simulates the resonance of particular Corti's apparatus threadbares using differential equations. Digital Signal Processing Window functions Dialogue systems Luděk Bártek ■ The short-time analysis methods assume that the signal Digital Signal Processing period doesn't change in the window surroundings. Time-domain Signal Processing ■ The error caused by this assumption is compensated by Frequency Domain Analysis using the "window". ■ Window - a sequence of weights of the samples in the micro segment. ■ The weights should correspond to the influencing the sample by its surrounding. ■ Commonly used window functions types:: ■ the rectangular window function ■ the Hamming's window function. Digital Signal Processing Hamming's Window Function Dialogue systems Luděk Bártek Time-domain Signal Processing Frequency Domain Analysis Assumes that the samples closer to the centre of the micro segment are less influenced by surroundings than the samples close to the micro segment boundaries. The weight function: 1/1/(n) = n = 0 ... N - 1 0,54 - 0,46cos(^y) n<0Vn>N 0 The weight function graph on the micro segment: 40 E.Ů GO □ gi - = Digital Signal Processing Rectangular window Dialogue systems Luděk Bártek Time-domain Signal Processing Frequency Domain Analysis It assumes that: either the micro segment samples are not influenced by micro segment surrounding for our purpose or all samples are influenced in the same way. All the micro segment samples has assigned the same weight: '(J < n < N 1 n<0Vn>N 0 1/1/(n) = Time-domain Digital Signal Analysis Dialogue systems Luděk Bártek Digital Signal Processing Time-domain Signal ■ Based on sample values not on spectral characteristics. Processing Frequency Domain Ana lysis ■ Methods: ■ short-time energy ■ short-time intensity ■ zero crossing rate ■ first order difference ■ autocorrelation function ■ ... Time-domain Analysis Short-time energy Dialogue systems Luděk Bártek Time-domain Signal Processing Frequency Domain Analysis Uses average energy function in the segment: OO k——co ■ s(/c) - time k sample ■ u(n — k) - time k weight according the weight function Calculates the window average energy. The square root increases the sound signal dynamics. Usage: ■ automatic separation of speech (signal) and silence ■ characteristic for simple word classifiers ■ separation of voiced and unvoiced speech parts. Time-domain Analysis Short Term Intensity Dialogue systems Luděk Bártek Function of signal intensity at given segment. Time-domain Signal Processing Frequency Domain Analysis oo /(n)= \s{k)\«>(n - k) k——oo ■ \s(k)\ - absolute value of sample in time k. ■ u(n — k) - value of weight window corresponding the time k Usage - same as the short term energy. Doesn't increase the speech signal dynamics as much as the short term energy. Time-domain Analysis Short-time Zero Crossing Rate Dialogue systems Luděk Bártek ■ Counts the digital signal signum changer. Digital Signal CO Processing Time-domain Signal Processing Z(n)= \sgn[s(k)] - sgn[s(k - l)]\u(n - k) Frequency Domain Analysis k——co ■ Variant - local extreme count. ■ Both methods may be negative affected by the background noise. ■ Usage: ■ silence detection ■ signal start and end detection even in noisy signal ■ formants approximation ■ simple word classifiers characteristic Time Domain Analysis Autocorrelation function Dialogue systems Luděk Bártek Time-domain Signal Processing Frequency Domain Analysis Returns similarity of sequences of the micro segment (the bigger value the more similar sequences shifted by m samples). oo R(m, n) = ^ (s(/cM'? ~ k))(s(k + m)uj(n - k + m)) k——oo When the signal period is P then the R(m,n) maximum is when m=0, P, 2P, ... Assumes the segment length 2P at least. Usage: ■ The Fq period. ■ Base for LPA coefficients calculation. Frequency Domain Signal Analysis Dialogue systems Luděk Bártek Digital Signal ■ Transforms digital signal from time domain into the Processing Time-domain Signal Processing frequency domain. Frequency Domain Analysis ■ It uses the Fourier transformation most often. ■ Commonly used types of frequency domain analysis: ■ short term Fourier transformation ■ short term discrete Fourier transformation ■ Fast Fourier transformation ■ cepstral analysis ■ linear predictive analysis ■ ... Frequency Domain Signal Analysis Short Term Fourier Transformation Dialogue systems ■ Based on Fourier transformation: Luděk Bártek Digital Signal Processing OO S(u,t)= s{k)w{t - k)e-'"k Time-domain Signal Processing k——oo Frequency Domain Analysis ■ Regular Fourier transformation can be obtained by fixating time t ■ |S(u;, t)\ - acoustic spectra component amplitude corresponding to frequency u on time t. ■ w(n) - window weight function. ■ It expects the periodic function on input - sound is periodic on short intervals only. ■ When using it we assume the micro segment is repeating periodically. <□►