Introduction to Stochastic
Processes
Michael Shadlen
Neubeh 545
February 2003
What is a stochastic process?
•  Stochastic just means random
•  Often, a random sequence of events
Stochastic point process
•   The events are stereotyped (points)
•   All that matters (to us) is when they occur
{t1? t2, t3,...}
Or the intervals between them
{tr0, t2-tl9 t3-t2, ...} = {Al9 A2, A3, ...}
•   If the intervals are independent and identically distributed (iid) the process is called a Renewal
•   Examples:
-  Radioactive decay
-  Time to failure of a part
-  Queuing (e.g., vesicle release)
-  Spikes
Information is coded by spikes
Variability of spike trains in cortex is a fundamental problem
Biophysics:
What accounts for variability?
Psychology:
Does it limit sensory fidelity? Motor precision?
Computational neuroscience:
What are the implications for the neural coding of information?
40 spikes per second
100 msec
™^-«SEI§gf£SK
500 ms
Spikes recorded on
214 repetitions of the
same random-dot
stimulus.
Instantaneous spike rate computed in 2 msec bins from average of all 214 trials
�30818
71
1802340958
500 ms
B
■■■.!■;. ■■■■.:■ i.-:!-:.....■■; .>■■■.■: ■■!■:■'.....':.'',■••
,:■  - . !i. ■.■ ;"■■■-■ i-.b- Y ,\:.' .-■ !i iaii: .'■-■   ii>.....'.'
■ !■;■   ľ" V '": '•'•'■• ! :'.'-1 :!i'.' S ' ■■"■ í"11!-1 ■■■'
■    IĽ         ■ J      ■■ .   I      ■■ ■       I .     ■■        ■       ■ ■   ■          ■    ■      ■■ ■      I          ■
-<?200i t/)
i 1001
co     0-
360    380    400    420    440    460
Time (ms)
�30818
71
™^-«SEI§gf£SK
500 ms
B
■■■.!■;. ■■■■.:■ i.-:!-:.....■■; .>■■■.■: ■■!■:■'.....':.'',■••
!'V ':iM':'.i\■■ -i ",!h-.■''■■   .'i.*, ■ ::!:i\i:-'; : i   i  ,, , ■b;1.- ..... i1 ::—;■■' |. .!, ir■-.■'i■■■-■ "-i--.
.■.." :'.,'■■ "■■ :■:. V:',,|,v ŕ ',V;'-' : í-.;!J '■ J.-.
i-;,   i-1 ■,"'": '.:'■■ : :,1'-1 !;■',',:." ■ n'1 í.'.11!-' ■■■'
II1         ■ j      ■■  .   I      ■■ ■        I  .     ■■         ■        ■  ■   ■          ■    ■       ■■  ■      I          ■
-<?200i to
31001
co     0-
360    380    400    420    440    460
Time (ms)
ST  SO]
0
10        20        30        40
IntenspikG interval frns)
�30818
71
1
Spike variability: temporal code or noise
40 spikes per second
Spike "I
^w
^~^
ŕ
46*
&ŕä"
"—^
ar" Code
Temporal pattern of spike intervals code features
Ensemble patterns possible
Discrete RVs are described by probability distributions
Random Val s are non-negative integers f(x) = P{X = x},   X e {0,1,2,...}
F (x) = P{X ^ x}
= Íf(n) The total probability is 1
oo
limF(*) = Y/(/i) = l
n=0
Examples of discrete probability
distributions
•   Bernoulli distribution
-   Single flip of a coin: 1 or 0
•   Binomial distribution
-   Number of heads out of N flips of a coin
•   Geometric distribution
-   Number of coin tosses before the first heads
•   Poisson distribution
-   Number of radioactive decays in 1 second
-   Number of silver grains in 1 square cm
Continuous RVs are described by probability
densities
• Random Vals are real numbers. Consider the cumulative distribution function (CDF)
F(x) = P{X z x}
Continuous RVs are described by probability
densities
• Random Vals are real numbers. Consider the cumulative distribution function (CDF)
F(x) = P{X < x}
There exists a probability density function, f(x), such that
x                                                                            °°c
F(x) = ff(x)dx          1}™F^ = JfWd* =l
-oo                                                                                                                      -00
f(x) = ^-F(x) ax
Continuous Probability density
function (PDF)
•  Continuous values (reals, positive reals, etc.)
-  Normal (Gaussian) distribution
-  T distribution
•   Sum of n RVs drawn from standard normal
-  Chi-square distribution
•   Sum of n squared RVs drawn from std normal
-  Exponential distribution
•  Waiting time to the next radioactive decay
-  Gamma distribution
•  Waiting time to the nih radioactive decay
-  Rayleigh distribution
•  Distance from bull's-eye of random dart
Some important properties of random variables
• Sum of independent RVs
- If X and Fare independent RVs, then if Z=X+Y, the expectation of Z is
E[ZJ = E[XJ + EfYJ
and the variance of z is
VarfZ] = VarfX] + VarfYJ
Some important properties of random variables
• Expectation
- This is just like an average, but it is convenient to think of it as a weighted sum (or integral)
- If X is an RV with ?D¥J(x), then the expectation of X is
00
E[X] = J xf(x)dx
— 00
Some important properties of random
variables
Variance
- This is the expectation of the RV minus its mean, squared. If X is an RV with PDF,/(xj,
Var[X] = E[(X-v)2]
oo
= J (x - /bi)2 f(x) dx
— 00
00
= J (x2 - 2/Ax + fJL2)f(x) dx
— 00
00                                                                            00                                                             00
= J x2f(x) dx - 2\i\ xf(x) dx + j [ŕ f(x) dx
— 00                                                                         —00                                                         —00
= E[X2]-2fŕ + [ŕ = E[X2]-E[X]2
Some important properties of random variables
• Distribution of sums
- If X and Y are independent RVs with PDFs fx(x) andfrfy), then if Z=X+ Y, the PDF, fz(z), is the convolution of fx and/}
00
fz(z) = ffx(x)fy(z-x)dx
—00
Some useful descriptive statistics
for point processes
* Coefficient of variation of the interspike interval
^A
Variance of the counts divided by the mean count (Fano factor)
pL
'N
Hazard function
• The conditional probability that an event happens at time t, given that it has not happened yet.
Entropy
The Poisson point process
Intervals distributed as Exponential Counts distributed as Poisson
Imagine an epoch, ŕ, divided into M bins.
Ar
0
T = MAt
If the expected count is pCT, where [i is the rate in events per sec,
then the probability of an occurrence in a bin of size At is fiAt. If
there are M bins, then At-TIM. The probability of getting no events
in all M bins is                                           ^T   .                .          .
Notice that as At gets tiny, M gets
large, and
P(0) = (1 - fjAt)
T
M
í          t\m
1- fl
M
TO = limfi -
M ^oo  \
M j
\M
= e
-\iT
The Poisson point process
Intervals distributed as Exponential Counts distributed as Poisson
Imagine an epoch, ŕ, divided into M bins.
Ar
0
T = MAt
The probability of getting 1 event is the product of the seeing an event in any one bin times not seeing it in any others
P(l) = M(/iAř)(l - MO
T
M-\
I          t\m-x
= fiT 1 - fi
\        M
Again, as At gets tiny, M gets large, and
/   ^M-1
p(X> = lim^ i - M ,
= \iTe
-\iT
The Poisson point process
•  Intervals distributed as Exponential
•  Counts distributed as Poisson
We are now imagining the limit, where M is very big and At is very small.
At    |_______|             |             |_______|             |             |_______|
0                                                                                                              T = M At
What is the waiting time to the 1st event? Let T be the waiting time. We know its cumulative distribution function
Fit) = P{T <t} = P{0 counts in epoch t} = e^