Chap. 3. Discrete time Markov chains

6 maio 2013, 08:00 • Manuel Cabral Morais

3.4 Limit behavior of irreducible Markov chains (from Definition 3.59)

3.5 Limit behavior of reducible Markov chains

3.6 Markov chains with costs/rewards

3.7 Time reversible Markov chains

Exercises: 3.60(hints), 3.65, 3.70, 3.72, 3.77, 3.80, 3.84, 3.88, 3.98

Chap. 3. Discrete time Markov chains

30 abril 2013, 13:00 • Manuel Cabral Morais

3.3 Classification of states; recurrent and transient states (from Definition 3.33)

3.4 Limit behavior of irreducible Markov chains (until Example/Exercise 3.58)

Exercises: 3.35, 3.42, 3.48, 3,57(b), 3.58

Chap. 3. Discrete time Markov chains

29 abril 2013, 08:00 • Manuel Cabral Morais

3.1/3.2 (revision)

3.3 Classification of states; recurrent and transient states (until Exercise 3.32)

Exercises: 3.7, 3.9(hints), 3.19, 3.22, 3.25(d), 3.32

Chap. 3. Discrete time Markov chains

23 abril 2013, 13:00 • Manuel Cabral Morais

3.1 Definitions and examples

3.2 Chapman-Kolmogorov equations; marginal and joint probabilities

3.3 Classification of states; recurrent and transient states (incomplete)

Exercises: 3.6, 3.11, 3.15, 3.16, 3.18, 3.25

Chap. 2. Renewal processes

22 abril 2013, 08:00 • Manuel Cabral Morais

2.8 Alternating renewal processes (from Exercise 2.109) 2.9 Delayed renewal processes 2.10 Regenerative processes (until Exercise 2.127) Exercises: 2.109, 2.110, 2.117, 2.127(hints)