Instructor: Tim Chumley
Office: Clapp 404b
Phone: 538-2299
e-mail: tchumley
Office Hours: Mondays 4:15-5:15, Tuesdays 2:30-5:00, Wednesdays 1:00-2:00 & 4:15-5:15, Thursdays 2:30-3:30

Textbook: Introduction to Stochastic Processes with R by Robert P. Dobrow, ISBN: 9781118740651;
on library reserve under QC20.7.S8 D63 2016


Announcements will be posted here throughout the semester.

Apr 27: My office hours for reading week are as follows:

  • Monday: 4:15-5:15
  • Tuesday: 3:30-5
  • Wednesday: 1-2
  • Thursday: 1:30-3

Apr 26: The final exam will be distributed in line with the final exam period. This means that it will be posted on May 4 at 7:00 pm, and will be due (under my office door) May 8 at 12:00 pm or before. You need not take the exam according the self-scheduled sessions. In order to allow everyone time to ask questions during the reading period, it won't be given out earlier, except under special circumstances.

Apr 14: Suggestions for the structure of the presentation are posted here.

Apr 11: The presentation calendar is now posted below.

Apr 7: Exam 2 is posted here.

Feb 9: A link to DataCamp and a reminder about bonus points is posted below under Resources.

Feb 7: As a reminder, please submit redo's the Tuesday after homework is returned. After that day, I'll post solutions and won't accept any more redo's.

Jan 30: One of our classmates pointed out to me that a full, searchable electronic copy of our book is available through the Mount Holyoke College library!


Check the syllabus for all the important class policies (grades, attendance, etc.).


There will be weekly homework assignments throughout the semester to be turned in, as well as suggested problems to be considered as additional practice.

  • Written assignments. A selection of problems taken from the textbook or other sources will be assigned to be due Thursdays at 5pm in the folder outside my office.
    • These assignments are given so that you can practice writing mathematics and receive feedback on your progress.
    • If you would like feedback on a particular steps in a problem, then you can indicate this on your assignment.
    • Each problem will be given a score of 4, 3, 2, or 1. Details about grading are here.
  • Suggested problems. The book has a good variety of problems, both in terms of content and difficulty level. As you study throughout the semester, you might consider browsing and trying the problems at the end of the chapter for extra practice. I'll always be happy to help over email or in office hours if you're stuck.
  • Redo's. You'll be allowed once chance to redo missed homework problems. Please turn in your redo's, with your previously graded solutions attached, the Tuesday after your graded solutions were returned.
Written Assignment Due Graders
Problem set 0 Jan 27 .
Problem set 1 (solutions) Feb 2 Amna, Jeanie
Problem set 2 (solutions) Feb 9 An, Carol, Thu
Problem set 3 (solutions) Feb 16 Regina, Young
Problem set 4 (solutions) Feb 23 Anisa, Madeline, Yiwen
Problem set 5 (solutions) Mar 9 Karen, Zhihong
Problem set 6 (solutions) Mar 23 Allison, Barbara
Problem set 7 (solutions) Mar 30 Emily, Marwa, Paula
Problem set 8 (solutions) Apr 6 Alex, Celine, Saadia
Problem set 9 (solutions) Apr 20 Cheng, Deepshikha


There will be three exams. The dates for the two mid-terms are subject to change slightly:

Exam Date Time and Location Material Study material Solutions
Exam 1 Mar 2 in class Ch 1, 2, 3.1-3.6 (pdf) solutions (pdf)
Exam 2 Apr 13 take home 3.7-3.8, 5.1-5.2, 5.6, 6.1-6.5 (pdf) .
Final May 4-8 take home cumulative (pdf) solutions .

Course plan

Below is a rough outline of the sections to be covered week by week through the semester, including suggested textbook problems. Some of this is subject to change, so please check regularly.

Week Sections In-class activities
Jan 23 - Jan 27 Chapter 1, 2 State spaces and index sets
Markov transition matrix
Jan 30 - Feb 3 Chapter 1, 2 n-step distribution (R code)
Simulation (R code)
Feb 6 - Feb 10 3.1 - 3.2 Communication classes
Feb 13 - Feb 17 3.2 - 3.3 Canonical decomposition
Feb 20 - Feb 24 3.4 - 3.8 Periodicity
Absorbing chains
Feb 27 - Mar 3 5.1 - 5.2 Metropolis-Hastings algorithm
MCMC and cryptography (main R code, reference text setup, visualization code)
Mar 6 - Mar 10 3.9, 5.6 Riffle shuffling
Mar 20 - Mar 24 6.1 - 6.2 Poisson process examples
More examples
Mar 27 - Mar 31 6.3 - 6.5 Thinning and superposition
Poisson processes and the uniform distribution
Apr 3 - Apr 7 7.1 - 7.2 R code for simulation of Poisson process
R code for simulation of CTMC examples
Apr 10 - Apr 14 7.3, 7.4 Infinitesimal generator (R code)
Apr 17 - Apr 21 Chapter 8
Apr 24 - Apr 28 Presentations


The presentations this semester serve as an opportunity to study, synthesize, and communicate on a topic of your choosing. Here are some suggested topics. The suggestions are mostly centered around applications of stochastic processes; this is done to emphasize the notion that the field of probability is deeply rooted in both pure and applied mathematics and statistics. Some suggestions on presentation structure are also posted.

A calendar of the presentation groups and topics is listed below. Please plan for 20 minute presentations, with 5 minutes after for questions and buffer time.

Group members Topic Presentation day Slides References
Allison, Karen, Zhihong Martingales, optional stopping Tuesday, April 25 (pdf) (pdf)
Anisa, Saadia, Young Applications to finance, pt 1 Tuesday, April 25 (pdf) (txt)
Carol Applications to finance, pt 2 Tuesday, April 25 (pdf) (pdf)
Alex, Emily, Regina Simulated annealing, optimization Thursday, April 27 (pdf) (txt)
Deepshikha, Jeanie, Paula Gibbs sampler Thursday, April 27 (pdf) (pdf)
Amna, Barbara, Madeline Cryptography Thursday, April 27 (pdf) (pdf)
An, Cheng, Thu Branching processes, pt 1 Friday, April 28 (pdf) (txt)
Celine, Marwa Branching processes, pt 2 Friday, April 28 (pdf) (txt)

Getting help

Here's a few ways to get help:

  • Office hours (listed at the top of this page). These are times I hold on my schedule to be available for students. I like to use these times as open drop in sessions where there may be multiple students getting help simultaneously, but it's expected that you come with specific questions about notes, homework problems, or the text that you have thought about already. If you need to schedule a time to meet with me one-on-one, either to discuss something private or because you can't make my office hours, please send me an email with times that you are available to meet, and I will find a time that works for both of us.
  • Study groups. I strongly encourage you to form small groups and work together on problems. My advice though is to try problems by yourself before discussing them. We have several spaces on the fourth floor of Clapp where students can work together for extended periods of time.


  • Everyone was invited to join DataCamp, which provides an introductory R tutorial. It's a convenient way to gain some familiarity with R, a useful tool for our course and beyond. Please complete the assigned tutorial by February 28 to get some bonus points. Let me know if you have any trouble with the invitation or need some help with the material in the tutorial.

  • Our textbook has a useful collection of R scripts available here; contained there are all the R code snippets you'll notice interspersed in the text.