Instructor: Tim Chumley
Office: Clapp 404b
Phone: 538-2299
e-mail: tchumley
Office Hours: tentatively Mondays 10:00–11:00, Tuesdays 4:00–5:00, Wednesdays 3:00–4:00, Thursdays 9:00–10:00, and by appointment

Textbook: Introduction to Stochastic Processes with R by Robert P. Dobrow, ISBN: 9781118740651;
on library reserve under QC20.7.S8 D63 2016;
available as a free e-text


Announcements will be posted here throughout the semester.

Feb 3: Some clarifying remarks about rewrites have been posted below in the Homework section.

Feb 2: Here is another modeling competition opportunity taking place at Amherst College April 21 called SCUDEM (in addition to the one I wrote to you about taking place next week).


Check the syllabus for all the important class policies (grades, attendance, etc.).


There will be weekly homework assignments throughout the semester to be turned in, as well as suggested problems to be considered as additional practice.

  • Written assignments. A selection of problems taken from the textbook or other sources will be assigned to be due Fridays at 7pm.
    • These assignments are given so that you can practice writing mathematics and receive feedback on your progress.
    • If you would like feedback on a particular steps in a problem, then you can indicate this on your assignment.
    • Each problem will be given a score out of 5. A score of 1 will be given to indicate that to stay on track you should get help from your instructor.
    • Please read these guidelines for writing up your homework solutions.
  • Submitting to Gradescope. We’ll be using an online submission tool called Gradescope to collect, grade, and return homework. Here are some notes:
    • Begin by signing up as a student on the Gradescope home page with Entry Code MWY6EK.
    • I’ve posted a short tutorial to help with Gradescope if you haven’t used it before.
    • Gradescope has also made a guide to help with scanning and submitting.
    • More information is available in this general help guide.
    • Please let me know of any issues that come up!
  • Suggested problems. The book has a good variety of problems, both in terms of content and difficulty level. As you study throughout the semester, you might consider browsing and trying the problems at the end of the chapter for extra practice. I’ll always be happy to help over email or in office hours if you’re stuck.
  • Rewrites. You’ll be allowed to redo some missed homework problems this semester. Here are some details:
    • You can rewrite one problem per problem set and earn back full points.
    • You should include with your rewrite an explanation that makes it easy to see what has changed in your answer (ie. what went wrong originally and what you’ve done to fix it).
    • You should turn in your rewrite in class on the Friday after the original due date. (Generally graded work will be released on Gradescope two or three days after the due date.)
Assignment Due
Problem set 0 (solutions) Jan 29
Problem set 1 (solutions) Feb 2
Problem set 2 (solutions) Feb 9
Problem set 3 Feb 16
Problem set 4 Feb 23


There will be three exams. The dates for the two mid-terms are subject to change slightly:

Exam Date Time and Location Material Study material Solutions
Exam 1 Mar 2 in class, take-home
Exam 2 Apr 6 in class, take-home
Final exam May 3 - 7 self-scheduled

Course plan

Our plan is to cover the textbook chapters 1 – 3, and parts of 4 – 7. Below is a rough outline of what is to be covered week by week through the semester. Please check back regularly for precise details on what is covered, as well as postings of in-class worksheets and other activities. Also note that the days of the week under Sections in the table below provide links to class notes.

Week Sections In-class activities Problems
Jan 22 - Jan 26 Wednesday 1.1, 1.2, 2.1
Friday: 2.2
First day info, State spaces, index sets
Markov transition matrix, graphs
1.1, 1.2, 1.3, 1.4, 1.9, 1.35
2.8, 2.9, 2.10
Jan 29 - Feb 2 Monday: 2.3, 2.4
Wednesday: 2.3
Friday: 2.5
n-step distribution (solutions)
Simulation, RMarkdown
More simulation
2.1, 2.2, 2.6, 2.25, 2.26
2.4, 2.5, 2.11, 2.17
2.23, 2.24, 2.27
Feb 5 - Feb 9 Monday: 3.1, 3.2
Wednesday: snow day
Friday: 3.2, 3.3
Stationary distribution

Communication classes
2.18, 3.1, 3.2, 3.5, 3.8, 3.10

3.13, 3.14a, 3.63
Feb 12 - Feb 16 Monday: 3.3
Wednesday: 3.3
Friday: 3.4
Canonical decomposition
3.28, 3.29
3.17, 3.22
3.14, 3.18, 3.34
Feb 19 - Feb 23 Monday: 3.5
Wednesday: 3.6, 3.8
Friday: 3.8

Ergodic chains
Feb 26 - Mar 2 4.1, review
Mar 5 - Mar 9 4.2 – 4.4
Mar 12 - Mar 16 spring break
Mar 19 - Mar 23 5.1, 5.2
Mar 26 - Mar 30 6.1, 6.2
Apr 2 - Apr 6 6.4, 6.5
Apr 9 - Apr 13 7.1, 7.2
Apr 16 - Apr 20 7.3, 7.4
Apr 23 - Apr 27 TBA
Apr 30 - May 4 review


We’ll devote the last week of the semester to a mini-symposium of short group presentations. Since the field of stochastic processes is rich with interesting examples and topics, more than we could cover in a single semester, each group of 2-3 students will choose a topic/application that we might otherwise not have time for in class. We’ll also plan to have a writing component to the project and some preliminary deliverables in the lead up to the final week. More details will be discussed in class later in the semester.

Getting help

Here’s a few ways to get help:

  • Office hours (listed at the top of this page). These are times I hold on my schedule to be available for students. I like to use these times as open drop in sessions where there may be multiple students getting help simultaneously, but it’s expected that you come with specific questions about notes, homework problems, or the text that you have thought about already. If you need to schedule a time to meet with me one-on-one, either to discuss something private or because you can’t make my office hours, please send me an email with times that you are available to meet, and I will find a time that works for both of us.
  • Study groups. I strongly encourage you to form small groups and work together on problems. My advice though is to try problems by yourself before discussing them. We have several spaces on the fourth floor of Clapp where students can work together for extended periods of time.


  • Everyone is invited to join DataCamp, which provides an introductory R tutorial. It’s a convenient way to gain some familiarity with R, a useful tool for our course and beyond. Our textbook also provides a thorough tutorial of some R basics in the appendix.

  • Our textbook also has a useful collection of R scripts available here; contained there are all the R code snippets you’ll notice interspersed in the text.