**Instructor**: Tim Chumley

**Office**: Clapp 404b

**Phone**: 538-2299

**e-mail**: tchumley

**Office Hours**: tentatively Mondays 10:00–11:00, Tuesdays 4:00–5:00, Wednesdays 3:00–4:00, Thursdays 9:00–10:00, and by appointment

Textbook: *Introduction to Stochastic Processes with R* by Robert P. Dobrow, ISBN: 9781118740651;

on library reserve under QC20.7.S8 D63 2016;

available as a free e-text

Announcements will be posted here throughout the semester.

**April 29**: Here are office hours for the coming week:

- Monday, April 30: 10:00 – 11:00
- Tuesday, May 1: 11:00 – 12:00, 4:00 – 5:00
- Wednesday, May 2: 2:00 – 3:00

and by appointment. Please note that I’ll be away Thursday and Friday, but I’ll be able to reply to email, albeit with some delay.

**April 28**: The final exam was handed out in class and is posted here.

**April 4**: The next homework due date was pushed back to April 11 to give you some time to study for Friday’s exam. Also, we’ll cancel class on April 13 for Senior Symposium.

**Feb 3**: Some clarifying remarks about rewrites have been posted below in the Homework section.

**Feb 2**: Here is another modeling competition opportunity taking place at Amherst College April 21 called SCUDEM (in addition to the one I wrote to you about taking place next week).

Check the syllabus for all the important class policies (grades, attendance, etc.).

There will be weekly homework assignments throughout the semester to be turned in, as well as suggested problems to be considered as additional practice.

**Written assignments**. A selection of problems taken from the textbook or other sources will be assigned to be due**Fridays at 7pm**.- These assignments are given so that you can practice writing mathematics and receive feedback on your progress.
- If you would like feedback on a particular steps in a problem, then you can indicate this on your assignment.

- Each problem will be given a score out of 5. A score of 1 will be given to indicate that to stay on track you should get help from your instructor.
- Please read these guidelines for writing up your homework solutions.

**Submitting to Gradescope**. We’ll be using an online submission tool called Gradescope to collect, grade, and return homework. Here are some notes:- Begin by signing up as a student on the Gradescope home page with
**Entry Code MWY6EK**. - I’ve posted a short tutorial to help with Gradescope if you haven’t used it before.
- Gradescope has also made a guide to help with scanning and submitting.
- More information is available in this general help guide.
- Please let me know of any issues that come up!

- Begin by signing up as a student on the Gradescope home page with
**Suggested problems**. The book has a good variety of problems, both in terms of content and difficulty level. As you study throughout the semester, you might consider browsing and trying the problems at the end of the chapter for extra practice. I’ll always be happy to help over email or in office hours if you’re stuck.**Rewrites**. You’ll be allowed to redo some missed homework problems this semester. Here are some details:- You can rewrite one problem per problem set and earn back full points.
- You should include with your rewrite an explanation that makes it easy to see what has changed in your answer (ie. what went wrong originally and what you’ve done to fix it).
- You should turn in your rewrite in class on the Friday after the original due date. (Generally graded work will be released on Gradescope two or three days after the due date.)

Assignment | Due |
---|---|

Problem set 0 (solutions) | Jan 29 |

Problem set 1 (solutions) | Feb 2 |

Problem set 2 (solutions) | Feb 9 |

Problem set 3 (solutions) | Feb 16 |

Problem set 4 (solutions) | Feb 23 |

Problem set 5 (solutions) | Mar 9 |

Problem set 6 (solutions) | Mar 23 |

Problem set 7 (solutions) | Mar 30 |

Problem set 8 (solutions) | Apr 11 |

1-2 page project topic summary | Apr 16 |

Problem set 9 (solutions) | Apr 20 |

Problem set 10 (solutions) | Apr 27 |

There will be three exams. The dates for the two mid-terms are subject to change slightly:

Exam | Date | Time and Location | Material | Study material | Solutions |
---|---|---|---|---|---|

Exam 1 | Mar 2 | in class, take-home | 1.1 – 3.6 | study guide (solutions) | in-class, take-home |

Exam 2 | Apr 6 | in class, take-home | 3.8, 4.1 – 4.4, 5.1, 5.2 | study guide (solutions) | in-class, take-home |

Final | May 3 - 7 | take-home | Ch. 6, 7 |

Our plan is to cover the textbook chapters 1 – 3, and parts of 4 – 7. Below is a rough outline of what is to be covered week by week through the semester. Please check back regularly for precise details on what is covered, as well as postings of in-class worksheets and other activities. Also note that the days of the week under Sections in the table below provide links to class notes.

Week | Sections | In-class activities | Problems |
---|---|---|---|

Jan 22 - Jan 26 | Wednesday 1.1, 1.2, 2.1 Friday: 2.2 |
First day info, State spaces, index sets Markov transition matrix, graphs |
1.1, 1.2, 1.3, 1.4, 1.9, 1.35 2.8, 2.9, 2.10 |

Jan 29 - Feb 2 | Monday: 2.3, 2.4 Wednesday: 2.3 Friday: 2.5 |
n-step distribution (solutions) Simulation, RMarkdown More simulation |
2.1, 2.2, 2.6, 2.25, 2.26 2.4, 2.5, 2.11, 2.17 2.23, 2.24, 2.27 |

Feb 5 - Feb 9 | Monday: 3.1, 3.2 Wednesday: snow day Friday: 3.2, 3.3 |
Stationary distribution Communication classes |
2.18, 3.1, 3.2, 3.5, 3.8, 3.10 3.13, 3.14a, 3.63 |

Feb 12 - Feb 16 | Monday: 3.3 Wednesday: 3.3 Friday: 3.4 |
Canonical decomposition Excursions |
3.28, 3.29 3.17, 3.22 3.14, 3.18, 3.34 |

Feb 19 - Feb 23 | Monday: 3.5 Wednesday: 3.6, 3.8 Friday: 3.8 |
Ergodic chains |
3.50, 3.51 3.52 |

Feb 26 - Mar 2 | Monday: 3.8 Wednesday: review Friday: exam |
Absorbing chains | 3.54 |

Mar 5 - Mar 9 | Monday: 5.1 Wednesday: snow day Friday: 5.2 |
TSP worksheet, TSP tutorial, RMarkdown Crytography, R demo |
5.1, 5.3 |

Mar 12 - Mar 16 | spring break | ||

Mar 19 - Mar 23 | Monday: 4.1, 4.2 Wednesday: 4.3 Friday: 4.4 |
Extinction in subcritical case Probability generating functions |
4.1, 4.2, 4.3, 4.4, 4.9 4.7, 4.12, 4.13 4.14, 4.16, 4.19 |

Mar 26 - Mar 30 | Monday: 6.1 Wednesday: 6.2 Friday: 6.2 |
Poisson process introduction Arrival times |
6.1, 6.3, 6.4, 6.6 6.2, 6.7 |

Apr 2 - Apr 6 | Monday: 6.4 Wednesday: review Friday: exam |
Thinning and superposition Review MCMC solution |
6.8, 6.13, 6.15, 6.16 |

Apr 9 - Apr 13 | Monday: 7.1, 7.2 Wednesday: 7.3 Friday: no class |
Transition rates, holding times, embedded chain | 7.1, 7.2 7.4, 7.5, 7.6 |

Apr 16 - Apr 20 | Monday: 7.3 Wednesday: 7.4 Friday: 7.6 |
Infinitesimal generator Limiting and stationary distributions |
7.7, 7.14, 7.23 7.11, 7.13 7.19, 7.25, 7.26 |

Apr 23 - Apr 27 | Monday: 7.6 Wednesday: presentations Friday: presentations |
Queueing theory (solution) | 7.20, 7.22, 7.24, 7.30, 7.33 |

Apr 30 - May 4 | Monday: presentations |

We’ll devote the last week of the semester to a mini-symposium of short group presentations. Since the field of stochastic processes is rich with interesting examples and topics, more than we could cover in a single semester, each group of 2-3 students will choose a topic/application that we might otherwise not have time for in class.

We’ll also plan to have a writing component to the project and some preliminary deliverables in the lead up to the final week. More details will be discussed in class later in the semester.

- Presentation information and suggested topics
- Upcoming dates:
**Friday, March 30**: select two potential topics and turn in 1-2 paragraphs about each. Give a synopsis of each topic and why you might like to present on it.**Monday, April 16**: submit 1-2 page (single spaced, written or typed) summary of the topic (one submission per group).**week of Monday, April 16**: meet with me with a draft of slides and/or plan for the talk.**April 25, 27, 30**: presentation days in class.**Monday, May 7**: submit 3-5 page report on the topic.

- Presentation rubric

Group members | Topic | Presentation day | Slides |
---|---|---|---|

Nicole, Sally, Younghoo | Moran process | Wednesday, April 25 | |

Jessica, Sunan, Weijia | Optimal stopping | Wednesday, April 25 | |

Nhu, Uyen | Martingales, betting strategies | Friday, April 27 | |

Emma, Juliana, Shirley | Card shuffling | Friday, April 27 | |

Faryal, Mirha, Momal | Birth and death processes | Monday, April 30 | |

Ranjini, Rebecca, Surabhi | Hidden Markov model | Monday, April 30 | |

Puyang | An application of MCMC to ecology | Monday, April 30 |

LaTeX is a typesetting system that makes it easy to type math. Here are some resources you might find useful as you type reports or make presentation slides.

MacTeX/TeXLive. An installation package for getting LaTeX running on your computer. MacTeX is for macOS and TeXLive is for other operating systems.

LaTeXiT. This small program for macOS lets you quickly make a mathematical expression using LaTeX commands and export it as an image or pdf file that can be inserted into a PowerPoint presentation. This is an alternative to making a full LaTeX document; it doesn’t require much time investment aside from first installing MacTeX or TeXLive. (LaTeXiT comes with a full MacTeX/TeXLive installation.)

quick reference. This file has a list of common commands.

LaTeX file template. This is a simple file that lets you get started making a full LaTeX document. This is the pdf file generated by the template.

Beamer template. If you’d like to use LaTeX to make presentation slides, there is a package called Beamer that is an alternative to systems like PowerPoint. Here is a template for a Beamer presentation. This is the pdf file generated by the template.

Here’s a few ways to get help:

**Office hours**(listed at the top of this page). These are times I hold on my schedule to be available for students. I like to use these times as open drop in sessions where there may be multiple students getting help simultaneously, but it’s expected that you come with specific questions about notes, homework problems, or the text that you have thought about already. If you need to schedule a time to meet with me one-on-one, either to discuss something private or because you can’t make my office hours, please send me an email with times that you are available to meet, and I will find a time that works for both of us.**Study groups**. I strongly encourage you to form small groups and work together on problems. My advice though is to try problems by yourself before discussing them. We have several spaces on the fourth floor of Clapp where students can work together for extended periods of time.

Everyone is invited to join DataCamp, which provides an introductory R tutorial. It’s a convenient way to gain some familiarity with R, a useful tool for our course and beyond. Our textbook also provides a thorough tutorial of some R basics in the appendix.

Our textbook also has a useful collection of R scripts available here; contained there are all the R code snippets you’ll notice interspersed in the text.