The Key Formula

Here's the formula you'll use in every probability DP problem: $dp[state] = \sum_{next} P(state \to next) \cdot dp[next]$ In words: the value at dp[pos] equals the sum over all possible next states of (probability of going there) times (value from there).

This is just expected value applied to DP. For probability problems, $dp[state]$ is the probability of some event. For expected value problems, $dp[state]$ is the expected outcome. The formula stays the same.