Use probability DP when: you need probability of an outcome after a sequence of random events. Signals: "probability of", "expected value", "random choice", "uniform distribution", "Monte Carlo" (but DP is exact).
State: typically or . Transitions: weighted by probabilities. The difference from counting DP: transitions multiply by probability instead of adding counts. But linearity of expectation often lets us sum contributions.