Title: Nonstationary zero-sum Markov games with the probability criterion
Speaker: 郭先平（中山大学）
Time: 1.9 (Friday), 10:10-10:50
Venue: 活动中心331
Abstract:
This paper studies a two-person non-stationary stochastic game under a probability criterion, which focuses on the probability that the accumulated rewards of player 1 (i.e., the costs of player 2) exceed a prescribed threshold before the first passage into a target set. We first present two illustrative examples. The first one shows that the probability criterion breaks the implication from a nonzero-sum Nash equilibrium to a zero-sum saddle point. The second demonstrates that the non-stationary game can not be transformed into an equivalent and stationary one via the standard state augmentation. Because of the non-stationariness, we introduce the notion of the n-th value of game from time n onwards. Under a mild condition, we prove that the sequence of the n-th values is the unique solution of the system of Shapley equations for the probability criterion. From the system of Shapley equations, we establish the existence of the value and a saddle-point for the game, give an iteration algorithm for computing the approximation value and ϵ-saddle-points of the game, and provide convergence guarantees together with explicit error bounds. Finally, an energy management numerical example is presented to illustrate the theoretical results and the effectiveness of the proposed algorithm.
