Access

You are not currently logged in.

Access your personal account or get JSTOR access through your library or other institution:

login

Log in to your personal account or through your institution.

If You Use a Screen Reader

This content is available through Read Online (Free) program, which relies on page scans. Since scans are not currently available to screen readers, please contact JSTOR User Support for access. We'll provide a PDF copy for your screen reader.

Controlled Markov Chains

Harry Kesten and Frank Spitzer
The Annals of Probability
Vol. 3, No. 1 (Feb., 1975), pp. 32-40
Stable URL: http://www.jstor.org/stable/2959261
Page Count: 9
  • Read Online (Free)
  • Download ($19.00)
  • Subscribe ($19.50)
  • Cite this Item
Since scans are not currently available to screen readers, please contact JSTOR User Support for access. We'll provide a PDF copy for your screen reader.
Controlled Markov Chains
Preview not available

Abstract

We propose a control problem in which we minimize the expected hitting time of a fixed state in an arbitrary Markov chains with countable state space. A Markovian optimal strategy exists in all cases, and the value of this strategy is the unique solution of a nonlinear equation involving the transition function of the Markov chain.

Page Thumbnails

  • Thumbnail: Page 
32
    32
  • Thumbnail: Page 
33
    33
  • Thumbnail: Page 
34
    34
  • Thumbnail: Page 
35
    35
  • Thumbnail: Page 
36
    36
  • Thumbnail: Page 
37
    37
  • Thumbnail: Page 
38
    38
  • Thumbnail: Page 
39
    39
  • Thumbnail: Page 
40
    40