If you need an accessible version of this item please contact JSTOR User Support

The Optimal Control of Partially Observable Markov Processes Over the Infinite Horizon: Discounted Costs

Edward J. Sondik
Operations Research
Vol. 26, No. 2 (Mar. - Apr., 1978), pp. 282-304
Published by: INFORMS
Stable URL: http://www.jstor.org/stable/169635
Page Count: 23
  • Download PDF
  • Cite this Item

You are not currently logged in.

Access your personal account or get JSTOR access through your library or other institution:

login

Log in to your personal account or through your institution.

If you need an accessible version of this item please contact JSTOR User Support
The Optimal Control of Partially Observable Markov Processes Over the Infinite Horizon: Discounted Costs
Preview not available

Abstract

This paper treats the discounted cost, optimal control problem for Markov processes with incomplete state information. The optimization approach for these partially observable Markov processes is a generalization of the well-known policy iteration technique for finding optimal stationary policies for completely observable Markov processes. The state space for the problem is the space of state occupancy probability distributions (the unit simplex). The development of the algorithm introduces several new ideas, including the class of finitely transient policies, which are shown to possess piecewise linear cost functions. The paper develops easily implemented approximations to stationary policies based on these finitely transient policies and shows that the concave hull of an approximation can be included in the well-known Howard policy improvement algorithm with subsequent convergence. The paper closes with a detailed example illustrating the application of the algorithm to the two-state partially observable Markov process.

Page Thumbnails

  • Thumbnail: Page 
282
    282
  • Thumbnail: Page 
283
    283
  • Thumbnail: Page 
284
    284
  • Thumbnail: Page 
285
    285
  • Thumbnail: Page 
286
    286
  • Thumbnail: Page 
287
    287
  • Thumbnail: Page 
288
    288
  • Thumbnail: Page 
289
    289
  • Thumbnail: Page 
290
    290
  • Thumbnail: Page 
291
    291
  • Thumbnail: Page 
292
    292
  • Thumbnail: Page 
293
    293
  • Thumbnail: Page 
294
    294
  • Thumbnail: Page 
295
    295
  • Thumbnail: Page 
296
    296
  • Thumbnail: Page 
297
    297
  • Thumbnail: Page 
298
    298
  • Thumbnail: Page 
299
    299
  • Thumbnail: Page 
300
    300
  • Thumbnail: Page 
301
    301
  • Thumbnail: Page 
302
    302
  • Thumbnail: Page 
303
    303
  • Thumbnail: Page 
304
    304