Access

You are not currently logged in.

Access your personal account or get JSTOR access through your library or other institution:

login

Log in to your personal account or through your institution.

If you need an accessible version of this item please contact JSTOR User Support

Entropy and the Consistent Estimation of Joint Distributions

Katalin Marton and Paul C. Shields
The Annals of Probability
Vol. 22, No. 2 (Apr., 1994), pp. 960-977
Stable URL: http://www.jstor.org/stable/2244900
Page Count: 18
  • Read Online (Free)
  • Download ($19.00)
  • Subscribe ($19.50)
  • Cite this Item
If you need an accessible version of this item please contact JSTOR User Support
Entropy and the Consistent Estimation of Joint Distributions
Preview not available

Abstract

The kth-order joint distribution for an ergodic finite-alphabet process can be estimated from a sample path of length n by sliding a window of length k along the sample path and counting frequencies of k-blocks. In this paper the problem of consistent estimation when k = k(n) grows as a function of n is addressed. It is shown that the variational distance between the true k(n)-block distribution and the empirical k(n)-block distribution goes to 0 almost surely for the class of weak Bernoulli processes, provided k(n) ≤ (log n)/(H + ε), where H is the entropy of the process. The weak Bernoulli class includes the i.i.d. processes, the aperiodic Markov chains and functions thereof and the aperiodic renewal processes. A similar result is also shown to hold for functions of irreducible Markov chains. This work sharpens prior results obtained for more general classes of processes by Ornstein and Weiss and by Ornstein and Shields, which used the d̄-distance rather than the variational distance.

Page Thumbnails

  • Thumbnail: Page 
960
    960
  • Thumbnail: Page 
961
    961
  • Thumbnail: Page 
962
    962
  • Thumbnail: Page 
963
    963
  • Thumbnail: Page 
964
    964
  • Thumbnail: Page 
965
    965
  • Thumbnail: Page 
966
    966
  • Thumbnail: Page 
967
    967
  • Thumbnail: Page 
968
    968
  • Thumbnail: Page 
969
    969
  • Thumbnail: Page 
970
    970
  • Thumbnail: Page 
971
    971
  • Thumbnail: Page 
972
    972
  • Thumbnail: Page 
973
    973
  • Thumbnail: Page 
974
    974
  • Thumbnail: Page 
975
    975
  • Thumbnail: Page 
976
    976
  • Thumbnail: Page 
977
    977