Access

You are not currently logged in.

Access your personal account or get JSTOR access through your library or other institution:

login

Log in to your personal account or through your institution.

Compressing Massive Geophysical Datasets Using Vector Quantization

Amy Braverman
Journal of Computational and Graphical Statistics
Vol. 11, No. 1 (Mar., 2002), pp. 44-62
Stable URL: http://www.jstor.org/stable/1391127
Page Count: 19
  • Download ($14.00)
  • Cite this Item
Compressing Massive Geophysical Datasets Using Vector Quantization
Preview not available

Abstract

This article presents a procedure for compressing massive geophysical datasets. A dataset is stratified geographically, and a penalized clustering algorithm applied to each stratum independently. The algorithm, called Monte Carlo extended ECVQ, is based on the entropy-constrained vector quantizer algorithm (ECVQ). ECVQ trades off error induced by compression against data reduction to produce a set of representative points, each of which stands for some number of input observations. Since the data are massive, a preliminary set of representatives is determined from a stratum sample, then the full stratum is clustered by assigning each observation to the nearest representative. After replacing the initial representatives by means of these clusters, the new representatives and their associated counts are a compressed version, or summary, of the original stratum data. With the initial set of representatives determined from a sample, the final summary is subject to sampling variation. A statistical model for the relationship between compressed and uncompressed data provides a framework for assessing this variability. Test data from the International Satellite Cloud Climatology Project are used to demonstrate the procedure.

Page Thumbnails

  • Thumbnail: Page 
44
    44
  • Thumbnail: Page 
45
    45
  • Thumbnail: Page 
46
    46
  • Thumbnail: Page 
47
    47
  • Thumbnail: Page 
48
    48
  • Thumbnail: Page 
49
    49
  • Thumbnail: Page 
50
    50
  • Thumbnail: Page 
51
    51
  • Thumbnail: Page 
52
    52
  • Thumbnail: Page 
53
    53
  • Thumbnail: Page 
54
    54
  • Thumbnail: Page 
55
    55
  • Thumbnail: Page 
56
    56
  • Thumbnail: Page 
57
    57
  • Thumbnail: Page 
58
    58
  • Thumbnail: Page 
59
    59
  • Thumbnail: Page 
60
    60
  • Thumbnail: Page 
61
    61
  • Thumbnail: Page 
62
    62