Your PDF has successfully downloaded.

You may be interested in finding more content on these topics:

Access

You are not currently logged in.

Access JSTOR through your library or other institution:

login

Log in through your institution.

Journal Article

“Superluminal” FITS File Processing on Multiprocessors: Zero Time Endian Conversion Technique

Satoshi Eguchi
Publications of the Astronomical Society of the Pacific
Vol. 125, No. 927 (May 2013), pp. 565-579
DOI: 10.1086/671105
Stable URL: http://www.jstor.org/stable/10.1086/671105
Page Count: 15
Were these topics helpful?
See somethings inaccurate? Let us know!

Select the topics that are inaccurate.

Cancel
  • Download PDF
  • Add to My Lists
  • Cite this Item
We're having trouble loading this content. Download PDF instead.

Abstract

ABSTRACT.The FITS is the standard file format in astronomy, and it has been extended to meet the astronomical needs of the day. However, astronomical datasets have been inflating year by year. In the case of the ALMA telescope, a ∼TB∼TB-scale four-dimensional data cube may be produced for one target. Considering that typical Internet bandwidth is tens of MB/sMB/s at most, the original data cubes in FITS format are hosted on a VO server, and the region which a user is interested in should be cut out and transferred to the user (Eguchi et al. 2012). The system will equip a very high-speed disk array to process a TB-scale data cube in 10 s, and disk I/O speed, endian conversion, and data processing speeds will be comparable. Hence, reducing the endian conversion time is one of issues to solve in our system. In this article, I introduce a technique named “just-in-time endian conversion”, which delays the endian conversion for each pixel just before it is really needed, to sweep out the endian conversion time; by applying this method, the FITS processing speed increases 20% for single threading and 40% for multi-threading compared to CFITSIO. The speedup tightly relates to modern CPU architecture to improve the efficiency of instruction pipelines due to break of “causality”, a programmed instruction code sequence.

Notes and References

This item contains 10 references.

REFERENCES
  • ['Amdahl, G. M. 1967, AFIPS ‘67 (Spring), 483']
  • ['Eguchi, S., Kawasaki, W., Shirasaki, Y., et al. 2012, preprint (arXiv: 1211.3790)']
  • ['Greisen, E. W., & Harten, R. H. 1981, ApJS, 44, 371']
  • ['Grosbol, P., Harten, R. H., Greisen, E. W., & Wells, D. C. 1988, ApJS, 73, 359']
  • ['Intel Corporation 2012, Intel 64 and IA-32 Architectures Software Developer’s Manuals, Volume\xa02A, 3–78']
  • ['Lucas, R., Richer, J., Shepherd, D., Testi, L., Wright, M., & Wilson, C. 2004, ALMA Memo, 501']
  • ['Matsumoto, M., & Nishimura, T. 1998, ACM Trans. Model. Comp. Simul., 8, 3']
  • ['Pence, W. D. 2010, Astrophysics Source Code Library, 10001']
  • ['Rohland, C. 2001, tmpfs Documentation, http://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt']
  • ['Wells, D. C., Greisen, E. W., & Harten, R. H. 1981, ApJS, 44, 363']