Big Data in Radio Astronomy: Scientific Data Processing for Advanced Radio Telescopes provides the latest research developments in big data methods and techniques for radio astronomy. Providing examples from such projects as the Square Kilometer Array (SKA), the world's largest radio telescope that generates over an Exabyte of data every day, the book offers solutions for coping with the challenges and opportunities presented by the exponential growth of astronomical data. Presenting state-of-the-art results and research, this book is a timely reference for both practitioners and researchers working in radio astronomy, as well as students looking for a basic understanding of big data in astronomy.
- Bridges the gap between radio astronomy and computer science
- Includes coverage of the observation lifecycle as well as data collection, processing and analysis
- Presents state-of-the-art research and techniques in big data related to radio astronomy
- Utilizes real-world examples, such as Square Kilometer Array (SKA) and Five-hundred-meter Aperture Spherical radio Telescope (FAST)
Part A: Fundamentals and Challenges 1. Fundamentals of Big Data in Radio Astronomy 2. Challenges of Big Data in Radio Astronomy
Part B: Signal Pre-processing 3. Pre-processing Pipeline 4. Pipeline Implementation
Part C: Data Recording 5. Data Reduction and Re-distribution 6. Correlation 7. Recording Pipeline
Part D: Data Calibration and Cleaning 8. Off-line and Online Calibration 9. Calibration Quality Monitoring 10. Antenna Quality Monitoring 11. RFI Detection and Mitigation
Part F: Imaging and Non-Imaging 12. Classic Imaging Pipelines 13. Classic Non-Imaging Pipelines 14. Limitations of Classic Pipelines to Process Astronomical Big Data
Part G: Scale-out Processing 15. Heterogeneous Computing Platform for Accelerating 16. Imaging Algorithm Optimization for Scale-out Processing 17. Applications of Artificial Intelligence in Non-Imaging Algorithms for Scale-out Processing 18. High Performance Computing for Astronomical Big Data
Part H: Evolving Digital Infrastructures over an Extended Telescope Lifetime 19. Operational Concepts 20. Processing and Data Transport 21. Storage, Data Archiving and Dissemination 22. Non-conventional solutions
Linghe Kong is currently a Research Professor in Department of Computer Science and Engineering at Shanghai Jiao Tong University and an engineer in the scientific data processing group in SKA China. Before that, he was a postdoctoral researcher at Columbia University and McGill University. He received his Ph.D. degree from Shanghai Jiao Tong University, China, his Masters degree from TELECOM SudParis, France, and his B. E. degree from Xidian University, China. His research interests include big data, Internet of things, and mobile computing systems. He has published more than 60 papers in refereed journals and conferences, such as ACM MobiCom, IEEE INFOCOM, IEEE RTSS, IEEE ICDCS, IEEE TMC, and IEEE TPDS. He serves on the editorial boards of several journals including Springer Telecommunication Systems and KSII Transactions on Internet and Information Systems. He organized several special issues such as in IEEE Communications Magazine and in the Computer Journal. He is a senior member of IEEE.
Tian Huang is Research Associate of the Astrophysics Group, Cavendish Lab, University of Cambridge. He takes part in multiple radio telescope array projects and mainly focuses on data preprocessing and quality metrics. In March 2016, he graduated from the School of Microelectronics at Shanghai Jiao Tong University, where he completed his PhD thesis. His main research interest is Data Mining for time series, including time series big data indexing, anomaly detecting, and computer architecture for time series data mining and statistical models for time series data. He has published 9 SCI journal and 18 EI conference papers. He has rich experience on software and hardware co-designing.
Yongxin Zhu is a full Professor at Shanghai Advanced Research Institute, Chinese Academy of Sciences (CAS). He is also an Adjunct Professor with the School of Microelectronics at the Shanghai Jiao Tong University (SJTU). He is currently the technical leader of Chinese Consortium of Science Data Processor (SDP) for Square Kilometre Array Telescope. He has published over 130 English journal and conference papers, 40 Chinese journal papers and 20 China patent approvals in the areas of computer architecture, embedded systems, and big data processing. With around 1,000 citations of these works in recent years, he has received recognition in China and Asia with IEEE best paper award, Shanghai innovation award, SJTU Annual Outstanding Teacher Award and Bilingual Teaching Award. To date, he has received around 20 million RMB in grants from various funding agencies and industrial partners in China. Prior to his tenure with CAS and SJTU, he worked as a research fellow with the National University of Singapore in 2002-2005, a senior consultant with S1 Incorporation (inventor of the 1st Internet banking in the world) in 1999-2002 and a teaching assistant with the Department of Computer Science and Engineering, SJTU in 1994-1995. He is a guest editor of Journal of Systems Architecture, senior member of IEEE and China Computer Federation (CCF). He has been also a Visiting Professor with National University of Singapore since 2013.
Chris Broekema is a research staff member at the Netherlands Institute for Radio Astronomy (ASTRON), focusing on the high-performance computing challenges faced by modern radio telescope. He has designed, built, procured, operated, and worked on high-performance computing systems for the LOFAR telescope, including some of the fastest supercomputers in the world at the time, since 2004. More recently he has shifted focus to the Square Kilometre Array Science Data Processor, where he is responsible for the hardware platform design. Apart from the direct work on existing and future radio telescopes, Broekema has published several papers focusing on aspects that impact the suitability and usability of computer- and data-transport systems for radio astronomy applications.
Shenghua Yu is an Associate Professor at National Astronomical Observatories, Chinese Academy of Sciences (CAS). He received his PhD degree in astrophysics from Queen's University of Belfast in 2012, and worked as a post-doc fellow at University of Western Australia in 2014-2015. His main research interests include gravitational wave astrophysics and astronomy, double compact objects, radio emission from ultracool dwarfs and radiation mechanisms. He has published ~13 SCI journal papers, 3 Chinese journal and conference papers, 2 China patent approvals, and 3 China computer software copyrights in the research areas.