Abstract
Parallel programming, whether imperative or functional, has long focused on arrays as the central data type. Meanwhile, typed functional programming has explored a variety of data types, including lists and various forms of trees. Generic functional programming decomposes these data types into a small set of fundamental building blocks: sum, product, composition, and their associated identities. Definitions over these few fundamental type constructions then automatically assemble into algorithms for an infinite variety of data types—some familiar and some new. This paper presents generic functional formulations for two important and well-known classes of parallel algorithms: parallel scan (generalized prefix sum) and fast Fourier transform (FFT). Notably, arrays pla…
Abstract
Parallel programming, whether imperative or functional, has long focused on arrays as the central data type. Meanwhile, typed functional programming has explored a variety of data types, including lists and various forms of trees. Generic functional programming decomposes these data types into a small set of fundamental building blocks: sum, product, composition, and their associated identities. Definitions over these few fundamental type constructions then automatically assemble into algorithms for an infinite variety of data types—some familiar and some new. This paper presents generic functional formulations for two important and well-known classes of parallel algorithms: parallel scan (generalized prefix sum) and fast Fourier transform (FFT). Notably, arrays play no role in these formulations. Consequent benefits include a simpler and more compositional style, much use of common algebraic patterns and freedom from possibility of run-time indexing errors. The functional generic style also clearly reveals deep commonality among what otherwise appear to be quite different algorithms. Instantiating the generic formulations, two well-known algorithms for each of parallel scan and FFT naturally emerge, as well as two possibly new algorithms.
Formats available
You can view the full content in the following formats:
References
[1]
Roland Backhouse, Jeremy Gibbons, Ralf Hinze, and Johan Jeuring. Datatype-Generic Programming: International Spring School, Revised Lectures . Lecture Notes in Computer Science. Springer Berlin Heidelberg, April 2007.
[2]
Jost Berthold, Mischa Dieterle, Oleg Lobachev, and Rita Loogen. Parallel FFT with Eden skeletons . In International Conference on Parallel Computing Technologies, PaCT ’09, pages 73–83, 2009.
[3]
Richard Bird and Lambert Meertens. Nested Datatypes . In Mathematics of Program Construction, pages 52–67, 1998.
[4]
Per Bjesse, Koen Claessen, Mary Sheeran, and Satnam Singh. Lava: Hardware design in Haskell . In International Conference on Functional Programming, pages 174–184, 1998.
[5]
Guy E. Blelloch. Prefix sums and their applications . Technical Report CMU-CS-90-190, School of Computer Science, Carnegie Mellon University, November 1990.
[6]
Guy E. Blelloch. Programming parallel algorithms . Communications of the ACM, 39:85–97, 1996.
[7]
R. P. Brent and H. T. Kung. A regular layout for parallel adders . IEEE Transactions on Computers, 31(3), March 1982.
[8]
James W. Cooley and John W. Tukey. An algorithm for the machine calculation of complex Fourier series. Mathematics of Computation, 19:297–301, 1965.
[9]
Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms. The MIT Press, 3rd edition, 2009.
[10]
G.C. Danielson and C. Lanczos. Some improvements in practical Fourier analysis and their application to X-ray scattering from liquids. Journal of the Franklin Institute, 233(5):435–452, 1942.
[11]
Fer-Jan de Vries. A functional program for the Fast Fourier Transform . SIGPLAN Notices, pages 67–74, January 1988.
[12]
Richard A. Eisenberg, Dimitrios Vytiniotis, Simon L. Peyton Jones, and Stephanie Weirich. Closed type families with overlapping equations . In Principles of Programming Languages, pages 671–684, 2014.
[13]
Richard A. Eisenberg, Stephanie Weirich, and Hamidhasan G. Ahmed. Visible type application . In European Symposium on Programming Languages and Systems, pages 229–254, 2016.
[14]
Conal Elliott. Denotational design with type class morphisms . Technical Report 2009-01, LambdaPix, March 2009.
[15]
Conal Elliott. Compiling to categories . Proc. ACM Program. Lang., 1(ICFP), September 2017.
[16]
Matteo Frigo. A fast Fourier transform compiler . In PLDI, volume 34, pages 169–180. ACM, May 1999.
[17]
Jeremy Gibbons. Upwards and downwards accumulations on trees . In Mathematics of Program Construction, 1993.
[18]
Jeremy Gibbons. Generic downwards accumulations . Science of Computer Programming, 37(1-3):37–65, May 2000.
[19]
Sergei Gorlatch. Programming with divide-and-conquer skeletons: A case study of FFT. The Journal of Supercomputing, 1998.
[20]
Pieter H. Hartel and Willem G. Vree. Arrays in a lazy functional language — a case study: The fast Fourier transform . In 2nd Arrays, functional languages, and parallel systems (ATABLE), 1992.
[21]
Michael T. Heideman, Don H. Johnson, and C. Sidney Burrus. Gauss and the history of the fast Fourier transform. IEEE ASSP Magazine, 1(4):14–21, October 1984.
[22]
Ralf Hinze. Memo functions, polytypically! In 2nd Workshop on Generic Programming, pages 17–32, 2000.
[23]
Ralf Hinze. An algebra of scans . In International Conference on Mathematics of Program Construction, pages 186–210, 2004.
Computing10.1145/3609024.3609410(1-14)Online publication date: 30-Aug-2023
Sikorski AWrosz ILewandowski M(2022)A Generalized Parallel Prefix Sums Algorithm for Arbitrary Size ArraysParallel Processing and Applied Mathematics10.1007/978-3-031-30442-2_3(30-39)Online publication date: 11-Sep-2022 Show More Cited By