arrow_back POC: How to slice, dice & search billions of users events in seconds (from scratch)
The many ways of parallel computing with Julia
Submitted by Viral B. Shah (@viralbshah) on Sunday, 14 June 2015
Introduce Julia for those who haven’t heard about it, and focus on parallel computing with Julia. Do some demos with hundreds of processors. The audience will get a feel for parallel computing with Julia and is strictly advised to “Try it at home.”
Julia is a high performance dynamic language, primarily designed for technical computing, but increasingly seeing applications in a wide variety of domains.
This talk will provide an overview of parallel computing in Julia. It will start with an introduction to using built-in Julia primitives for parallel processing, such as pmap, @parallel, remotecall, spawn, fetch, etc. Based on this low-level primitives, shared arrays and distributed arrays have been built. We will try some Parallel Linear Algebra using packages such as ScaLapack along with some MPI programming. We will also look at the possibilities of data processing with data loaded from the Hadoop file system (HDFS) and/or S3. We will also preview the upcoming multi-threading capabilities in Julia.
Not only will we show how the compute can be efficiently carried out in Julia, but using tools such as Escher, we will also show how the findings can be beautifully packaged for presentation.