Easy ruby paralellism with the ‘parallel’ gem

Concurrency is a hard topic in programming with many advanced concepts. We love libraries and solutions which help us during our everyday simple tasks. The ‘parallel’ gem is one of these – well, gems -. You can find it at github. The author’s own description:

Run any code in parallel Processes(> use all CPUs) or Threads(> speedup blocking operations).
Best suited for map-reduce or e.g. parallel downloads/uploads.

Let’s see some simple example of its usage.

Sample code:

require 'rubygems'
require 'rubygems'
require 'parallel'
require 'net/http'
require 'benchmark'
 
 
ping_hosts = lambda {|host|
  10.times { Net::HTTP.new(host).head('/') }
}
 
puts "#{Parallel.processor_count} procesor(s)"
hosts = ['www.ruby-lang.org', 'programmingzen.com', 'blog.headius.com', 'chadfowler.com', 'drnicwilliams.com', 'igvita.com']
 
Benchmark.bm do |x|
  x.report('normal') { 
    hosts.each &ping_hosts
  }
  x.report('thread') { 
    Parallel.each(hosts, :in_threads => Parallel.processor_count, &ping_hosts)
  }
  x.report('fork') { 
    Parallel.each(hosts, &ping_hosts)
  }
end

This code basically gets the root document of the http servers listening on port 80 on the list of domains, 10 times.
The first run in a simple sequential run, we call this ‘normal‘ in the benchmark.
Then we instruct the parallel library to use as many threads as our CPU core count is, this is the run called ‘thread‘
Finally we use forking (the default operation of the library), naturally this run is called ‘fork‘.

The results on my machine are:

8 procesor(s)
      user     system      total        real
normal  0.070000   0.080000   0.150000 ( 33.270311)
thread  0.040000   0.030000   0.070000 (  7.227803)
fork  0.010000   0.030000   0.330000 ( 10.276050)

I think this shows how easy and straightforward it is to use this library for everyday parallel tasks. Of course we haven’t covered advanced contexts as locks, mutexes, shared data, etc. – this library is not made for that.

Concurrency is a hard topic in programming with many advanced concepts. We love libraries and solutions which help us during our everyday simple tasks. The 'parallel' gem is one of these - well, gems -. You can find it at github. The author's own description: Run any code in parallel Processes(> use all CPUs) or Threads(> speedup blocking operations). Best suited for map-reduce or e.g. parallel downloads/uploads. Let's see some simple example of its usage.   Sample code: require 'rubygems' require 'rubygems' require 'parallel' require 'net/http' require 'benchmark' ping_hosts = lambda {|host| 10.times { Net::HTTP.new(host).head('/') } } puts "#{Parallel.processor_count} procesor(s)" hosts = ['www.ruby-lang.org', 'programmingzen.com', 'blog.headius.com', 'chadfowler.com', 'drnicwilliams.com', 'igvita.com'] Benchmark.bm do |x| x.report('normal') { hosts.each &ping_hosts } x.report('thread') { Parallel.each(hosts, :in_threads => Parallel.processor_count, &ping_hosts) } x.report('fork') { Parallel.each(hosts, &ping_hosts) } end This code basically gets the root document of the http servers listening on port 80 on the list of domains, 10 times. The first run in a simple sequential run, we call this 'normal' in the benchmark. Then we instruct the parallel library to use as many threads as our CPU core count is, this is the run called 'thread' Finally we use forking (the default operation of the library), naturally this run is called 'fork'. The results on my machine are: 8 procesor(s) user system total real normal 0.070000 0.080000 0.150000 ( 33.270311) thread 0.040000 0.030000 0.070000 ( 7.227803) fork 0.010000 0.030000 0.330000 ( 10.276050) I think this shows how easy and straightforward it is to use this library for everyday parallel tasks. Of course we haven't covered advanced contexts as locks, mutexes, shared data, etc. - this library is not made for that.