git clone 'https://github.com/damballa/parkour.git'

(ql:quickload :damballa.parkour)


Build Status

Hadoop MapReduce in idiomatic Clojure. Parkour takes your Clojure code’s functional gymnastics and sends it free-running across the urban environment of your Hadoop cluster.

Parkour is a Clojure library for writing distributed programs in the MapReduce pattern which run on the Hadoop MapReduce platform. Parkour does its best to avoid being yet another “framework” – if you know Hadoop, and you know Clojure, then you’re most of the way to knowing Parkour. By combining functional programming, direct access to Hadoop features, and interactive iteration on live data, Parkour supports rapid development of highly efficient Hadoop MapReduce applications.


Parkour is available on Clojars. Add this :dependency to your Leiningen project.clj:

[com.damballa/parkour "0.6.3"]


The Parkour introduction contains an overview of the key concepts, but here is the classic “word count” example, in Parkour:

(defn word-count-m
  (->> coll
       (r/mapcat #(str/split % #"\s+"))
       (r/map #(-> [% 1]))))

(defn word-count
  [conf lines]
  (-> (pg/input lines)
      (pg/map #'word-count-m)
      (pg/partition [Text LongWritable])
      (pg/combine #'ptb/keyvalgroups-r #'+)
      (pg/output (seqf/dsink [Text LongWritable]))
      (pg/fexecute conf `word-count)))


Parkour’s documentation is divided into a number of separate sections:


There is a Parkour mailing list hosted by Librelist for announcements and discussion. To join the mailing list, just email parkour@librelist.org; your first message to that address will subscribe you without being posted. Please report issues on the GitHub issue tracker. And of course, pull requests welcome!


Copyright © 2013-2015 Marshall Bockrath-Vandegrift & Damballa, Inc.

Distributed under the Apache License, Version 2.0.