japonophile.synaptic

https://github.com/japonophile/synaptic.git

git clone 'https://github.com/japonophile/synaptic.git'

(ql:quickload :japonophile.synaptic)
91

synaptic

Synaptic is a Neural Networks library written in Clojure.

It is intendend to be used for experimenting with various neural network architectures and learning algorithms, as well as to solve real-life problems.

It is largely inspired by Geoffrey Hinton's class “Neural Networks for Machine Learning” available online on Coursera.

Features

Synaptic allows to create multilayer feedforward networks and train them using various algorithms such as: - perceptron learning rule - backpropagation - L-BFGS (approximation of Newton's method) - R-prop - RMSprop - L1 & L2 regularization - convolution / pooling layers

It also allows to play with various training parameters like learning rate and momentum, and supports adaptive learning rate and variants of the momentum method (Nesterov's momentum).

Usage

To use Synaptic, first add this to your project.clj:

Clojars Project

You can then experiment in the REPL.

First, make sure you have a directory called data in your project, where you copy the training set (you can use the mnist10k training set provided in Synaptic git repo).

$ ls data/
data/trainingset.mnist10k

Next, open your REPL, load the training set and create a neural net, as follows:

(require '[synaptic.core :refer :all])

(def trset (load-training-set "mnist10k"))   ; load MNIST training set
(def net (mlp [784 100 10]             ; net with 784 inputs, 100 hidden units
              [:sigmoid :softmax]      ; and 10 (softmax) outputs
              (training :backprop)))   ; to be trained with backpropagation

(train net trset 10)              ; train the network for 10 epochs (returns a future)
@*1                               ; (deref to wait for the future to complete)

(-> @net :training :stats)        ; show training statistics

Note: you may want to do export JVM_OPTS="-Xmx1024m" before starting your REPL to make sure you have enough memory to load the training set.

Synaptic github repository includes a subset of the MNIST handwritten digit training data available on Yann LeCun's website, so you can experiment.

But you can also easily create your own training set:

(def trset (training-set samples labels))

There are many options you can specify to customize the training algorithm to your taste.

Examples

(require '[synaptic.core :refer :all])

(def trset (load-training-set "mnist10k"))
(def net (mlp [784 10] :binary-threshold (training :perceptron)))
(def net (mlp [784 100 10] [:sigmoid :softmax]
              (training :backprop {:learning-rate {:epsilon 0.001}})))
(def net (neural-net [{:type :input :fieldsize [28 28]}
                      {:type :convolution :act-fn :hyperbolic-tangent
                       :feature-map {:size [9 9] :k 3}
                       :pool {:kind :max :size [2 2]}}
                      {:type :fully-connected :act-fn :sigmoid :n 10}]
                     (training :backprop {:learning-rate {:epsilon 0.01}})))
(def net (mlp [784 100 10] [:sigmoid :softmax]
              (training :backprop
              {:learning-rate {:epsilon 0.01 :adaptive true
                               :ming 0.01 :maxg 10.0}})))
(def net (mlp [784 100 10] [:sigmoid :softmax] (training :lbfgs)))

How to monitor training

As mentioned above, the train function returns a future which will be completed when the training stops. You can monitor the training by adding a watch on the net (it is an atom) so you can see when its weights change or when training state or training stats are updated (typically every epoch).

To Do

Feedback

This library is still in its infancy. Your feedback on how it can be improved, and contributions are welcome.

License

Copyright © 2014-2015 Antoine Choppin

Distributed under the Eclipse Public License, the same as Clojure.