TheBrain is a small C++ library for artificial neural networks.
Currently I implemented a feedforward, a recurrent neural net and wrappers for GEM, which calculate audio signals out of video frames.
TheBrain consists of the following 2 objects:
pix_linNN (with a linear feedforward neural net) and pix_recNN (with a recurrent neural net).
pix_recNN/pix_linNN are thought as an instument/interface.
This instrument should be useful as a general experimental video interface to generate audio. You can train an artificial neural net with playing audio samples to specific video frames in real-time - so you are able to produce specific sounds to specific video frames and you can control the sound with making some movements, colors, ... (whatever) in front of the camera.
The main interest for me was not to train the net exactly to reproduce these samples, but to make experimental sounds, which are "between" all the trained samples.
pix_linNN has one neuron per audio sample: this neuron has three inputs (a RGB-signal), a weight vector for each of the inputs, a bias value and a linear output function.
pix_recNN uses a 2 layer recurrent neural net (which is much better for time-based information like video/music).
