Legal Issues  «Prev 

United States Copyright Office Website


What is copyright?


What does copyright protect?


Copyright Law

Generating Synthetic Examples

As computers become faster, the other way of putting in knowledge, which is by generating synthetic examples, begins to look better. Generating synthetic examples allows optimization to discover clever ways of using the multilayer network that we did not think of. In fact, we might never fully understand how the multilayer network does it. If we just want good solutions to a problem, generating synthetic examples may be appropriate. Using the idea of synthetic data, there is a brute force approach to handwritten digit recognition. Lenet5 uses knowledge about invariances to design the connectivity and the weight sharing and the pooling that achieves about 80 errors. Adding additional techniques including synthetic data, Ranzato 2008 was able to get the results down to about 40 errors.
A group in Switzerland led by Juergen Schmidhuber implemented this on a large scale with injecting knowledge by putting in synthetic data. They worked on creating instructive synthetic data. For every real training case, they transformed it to make more training examples. They then trained a large neural net with many units per layer using many layers on a graphic processor unit. The graphics processor unit gave them a factor of thirteen computation, and because of all the synthetic data they put in, it did not overfit. If they just use a large neural net with a GPU, it would have been a disaster that would have over fitted terribly, that would have performed fine on the training data and performed terribly on the test data.