CSCI 335 - Artificial Intelligence

Spring 2020

Programming Project #4: Handwriting Recognition with Multilayer Perceptrons

Overview

You will implement the backpropagation algorithm for training a multi-layer perceptron. You will experiment with different configurations of the neural network to find the best results for a handwriting recognition problem.

Setup

Download perceptron.zip.
Set up the files as an IntelliJ project.

Programming Assignment

The program executable is DrawingEditor, in the handwriting.gui package.
Most of the classes you will modify are in the handwriting.learners.perceptron package. I suggest working on them in the following order:
- Perceptron.java: compute() and applyWeightUpdates()
- PerceptronTrainer.java: calculateError() and recordWeightUpdate()
  - Once these two files have been completed correctly, PerceptronTest.testAnd() and testOr() should pass.
- Perceptron2.java: computeHiddenAndOutput()
- Perceptron2Trainer.java: calculateBackpropagatedError()
  - Once these two files have been completed correctly, PerceptronTest.testXor() should pass.
- Perceptron2RecognizerAI.java: encodeSample()
  - LabelEncoderDecoderTest.testEncodeSample() should pass once it is completed.
- LabelEncoderDecoder.java: decode()
  - LabelEncoderDecoderTest.testLED() should pass once it is completed.
The DrawingEditor program looks for fully specified neural network objects in the handwriting.learners package. Iter1000Rate10Hide20 is given as an example. It will create and employ a 2-layer perceptron trained with 1000 iterations, a learning rate of 0.1, and 20 hidden nodes. Copy and modify this file to create your own experimental variations.

Generating Data

Using the drawing editor, draw 10 samples each of two letters. For each drawing, click the "Record" button when it is complete. For the label, use the letter that you drew. Once this is complete, save the file (using the Save command on the File menu).
Create a second set of samples of your two letters, and save them under a different filename.
Create a neural network using the first file as the training set. Test its performance using the second file. Then build a new network using the second file for training and the first file for testing. How well does each network perform on its testing set?
Once you can build a network that distinguishes two letters, expand your training and test sets to train it to distinguish three letters. Continue iterating this process until you can build a network that can distinguish at least eight different letters.

Presentations

Tuesday, February 25:
- Give testing-set results for a two-letter experiment. Include the following details:
  - Number of training iterations
  - Number of hidden nodes employed
  - Percentage of testing-set examples correctly classified
Thursday, February 27:
- Describe your overall results for all experiments, up to eight letters

Paper

When you are finished with your experiments, write a paper summarizing your findings. Include the following:

An analysis and discussion of your data. (Be sure to include the data as well.)
How well does the multi-layer perceptron perform for this task?
- I will award bonus points for benchmarking its performance against kNN and/or Naive Bayes, which I encourage you to re-implement for this project.
What effect did variations in the number of training iterations, the learning rate and numbers of hidden nodes have on the results?
Beyond the actual results, what other issues are noteworthy?

Grading criteria

Achievement	Points
Passes PerceptronTest.testAnd()	3
Passes PerceptronTest.testOr()	3
Passes PerceptronTest.testXor()	11
Passes LabelEncoderDecoderTest.testEncodeSample()	4
Passes LabelEncoderDecoderTest.testLED()	4
Created 10 samples of eight letters, twice over	10
Ran fourteen experiments, from 2 to 8 letters, alternating training/testing sets	10
Paper quantitatively assesses each of the fourteen experiments	14
Paper insightfully discusses how the MLP performs with respect to their data set	3
Paper insightfully discusses the distinctions in performance as the number of letters increases	3
Paper insightfully discusses the impact of variations in the number of training iterations, the learning rate and numbers of hidden nodes	5