CSCI 335 - Programming Project #6

Spring 2020

robosim.zip
Apply Q-Learning to automatically learn the following task:
- The simulated robot should drive forward as much as possible.
- It should avoid hitting objects.
- Devise a state encoding for the sonar and a reward scheme to bring this about.
Create at least three different maps to evaluate the robot's performance.
For each map, run at least three trials for each combination of Q-Learning parameters you choose to try.
- Try at least three combinations of Q-Learning parameters.
- Be sure to run both the random explorer and the hand-coded obstacle explorer on each map.
From your experiments, you will need to determine ideal values for:
- The discount rate γ
- The target number of visits
- The learning rate constant
- The amount of time for the robot to run before stopping
Presentations on 3/17 (intermediate) and 3/19 (final)
Write a report detailing your results:
- Discuss evidence indicating that the learned behavior somehow represents an improvement on random action selection.
- Also discuss how well the learned behavior performs in comparison with the hand-coded obstacle avoider.

Achievement	Points
Passes `testLeastVisitedAction()`	2
Passes `testLearningRate()`	2
Passes `testIsExploring()`	2
Passes `testBestAction()`	2
Passes `testQ()`	7
A class implementing the Controller interface employs a QTable to determine all actions	10
Ran experiments with three different maps, with three trials per experiment, with at least three combinations of parameters, also using both baseline controllers	20
Paper insightfully discusses how the Q-Learner performs in comparison with the baseline controllers	5
Paper insightfully discusses the impact of each of the four Q-Learning parameters under consideration	10
Paper insightfully discusses the relative utility of Q-Learning vs explicit programming for this task	5
Paper speculates in an informed manner about the general utility of Q-Learning in comparison with explicit programming	5