Milestone 4

Milestone Goals

Improve the image processor to add the following functionality:

Treasure recognition
Treasure color detection
Treasure shape detection
FPGA-to-Arduino notification of detected treasure

Troubleshooting

Issues
During Lab 4, and throughout the past few weeks working on Milestone 4, our group became aware of issues affecting our camera-FPGA setup. The performance of the camera would drastically degrade over time. The longer we ran the camera, the more dim it would become. Noise, visible as rainbow bars on the camera output, would appear and eventually consume the image.

Checking the Arduino
To confirm that the Arduino code used to setup the cameras was not to blame, we checked the register values with the TAs, as well as against other groups. Our registers, however, appear to be set correctly.

Checking the Verilog
Next, we investigated our Verilog code. After spending over two weeks with little to no progress, we decided to test our system with another group's working code. We used Verilog code from both Team 3 and Team 5, with the only edits being the I/O pin numbers to match our physical setup. Unfortunately, neither group's code worked properly, which confirmed our suspicions of hardware issues.

Checking the Hardware
Our group did not receive a ribbon cable during Lab 4. Since there were no more female-to-female cables available, we were forced to cut and crimp our own wires. Although we have tested for connectivity, the wiring is a major concern for us. In addition, we fear that our camera might be broken, but since there are no cameras left, we are unable to test this hypothesis

Conclusion
As a result, we have been unable to fully implement or test our Tresure Detection. Nonetheless, we have came up with the algorithms and ideas that we would implement if we had functioning hardware. In addition, we have taken the steps to integrate Treasure Detection into our full Arduino system. We connected 3 GPIO pins from the FPGA to the Arduino to transmit the information, updated our Arduino's Radio Communication protocoal to handle the additional information, and modified our update_GUI() function (used by the Base Station) to output treasure information.

Image Processing and Storage

Inspired by Team 3 (Pulse)
Break up image into 3 sections diagonally

Create six counters: one for each color (blue and red) for each of the three sections on the diagonal
In each section, increment the blue or red counter if a majority blue or majority red pixel was seen respectively

Image Analysis for Color and Shape Detection

Based on the pixel/color counts, we can determine which shape and which color was detected

If only the center has lots of blue (or red) majority pixels, then was most likely a blue (or red) diamond
If the center and bottom right sections contain blue (or red) majority pixels, most likely triangle
If all three sections contain blue (or red) majority pixels, most likely square

Encoding

We connected 3 GPIO pins on the FPGA to 3 digital I/O pins on the Arduino
The Arduino periodically polls these inputs

We have a variable for each of the 6 possible treasure types
To prevent false-positives, upon detecting a treasure we increment the counter for that treasure type and reset the other counters to 0
We only declare that a treasure has truly been found if the counter rises above a threshold
If none of the counters are greater than the threshold, there is no treasure present
This information is then re-encoded into 3 bits and sent to the Base Station using our Radio

The following encoding scheme is used throughout the robot:

Treasure Detection Demonstration

This video demonstrates our robot completing all required behaviors:

Robot capable of detecting when there are/are not treasures
Robot capable of distinguishing between red and blue treasures
Robot capable of distinguishing a square, triangle, and diamond shape
Transmitting treasure detection information from the FPGA to the Arduino

Unfortunately, as mentioned above, we do not have a video demonstration at this time :'(