ASL Artificial Intelligence Model - Bounded Box

[anthonybazhenov]

Planning for Trimester Long Project

Goals

Use commonly used techniques to increase accuracy of the model especially with an increased number of classes. We decided to go with bounded box because of its versatility and its common use in Artificial Intelligence, providing a box around the area being tested on to pinpoint and track down the exact location the hand is for more accurate testing.

How does it work

We take each of the frames and sets a skin color range that it looks for within the frame and once it finds it, it creates a mask of just those colors. After with this new "skin" mask we find the largest contours, or biggest differences in neighboring pixels, which is most likely to be the hand.

Design

The bounded box will most likely look like this when run with a giant box following your hand around the webcam no matter how you orientate it or how far back or forward you move it.

Bounded Box Progress

• Finish instillation of openCV in order to get my webcam to work let testing be easy on with taking individual frames or drawing rectangles being made easy with it's use.
• Track information about the skin color range of the person who signed up in order to allow it to just track their hand while logging in furthering the logins specificity and security because the box wouldn't appear if you don't share a similar skin color to the person logging in.
• Finish recording everyones parts and introduce a way, with the help of Tay Kim to increase the size of the class array when a new person signs up and change the number of iterations of the for loops that go through every class.

- [ ] Change the number of pixels that we train and test on because the bounded box would make the data that we are training to be less that 256 by 256 and have it instead train on a more specified image of the hand but only around 100 by 100 pixels.

Day by Day Plan

Monday

• Finish plan for checkpoint.
• Finish the logic for hand detection using contours and differences in color.

Tuesday

• Finish instillation of openCV in order to get my webcam to work let testing be easy on with taking individual frames or drawing rectangles being made easy with it's use.

Wednesday

• Track information about the skin color range of the person who signed up in order to allow it to just track their hand while logging in furthering the logins specificity and security because the box wouldn't appear if you don't share a similar skin color to the person logging in.

Thursday

• Finish recording everyones parts and introduce a way, with the help of Tay Kim to increase the size of the class array when a new person signs up and change the number of iterations of the for loops that go through every class.
• Change the number of pixels that we train and test on because the bounded box would make the data that we are training to be less that 256 by 256 and have it instead train on a more specified image of the hand but only around 100 by 100 pixels.

Friday

• Test everything as much as possible

TDD

For the demo I plan to show using the bounded box with the webcam and show the individual images that are tested on with the help of the increased specificity of the bounded box. Additionally, show progressions of the data trained on before and after using bounded boxes.