AP Week and Big Project 3 Plans

[JishnuS420]

RDS and SQLite Legacy Challenge:

• For RDS, we want to have Adi, who has had prior knowledge on the topic working on it, and alongside Adi, we also have Ethan Zhao and Alex Lu helping him to complete the challenge

Big Project:

For the big project, a large majority of the backend is completed and the key features are primarily done apart from things for the leaderboard and other aparts of the stock game and so, the frontend is the primary focus now.

Links to individual issues are highlighted in blue.

• David <3 will be moving over to Frontend but primarily to just correct things and fine tune them to the backend

• Jishnu will be countinuing to improve both graphs for the live stats and also the LSTM prediction and will be working to add someother features to the live stats and will work with David to fix it up

• Emaad will be fixing up the styling page wide and also making sure that the pages are consisstent amongst all the pages and have a proper working global navbar and will work with all the frontend components

• Ethan Tran and Krishiv will be working to fix up the leaderboard things and work on making it work on both backend and frontend, working alongside James

• Anthony will be working on the backend fixing up the perceptron model and making sure that model is making it more accurate and will be working with Tay

• Tay is backing up Anthony, also working on the backend making the model more accurate and also working on the frontend of the perceptron model

• James is working on the frontend for the stock game and will be working along with Ethan and Krishiv to make the stock game close to completion

• Yuri will be finishing up the homepage and making it fluid and smooth as well as work on a little bit of blogging

Features

Ethan T

Our backend has...

• A working AI model that makes predictions on what sign the user is making with their hands (ASL Alphabet and Custom signs)
• Data storage for each frame from the backend (what we use to make predictions)

Hand Recognition Generalization

Improve accuracy of the model (further training of the model for accuracy)
OpenCV's bounding box capabilities rely solely on geometric properties and the pixels taken from the frontend in the bounded area. Through Generalization, the hand's appearance in different environments such as lighting.

Timeline

• Research generalization, understand how it works and why it's implementation will allow for adaptability in the machine learning model
• Code for Data Augmentation of original frames
  I am working on code that takes each frame and creates new data based off the original frame in order to account for variability. This would allow the model to be trained on each frame in different lighting scenarios and allow for an accurate generalization/prediction.

    // method to create variations of an image
    private BufferedImage[] createAugmentedImages(BufferedImage originalImage) throws IOException {
        BufferedImage[] augmentedImages = new BufferedImage[9]; // array size remains 9 for 9 variations
        augmentedImages[0] = originalImage; // first image remains unchanged
        augmentedImages[1] = changeBrightness(originalImage, 0.8f); // decrease brightness by 20%
        augmentedImages[2] = changeBrightness(originalImage, 1.2f); // increase brightness by 20%
        augmentedImages[3] = addNoise(originalImage, 0.1); // add low level of noise
        augmentedImages[4] = addNoise(originalImage, 0.2); // add higher level of noise
        augmentedImages[5] = adjustHue(originalImage); // randomly adjust hue
        augmentedImages[6] = rotateImage(originalImage, 30); // rotate image by 30 degrees
        augmentedImages[7] = flipImage(originalImage, true); // flip image horizontally
        augmentedImages[8] = flipImage(originalImage, false); // flip image vertically
    
        // save each augmented image to a file using i+1 as filename
        for (int i = 0; i < augmentedImages.length; i++) {
            saveImage(augmentedImages[i], "augmented_image_" + (i + 1) + ".png");
        }
    
        return augmentedImages;
    }    

    // method to change brightness of an image
    private BufferedImage changeBrightness(BufferedImage image, float factor) {
        BufferedImage newImage = new BufferedImage(image.getWidth(), image.getHeight(), image.getType());
        Graphics2D g = newImage.createGraphics();
        g.drawImage(image, 0, 0, null);
        g.dispose();

        // adjust brightness for each pixel
        for (int y = 0; y < newImage.getHeight(); y++) {
            for (int x = 0; x < newImage.getWidth(); x++) {
                int rgba = newImage.getRGB(x, y);
                int alpha = (rgba >> 24) & 0xff;
                int red = (int)Math.min(255, ((rgba >> 16) & 0xff) * factor);
                int green = (int)Math.min(255, ((rgba >> 8) & 0xff) * factor);
                int blue = (int)Math.min(255, (rgba & 0xff) * factor);
                newImage.setRGB(x, y, (alpha << 24) | (red << 16) | (green << 8) | blue);
            }
        }
        return newImage;
    }

    // method to add noise to an image
    private BufferedImage addNoise(BufferedImage image, double noiseLevel) {
        BufferedImage newImage = new BufferedImage(image.getWidth(), image.getHeight(), image.getType());
        Graphics2D g = newImage.createGraphics();
        g.drawImage(image, 0, 0, null);
        g.dispose();

        // add noise to each pixel
        for (int y = 0; y < newImage.getHeight(); y++) {
            for (int x = 0; x < newImage.getWidth(); x++) {
                int rgba = newImage.getRGB(x, y);
                int alpha = (rgba >> 24) & 0xff;
                int red = (int)Math.min(255, Math.max(0, (int)(((rgba >> 16) & 0xff) * (1 + (Math.random() - 0.5) * noiseLevel))));
                int green = (int)Math.min(255, Math.max(0, (int)(((rgba >> 8) & 0xff) * (1 + (Math.random() - 0.5) * noiseLevel))));
                int blue = (int)Math.min(255, Math.max(0, (int)((rgba & 0xff) * (1 + (Math.random() - 0.5) * noiseLevel))));
                newImage.setRGB(x, y, (alpha << 24) | (red << 16) | (green << 8) | blue);
            }
        }
        return newImage;
    }

    // method to adjust hue of an image
    private BufferedImage adjustHue(BufferedImage image) {
        BufferedImage newImage = new BufferedImage(image.getWidth(), image.getHeight(), image.getType());
        Graphics2D g = newImage.createGraphics();
        g.drawImage(image, 0, 0, null);
        g.dispose();

        // adjust hue for each pixel
        for (int y = 0; y < newImage.getHeight(); y++) {
            for (int x = 0; x < newImage.getWidth(); x++) {
                int rgba = newImage.getRGB(x, y);
                int alpha = (rgba >> 24) & 0xff;
                int red = Math.min(255, (int)(((rgba >> 16) & 0xff) * (0.8 + Math.random() * 0.4)));
                int green = Math.min(255, (int)(((rgba >> 8) & 0xff) * (0.8 + Math.random() * 0.4)));
                int blue = Math.min(255, (int)((rgba & 0xff) * (0.8 + Math.random() * 0.4)));
                newImage.setRGB(x, y, (alpha << 24) | (red << 16) | (green << 8) | blue);
            }
        }
        return newImage;
    }

    // method to rotate an image
    private BufferedImage rotateImage(BufferedImage image, double angle) {
        // create a transformation matrix for rotating the image
        AffineTransform tx = new AffineTransform();
        tx.rotate(Math.toRadians(angle), image.getWidth() / 2.0, image.getHeight() / 2.0);
    
        // apply the transformation to the image
        AffineTransformOp op = new AffineTransformOp(tx, AffineTransformOp.TYPE_BILINEAR);
        return op.filter(image, null);
    }
    
    // method to flip an image horizontally or vertically
    private BufferedImage flipImage(BufferedImage image, boolean horizontal) {
        // create a transformation matrix for flipping the image
        AffineTransform tx = new AffineTransform();
        if (horizontal) {
            tx.scale(-1, 1); // flip horizontally
            tx.translate(-image.getWidth(), 0);
        } else {
            tx.scale(1, -1); // flip vertically
            tx.translate(0, -image.getHeight());
        }
    
        // apply the transformation to the image
        AffineTransformOp op = new AffineTransformOp(tx, AffineTransformOp.TYPE_NEAREST_NEIGHBOR);
        return op.filter(image, null);
    }

• Implement Generalization to the model itself
  After completing the code that will take into account the variations ie. lighting, horizontal, tilt and form the next step is to utilize the mnist data from the model and use this in the model which will make the predictions

    private List<List<List<Integer>>> generateAugmentedFrames(String imageData) throws IOException {
        byte[] imageBytes = Base64.getDecoder().decode(imageData);
        ByteArrayInputStream bis = new ByteArrayInputStream(imageBytes);
        BufferedImage originalImage = ImageIO.read(bis);

        List<List<List<Integer>>> mnistDataList = new ArrayList<>();
        BufferedImage[] augmentedImages = createAugmentedImages(originalImage);

        for (BufferedImage image : augmentedImages) {
            mnistDataList.add(convertToMNIST(image));
        }

        return mnistDataList;
    }

Krishiv

LeaderBoard Key Features

---

Backend

• Object: Declare variables and instantiate them and create columns for the SQL DB in order to have raw data to be able to sort specific users based on account growth and give them a rating.
• Collectable: Used as an abstract class( a class which isn't instantiated on its own) to set key types and variables for the performance, here we also initialize the specific variables.
• Itterator: The iterator manages the object through an interface for the performance object and as an iterator the function of the file is to The class includes methods to get the list size, update the key type for sorting purposes, and perform a merge sort on the list. This is to sort by the key value of account value that was set in the collectable and make the full dataset be active and work.

Working Feature

USE OF SORTING IN THE ITTERATOR

SQL IMPLEMENTATION

POSTMAN REQUESTS

FRONTEND FETCH AND WORKING TABLE!

ERRORS THAT WERE FACED!

1. POM.XML; with working on different versions, since everyones codebase was constantly changing using wrong headers and imports often left dependencies to fail and build not working in the front-end, however this was fixed through better communication and better branch management.

2. Being able to have test data: every time we tried to make a postman request to post new data and later on make a get request so that it sorted the list by a specified attribute, we would get some sort of error, such as a 405 Not Allowed error or even an Internal Server Error. Later on, however, we found that this was because we were missing PostMapping in our API controller for the performance leaderboard, so we added that along with a new endpoint so that it could create new users. Now we are able to add users to the list along with their other attributes.

(used the wrong endpoint and format when adding the data)

Jishnu

Things to tend to:

Live Stats page (primary priority):

• Stats live that cleans up the graph and then updates to the tike range selected (1D, 1W, 1M, 1Y, Alltime)
• Stats live has a carousel slider that displays similar stocks and popular stocks, Example:
  
• Stats for a company is pulled and then also provides detail displaying the recommendations, Example:
  
• Stats also shows the popular, gainers, and other relevant information

Prediction page (primary priority):

• Fix up the date scaling (change the graph to an irregular timeseries) example below:
  

Stock Game page (secondary priority, cannot be implmented without the above):

• Implement the live graph into the game
• Have the predicitons show up with the player's performance
• The player's predcition are compared with the AI's predictions, and the real prices, award points accordingly

Timeline and CheckPoints:

• Tuesday 5/21, check in with David after completed stats live and start steady implementation into the game
• Thursday 5/23, check in with David for prediciton implementation and further work to be done