Skip to content

Latest commit

 

History

History
74 lines (64 loc) · 2.97 KB

README.md

File metadata and controls

74 lines (64 loc) · 2.97 KB
Raw

Guitar
Finguitar 🎸

Finguitar - This is an online instrument which you can play using just your fingers.
CodeOdyssey Hack-a-thon Winners || Team NeoStar Aaghadi

This web-app is called Finguitar (Finger-Guitar). It uses Computer Vision to build an online instrument which you can play using just your fingers.
You can hold up a gesture with your hand and the web-app will record it using your webcam and play a music note for every gesture! The idea is to then give the user the ability to combine these gestures to make a tune out of it. So, it's like playing a guitar... but using Computer Vision!
Currently we recognize 10 gestures to play different sounds:
  • Palm
  • Fist
  • Thumb
  • Index
  • Middle 2 fingers
  • Pinky
  • Call
  • Rock
  • LShape(finger+thumb)
  • Ok

Resources

🤖Tech-Stack

Front-end

  • ReactJS

Back-end

  • Flask

Computer vision

  • OpenCV

📲Communication b/w the web-app and the model

The communication is carried out on a WebSocket endpoint run by the server. The web-app uses the browser's Stream API to emit the current frame encoded as a base64 string via the WebSocket. The server receives the string, decodes it and runs the OpenCV model on it. The gesture recognized is then relayed back on the same endpoint as a response.

WebSockets

🔮Further additions

  • Host the web-app and the model
  • Improve the animations on the web-app
  • Add a gamified experience for the user, and provide him with a score as a feedback
  • Add a tutorial run, where the user can first train on the set of gestures and get familiar with the sounds.

👨‍💻Team Members