initial with npm install

Author: prateeksawhney97

README.md

@@ -1 +1,202 @@
-# People-Counter-Application-Using-Intel-OpenVINO-Toolkit
+# Deploy a People Counter App at the Edge
+
+| Details            |              |
+|-----------------------|---------------|
+| Programming Language: |  Python 3.5 or 3.6 |
+
+![people-counter-python](./images/people-counter-image.png)
+
+## What it Does
+
+The people counter application will demonstrate how to create a smart video IoT solution using Intel® hardware and software tools. The app will detect people in a designated area, providing the number of people in the frame, average duration of people in frame, and total count.
+
+## How it Works
+
+The counter will use the Inference Engine included in the Intel® Distribution of OpenVINO™ Toolkit. The model used should be able to identify people in a video frame. The app should count the number of people in the current frame, the duration that a person is in the frame (time elapsed between entering and exiting a frame) and the total count of people. It then sends the data to a local web server using the Paho MQTT Python package.
+
+You will choose a model to use and convert it with the Model Optimizer.
+
+![architectural diagram](./images/arch_diagram.png)
+
+## Requirements
+
+### Hardware
+
+* 6th to 10th generation Intel® Core™ processor with Iris® Pro graphics or Intel® HD Graphics.
+* OR use of Intel® Neural Compute Stick 2 (NCS2)
+* OR Udacity classroom workspace for the related course
+
+### Software
+
+*   Intel® Distribution of OpenVINO™ toolkit 2019 R3 release
+*   Node v6.17.1
+*   Npm v3.10.10
+*   CMake
+*   MQTT Mosca server
+  
+        
+## Setup
+
+### Install Intel® Distribution of OpenVINO™ toolkit
+
+Utilize the classroom workspace, or refer to the relevant instructions for your operating system for this step.
+
+- [Linux/Ubuntu](./linux-setup.md)
+- [Mac](./mac-setup.md)
+- [Windows](./windows-setup.md)
+
+### Install Nodejs and its dependencies
+
+Utilize the classroom workspace, or refer to the relevant instructions for your operating system for this step.
+
+- [Linux/Ubuntu](./linux-setup.md)
+- [Mac](./mac-setup.md)
+- [Windows](./windows-setup.md)
+
+### Install npm
+
+There are three components that need to be running in separate terminals for this application to work:
+
+-   MQTT Mosca server 
+-   Node.js* Web server
+-   FFmpeg server
+     
+From the main directory:
+
+* For MQTT/Mosca server:
+   ```
+   cd webservice/server
+   npm install
+   ```
+
+* For Web server:
+  ```
+  cd ../ui
+  npm install
+  ```
+  **Note:** If any configuration errors occur in mosca server or Web server while using **npm install**, use the below commands:
+   ```
+   sudo npm install npm -g 
+   rm -rf node_modules
+   npm cache clean
+   npm config set registry "http://registry.npmjs.org"
+   npm install
+   ```
+
+## What model to use
+
+It is up to you to decide on what model to use for the application. You need to find a model not already converted to Intermediate Representation format (i.e. not one of the Intel® Pre-Trained Models), convert it, and utilize the converted model in your application.
+
+Note that you may need to do additional processing of the output to handle incorrect detections, such as adjusting confidence threshold or accounting for 1-2 frames where the model fails to see a person already counted and would otherwise double count.
+
+**If you are otherwise unable to find a suitable model after attempting and successfully converting at least three other models**, you can document in your write-up what the models were, how you converted them, and why they failed, and then utilize any of the Intel® Pre-Trained Models that may perform better.
+
+## Run the application
+
+From the main directory:
+
+### Step 1 - Start the Mosca server
+
+```
+cd webservice/server/node-server
+node ./server.js
+```
+
+You should see the following message, if successful:
+```
+Mosca server started.
+```
+
+### Step 2 - Start the GUI
+
+Open new terminal and run below commands.
+```
+cd webservice/ui
+npm run dev
+```
+
+You should see the following message in the terminal.
+```
+webpack: Compiled successfully
+```
+
+### Step 3 - FFmpeg Server
+
+Open new terminal and run the below commands.
+```
+sudo ffserver -f ./ffmpeg/server.conf
+```
+
+### Step 4 - Run the code
+
+Open a new terminal to run the code. 
+
+#### Setup the environment
+
+You must configure the environment to use the Intel® Distribution of OpenVINO™ toolkit one time per session by running the following command:
+```
+source /opt/intel/openvino/bin/setupvars.sh -pyver 3.5
+```
+
+You should also be able to run the application with Python 3.6, although newer versions of Python will not work with the app.
+
+#### Running on the CPU
+
+When running Intel® Distribution of OpenVINO™ toolkit Python applications on the CPU, the CPU extension library is required. This can be found at: 
+
+```
+/opt/intel/openvino/deployment_tools/inference_engine/lib/intel64/
+```
+
+*Depending on whether you are using Linux or Mac, the filename will be either `libcpu_extension_sse4.so` or `libcpu_extension.dylib`, respectively.* (The Linux filename may be different if you are using a AVX architecture)
+
+Though by default application runs on CPU, this can also be explicitly specified by ```-d CPU``` command-line argument:
+
+```
+python main.py -i resources/Pedestrian_Detect_2_1_1.mp4 -m your-model.xml -l /opt/intel/openvino/deployment_tools/inference_engine/lib/intel64/libcpu_extension_sse4.so -d CPU -pt 0.6 | ffmpeg -v warning -f rawvideo -pixel_format bgr24 -video_size 768x432 -framerate 24 -i - http://0.0.0.0:3004/fac.ffm
+```
+If you are in the classroom workspace, use the “Open App” button to view the output. If working locally, to see the output on a web based interface, open the link [http://0.0.0.0:3004](http://0.0.0.0:3004/) in a browser.
+
+#### Running on the Intel® Neural Compute Stick
+
+To run on the Intel® Neural Compute Stick, use the ```-d MYRIAD``` command-line argument:
+
+```
+python3.5 main.py -d MYRIAD -i resources/Pedestrian_Detect_2_1_1.mp4 -m your-model.xml -pt 0.6 | ffmpeg -v warning -f rawvideo -pixel_format bgr24 -video_size 768x432 -framerate 24 -i - http://0.0.0.0:3004/fac.ffm
+```
+
+To see the output on a web based interface, open the link [http://0.0.0.0:3004](http://0.0.0.0:3004/) in a browser.
+
+**Note:** The Intel® Neural Compute Stick can only run FP16 models at this time. The model that is passed to the application, through the `-m <path_to_model>` command-line argument, must be of data type FP16.
+
+#### Using a camera stream instead of a video file
+
+To get the input video from the camera, use the `-i CAM` command-line argument. Specify the resolution of the camera using the `-video_size` command line argument.
+
+For example:
+```
+python main.py -i CAM -m your-model.xml -l /opt/intel/openvino/deployment_tools/inference_engine/lib/intel64/libcpu_extension_sse4.so -d CPU -pt 0.6 | ffmpeg -v warning -f rawvideo -pixel_format bgr24 -video_size 768x432 -framerate 24 -i - http://0.0.0.0:3004/fac.ffm
+```
+
+To see the output on a web based interface, open the link [http://0.0.0.0:3004](http://0.0.0.0:3004/) in a browser.
+
+**Note:**
+User has to give `-video_size` command line argument according to the input as it is used to specify the resolution of the video or image file.
+
+## A Note on Running Locally
+
+The servers herein are configured to utilize the Udacity classroom workspace. As such,
+to run on your local machine, you will need to change the below file:
+
+```
+webservice/ui/src/constants/constants.js
+```
+
+The `CAMERA_FEED_SERVER` and `MQTT_SERVER` both use the workspace configuration. 
+You can change each of these as follows:
+
+```
+CAMERA_FEED_SERVER: "http://localhost:3004"
+...
+MQTT_SERVER: "ws://localhost:3002"
+```

WRITEUP.md

@@ -0,0 +1,59 @@
+# Project Write-Up
+
+You can use this document as a template for providing your project write-up. However, if you
+have a different format you prefer, feel free to use it as long as you answer all required
+questions.
+
+## Explaining Custom Layers
+
+The process behind converting custom layers involves...
+
+Some of the potential reasons for handling custom layers are...
+
+## Comparing Model Performance
+
+My method(s) to compare models before and after conversion to Intermediate Representations
+were...
+
+The difference between model accuracy pre- and post-conversion was...
+
+The size of the model pre- and post-conversion was...
+
+The inference time of the model pre- and post-conversion was...
+
+## Assess Model Use Cases
+
+Some of the potential use cases of the people counter app are...
+
+Each of these use cases would be useful because...
+
+## Assess Effects on End User Needs
+
+Lighting, model accuracy, and camera focal length/image size have different effects on a
+deployed edge model. The potential effects of each of these are as follows...
+
+## Model Research
+
+[This heading is only required if a suitable model was not found after trying out at least three
+different models. However, you may also use this heading to detail how you converted 
+a successful model.]
+
+In investigating potential people counter models, I tried each of the following three models:
+
+- Model 1: [Name]
+  - [Model Source]
+  - I converted the model to an Intermediate Representation with the following arguments...
+  - The model was insufficient for the app because...
+  - I tried to improve the model for the app by...
+  
+- Model 2: [Name]
+  - [Model Source]
+  - I converted the model to an Intermediate Representation with the following arguments...
+  - The model was insufficient for the app because...
+  - I tried to improve the model for the app by...
+
+- Model 3: [Name]
+  - [Model Source]
+  - I converted the model to an Intermediate Representation with the following arguments...
+  - The model was insufficient for the app because...
+  - I tried to improve the model for the app by...

fac.ffm

@@ -0,0 +1,46 @@
+# Server
+HTTPPort 3004
+HTTPBindAddress 0.0.0.0
+MaxHTTPConnections 200
+MaxClients 100
+MaxBandwidth 54000
+CustomLog -
+
+# Feed/Raw video
+<Feed fac.ffm>
+File fac.ffm
+FileMaxSize 16M
+ACL allow 127.0.0.1
+</Feed>
+
+# Stream
+<Stream facstream.mjpeg>
+Feed fac.ffm
+Format mpjpeg
+VideoBitRate    8192
+VideoBufferSize 8192
+VideoFrameRate  25
+VideoSize       hd480
+#VideoQMin       2
+#VideoQMax       8
+NoAudio
+Strict          -1
+
+ACL allow 192.168.0.0 192.168.255.255
+ACL allow localhost
+ACL allow 127.0.0.1
+</Stream>
+
+# Special streams
+# Server status
+<Stream stat.html>
+Format status
+ACL allow localhost
+ACL allow 127.0.0.1
+ACL allow 192.168.0.0 192.168.255.255
+</Stream>
+
+# Redirect index.html to the appropriate site
+<Redirect index.html>
+URL http://www.github.com/intel-iot-devkit
+</Redirect>

ffmpeg/server.conf

@@ -0,0 +1,67 @@
+#!/usr/bin/env python3
+"""
+ Copyright (c) 2018 Intel Corporation.
+
+ Permission is hereby granted, free of charge, to any person obtaining
+ a copy of this software and associated documentation files (the
+ "Software"), to deal in the Software without restriction, including
+ without limitation the rights to use, copy, modify, merge, publish,
+ distribute, sublicense, and/or sell copies of the Software, and to
+ permit persons to whom the Software is furnished to do so, subject to
+ the following conditions:
+
+ The above copyright notice and this permission notice shall be
+ included in all copies or substantial portions of the Software.
+
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+"""
+
+import os
+import sys
+import logging as log
+from openvino.inference_engine import IENetwork, IECore
+
+
+class Network:
+    """
+    Load and configure inference plugins for the specified target devices 
+    and performs synchronous and asynchronous modes for the specified infer requests.
+    """
+
+    def __init__(self):
+        ### TODO: Initialize any class variables desired ###
+
+    def load_model(self):
+        ### TODO: Load the model ###
+        ### TODO: Check for supported layers ###
+        ### TODO: Add any necessary extensions ###
+        ### TODO: Return the loaded inference plugin ###
+        ### Note: You may need to update the function parameters. ###
+        return
+
+    def get_input_shape(self):
+        ### TODO: Return the shape of the input layer ###
+        return
+
+    def exec_net(self):
+        ### TODO: Start an asynchronous request ###
+        ### TODO: Return any necessary information ###
+        ### Note: You may need to update the function parameters. ###
+        return
+
+    def wait(self):
+        ### TODO: Wait for the request to be complete. ###
+        ### TODO: Return any necessary information ###
+        ### Note: You may need to update the function parameters. ###
+        return
+
+    def get_output(self):
+        ### TODO: Extract and return the output results
+        ### Note: You may need to update the function parameters. ###
+        return