Skip to content

Set up the DAQ dashboard

Jump to bottom
Francesco Anselmo edited this page Nov 19, 2019 · 9 revisions

The DAQ command line interface (CLI) output is very informative and, together with the report output, gives detailed information about which tests have been successful and which ones have failed.

In addition to the CLI output and reports, DAQ offers the possibility to show the status of the tests via a web based application that uses the Google Cloud Platform and Firebase infrastructure.

The web based application is particularly useful when continuously testing multiple devices, since it provides a mean to do remote monitoring while tests are being executed.

In this tutorial you will set up the Firebase-hosted DAQ dashboard.

The dashboard running in-browser is written in vanilla JS; the only dependencies are firebase JavaScript libraries imported in the <head> tag of index.html.

Pre-requisites

You will need to have a [Google Cloud Platform] (GCP) account.

Cloud setup

  • Set up a dedicated GCP project.

  • Goto the Firebase Console and add a new project. You don't need to sign up to Firebase separately after signing up for GCP.

  • Add the hosting GCP project to link it to this Firebase setup.

  • Enable storage on Firebase (simply click on the storage tab and follow the on-screen instructions):

enable-firebase-storage
  • If you see an error concerning the Firebase API:

firebase-api-error

you will need to enable 'Native Mode' in the 'Cloud Firestore' tab on GCP

  • Navigate to the [Google Cloud Platform (GCP) service accounts page] (https://console.cloud.google.com/iam-admin/serviceaccounts?project=daq-project) and create a new service account with a meaningful name.

  • Add the Pub/Sub Admin, Storage Admin, Cloud Datastore User, and Firebase Admin roles.

  • Create a new private key.

  • Finally, you will need to copy your module_config.json from misc/ to local/site

DAQ setup

Install the downloaded key into DAQ.

Copy the download JSON key file to the daq/local/ directory.

Edit daq/local/system.conf to specify the gcp_cred setting to point to the downloaded file (with a path relative to the daq/ install directory), e.g. gcp_cred=local/daq-testlab-de56aa4b1e47.json.

Start DAQ (cmd/run)

There should be something in the top 10-20 startup log lines that look something like:
INFO:gcp:Loading gcp credentials from local/daq-testlab-de56aa4b1e47.json
INFO:gcp:Initialized gcp publisher client daq-project:daq-testlab

Follow the Firebase CLI setup instructions. It is strongly recommended that you use nvm to install Node.js. nvm is a very powerful bash script that allows multiple versions of Node.js to be installed on one host machine.

Run npm install -g firebase-functions firebase-admin

Goto the 'daq/firebase/functions` directory.

Run npm install

Goto the 'daq/firebase/` directory.

Run firebase use --add to set the GCP project to use (as created above).

Follow the relevant parts of the https://console.firebase.google.com/

Select your project. Select "+ Add app" Select "</>" (Web) Use a clever nickname and register app.

Copy the var firebaseConfig = { ... } snippet to local/firebase_config.js

Add an API Key Restriction for an HTTP Referrer, which will be the https:// address of the daq hosted web app.

Enable Google sign-in from https://console.firebase.google.com/

Select your project. Select "Authentication" Select "Sign-in method" Enable "Google" sign-in.

Run firebase/deploy.sh to deploy the necessary parts of the system.

Follow the link to the indicated Hosting URL to see the newly installed pages.

When the system starts up and runs, there should be some descriptive log messages that highlight operation, with obvious error messages when something goes wrong:

...
INFO:gcp:Loading gcp credentials from local/daq-testlab-de56aa4b1e47.json
INFO:gcp:Initialized gcp pub/sub daq-project:daq-testlab
INFO:gcp:Initialized gcp firestore daq-project:daq-testlab
...
INFO:gcp:Uploaded test report to inst/report_9a02571e8f00.txt
...

On the GCP PubSub Topics page, there should be an entry for a projects/daq-project/topics/daq_runner topic, with at least one subscription to something like projects/daq-project/subscriptions/gcf-daq_firestore-daq_runner, which is the linked Firestore cloud function.

The GCP Cloud Functions page should show a daq_firestore function, and if you look at the logs there should be info events for each reported stage of DAQ test, e.g.:

I  daq_firestore 210626605171528 updating 1537561337615 daq-laptop port-undefined undefined status daq_firestore 210626605171528
I  daq_firestore 210625487917242 updating 1537561337804 daq-laptop port-1 5ba552f2 sanity daq_firestore 210625487917242
I  daq_firestore 210621644751867 updating 1537561338017 daq-laptop port-1 5ba552f2 info daq_firestore 210621644751867
I  daq_firestore 210625202516197 updating 1537561338223 daq-laptop port-1 5ba552f2 dhcp daq_firestore 210625202516197
I  daq_firestore 210621107285036 updating 1537561362562 daq-laptop port-1 5ba552f2 info daq_firestore 210621107285036
I  daq_firestore 210617247716870 updating 1537561362716 daq-laptop port-1 5ba552f2 dhcp daq_firestore 210617247716870
I  daq_firestore 210615717506431 updating 1537561362986 daq-laptop port-1 5ba552f2 base daq_firestore 210615717506431

The same Cloud Functions are represented in Firebase and can be viewed as part of the Firebase Functions Usage Logs (note that you'll have to manually replace daq-project with the appropriate {projectId} in order for that link to work). The logs there will also show any signficant errors highlighting most problems.

The data ends up in the Firestore Database (again replacing daq-project with the appropraite {projectId}) and filed under origin/{accountId} (the name of the service account) of the DAQ install supplying the data.

The test Web Application (again, will have to substitute the appropriate projectId), should show a list of all accounts with ingested data. If nothing is showing here, or the accountId is missing, check the JS console in your browser to see if there's any obvious errors.

Go to the next tutorial > Using DAQ with an OpenFlow switch

Clone this wiki locally