Skip to content

Release Testing

Jump to bottom
Erik Jaegervall edited this page Jun 10, 2024 · 14 revisions

This page intends to list the tests that we shall/must do before we do a release for KUKSA Databroker. It is based on the old kuksa.val release testing page.

The intention of the release test is to do a reasonable regression test. It is assumed that new features of individual components have been thoroughly tested, the focus of the release test is to verify that there are no regressions. For capacity reasons the tests have no intention to cover all possible use cases.

Prerequisites

  • No outstanding Pull Requests that may affect component behavior.
  • CI Docker builds on latest master has succeeded
  • Pre-release of kuksa-client exists on PyPI (If dependencies exist)

Basic testing with Databroker (Insecure, No TLS)

erik@debian3:~/kuksa.val/kuksa_databroker$ cargo run --bin databroker -- --metadata data/vss-core/vss_release_4.0.json --insecure

Python Client test

(kuksa-client) erik@debian3:~/kuksa.val/kuksa-client$ kuksa-client grpc://127.0.0.1:55555


Test Client> setValue Vehicle.Speed 33
OK

Test Client> getValue Vehicle.Speed

Databroker CLI

  • NOTE: Verify that correct version number is shown in the "hello message!*
erik@debian3:~/kuksa.val/kuksa_databroker$ cargo run --bin databroker-cli
...
kuksa.val.v1 > publish Vehicle.Speed 44
[publish]  OK  
kuksa.val.v1 > get Vehicle.Speed
[get]  OK  
Vehicle.Speed: 44.00 km/h

Feeder smoke tests

DBC2VAL

They rely on the client, so some testing is necessary. Two options exist, either build and install kuksa-client locally, or upload an alpha package to pypi. Testing with an uploaded package is preferable as that also tests that Pypi packaging works as expected.

Start DBC Feeder using default settings

erik@debian3:~/kuksa.val.feeders/dbc2val$ ./dbcfeeder.py

Let it run for a while, verify with Python client (or databroker CLI) that Vehicle.Speed change now and then (it takes some time to see the first change).

Test Client> getValue Vehicle.Speed                          
{
    "path": "Vehicle.Speed",
    "value": {
        "value": 77.0,
        "timestamp": "2023-07-20T13:17:26.946733+00:00"
    }
}

Test Client> getValue Vehicle.Speed
{
    "path": "Vehicle.Speed",
    "value": {
        "value": 54.0,
        "timestamp": "2023-07-20T13:17:33.924064+00:00"
    }
}

Basic testing with Databroker (Tokens and TLS)

erik@debian3:~/kuksa.val/kuksa_databroker$ cargo run --bin databroker -- --metadata data/vss-core/vss_release_4.0.json --tls-cert certificates/Server.pem --tls-private-key certificates/Server.key --jwt-public-key certificates/jwt/jwt.key.pub

Kuksa-client

Verify with Kuksa-client that connection works

erik@debian4:~/kuksa-python-sdk/kuksa-client$ kuksa-client grpcs://127.0.0.1:55555 --cacertificate /home/erik/kuksa-common/tls/CA.pem --tls-server-name Server

     ⢀⣤⣶⣾⣿⢸⣿⣿⣷⣶⣤⡀
    ⣴⣿⡿⠋⣿⣿   ⠈⠙⢿⣿⣦
   ⣾⣿⠋  ⣿⣿  ⣶⣿  ⠙⣿⣷
  ⣸⣿⠇   ⣿⣿⠠⣾⡿⠃   ⠸⣿⣇  ⣶ ⣠⡶⠂ ⣶  ⢰⡆ ⢰⡆⢀⣴⠖ ⢠⡶⠶⠶⡦   ⣰⣶⡀
  ⣿⣿    ⠿⢿⣷⣦⡀     ⣿⣿  ⣿⢾⣏   ⣿  ⢸⡇ ⢸⡷⣿⡁  ⠘⠷⠶⠶⣦  ⢠⡟⠘⣷
  ⢹⣿⡆   ⣿⣶⠈⢻⣿⡆   ⢰⣿⡏  ⠿ ⠙⠷⠄ ⠙⠷⠶⠟⠁ ⠸⠇⠈⠻⠦ ⠐⠷⠶⠶⠟ ⠠⠿⠁ ⠹⠧
   ⢿⣿⣄  ⣿⣿  ⠿⣿  ⣠⣿⡿
    ⠻⣿⣷⡄⣿⣿   ⢀⣠⣾⣿⠟    kuksa-client CLI
     ⠈⠛⠇⢿⣿⣿⣿⣿⡿⠿⠛⠁     0.0.0

Default tokens directory: /home/erik/kuksa-python-sdk/submodules/kuksa.val/kuksa_certificates/jwt

Connecting to VSS server at 127.0.0.1 port 55555 using KUKSA GRPC protocol.
TLS will be used.
INFO 2023-11-16 10:36:56,850 kuksa_client.grpc Using TLS with Root CA from /home/erik/kuksa-common/tls/CA.pem
INFO 2023-11-16 10:36:56,855 kuksa_client.grpc Using client private key and certificates, mutual TLS supported if supported by server
INFO 2023-11-16 10:36:56,855 kuksa_client.grpc.aio Establishing secure channel
INFO 2023-11-16 10:36:56,855 kuksa_client.grpc.aio Using TLS server name Server
INFO 2023-11-16 10:36:56,909 kuksa_client.grpc.aio Unauthenticated channel started
Secure gRPC channel connected.


Test Client> authorize /home/erik/kuksa-common/jwt/actuate-provide-all.token 
"Authenticated"

Test Client> setValue Vehicle.Speed 33
OK

Test Client> getValue Vehicle.Speed
{
    "path": "Vehicle.Speed",
    "value": {
        "value": 33.0,
        "timestamp": "2023-11-16T09:37:49.769857+00:00"
    }
}

Databroker-cli

erik@debian3:~/kuksa.val/kuksa_databroker$ cargo run --bin databroker-cli -- --token-file jwt/provide-all.token --ca-cert certificates/CA.pem
    Finished dev [unoptimized + debuginfo] target(s) in 0.20s
     Running `/home/erik/kuksa.val/target/debug/databroker-cli --token-file ../jwt/provide-all.token --ca-cert ../kuksa_certificates/CA.pem`

...

sdv.databroker.v1 > feed Vehicle.Speed 5
[feed]  OK  
sdv.databroker.v1 > get Vehicle.Speed
[get]  OK  
Vehicle.Speed: 5.00
sdv.databroker.v1 >

Databroker Docker

Verify first that Databroker:main is recently built and there is no pending/queued builds.

erik@debian3:~/kuksa.val/kuksa_databroker$ docker pull ghcr.io/eclipse-kuksa/kuksa-databroker:main
erik@debian3:~/kuksa.val/kuksa_databroker$ docker run --rm -it  -p 55555:55555/tcp -v /home/erik/kuksa.val/kuksa_certificates:/certs   ghcr.io/eclipse-kuksa/kuksa-databroker:main --tls-cert /certs/Server.pem --tls-private-key /certs/Server.key --jwt-public-key /certs/jwt/jwt.key.pub

Then repeat test for local build

databroker-cli docker

erik@debian4:~/kuksa.val$ docker run --rm -it --net=host -v /home/erik/kuksa.val/kuksa_certificates:/certs -v /home/erik/kuksa.val/jwt:/jwt ghcr.io/eclipse-kuksa/kuksa-databroker-cli:main --token-file /jwt/provide-all.token --ca-cert /certs/CA.pem

KUKSA VISS Interface

Run

cargo run --bin databroker --features viss -- --enable-viss --metadata data/vss-core/vss_release_4.0.json --insecure

Then use two instances of kuksa-client (as VISS interface ONLY supports setting of target value and reading of actual value)

Step VISS client gRPC client
Start client kuksa-client ws://127.0.0.1:8090 kuksa-client
Set Speed Test Client> setValue Vehicle.Speed 332
Read Speed Test Client> getValue Vehicle.Speed
Set target tripmeter Test Client> setTargetValue Vehicle.TripMeterReading 32
Read target tripmeter Test Client> getTargetValue Vehicle.TripMeterReading
Clone this wiki locally