Fully updated readme file

nareshnavinash · nareshnavinash · commit 74834ed1dd99 · 2020-02-26T19:26:27.000+05:30
diff --git a/Library/api.py b/Library/api.py
@@ -126,5 +126,14 @@ def get_params_from_response(path):
         path_array = path.split(",")
         result = Store.current_response.json()
         for key in path_array:
-            result = result[key]
+            actual_key = Api.actual_key(key)
+            result = result[actual_key]
+        return result
+
+    @staticmethod
+    def actual_key(key):
+        try:
+            result = int(key)
+        except:
+            result = key
         return result
diff --git a/README.md b/README.md
@@ -40,11 +40,11 @@ For a simple run of all the test files in normal mode, try
 pytest
 ```
 
-To run the tests in snap mode (to save the UI texts to the dynamic file)
+To run the tests in snap mode (to save the Response JSON to the test data file or to save the response image to the stored image)
 ```
 snap=1 pytest
 ```
-Once the changes are saved to the file run the tests with `pytest` to get the test running against the saved data. To verify this feature I intentionally added two locator texts which will be changing continuously.
+Once the changes are saved to the file run the tests with `pytest` to get the test running against the saved data.
 
 To Run the tests in parallel mode or multi thread run for the available test files, try (To have parallel run you need to have atleast 2 tests inside your folder structure)
 
@@ -68,15 +68,15 @@ allure serve reports/allure
 
 
 ## Reports
-For better illustration on the testcases, allure reports has been integrated. Allure reports can also be integrated with jenkins to get a dashboard view. Apart from allure, pytest's default reporting such as html file has been added to the `reports/` folder.
+For better illustration on the testcases, allure reports has been integrated. Allure reports can also be integrated with jenkins to get a dashboard view. Apart from allure, pytest's default reporting html file has been added to the `reports/` folder.
 
-If there is a failure while comparing the images, allure report will have all the files attached to it. The difference between the two images is generated in run time and attached to the allure report for our referenece.
+If there is a failure while comparing the images, allure report will have all the files attached to it. The difference between the two images is generated in run time and attached to the allure report for our reference.
 
 ![alt text](Library/diff_image.png)
  
 
 ## Jenkins Integration with Docker images
-Get any of the linux with python docker image as the slaves in jenkins and use the same for executing the UI automation with this framework (Sample docker image - `https://hub.docker.com/_/python`). From the jenkins bash Execute the following to get the testcases to run,
+Get any of the linux with python docker image as the slaves in jenkins and use the same for executing tests with this framework (Sample docker image - `https://hub.docker.com/_/python`). From the jenkins bash Execute the following to get the testcases to run,
 
 ```
 #!/usr/bin/python3
@@ -104,18 +104,159 @@ pipeline {
 }
 ```
 
+# Break down into end to end tests
 
+## Creating a test file
 
+* Tests can be created directly within the `Tests/` folder with the file prefix as `test_` so that those files alone will be taken during test run. This is configured in `pytest.ini` file.
 
+```
+[pytest]
+markers =
+    sanity: sanity tests marker
+    regression: regression tests marker
+    snap: Snap feature enabled for this case, should have separate file for validating the response
+    plain: Snap feature is not recommended since the expected JSON has some custom values
+python_files=*.py
+python_functions=test_*
+addopts = -rsxX
+          -q
+          -v
+          --self-contained-html
+          --html=reports/html_report.html
+          --cov=Tests
+          --alluredir reports/allure
+          --clean-alluredir
+``` 
+
+* Do import the needed modules inside the test file. Since we have imitated karate framework's approach of testing, we actually need to just use the commands to test the REST API endpoints. Those commands and the features can be discussed below.
+
+```
+import allure
+import pytest
+from Library.api import Api
+from Library.images import Img
+```
+
+* Do set the URL on which you want your automation suite to run in the `/Data/GlobalData/global_data.yml` file. You can also set add other project level data in this file and then call those by using `Var` method.
+
+```
+URL: https://naresh.free.beeceptor.com
+timeout: 10
+tolerance: 0.01
+```  
+In this project I have set the URL on which automation is going to run, maximum timeout which is allowed, and the tolerance which is allowed while comparing the images.
+
+In order to change the URL against which the suite is running, one could always set the environment variable while executing the suite. Always environment variable gets the higher precedence, so even if we have URL set in the global variable data, the URL which we give from command line will be taken for execution.
+```
+URL=https://customurl.inruntime.com pytest
+``` 
+
+* While starting to draft a test case, do add the following tags to the test case, which will be helpful in reporting part.
+
+```
+@allure.feature("Sample get request") # Title for the test case
+@allure.severity('Critical') # Set the severity for the case
+@pytest.mark.regression  # Custom pytest marker to run the test cases with ease on demand
+@pytest.mark.snap  # Custom pytest marker to run the test cases with ease on demand
+``` 
+
+### Simple test case with an endpoint
+
+For a very simple basic get request and to validate the response code we could do,
+
+```
+Api.get("/name")
+Api.verify_response_code(200)
+```
+On calling only these two methods from the `Api` library, all the allure report actions, attaching the request and the response file to the reports, and asserting the response code of the response is taken care off.
+
+### Simple test case with validating the response with test data
+
+To validate the response json with a test data, one could do the following,
+
+```
+Api.get("/name")
+Api.verify_response_code(200)
+Api.verify_response_json("sample.yml", "test_sample_get_request_001")
+```
+Here, we are trying to take the sample.yml file under `/Data/DynamicData/` folder and then fetch the data for the key `test_sample_get_request_001`. 
+After getting the data from the stored file, we will compare that with the response data and generate the allure reports along with necessary attachments.
+
+The YAML file will be looking like,
+```
+test_sample_get_request_001:
+  age: 20
+  name: Naresh
+```
+
+While fetching the key from a yaml file, the above file structure will return the data in JSON format. This in turn gives us the edge while creating the test data. One can always save the key value in direct JSON format as well.
+
+```
+test_sample_get_request_001:
+  age: 20
+  name: Naresh
+
+OR
+
+test_sample_get_request_001: { "age": 20, "name": "Naresh" }
+```
+In either way JSON parser will get the values in JSON format. Whereas when we use `snap` mode, the file will be saved in the first format which we can see in detail below.
 
+### Simple test case with validating the response with test data and ignoring few keys
 
+While validating an api response, we may encounter a scenario where we don't want to validate few keys. In such scenario one can do the following,
+```
+Api.get("/name")
+Api.verify_response_code(200)
+Api.ignore_keys("age")
+Api.verify_response_json("sample.yml", "test_sample_get_request_001")
+```
+The above code will validate the response status code, response json values except `age` key. If you want to have more keys that are supposed to be ignored, have that in the comma separated format,
+```
+Api.get("/name")
+Api.verify_response_code(200)
+Api.ignore_keys("age,name")
+Api.verify_response_json("sample.yml", "test_sample_get_request_001")
+```
+This will ignore the keys `age` and `name` while validating the response with the stored data.
 
+### Simple test case with validating the response with test data and custom markers
 
+While validating an api response, we may encounter a scenario where we need to validate whether a key is present or not but not the value for that key. In that case one can always have that marked in their test data with the unique markers specified with `$` symbol.
 
+Test File:
+```
+Api.get("/name")
+Api.verify_response_code(200)
+Api.verify_response_json("sample.yml", "test_sample_get_request_001")
+```
+Data File:
+```
+test_sample_get_request_001:
+  age: $notnull
+  name: Naresh
+```
+The above combination will validate the response as,
+1. Whether `age` key is present without Null value in it.
+2. And `name` is present with the exact same value `Naresh` in it.
 
+We can also make the validation so specific for the `age` field in the above example by mentioning that value corresponds to `age` should be a `number`. To achieve this we need to have the following combination.
 
+Test File:
+```
+Api.get("/name")
+Api.verify_response_code(200)
+Api.verify_response_json("sample.yml", "test_sample_get_request_001")
+```
+Data File:
+```
+test_sample_get_request_001:
+  age: $number
+  name: Naresh
+```
 
-The supported markers are the following:
+Apart from the above two there are multiple markers available which are listed as follows,
 
 Marker | Description
 ------ | -----------
@@ -127,15 +268,52 @@ Marker | Description
 `$string` | Expects actual value to be a string
 `$uuid` | Expects actual (string) value to conform to the UUID format
  
+### Test cases with validation of images
+
+In few scenarios if we need to validate the image file from the response, first we need to hit the endpoint and get the image URL, after which we need to download the image from the URL and store that in temporary folder, and then compare the image with the stored image. To do this,
+```
+Api.get("/image")
+Api.verify_response_code(200)
+Api.verify_response_json("sample.yml", "test_sample_get_request_003")
+image_url = Api.get_params_from_response("image")
+Img.download_image(image_url, "downloaded_file")
+Img.is_equal("Naresh", "downloaded_file")
+``` 
+The above code will save a value from the response json through `Api.get_params_from_response`. If the URL is present inside the nested json one can always give the path to the image url using comma separated value like,
+```
+Api.get_params_from_response("image,0,user,profile,image")
+```
+After getting the Image URL, we need to download it and save it in the temporary folder under `reports/images`. We are also supposed to send the name for the downloading image file. All the download and comparison of images are happening in png format. We need a change in framework if we want to compare images with some other format.
+
+Now after downloading, directly give the image name against which we need to compare the downloaded image. The stored image must be under the folder `/Data/Images/`.
+
+The method `Img.is_equal` takes care of all the allure reporting part, attaching the images to the report and if there is a mismatch between the images, difference between two images also will be attached to the allure report. as mentioned in the above allure report topic. 
+
+### Test cases with validation of images along with tolerance
+
+In few scenarios if we need to validate the image file from the response along with the allowed tolerance. The above method will result in failure even if there is a minute change in the image file. To validate the images along with tolerance one has to change,
+
+From: 
+```
+Img.is_equal("Naresh", "downloaded_file")
+``` 
+To:
+```
+Img.is_equal_with_tolerance("Naresh", "downloaded_file")
+```
+This will take the tolerance level from the global data file and validate. Its always recommended to use same tolerance level across the project, but in few cases if one need to have custom tolerance level to an image compare one has to do,
+```
+Img.is_equal_with_tolerance("Naresh", "downloaded_file", 0.5)
+```
+The above code will validate the images with 0.5 percent tolerance level.
+
  
- 
- 
-### Data sets:
+## Data sets:
 
 In order to have distinguished set of data I have used three types of data.
 
 * **Global** - Global configuration for the whole project. Here mode of run, browsers to use, browser configurations etc., are specified.
-* **Test Data** - This is to store the module level data. Ideally for each test file we need to have a test data file, but that depends on the requirement.
+* **Static Data** - This is to store the module level data. Ideally for each test file we need to have a test data file, but that depends on the requirement.
 * **Dynamic Data** - This is to store the dynamic data. Files in this folder are supposed to change when we run with `snap=1 pytest`. This is separated from the other data files so that other static files are not disturbed during the run.
 * **Images** - This folder is to store all the image files that are needed to compare with the response Image files 
 
diff --git a/Tests/test_sample.py b/Tests/test_sample.py
