Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLOUDP-283086: POC validator with custom function #287

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions openapi/v2.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@
format: date-time
type: string
responses:
accepted:

Check warning on line 167 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
description: Accepted.
badRequest:
content:
Expand Down Expand Up @@ -199,7 +199,7 @@
schema:
$ref: '#/components/schemas/ApiError'
description: Forbidden.
gone:

Check warning on line 202 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
content:
application/json:
example:
Expand Down Expand Up @@ -232,7 +232,7 @@
schema:
$ref: '#/components/schemas/ApiError'
description: Method Not Allowed.
noBody:

Check warning on line 235 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
description: This endpoint does not return a response body.
notFound:
content:
Expand Down Expand Up @@ -517,7 +517,7 @@
type: string
title: AWS
type: object
AWSCreateDataProcessRegionView:

Check warning on line 520 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
allOf:
- $ref: '#/components/schemas/CreateDataProcessRegionView'
- properties:
Expand Down Expand Up @@ -549,7 +549,7 @@
required:
- enabled
type: object
AWSDataProcessRegionView:

Check warning on line 552 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
allOf:
- $ref: '#/components/schemas/DataProcessRegionView'
- properties:
Expand Down Expand Up @@ -709,7 +709,7 @@
type: integer
title: AWS Cluster Hardware Settings
type: object
AWSInterfaceEndpoint:

Check warning on line 712 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
description: Group of Private Endpoint settings.
properties:
cloudProvider:
Expand Down Expand Up @@ -826,7 +826,7 @@
readOnly: true
type: boolean
type: object
AWSPrivateLinkConnection:

Check warning on line 829 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
description: Group of Private Endpoint Service settings.
properties:
cloudProvider:
Expand Down Expand Up @@ -1482,7 +1482,7 @@
example: ALERT_CONFIG_ADDED_AUDIT
title: Alert Audit Types
type: string
AlertConfigView:

Check warning on line 1485 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
description: Alert settings allows to select which conditions trigger alerts and how users are notified.
properties:
created:
Expand Down Expand Up @@ -1856,7 +1856,7 @@
- TOKYO_JPN
- SINGAPORE_SGP
type: string
ApiAtlasDataLakeStorageView:

Check warning on line 1859 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
$ref: '#/components/schemas/DataLakeStorage'
ApiAtlasFTSAnalyzersViewManual:
description: Settings that describe one Atlas Search custom analyzer.
Expand Down Expand Up @@ -4245,7 +4245,7 @@
type: string
title: Azure
type: object
AzureCreateDataProcessRegionView:

Check warning on line 4248 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
allOf:
- $ref: '#/components/schemas/CreateDataProcessRegionView'
- properties:
Expand All @@ -4257,7 +4257,7 @@
type: string
type: object
type: object
AzureDataProcessRegionView:

Check warning on line 4260 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
allOf:
- $ref: '#/components/schemas/DataProcessRegionView'
- properties:
Expand Down Expand Up @@ -4589,7 +4589,7 @@
- vnetName
title: AZURE
type: object
AzurePrivateEndpoint:

Check warning on line 4592 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
description: Group of Private Endpoint settings.
properties:
cloudProvider:
Expand Down Expand Up @@ -4635,7 +4635,7 @@
- cloudProvider
title: AZURE
type: object
AzurePrivateLinkConnection:

Check warning on line 4638 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
description: Group of Private Endpoint Service settings.
properties:
cloudProvider:
Expand Down Expand Up @@ -5682,7 +5682,7 @@
- $ref: '#/components/schemas/ApiStreamsAWSRegionView'
- $ref: '#/components/schemas/ApiStreamsAzureRegionView'
type: object
BasicBSONList:

Check warning on line 5685 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
description: List that contains the search criteria that the query uses. To use the values in key-value pairs in these predicates requires **Project Data Access Read Only** permissions or greater. Otherwise, MongoDB Cloud redacts these values.
items:
description: List that contains the search criteria that the query uses. To use the values in key-value pairs in these predicates requires **Project Data Access Read Only** permissions or greater. Otherwise, MongoDB Cloud redacts these values.
Expand Down Expand Up @@ -6945,7 +6945,7 @@
required:
- providerName
type: object
CloudProviderAccessAWSIAMRoleRequestUpdate:

Check warning on line 6948 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
allOf:
- $ref: '#/components/schemas/CloudProviderAccessRoleRequestUpdate'
- properties:
Expand Down Expand Up @@ -7102,7 +7102,7 @@
required:
- providerName
type: object
CloudProviderAccessAzureServicePrincipalRequestUpdate:

Check warning on line 7105 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
allOf:
- $ref: '#/components/schemas/CloudProviderAccessRoleRequestUpdate'
- properties:
Expand Down Expand Up @@ -7153,7 +7153,7 @@
required:
- providerName
type: object
CloudProviderAccessDataLakeFeatureUsage:

Check warning on line 7156 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
allOf:
- $ref: '#/components/schemas/CloudProviderAccessFeatureUsage'
- properties:
Expand All @@ -7162,7 +7162,7 @@
type: object
description: Details that describe the Atlas Data Lakes linked to this Amazon Web Services (AWS) Identity and Access Management (IAM) role.
type: object
CloudProviderAccessEncryptionAtRestFeatureUsage:

Check warning on line 7165 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
allOf:
- $ref: '#/components/schemas/CloudProviderAccessFeatureUsage'
- properties:
Expand All @@ -7171,7 +7171,7 @@
type: object
description: Details that describe the Key Management Service (KMS) linked to this Amazon Web Services (AWS) Identity and Access Management (IAM) role.
type: object
CloudProviderAccessExportSnapshotFeatureUsage:

Check warning on line 7174 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
allOf:
- $ref: '#/components/schemas/CloudProviderAccessFeatureUsage'
- properties:
Expand Down Expand Up @@ -7251,7 +7251,7 @@
readOnly: true
type: string
type: object
CloudProviderAccessGCPServiceAccount:

Check warning on line 7254 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
allOf:
- $ref: '#/components/schemas/CloudProviderAccessRole'
- properties:
Expand Down Expand Up @@ -7317,14 +7317,14 @@
required:
- providerName
type: object
CloudProviderAccessGCPServiceAccountRequestUpdate:

Check warning on line 7320 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
allOf:
- $ref: '#/components/schemas/CloudProviderAccessRoleRequestUpdate'
description: Details that describe the features linked to the GCP Service Account.
required:
- providerName
type: object
CloudProviderAccessPushBasedLogExportFeatureUsage:

Check warning on line 7327 in openapi/v2.yaml

View workflow job for this annotation

GitHub Actions / Lint (pull_request)

oas3-unused-component

Potentially unused component has been detected.
allOf:
- $ref: '#/components/schemas/CloudProviderAccessFeatureUsage'
- properties:
Expand Down Expand Up @@ -47360,6 +47360,9 @@
- Service Accounts
x-xgen-owner-team: apix
/api/atlas/v2/groups/{groupId}/serviceAccounts/{clientId}/accessList:
x-xgen-IPA-exception:
xgen-IPA-104-resource-has-GET:
reason: "Testing"
get:
description: Returns all access list entries that you configured for the specified Service Account for the project. Available as a preview feature.
operationId: listProjectServiceAccountAccessList
Expand Down Expand Up @@ -47485,6 +47488,9 @@
- Service Accounts
x-xgen-owner-team: apix
/api/atlas/v2/groups/{groupId}/serviceAccounts/{clientId}/secrets:
x-xgen-IPA-exception:
xgen-IPA-104-resource-has-GET:
reason: "Testing"
post:
description: Create a secret for the specified Service Account in the specified Project. Available as a preview feature.
operationId: createProjectServiceAccountSecret
Expand Down Expand Up @@ -51065,6 +51071,9 @@
- Service Accounts
x-xgen-owner-team: apix
/api/atlas/v2/orgs/{orgId}/serviceAccounts/{clientId}/accessList:
x-xgen-IPA-exception:
xgen-IPA-104-resource-has-GET:
reason: "Testing"
get:
description: Returns all access list entries that you configured for the specified Service Account for the organization. Available as a preview feature.
operationId: listServiceAccountAccessList
Expand Down Expand Up @@ -51226,6 +51235,9 @@
- Service Accounts
x-xgen-owner-team: apix
/api/atlas/v2/orgs/{orgId}/serviceAccounts/{clientId}/secrets:
x-xgen-IPA-exception:
xgen-IPA-104-resource-has-GET:
reason: "Testing"
post:
description: Create a secret for the specified Service Account. Available as a preview feature.
operationId: createServiceAccountSecret
Expand Down
2,722 changes: 2,613 additions & 109 deletions package-lock.json

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@
"scripts": {
"format": "npx prettier . --write",
"format-check": "npx prettier . --check",
"lint-js": "npx eslint **/*.js"
"lint-js": "npx eslint **/*.js",
"ipa-validation": "spectral lint ./openapi/v2.yaml --ruleset=./tools/ipa/ipa-spectral.yaml -v"
},
"dependencies": {
"@stoplight/spectral-cli": "^6.14.2",
"openapi-to-postmanv2": "4.24.0"
},
"devDependencies": {
Expand Down
2 changes: 2 additions & 0 deletions tools/ipa/ipa-spectral.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
extends:
- ./rulesets/IPA-104.yaml
15 changes: 15 additions & 0 deletions tools/ipa/rulesets/IPA-104.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# IPA-104: Get
# http://go/ipa/104

functions:
- eachResourceHasGetMethod

rules:
xgen-IPA-104-resource-has-GET:
description: "APIs must provide a get method for resources. http://go/ipa/104"
message: "{{error}} http://go/ipa/117"
severity: error
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning :P

given: "$.paths"
then:
field: "@key"
function: "eachResourceHasGetMethod"
47 changes: 47 additions & 0 deletions tools/ipa/rulesets/functions/eachResourceHasGetMethod.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
import { hasExemption } from './utils/exemptions.js';
import {
hasGetMethod,
isChild,
isCustomMethod,
isStandardResource,
isSingletonResource,
getResourcePaths,
} from './utils/resourceEvaluation.js';

const RULE_NAME = 'xgen-IPA-104-resource-has-GET';
const ERROR_MESSAGE = 'APIs must provide a get method for resources.';

export default (input, _, context) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we document what version of ES can we use?

if (isChild(input) || isCustomMethod(input)) {
return;
}

const oas = context.documentInventory.resolved;
const resourceObject = oas.paths[input];

if (hasExemption(RULE_NAME, resourceObject)) {
return;
}

const resourcePaths = getResourcePaths(input, Object.keys(oas.paths));

if (isSingletonResource(resourcePaths)) {
// Singleton resource, may have custom methods
if (!hasGetMethod(oas.paths[resourcePaths[0]])) {
return [
{
message: ERROR_MESSAGE,
},
];
}
} else if (isStandardResource(resourcePaths)) {
// Normal resource, may have custom methods
if (!hasGetMethod(oas.paths[resourcePaths[1]])) {
return [
{
message: ERROR_MESSAGE,
},
];
}
}
};
15 changes: 15 additions & 0 deletions tools/ipa/rulesets/functions/utils/exemptions.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
const EXEMPTION_EXTENSION = 'x-xgen-IPA-exception';

/**
* Checks if the object has an exemption set for the passed rule name by checking
* if the object has a field "x-xgen-IPA-exception" containing the rule as a
* field.
*
* @param ruleName the name of the exemption
* @param object the object to evaluate
* @returns {boolean}
*/
export function hasExemption(ruleName, object) {
const exemptions = object[EXEMPTION_EXTENSION];
return exemptions !== undefined && Object.keys(exemptions).includes(ruleName);
}
63 changes: 63 additions & 0 deletions tools/ipa/rulesets/functions/utils/resourceEvaluation.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
export function isChild(path) {
return path.endsWith('}');
}

export function isCustomMethod(path) {
return path.includes(':');
}

/**
* Checks if a resource is a singleton resource based on the paths for the
* resource. The resource may have custom methods.
*
* @param resourcePaths all paths for the resource as an array of strings
* @returns {boolean}
*/
export function isSingletonResource(resourcePaths) {
if (resourcePaths.length === 1) {
return true;
}
const additionalPaths = resourcePaths.slice(1);
return !additionalPaths.some((p) => !isCustomMethod(p));
}

/**
* Checks if a resource is a standard resource based on the paths for the
* resource. The resource may have custom methods.
*
* @param resourcePaths all paths for the resource as an array of strings
* @returns {boolean}
*/
export function isStandardResource(resourcePaths) {
if (resourcePaths.length === 2 && isChild(resourcePaths[1])) {
return true;
}
if (resourcePaths.length < 3 || !isChild(resourcePaths[1])) {
return false;
}
const additionalPaths = resourcePaths.slice(2);
return !additionalPaths.some((p) => !isCustomMethod(p));
}

/**
* Checks if a path object has a GET method
*
* @param pathObject the path object to evaluate
* @returns {boolean}
*/
export function hasGetMethod(pathObject) {
return Object.keys(pathObject).some((o) => o === 'get');
}

/**
* Get all paths for a resource based on the parent path
*
* @param parent the parent path string
* @param allPaths all paths as an array of strings
* @returns {*} a string array of all paths for a resource, including the parent
*/
export function getResourcePaths(parent, allPaths) {
const childPathPattern = new RegExp(`^${parent}/{[a-zA-Z]+}$`);
const customMethodPattern = new RegExp(`^${parent}/{[a-zA-Z]+}:+[a-zA-Z]+$`);
return allPaths.filter((p) => parent === p || childPathPattern.test(p) || customMethodPattern.test(p));
}
Loading