Skip to content

Update Cortex Xpanse asm-list-external-websites command to include pa… #40140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: contrib/zdrouse_zdrouse
Choose a base branch
from

Conversation

zdrouse
Copy link
Contributor

@zdrouse zdrouse commented May 29, 2025

…gination support.

Contributing to Cortex XSOAR Content

Make sure to register your contribution by filling the contribution registration form

The Pull Request will be reviewed only after the contribution registration form is filled.

Status

  • In Progress
  • Ready
  • In Hold - (Reason for hold)

Related Issues

Currently, asm-list-external-websites can only ever retrieve the first 500 websites and organizations have way more than that.

Description

Update asm-list-external-websites command to allow for pagination of requests.

API Reference: https://docs-cortex.paloaltonetworks.com/r/Cortex-Xpanse-REST-API/Get-All-Websites

Must have

  • Tests
  • Documentation

@content-bot content-bot added Contribution Thank you! Contributions are always welcome! External PR Xsoar Support Level Indicates that the contribution is for XSOAR supported pack labels May 29, 2025
@content-bot content-bot changed the base branch from master to contrib/zdrouse_zdrouse May 29, 2025 19:29
@content-bot content-bot requested a review from itssapir May 29, 2025 19:29
@content-bot
Copy link
Collaborator

Thank you for your contribution. Your generosity and caring are unrivaled! Make sure to register your contribution by filling the Contribution Registration form, so our content wizard @itssapir will know the proposed changes are ready to be reviewed.
For your convenience, here is a link to the contributions SLAs document.

@content-bot
Copy link
Collaborator

Hi @zdrouse, thanks for contributing to the XSOAR marketplace. To receive credit for your generous contribution please follow this link.

@content-bot content-bot added Community Contribution Form Filled Whether contribution form filled or not. labels May 29, 2025
@johnnywilkes
Copy link
Contributor

@zdrouse , thank you for the submission. What were your thoughts on using this? With raw-response=true only?

@zdrouse
Copy link
Contributor Author

zdrouse commented Jun 5, 2025

@zdrouse , thank you for the submission. What were your thoughts on using this? With raw-response=true only?

@johnnywilkes you are asking my thoughts on giving this command support for paging? We need to retrieve all of the external websites in Xpanse from XSOAR. Without paging support, we cannot do this. From there, a subplaybook or automation can be built to repeatedly call the command using the next_page_token until the last page is reached.

@johnnywilkes
Copy link
Contributor

@zdrouse , I understand why you want paging support. However, I am not sure with your change how it is usable. The command doesn't seem to expose next_page_token to context. Can you explain how you are using the command as-is?

If not, please look into changes to add next_page_token to context

…r ExternalWebsiteReply that contains contextual information about the request for external websites.
@cah-zachary-rouse
Copy link
Contributor

@johnnywilkes I see. Yes, usually when I am using commands on playbook tasks I typically utilize extend-context with the raw response in order to see all the data or pick and choose what I want from the raw response. I find often that I cannot rely on the implemented outputs of the command results because it is either missing data or doesn't provide it in the format I need which I suppose is the point you are making. I've adjusted the output in the integration for this command to provide another node in context for this information. I had to adjust the outputs_prefix in order to preserve the existing context expected from the command.

@cah-zachary-rouse
Copy link
Contributor

Screenshot 2025-06-06 at 12 57 38 PM

Adding a screenshot of context adjustments for reference.

@johnnywilkes
Copy link
Contributor

@johnnywilkes
Copy link
Contributor

@zdrouse , reaching out on DFIR slack

@johnnywilkes
Copy link
Contributor

Tests failing because context output change, reaching out on DFIR

… since we are unable to change existing functionality of outputs_prefix of the original CommandResults. The new CommandResults object provides an additional about with context information about the request including next page token for pagination, the total results and the result count from the request. CommandResults is also using so that context is overwritten with the most recent command execution per recommended usage of .
@zdrouse
Copy link
Contributor Author

zdrouse commented Jun 18, 2025

@johnnywilkes updated to include an additional CommandResults object for ASM.ExternalWebsiteReply which is also using replace_existing=True per recommendations from CommandResults.

Works only if outputs_prefix is a path to a nested value i.e., contains a period.

@johnnywilkes
Copy link
Contributor

@zdrouse , seeing an issue, will message on slack

@johnnywilkes
Copy link
Contributor

seems to be underlying API issue that first response returns one less form limit:
testing commands and screenshots:
!asm-list-external-websites limit=2 use_page_token=true
!asm-list-external-websites next_page_token=
!asm-list-external-websites next_page_token=

image image image Token changes everytime

Copy link
Contributor

@johnnywilkes johnnywilkes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Packs/CortexXpanse/Integrations/CortexXpanse/CortexXpanse_test.py
returns from list_external_websites_command() are now a list so need to change

    assert response.outputs == EXTERNAL_WEBSITES_RESULTS.get("ExternalWebsite", {}).get("websites")
    assert response.outputs_prefix == "ASM.ExternalWebsite"

to

    assert response[0].outputs == EXTERNAL_WEBSITES_RESULTS.get("ExternalWebsite", {}).get("websites")
    assert response[0].outputs_prefix == "ASM.ExternalWebsite"

@@ -1120,25 +1120,33 @@ def get_incident_command(client: Client, args: dict[str, Any]) -> CommandResults

def list_external_websites_command(client: Client, args: Dict[str, Any]) -> CommandResults:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def list_external_websites_command(client: Client, args: Dict[str, Any]) -> CommandResults:
def list_external_websites_command(client: Client, args: Dict[str, Any]) -> list[CommandResults]:

for type checking


Args:
client (Client): CortexXpanse client to use.
args (dict): all command arguments, usually passed from ``demisto.args()``.
``args['filter']`` Used for filter websites based on authentication type
``args['limit']`` Used for limit num of results
``args['use_page_token']`` Boolean value used to specify pagination
``args['next_page_token']`` String value of a page token to get the next page of results

Returns:
CommandResults: A ``CommandResults`` object that is then passed to ``return_results``
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
CommandResults: A ``CommandResults`` object that is then passed to ``return_results``
CommandResults: A list of `CommandResults` objects that is then passed to ``return_results``

update description

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Community Contribution Form Filled Whether contribution form filled or not. Contribution Thank you! Contributions are always welcome! External PR Xsoar Support Level Indicates that the contribution is for XSOAR supported pack
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants