Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FMC ACP rule hit count - feature request #39

Open
NetDevAutomate opened this issue Aug 9, 2019 · 22 comments
Open

FMC ACP rule hit count - feature request #39

NetDevAutomate opened this issue Aug 9, 2019 · 22 comments

Comments

@NetDevAutomate
Copy link

Are there any plans to use the 6.4 features to be able to get ACP rule hit counts?
@daxm
Copy link
Collaborator

daxm commented Aug 11, 2019

Our goal is to eventually support access to ALL FMC API objects. That said, we do them as we can. Are you interested/willing to help the project out and work on that?

@NetDevAutomate
Copy link
Author

I can certainly try, I would love to be able to contribute as a great project!

@daxm
Copy link
Collaborator

daxm commented Sep 10, 2019

Any progress on this feature? If not, I'll give it a try.

@daxm
Copy link
Collaborator

daxm commented Sep 12, 2019

I've got HitCount to work for 'device' and 'fetchZeroHitCount' but the prefilter doesn't work because I don't have a Class for prefilter yet. It is in the Develop branch currently: https://github.com/daxm/fmcapi/tree/develop

@daxm
Copy link
Collaborator

daxm commented Sep 12, 2019

NOTE: Only for the GET method. I'm still working on PUT and DELETE methods. FYI

@NetDevAutomate
Copy link
Author

NetDevAutomate commented Sep 12, 2019 via email

@daxm
Copy link
Collaborator

daxm commented Sep 13, 2019

hitcount and prefilter_policy GET features are now in Master branch. However, hitcount also can do PUT or DELETE but I haven't written those features into fmcapi yet. Test it out and see what you think!

@NetDevAutomate
Copy link
Author

Do you have an example/unit test for hit count? There was nothing in the repo and the code base has had a pretty major refactor.

@daxm
Copy link
Collaborator

daxm commented Oct 26, 2019 via email

@daxm
Copy link
Collaborator

daxm commented Oct 27, 2019

Yes, we did a major refactor to better organize the code. Hopefully (other than trying to find things) this reorg didn't break your workflow!

I must have lost some code or "thought" HitCount was done. I just spent the day getting the GET features to work. There is still an issue that I can't seem to resolve (you can't add an AccessRule to the HitCount's filter via the rule's name... it provides/expects the wrong value!)
Anyway, here is the sample code: https://github.com/daxm/fmcapi/blob/master/unit_tests/hit_counts.py

Note: These changes aren't published to PyPi yet so you'll have to clone the repo to try them out.

@NetDevAutomate
Copy link
Author

NetDevAutomate commented Nov 4, 2019 •
edited
Loading

When I run the test, sadly I get:

WARNING:root:Either:
1. Payload too large. FMC can only handle a payload of 2048000 bytes.
2.The payload contains an unprocessable or unreadable entity such as a invalid attribut name or incorrect JSON syntax

The code is largely untouched but removed the add acp rule but did add a caching function as found it makes a huge difference when running more than one task on a requests dataset:

import logging
import fmcapi
import time
import requests_cache

# ### Set these variables to match your environment. ### #

host = ""
username = ""
password = ""
autodeploy = False
logname = "TestingUserScript.log"
pagelimit = 500
debug = False
device = ""
cache_name = "fmcacpi_cache"

requests_cache.install_cache(
    cache_name=cache_name, expire_after=21600
)

def main():
    with fmcapi.FMC(
        host=host,
        username=username,
        password=password,
        autodeploy=autodeploy,
        limit=pagelimit,
        file_logging=logname,
        debug=debug,
    ) as fmc1:
        logging.info("# ### Starting!!! ### #\n\n")
        hitcounts(fmc=fmc1, device_name=device)

def hitcounts(fmc, device_name="", prefilter_id=""):
    if not device_name and not prefilter_id:
        return f"Name of an actual device or prefilter ID is required for the HitCounts test to work... skipping test."
    # logging.info(
    #     "In preparation for testing HitCounts method, set up some known objects in the FMC."
    # )
    # starttime = str(int(time.time()))
    # namer = f"test__hitcounts_{starttime}"

    # Get the device
    device1 = fmcapi.DeviceRecords(fmc=fmc, name=device_name)
    device1.get()

    # In case there is no ACP Rule build a temp one.
    acprule1 = fmcapi.AccessRules(fmc=fmc, acp_id=device1.accessPolicy["id"])
    # acprule1 = fmcapi.AccessRules(fmc=fmc, acp_name=device1.accessPolicy['name'])
    # acprule1.name = namer
    # acprule1.action = "ALLOW"
    # acprule1.post()
    # time.sleep(1)
    acprule1.get()

    hitcounter1 = None
    if prefilter_id:
        hitcounter1 = fmcapi.HitCounts(
            fmc=fmc, prefilter_id=prefilter_id, device_name=device_name
        )
    else:
        hitcounter1 = fmcapi.HitCounts(
            fmc=fmc, acp_id=device1.accessPolicy["id"], device_name=device_name
        )
        """ 
        Searching for AccessRule by name returns the "correct" ID for the rule but HitCount shows a completely
        different ID so it doesn't match.
        # hitcounter1.acp_rules(action="add", name="Permit HQ LAN")
        If you know the ID that HitCount is looking for a specific rule:
        # hitcounter1.acp_rules(action="add", acp_rule_id="005056B5-44E6-0ed3-0000-000268434433")
        """
    for result in hitcounter1.get():
        print(result)

    logging.info("Test HitCount done.")

    logging.info("Cleanup of testing HitCount methods.")
    # acprule1.delete()
    time.sleep(1)
    logging.info("Cleanup of objects for HitCount test done.\n\n")

if __name__ == "__main__":
    main()

@daxm
Copy link
Collaborator

daxm commented Nov 4, 2019 •
edited
Loading

Note the """ quoted note near the end of the code. I haven't figured out why but the HitCount "ID" for the AccessRule doesn't match what the AccessRule's ID is.

Correction: You need to find the acp_rule_id that HitCount wants... which isn't the same thing as the AccessRule id.

@daxm
Copy link
Collaborator

daxm commented Nov 4, 2019

Check out the debug output for what is returned from GET'ting the AccessRule vs what the HitCounts sees as the id for that particular AccessRule.

@NetDevAutomate
Copy link
Author

NetDevAutomate commented Nov 4, 2019 •
edited
Loading

Same problem when using name when using:

acprule1 = fmcapi.AccessRules(fmc=fmc, acp_name=device1.accessPolicy['name'])

Log file:

2019/11/04-15:53:36 - INFO:apiclasstemplate.py:135 - GET success. Object with name: "<device name>" and id: "<device id>" fetched from FMC.
2019/11/04-15:53:36 - INFO:apiclasstemplate.py:135 - GET success. Object with name: "<policy name>" and id: "<policy id>" fetched from FMC.
2019/11/04-16:09:27 - INFO:apiclasstemplate.py:135 - GET success. Object with name: "<device name>" and id: "<device id>" fetched from FMC.
2019/11/04-16:09:27 - WARNING:fmc.py:245 - Either:
	1. Payload too large.  FMC can only handle a payload of 2048000 bytes.
	2.The payload contains an unprocessable or unreadable entity such as a invalid attribut name or incorrect JSON syntax
2019/11/04-16:09:27 - ERROR:fmc.py:253 - Error in POST operation --> 422 Client Error: Unprocessable Entity for url: https://<IP address>/api/fmc_config/v1/domain/<domain>/policy/accesspolicies/<ACP>/operational/hitcounts?filter=%22deviceId:<device id>%22&expanded=true&limit=500?filter=%22deviceId:<device id>%22&expanded=true&limit=500
2019/11/04-16:09:27 - ERROR:fmc.py:254 - json_response -->	{'error': {'category': 'VALIDATION', 'messages': [{'description': 'The limit in query is not a number.', 'code': 'invalidLimit', 'location': 'limit'}], 'severity': 'ERROR'}}
2019/11/04-16:09:27 - INFO:fmc.py:154 - Auto deploy changes set to False.  Use the Deploy button in FMC to push changes to FTDs.

@daxm
Copy link
Collaborator

daxm commented Nov 4, 2019 via email

@NetDevAutomate
Copy link
Author

NetDevAutomate commented Nov 4, 2019 •
edited
Loading

When using acprule1 = fmcapi.AccessRules(fmc=fmc, acp_name=device1.accessPolicy["name"])

The log shows the FTD device and ACP rule IDs, using acp_id=device1.accessPolicy["id"]) only the FTD device ID is found.

Neither option get's as far as showing HitCount logs.

ERROR:fmc.py:254 - json_response -->     {'error': {'category': 'VALIDATION', 'messages': [{'description': 'The limit in query is not a number.', 'code': 'invalidLimit', 'location': 'limit'}], 'severity': 'ERROR'}}

However the limit does appear to be a number:

ERROR:fmc.py:253 - Error in POST operation --> 422 Client Error: Unprocessable Entity for url: https://<snip>&expanded=true&limit=500

@daxm
Copy link
Collaborator

daxm commented Nov 5, 2019 via email

@NetDevAutomate
Copy link
Author

NetDevAutomate commented Nov 5, 2019 •
edited
Loading

Thank you so much, you have been incredibly helpful as always! Happy to perform any testing as where I am working we have large policies so a good test case.

@daxm
Copy link
Collaborator

daxm commented Nov 5, 2019 via email

@NetDevAutomate
Copy link
Author

I knocked up some rough python and used:

api_path = f"/api/fmc_config/v1/domain/{domain}/policy/accesspolicies/{acp}/operational/hitcounts?filter=%22deviceId%3A{deviceID}%3BfetchZeroHitCount%3A{ZeroCount}%22&expanded={expanded}&limit={LIMIT}"

acp = ACP ID

The code isn't pretty but did give me hit counts for each ACP entry.

When I set ZeroCount to true oddly enough got nothing, setting it to false I got everything including ZeroCount which is actually the only thing I was interested in.

@MysticRyuujin
Copy link
Contributor

MysticRyuujin commented Nov 17, 2020 •
edited
Loading

So I just tried to use this, and it works...but there are a few things wrong with it (and one thing wrong with the actual API itself...)

First what's wrong with the API itself? Using version 6.6.1 of the FMC, the hitcounts endpoint will not return more than 1,000 objects, even if the limit is set to be greater than 1,000. Additionally, it does not page. It will always say that the limit was set to 1,000 in the reply and will only return 1,000 objects. Basically if you send limit=2000, the response link to self will say limit=1000 and the paging property will say there's only 1 page.

Now, back to this module...

There are 2 things not working:

  1. There is no offset or index number available. This is important because of the issue above, if I get back 1,000 objects, there's a very good chance that there are more objects that I need to go get, but I can't because I can't set an offset/index. The module doesn't know there are more objects to get because the paging is broken 😄 so there needs to be a way to set an offset in the module...

  2. You cannot get an HA pair device using its name. I don't know if this is a bigger problem than just the hitcounts endpoint, but using the device_name only works for non-HA pairs. You must use the deviceId of the HA pair as the containerId

@Niltak
Copy link

Niltak commented Sep 15, 2022 •
edited
Loading

Offset was added in #127

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@daxm @MysticRyuujin @NetDevAutomate @Niltak