After upgrade from 2.16.1 to 2.18, pool pointing to wrong endpoint

[jhennf5]

Setup Details

CIS Version : 2.16.1 -> 2.18.0
Build: f5networks/k8s-bigip-ctlr:latest
BIGIP Version: Big IP 16.1.4.3
AS3 Version: 3.x
Agent Mode: AS3/CCCL
Orchestration: K8S/OSCP
Orchestration Version:
Pool Mode: Cluster
Additional Setup details: OpenShift

Description

F5 Case 00705492
Data sanitized

Service

Port: test-web 8080/TCP
TargetPort: 8080/TCP
NodePort: test-web 32348/TCP
Endpoints: 10.10.10.10:8080
Port: test-web-ssl 443/TCP
TargetPort: 8443/TCP
NodePort: test-web-ssl 31525/TCP
Endpoints: 10.10.10.10:8443
Session Affinity: None

Config Map

        "class": "Pool",
        "monitors": ["https"],
             "members": [
              {
                "servicePort": 8443,
                "serverAddresses": []
              }
          ]
         },

2.16.1 Controller logs

[DEBUG] [CORE] Discovered members for service test/test-console-svc is [{10.10.10.10 8443 443 0} {10.10.10.10 8080 8080 0}]

2.18.0 Controller logs

[DEBUG] [CORE] Discovered members for service test/test-console-svc is [{10.10.10.10 8443 443 enable 0 0} {10.10.10.10 8080 8080 enable 0 0}]

Behavior on upgrade to 2.18.0:

obj_delete { pool_member { pool_member_pool_name "/test-configmap/test/test-console-svc" pool_member_node_name "/test-configmap/10.10.10.10" pool_member_port 8443 } } [Status=Command OK]
create_if { pool_member { pool_member_pool_name "/test-configmap/test/test-console-svc" pool_member_node_name "/test-configmap/10.10.10.10" pool_member_port 8080 pool_member_new_session_enable 2 pool_member_monitor_state 3 pool_member_rate_limit 0 pool_member_dynamic_ratio 1 pool_member_priority 0 pool_member_ratio 1 pool_member_conn_limit 0 pool_member_autoscale 0 } } [Status=Command OK]

Behavior on revert to 2.16.1:

obj_delete { pool_member { pool_member_pool_name "/test-configmap/test/test-console-svc" pool_member_node_name "/test-configmap/10.10.10.10" pool_member_port 8080 } } [Status=Command OK]
create_if { pool_member { pool_member_pool_name "/test-configmap/test/test-console-svc" pool_member_node_name "/test-configmap/10.10.10.10" pool_member_port 8443 pool_member_new_session_enable 2 pool_member_monitor_state 3 pool_member_rate_limit 0 pool_member_dynamic_ratio 1 pool_member_priority 0 pool_member_ratio 1 pool_member_conn_limit 0 pool_member_autoscale 0 } } [Status=Command OK]

Controller flags:
- '--pool-member-type=cluster'
- '--insecure=true'
- '--log-as3-response=true'
- '--openshift-sdn-name=-----'
- '--namespace=-----'
- '--namespace=test'
- '--share-nodes=true'
- '--as3-post-delay=30'
- '--disable-teems=true'
- '--log-level=INFO'

Steps To Reproduce

1. Upgrade/revert controller

Expected Result

Pool member should use port 8443

Actual Result

Pool member uses port 8080

Diagnostic Information

Internal F5 data in case 00705492

[trinaths]

@jhennf5 Please share sample manifest YAML to reproduce this issue.

[trinaths]

Created [CONTCNTR-4947] for internal tracking

[jhennf5]

@jhennf5 Please share sample manifest YAML to reproduce this issue.

Shared internally

[arzzon]

@jhennf5
In this case, Port 8443 is used as the servicePort in the poolMemeber present in the AS3 configmap, where as the following ports are used in the service:
servicePort: 443
targetPort: 8443
As per the documentation mentioned below, the servicePort(8443) defined in the AS3 ConfigMap doesn't match the actual servicePort(443). So CIS takes the resource's port on index 0.
https://clouddocs.f5.com/containers/latest/userguide/config-map.html#cis-service-discovery
5. CIS performs service discovery using extracted tenant (Tenant-1), Application (APP1) and Pool (web_pool) details, and fetches service endpoints. If servicePort, defined in the AS3 ConfigMap, matches the actual servicePort in the service we add the members using targetPort in cluster mode. If servicePort, defined in the AS3 ConfigMap, does not match the actual servicePort in the service, we add the service’s corresponding endpoint and the resource’s port on index 0 to support backward compatibility.

Port 8443 was used by CIS 2.16.1 because in case when service ports don't match, CIS uses the port on index 0, however CIS 2.18.1 now maintains a sorted order for the pool members which has been done to fix an issue of multiple posting of declaration. So, the port at index 0 is now changed due to sorting and CIS used the port 8080 instead.

Please use port 443 for the pool member in the AS3 configMap, to ensure the service ports match and port 8443 is used for the pool member.