xdsclient: update watcher API as per gRFC A88

[purnesh42H]

This is the first part of implementing gRFC A88 (grpc/proposal#466).

This introduces the new watcher API but does not change any of the existing behavior. This table summarizes the API changes and behavior for each case:

Case	Old API	New API	Behavior
Resource timer fires	OnResourceDoesNotExist()	OnResourceChanged(NOT_FOUND)	Fail data plane RPCs
LDS or CDS resource deletion	OnResourceDoesNotExist()	OnResourceChanged(NOT_FOUND)	Drop resource and fail data plane RPCs
xDS channel reports TRANSIENT_FAILURE	OnError()	OnResourceChanged(status) if resource NOT already cached; OnAmbientError(status) otherwise	Continue using cached resource, if any; otherwise, fail data plane RPCs
ADS stream terminates without receiving a response	OnError()	OnResourceChanged(status) if resource NOT already cached; OnAmbientError(status) otherwise	Continue using cached resource, if any; otherwise, fail data plane RPCs
Invalid resource update (client NACK)	OnError()	OnResourceChanged(status) if resource NOT already cached; OnAmbientError(status) otherwise	Continue using cached resource, if any; otherwise, fail data plane RPCs
Valid resource update	OnUpdate(resource)	OnResourceChanged(resource)	use the new resource

RELEASE NOTES:

• xds: TBD

[codecov]

Codecov Report

Attention: Patch coverage is 81.36646% with 30 lines in your changes missing coverage. Please review.

Project coverage is 82.45%. Comparing base (724f450) to head (b9d2a92).
Report is 44 commits behind head on master.

Files with missing lines	Patch %	Lines
xds/internal/testutils/resource_watcher.go	20.00%	11 Missing and 1 partial ⚠️
xds/internal/server/listener_wrapper.go	42.10%	9 Missing and 2 partials ⚠️
.../balancer/clusterresolver/resource_resolver_eds.go	78.57%	2 Missing and 1 partial ⚠️
xds/internal/resolver/xds_resolver.go	66.66%	2 Missing ⚠️
xds/internal/xdsclient/clientimpl_watchers.go	33.33%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7977      +/-   ##
==========================================
+ Coverage   82.28%   82.45%   +0.17%     
==========================================
  Files         381      388       +7     
  Lines       38539    39048     +509     
==========================================
+ Hits        31712    32198     +486     
+ Misses       5535     5530       -5     
- Partials     1292     1320      +28     

Files with missing lines	Coverage Δ
xds/internal/balancer/cdsbalancer/cdsbalancer.go	81.69% <100.00%> (-0.07%)	⬇️
...s/internal/balancer/cdsbalancer/cluster_watcher.go	100.00% <100.00%> (ø)
xds/internal/resolver/watch_service.go	100.00% <100.00%> (+8.57%)	⬆️
xds/internal/server/rds_handler.go	90.69% <100.00%> (-0.61%)	⬇️
xds/internal/xdsclient/authority.go	77.21% <100.00%> (+0.34%)	⬆️
...nal/xdsclient/xdsresource/cluster_resource_type.go	78.37% <100.00%> (+1.23%)	⬆️
...l/xdsclient/xdsresource/endpoints_resource_type.go	76.47% <100.00%> (+1.47%)	⬆️
...al/xdsclient/xdsresource/listener_resource_type.go	86.44% <100.00%> (+0.47%)	⬆️
...ds/internal/xdsclient/xdsresource/resource_type.go	100.00% <ø> (ø)
...dsclient/xdsresource/route_config_resource_type.go	76.47% <100.00%> (+1.47%)	⬆️
... and 5 more

... and 58 files with indirect coverage changes

[easwars]

This also depends on a number of things like the server feature being set etc. So, I think we can get rid of this block as well. Somewhere in ResourceWatcher documentation, we should link to A88 (probably at the top) and say that it contains an exhaustive list of what method is invoked under what conditions.

[easwars]

Please update the documentation of the individual watchers as well based on the two comments here. Thanks.

[purnesh42H]

I have removed it. I have mentioned gRFC A88 at the top to refer for exhaustive list. I saw other places in code where we just mention the grfc and don't link. Also, the proposal is not merged yet.

One more i felt worth mentioning is about whether to continue reusing the existing resource or not. For OnAmbientError(), they may continue using it.

[easwars]

The language from the gFRC is much stricter than yours:

In particular, the watcher should not stop using the previously seen resource, and the XdsClient will not remove the resource from its cache

So, please mention that the watcher should not stop using the previously seen resource and not the watcher may continue using the previously seen resource

[purnesh42H]

Thanks. Updated.

[purnesh42H]

@easwars ptal

[easwars]

The comment at the top of this method needs to be updated. This method now handles not only resource-not-found errors, but other types of errors as well that instruct the LB policy to stop using the previously received resource.

[purnesh42H]

Updated. Also, updated the onClusterAmbient error to not stop using the previously seen resource

[easwars]

Based on the PR description, for the following two cases:

• xDS channel reports TRANSIENT_FAILURE |
• ADS stream terminates without receiving a response
  the old behavior was to call OnError(), while the new behavior is to call OnResourceChanged(status) if resource NOT already cached, and OnAmbientError(status) otherwise.

But, we are only calling the latter here. Am I missing something?

[easwars]

This came up during some other discussion with @dfawley.

1. Callback methods named OnXxx are more of a C++ style. In Go, we would probably just name them Xxx.
2. Whether we should have two callback methods, one named ResourceChanged(ResourceData), and the other named ResourceError(error). This is to work around the fact that we don't have a StatusOr type in Go, and without that, having two APIs is better than one API (where we need to add a new struct to hold one of two possible values, with no guarantees that only one of the two values is set).