Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to get server stream arg in return probe of gRPC #1547

Open
minimAluminiumalism opened this issue Jan 6, 2025 · 0 comments
Open

Failed to get server stream arg in return probe of gRPC #1547

minimAluminiumalism opened this issue Jan 6, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@minimAluminiumalism
Copy link
Contributor

minimAluminiumalism commented Jan 6, 2025

Describe the bug

Let's see the grpc bpf code snippet:

SEC("uprobe/server_handleStream2")
int uprobe_server_handleStream2_Returns(struct pt_regs *ctx) {
    u64 server_stream_pos = 4;
    void *server_stream_ptr = get_argument(ctx, server_stream_pos);
    if (server_stream_ptr == NULL) {
        bpf_printk("grpc:server:uprobe/server_handleStream2Return: failed to get ServerStream arg");
        return -1;
    }
}

The original probe func is:

func (s *Server) handleStream(t transport.ServerTransport, stream *transport.ServerStream)

Obviously this func is a return probe which will be triggered when handleStream() returns. I think we cannot obtain the server_stream_pos(using get_arugment()) through BPF instrumentation when the function returns, which is different from retrieving this param at the very beginning of the function execution.

Suppose there is a simple function like this:

func f(a, b int) (c int) {}

In return probe, we try to retrieve a param: get_argument(ctx, 1), actually the param we get is the return param c rather than a.

WDYT? @MrAlias

To Reproduce

I tested gRPC demo and found the rpc instrument always failed, and the trace log is server-995136 [000] d...1 763711.253233: bpf_trace_printk: grpc:server:uprobe/server_handleStream2Return: failed to get ServerStream arg.
Clipboard_Screenshot_1736157292

To further verify this, I tried to change u64 server_stream_pos = 4; to u64 server_stream_pos = 1;, and still get NULL. I suppose this is because the function handleStream() does not return any value.

I think the correct approach is to save the address of the ServerStream in a BPF map at the beginning of the func execution. When the function returns, look up the map and ensure the addr of ServerStream.

@minimAluminiumalism minimAluminiumalism added the bug Something isn't working label Jan 6, 2025
@minimAluminiumalism minimAluminiumalism changed the title Failed to get server stream arg in return probe of gPRC Failed to get server stream arg in return probe of gRPC Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant