bug: delegation token parameter should be renamed when using WebHDFS as storage

[jason-heo]

Search before asking

• I had searched in the issues and found no similar issues.

Version

v1.2.615-a8da519a63(rust-1.81.0-nightly-2024-08-19T03:29:16.412442905Z)

What's Wrong?

Hello, Im new to Databend.

I'm trying to test Databend with my data on secure hadoop.

I've setup Query node as describe in Deploying with HDFS using WebHDFS

My current setting is like this:

[storage.webhdfs]
endpoint_url = "https://<hostname>:<port>"
root = "/path/to"
# if your webhdfs needs authentication, uncomment and set with your value
delegation = "<token>"

When I run INSERT INTO I got an error:

error: APIError: ResponseError with 3002: PermissionDenied (persistent) at Writer::close, context: {
  uri: https://<hostname>:<port>/webhdfs/v1/path/to/xxx.parquet?op=CREATE&...&delegation_token=<token>,
  response: Parts {
    status: 401,
    version: HTTP/1.1,
    headers: {
      ...
    }
  }
}

I think delegation_token should be renamed to delegation.

You can see not delegation_token but delegation on WebHDFS REST API -> Authentication

I've tested using curl and confirm that following url worked well.

$ curl -i "https://<hostname>:<port>/webhdfs/v1/path/to?op=LISTSTATUS&delegation=<token>"

How to Reproduce?

Described above.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[jason-heo]

It seems that this issue is related to OpenDAL.

After checking reproducable via OpenDAL API, let me report to OpenDAL.

[Xuanwo]

Thank you @jason-heo for this. Maybe we can also accpet delegation during parsing the url.

[Xuanwo]

This change should be simple that parse delegation as delegation_token.

[jason-heo]

Hello, @Xuanwo

If my understanding is correct, to resolve this isssue one needs to replace delegation_token to delegation in url parameter in some codes.

Could you point me out which source code should be modified for this issue?

I tried to find delegation_token but it seems that only test codes have delegation_token.

git grep -i delegation_token
src/meta/proto-conv/tests/it/user_proto_conv.rs:                delegation: "<delegation_token>".to_string(),
src/meta/proto-conv/tests/it/user_stage.rs:                delegation: "<delegation_token>".to_string(),
src/meta/proto-conv/tests/it/v030_user_stage.rs:                delegation: "<delegation_token>".to_string(),
src/meta/proto-conv/tests/it/v031_copy_max_file.rs:                delegation: "<delegation_token>".to_string(),

Although my background is Java, Scala and Python, I'll try to resolve this is if simple.

Sorry, my team's task priority has been changed.

Thanks.

[notauserx]

I reviewed the issue, and it seems the solution lies in the WebhdfsBuilder implementation in OpenDAL.

https://github.com/apache/opendal/blob/0f45b5d34a95d21760a7bb7339a9103ae3311a53/core/src/services/webhdfs/backend.rs#L177

I have a proposed fix in the PR: https://github.com/apache/opendal/pull/5342/files

[Xuanwo]

I reviewed the issue, and it seems the solution lies in the WebhdfsBuilder implementation in OpenDAL.

Hi, please fix the builder at databend side instead.

[notauserx]

Hi, thanks for pointing that out! Could you please guide me on where exactly the builder is located on the Databend side? A pointer to the relevant file or module would be helpful.

Also what is the rationale behind keeping the auth value as delegation_token on the OpenDAL side?
https://github.com/apache/opendal/blob/0f45b5d34a95d21760a7bb7339a9103ae3311a53/core/src/services/webhdfs/backend.rs#L240

[Xuanwo]

Hi, thanks for pointing that out! Could you please guide me on where exactly the builder is located on the Databend side? A pointer to the relevant file or module would be helpful.

databend/src/query/sql/src/planner/binder/location.rs

Line 400 in ed6a0c7

let delegation = l.connection.get("delegation").cloned().unwrap_or_default();

---

Oh, I see. Databend does use delegation, the real issue is opendal doesn't build the correct request.

[Xuanwo]

This issue will be fixed by apache/opendal#5342