Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP feat:Init commit for rust backend #1180

Draft
wants to merge 18 commits into
base: master
Choose a base branch
from

Conversation

Aisuko
Copy link
Collaborator

@Aisuko Aisuko commented Oct 17, 2023

Description

This PR relates to #939

Notes for Reviewers

Signed commits

  • Yes, I signed my commits.

@Aisuko Aisuko marked this pull request as draft October 17, 2023 02:38
@Aisuko Aisuko self-assigned this Oct 17, 2023
@Aisuko Aisuko force-pushed the feat/rust_grpc branch 2 times, most recently from aacaf4e to afcd7bd Compare October 17, 2023 04:53
@mudler
Copy link
Owner

mudler commented Oct 17, 2023

cc @lu-zero

.gitignore Outdated Show resolved Hide resolved
backend/rust/Makefile Outdated Show resolved Hide resolved
backend/rust/src/main.rs Outdated Show resolved Hide resolved
backend/rust/src/main.rs Outdated Show resolved Hide resolved
Aisuko and others added 3 commits October 18, 2023 10:47
Co-authored-by: Luca Barbato <[email protected]>
Signed-off-by: Aisuko <[email protected]>
Signed-off-by: GitHub <[email protected]>
Signed-off-by: Aisuko <[email protected]>
@Aisuko Aisuko requested a review from lu-zero October 18, 2023 08:00
@Aisuko Aisuko requested a review from lu-zero October 19, 2023 04:51
Co-authored-by: Luca Barbato <[email protected]>
Signed-off-by: Aisuko <[email protected]>
backend/rust/burn/src/main.rs Outdated Show resolved Hide resolved
Signed-off-by: Aisuko <[email protected]>
@lu-zero
Copy link
Collaborator

lu-zero commented Oct 31, 2023

it seems to look for libtorch and fails to find it. if you use the ndarray backend does it work?

@Aisuko
Copy link
Collaborator Author

Aisuko commented Oct 31, 2023

it seems to look for libtorch and fails to find it. if you use the ndarray backend does it work?

Will try it and give a feedback

Update

ndarary backend can be used to debug in IDE. And the torch backend has some issues on Mac M1. Here I am trying to set up LIBTORCH_USE_PYTORCH=1 as env with the conda env which is installed PyTorch. However, it is still hit other issues on M1 environment. So, I'm going to use ndarray to help me debug the conversion part code.

@lu-zero
Copy link
Collaborator

lu-zero commented Nov 1, 2023

On the M1 probably the wgpu backend is the nicest to use, but ndarray is the one that does not depend on the host system.

@Aisuko
Copy link
Collaborator Author

Aisuko commented Nov 1, 2023

On the M1 probably the wgpu backend is the nicest to use, but ndarray is the one that does not depend on the host system.

Thanks a lot. I have made some change here. I have been migrated the code which is included Llama2 to fork repo, and I am working on the a more simpler model. Here are some reasons:

  • A simpler model can be more effecient to debug than Llama2, less parameters, and less memory used. (Only load half of Llama2 parameters to tensor can cost at least 13min in my local env now)
  • We can move faster on this PR. It is good for us to refractor the code, project structure and abstract some common traits.
  • Easy for code reviewing
  • Easy for adding some test cases(CI).

Here I hit an issue on reshaping of the Tensor. So, we can try to implement a simple one instead of getting stuck on the Llama2.
Screenshot 2023-11-01 at 5 32 21 pm

@Aisuko Aisuko force-pushed the feat/rust_grpc branch 3 times, most recently from d9f1f7d to da3a0d8 Compare November 3, 2023 11:50
@Aisuko Aisuko force-pushed the feat/rust_grpc branch 4 times, most recently from cb216fa to ed95d9c Compare November 4, 2023 02:26
// And now the nonlinear scale
let min_log_hz = 1000.0; // beginning of log region (Hz)
let min_log_mel = (min_log_hz - f_min) / f_sp;
let logstep = (6.4f64).ln() / 27.0; // step size for log region
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

those constants are repeated, being always f64 you can just keep them as consts

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you, will do.

@Aisuko Aisuko force-pushed the feat/rust_grpc branch 3 times, most recently from a6ff963 to b91b79c Compare November 18, 2023 01:35
Signed-off-by: Aisuko <[email protected]>
Copy link

netlify bot commented Nov 23, 2023

Deploy Preview for localai failed.

Name Link
🔨 Latest commit c990112
🔍 Latest deploy log https://app.netlify.com/sites/localai/deploys/655ea7b3d02aec0008ca4cdf


let tensor3=tensor2.transpose();

let tensor41=tensor3.repeat(2, 2);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lu-zero Here, I am going to use wgpu backend instead of tch. However, I the repeat function here only support 2 dimensions tensor, (Can only repeat dimension with dim=1) https://github.com/Tracel-AI/burn/blob/b86bc5876149bd73bc59cb5197fd3ee8b92509d4/burn-tensor/src/tensor/ops/tensor.rs#L222C7-L222C7.

I have been tried several solutions, like use swap_dims and flattern these internal function of Tensor, but here hard to say it is correct and also causes other issues. Is there a better example for this?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

asking upstream probably it is the best route (sorry for the belated reply, I got very busy and the message got lost in the mailbox)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries, thanks for your support. I will continue to work on this one after I applied PhD successfully. Currently, sooo busy. But I still want to get this PR to merged.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once you are more free please contact me, probably a good deal of the issues will be ironed out by upstream meanwhile :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants