Add SAM. #115
Add SAM. #115
Conversation
Silver0930
force-pushed
the
master
branch
3 times, most recently
from
6852ce9
to
adb7337
Compare
| Clone the repository locally and install with | ||
|
|
||
| ```shell | ||
| git clone https://github.com/Mark-ZhouWX/models.git |
| ## Installation | ||
|
|
||
| The code requires `python>=3.7` and supports Ascend platform, some important pre-dependencies is: | ||
| 1. mindspore: Please follow the instructions [here](https://www.mindspore.cn/install) to install mindspore dependencies. |
There was a problem hiding this comment.
版本信息用表格呈现：
MindSPore CANN Mindformers
2.3.1 version version
done
|
|
||
| | pretrained_model | dataset | epochs | mIOU | ckpt | | ||
| |:----------------:| -------- |:-------------:|------|--------------------------------------------------------------------------------------------------------------| | ||
| | sam-vit-b | COCO2017 | 0 (zero-shot) | 74.5 | | |
|
|
||
| for distributed finetune of COCO dataset, please run: | ||
| ```shell | ||
| msrun --worker_num=8 --local_worker_num=8 train.py -c configs/coco_box_finetune.yaml -o amp_level=O2 |
|
|
||
| for distributed finetune, please run: | ||
| ```shell | ||
| msrun --worker_num=8 --local_worker_num=8 train.py -c configs/sa1b_text_finetune_blip2.yaml |
| pycocotools | ||
| opencv-python | ||
| omegaconf==2.0.0 | ||
| mindformers=1.0.0 |
Silver0930
force-pushed
the
master
branch
2 times, most recently
from
feacfe1
to
fa91c01
Compare
| python demo/inference_with_amg.py --model-type vit_h | ||
| ``` | ||
|
|
||
| See `python demo/inference_with_amg.py --help` to explore more custom settings. |
There was a problem hiding this comment.
lite推理的代码有但是readme没有体现
| python text_inference.py --checkpoint=your/path/to/ckpt --text-prompt your_prompt | ||
| ``` | ||
|
|
||
| ## Demo |
| @@ -0,0 +1,12 @@ | |||
| def create_group_param(params, **kwargs): | |||
There was a problem hiding this comment.
remove it if not use
AdmW 类需要用到此方法
|
|
||
| def main(args) -> None: | ||
| # Step1: initialize environment | ||
| ms.context.set_context(mode=args.mode, device_target=args.device, pynative_synchronize=True, max_device_memory="50GB") |
There was a problem hiding this comment.
why set pynative_synchronize=True? isn't it mainly for debug?
why limite max_device_memory to 50GB?
There was a problem hiding this comment.
why set pynative_synchronize=True? isn't it mainly for debug? why limite max_device_memory to 50GB?
pynative_synchronize=True 是之前测过pynative没有删掉，这个我去掉
max_device_memory=50GB，是之前如果不限制会报OOM，我再测下去掉还会不会oom
There was a problem hiding this comment.
why set pynative_synchronize=True? isn't it mainly for debug? why limite max_device_memory to 50GB?
已更新
Silver0930
force-pushed
the
master
branch
3 times, most recently
from
086cfbe
to
af39ae9
Compare
|
|
||
| ## Installation | ||
|
|
||
| The code supports Ascend platform, some important pre-dependencies is: |
There was a problem hiding this comment.
are
Here are some important dependency
|
|
||
| | model name | cards | batch size | graph compile | recompute | sink | jit level | s/step | img/s | | ||
| |:----------:| :---: | :--------: | :-----------: | :-------: | :---: | :------: | :----: | :---: | | ||
| | sam-vit-b finetune on COCO2017 | 1 | 16 | 3~4 mins | OFF | ON | O2 | 2.59 | 6.18 | |
There was a problem hiding this comment.
better add a dataset column instead of writing the dataset name in model_name column
|
|
||
| ## Installation | ||
|
|
||
| The code supports Ascend platform, some important pre-dependencies is: |
|
|
||
| | mindspore-lite | mindspore | ascend driver | firmware | cann tookit/kernel | | ||
| | :------------: | :-------: | :-----------: | :------: | :----------------: | | ||
| | 2.3.1 | 2.3.1 | Same as Finetune | Same as Finetune | Same as Finetune | |
There was a problem hiding this comment.
Same as Finetune?
use hyperlink or simply specify the version code
|
|
||
| | model name | s/img | | ||
| |:----------:| :---: | | ||
| | sam-vit-b finetune on COCO2017 | 5.35 | |
|
|
||
| predictor = SamPredictor(sam) | ||
|
|
||
| image = cv2.imread('https://github.com/user-attachments/assets/022bd54a-4e28-4c22-82d4-45113bbe49ee') |
There was a problem hiding this comment.
Allow setting input image via argparser, same as lite inference --image_path
| # image_npnew = cv2.imread('./images/tumor1.png')/255.0 # (1024, 1024, 3) | ||
| # diff = image_npnew - image_np | ||
|
|
||
| points_np = np.array([[[868, 622]], [[724, 466]], [[501, 543]], [[565, 505]], [[262, 456]]]) # (bp, n, 2), in [w, h] |
There was a problem hiding this comment.
is this point prompt bound with images/tumor1.png?
what if user change image_path to another image?
better allow setting point prompt via argparser too.
No description provided.