GLM4教程中“02-GLM-4-9b-Chat 接入 LangChain”代码运行报错

[meisongzhu]

按照教程直接运行代码时，会出现以下报错信息：
/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/transformers/generation/utils.py:1885: UserWarning: You are calling .generate() with the input_idsbeing on a device type different than your model's device.input_idsis on cpu, whereas the model is on cuda. You may experience unexpected behaviors or slower generation. Please make sure that you have putinput_idsto the correct device by calling for example input_ids = input_ids.to('cuda') before running.generate(). warnings.warn( Traceback (most recent call last): File "/home/pdl/Workspaces/GLM4_Demo/langchain_basic.py", line 8, in <module> print(llm.invoke("你是谁").to("cuda")) File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 385, in invoke self.generate_prompt( File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 750, in generate_prompt return self.generate(prompt_strings, stop=stop, callbacks=callbacks, **kwargs) File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 944, in generate output = self._generate_helper( File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 787, in _generate_helper raise e File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 774, in _generate_helper self._generate( File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 1508, in _generate self._call(prompt, stop=stop, run_manager=run_manager, **kwargs) File "/home/pdl/Workspaces/GLM4_Demo/LLM.py", line 40, in _call generated_ids = self.model.generate(**model_inputs, **self.gen_kwargs) File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/transformers/generation/utils.py", line 2024, in generate result = self._sample( File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/transformers/generation/utils.py", line 2982, in _sample outputs = self(**model_inputs, return_dict=True) File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, **kwargs) File "/home/pdl/.cache/huggingface/modules/transformers_modules/glm-4-9b-chat/modeling_chatglm.py", line 994, in forward transformer_outputs = self.transformer( File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, **kwargs) File "/home/pdl/.cache/huggingface/modules/transformers_modules/glm-4-9b-chat/modeling_chatglm.py", line 878, in forward inputs_embeds = self.embedding(input_ids) File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, **kwargs) File "/home/pdl/.cache/huggingface/modules/transformers_modules/glm-4-9b-chat/modeling_chatglm.py", line 814, in forward words_embeddings = self.word_embeddings(input_ids) File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, **kwargs) File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/torch/nn/modules/sparse.py", line 164, in forward return F.embedding( File "/home/pdl/Applications/miniforge-pypy3/envs/glm4_env/lib/python3.10/site-packages/torch/nn/functional.py", line 2267, in embedding return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse) RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper_CUDA__index_select)

应该是输入参数生成的张量跟模型张量没有分配到同一个设备上。按照以下方式修改可以正确运行：
在LLM.py中ChatGLM4_LLM类的_call方法中进行修改，在
generated_ids = self.model.generate(**model_inputs, **self.gen_kwargs) generated_ids = [ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs['input_ids'], generated_ids) ]
之前增加
device = next(self.model.parameters()).device model_inputs = {key: value.to(device) for key, value in model_inputs.items()}