Bug: Metal GPU fails on iPad M5 with bfloat/half type mismatch

[benoitdelorme]

Environment

• Device: iPad Pro M5
• iPadOS version: 26.2
• llama.rn version: 0.9.3
• React Native version: 0.76.x
• Model tested: Llama-3.2-3B-Instruct-Q8_0.gguf

Description

Metal GPU initialization fails on iPad M5 with a shader compilation error related to bfloat and half type mismatch in MetalPerformancePrimitives framework.

The model loads successfully on CPU fallback, but GPU acceleration is not working.

Error Message

lm_ggml_metal_init: picking default device: Apple M5 GPU
lm_ggml_metal_init: the device does not have a precompiled Metal library - this is unexpected
lm_ggml_metal_init: will try to compile it on the fly
lm_ggml_metal_library_init: default.metallib not found, loading from source
error: static_assert failed due to requirement 'tensor_ops_detail::_is_same_v<bfloat, half>'
"Input types must match cooperative tensor types"
static_assert(tensor_ops_detail::_is_same_v<_leftType, leftValueType>, "Input types must match cooperative tensor types");
lm_ggml_metal_init: error: failed to initialize the Metal library
lm_ggml_backend_metal_device_init: error: failed to allocate context
llama_init_from_model: failed to initialize the context: failed to initialize Metal backend

Configuration Used

const params = {
  model: MODEL_PATH,
  use_mlock: true,
  n_ctx: 2048,
  n_batch: 512,
  n_gpu_layers: 99,
  cache_type_k: 'f16',
  cache_type_v: 'f16',
  flash_attn: false,
};

Steps to Reproduce

1. Install llama.rn 0.9.3 on a React Native project
2. Build and run on iPad M5 (iPadOS 18)
3. Try to initialize a model with n_gpu_layers > 0
4. Observe the Metal compilation error

Related

• llama.cpp PR #16634: metal : initial Metal4 tensor API support

Additional Context

• The fix seems to work on macOS but not on iPadOS
• CPU fallback works correctly

[jhen0409]

The similar problem also happened on my iPhone Air after upgrade iOS from 26.1 to 26.2 Public Beta. I'll investigate this later.

Until this issue fixed, we can temporarily disable bf16 support in AppDelegate:

if #available(iOS 26.2, *) {
    setenv("LM_GGML_METAL_BF16_DISABLE", "1", 1)
}

For more specific check we can confirm whether the chip is M5 or A19.

[Cdingram]

Yeah I can't load any model on iOS 26.2 now that I've updated, iPhone 17 pro.

[jhen0409]

Confirmed it's fixed by ggml-org/llama.cpp#18456. (v0.10.0-rc.4)

Tested on iPhone Air (A19 Pro).