[hhajimol]

Summary

• Add 33 KernelBench-derived torch2hip tasks (Levels 1–3) under
  tasks/torch2hip/kernelbench/, each with its config.yaml and eval harness
  (eval_tools/), PyTorch module/functional references, and HIP kernel stub.
• Fix 14539_GELU, which previously computed ReLU instead of GELU.
• Add the KernelBench→torch2hip generator (tools/kernelbench_to_torch2hip.py),
  the upstream KernelBench reference sources (tools/kernelbench_src/), and a
  CPU functional-equivalence checker (tools/validate_functional_cpu.py).

Test plan

• Run a sample task via python main.py (config targeting a
  torch2hip/kernelbench task)
• New tasks pass the task_validator agent

[sharareh-y]

Thanks for adding these — nice to have KernelBench coverage in the torch2hip set. The generated tasks are wired up correctly (naming, functional/module split, eval_tools all check out). A few things I'd like to sort out before merging:

1. Multiple input configs per task.
Right now each KernelBench task has a single-case get_inputs() (return [x]), whereas our existing gpumode torch2hip tasks yield ~5 different shapes. This matters because both the correctness check and the perf harness iterate over every case get_inputs() yields:

• Correctness only runs on one shape, so a HIP kernel hard-coded to that exact shape/launch config still passes. The multi-shape pattern is what guards against that overfitting.
• The speedup is measured on a single shape (no averaging across sizes), so it's noisier and less representative.

Could we have these emit a few shape variants like the gpumode tasks do? (The converter appends KernelBench's get_inputs verbatim, and upstream only ships one case — so it'd be a small change in the generator to produce a multi-shape generator.)

2. Please don't commit the migration tooling.
tools/kernelbench_to_torch2hip.py, tools/validate_functional_cpu.py, and tools/kernelbench_src/ are essentially a one-time migration tool that converted KernelBench into our task format. Once the tasks are generated we don't need it living in the main repo (and it also drags in the vendored KernelBench sources + their licensing/attribution question). Suggest keeping it in your fork and only committing the generated tasks/ + the config.yaml registration here. If we ever want to regen, the tool's still in your fork.

3. Group tasks into l1/l2/l3 subdirectories.
Instead of flat l1n1_..., l2n6_..., l3n44_... names, could we nest them as kernelbench/level1/..., kernelbench/level2/..., kernelbench/level3/...? Cleaner to browse and the level grouping is more obvious. (Just note the eval_tools kernel_name derivation splits on _, so we'd want to keep the per-task folder/file naming consistent with however that's resolved.)

[hhajimol]

Thanks for the detailed review! I've pushed commits addressing all three points.

1. Multiple input configs per task

Every KernelBench task's get_inputs() is now a generator yielding 5 geometrically-escalating shape cases (following the gpumode torch2hip style), so correctness runs across multiple shapes and the speedup is averaged across sizes instead of a single one.

I kept dimensions that are baked into the weights/init (channels, in_features, embed_dim/n_embd, LayerNorm normalized shape, etc.) fixed, and only varied the free dims (batch, spatial H/W, sequence length, matmul outer dims) — varying an init-tied dim would break the functional model's parameter alignment with the reference module.

A few task-specific notes following the gpumode logic:

l1n26_GELU_ spans multiple ranks (1D→4D), mirroring its gpumode twin 14539_GELU.
l1n95_CrossEntropyLoss varies num_classes as well as batch (it has no init, so the class dim is free — like gpumode's CE/KD-loss tasks).
Conv tasks (which have no gpumode counterpart) follow the 10190_FusedLeakyReLU 4D [N,C,H,W] pattern: in-channels pinned, batch/spatial escalated.
I kept KernelBench's large, perf-relevant sizes rather than gpumode's toy sizes, since these tasks exist to benchmark performance. Validated module == functional on an MI300X for all 33 tasks (5 cases each).

2. Migration tooling removed

Dropped tools/kernelbench_to_torch2hip.py, tools/validate_functional_cpu.py, and tools/kernelbench_src/ from the PR.

3. Level grouping

Tasks are now nested under kernelbench/level1/, level2/, level3/ (moved via git mv, history preserved). I intentionally kept the per-task folder/file names unchanged (e.g. level1/l1n26_GELU_/hip/hip_l1n26_GELU_.hip) so the eval_tools kernel_name derivation (hip_file.split('.hip')[0].split('_', 2)[-1]) still resolves to the class name. config.yaml registration updated to the nested paths; all 33 resolve via task discovery.

Happy to strip the now-redundant l1n/l2n/l3n prefixes (e.g. level1/26_GELU_/) if you'd prefer — works either way.
Note that there are still some level4 tasks that I am currently working on that are not included in the current PR.
Let me know if anything else needs adjusting.

[irvineoy]

Hi, quick heads-up: we are currently working on a larger refactor #48 that moves the benchmark/runtime flow to a Docker-first environment. This touches a lot of shared infrastructure, so reviews and merges may be slower than usual for a bit.

Once that refactor lands, I’ll help rebase open PRs onto that one. Sorry for the delay, and thanks for your patience.