I followed the instruction in the skyrl-train/examples/livecodebench/lcb.md and also created a venv as explained.
When I try to run the following training command it fails on loading the dataset:

`DATA_DIR="/data/yarin_shaked7/SkyRL"
train_data="['/data/yarin_shaked7/SkyRL/deepcoder_train.json']"
val_data="['/data/yarin_shaked7/SkyRL/test_livecodebench.json']"

uv run --isolated --frozen --extra vllm -m skyrl_train.entrypoints.main_base
trainer.algorithm.advantage_estimator="grpo"
data.train_data=$train_data
data.val_data=$val_data
trainer.policy.model.path="Qwen/Qwen3-0.6B"
trainer.placement.colocate_all=true
trainer.strategy=fsdp2
trainer.policy.optimizer_config.max_grad_norm=0.5
trainer.placement.policy_num_gpus_per_node=4
trainer.placement.ref_num_gpus_per_node=4
generator.num_inference_engines=1
generator.inference_engine_tensor_parallel_size=4
trainer.policy_mini_batch_size=4
trainer.train_batch_size=16
trainer.micro_forward_batch_size_per_gpu=16
trainer.micro_train_batch_size_per_gpu=2
trainer.max_prompt_length=29000
generator.max_input_length=29000
generator.sampling_params.max_generate_length=3000
trainer.policy.optimizer_config.lr=1.0e-6
trainer.algorithm.use_kl_loss=true
trainer.algorithm.kl_loss_coef=0.001
trainer.ckpt_interval=100000
generator.backend=vllm
generator.run_engines_locally=true
generator.weight_sync_backend=nccl
generator.async_engine=true
generator.batched=false
environment.env_class=lcb
generator.n_samples_per_prompt=8
generator.gpu_memory_utilization=0.7
generator.sampling_params.temperature=0.6
generator.sampling_params.top_p=0.95
trainer.logger="wandb"
trainer.project_name="skyrl"
trainer.run_name="skyrlcode_test"
trainer.resume_mode=null
trainer.ckpt_path="/data/yarin_shaked7/SkyRL/checkpoints"
trainer.eval_batch_size=1024
trainer.eval_before_train=true
trainer.eval_interval=5
$@`

I get the following error:
Installed 188 packages in 270ms
/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/transformers/utils/hub.py:110: FutureWarning: Using TRANSFORMERS_CACHE is deprecated and will be removed in v5 of Transformers. Use HF_HOME instead.
warnings.warn(
2025-10-21 17:27:14.171 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:523 - VLLM_USE_V1 is not specified, setting VLLM_USE_V1 to 1. To override, set VLLM_USE_V1 explicitly
2025-10-21 17:27:14.253 | INFO | skyrl_train.utils.utils:prepare_runtime_environment:551 - Exporting wandb api key to ray runtime env
2025-10-21 17:27:15,651 INFO worker.py:1927 -- Started a local Ray instance.
2025-10-21 17:27:15,748 INFO packaging.py:588 -- Creating a file package for local module '/home/dsi/yarin_shaked7/thesis/CodeExp/SkyRLExp/SkyRL/skyrl-train'.
2025-10-21 17:27:15,884 INFO packaging.py:380 -- Pushing file package 'gcs://_ray_pkg_8e55505503fd8b05.zip' (16.25MiB) to Ray cluster...
2025-10-21 17:27:15,918 INFO packaging.py:393 -- Successfully pushed file package 'gcs://_ray_pkg_8e55505503fd8b05.zip'.
(raylet) Building skyrl-train @ file:///tmp/ray/session_2025-10-21_17-27-14_330225_3898464/runtime_resources/working_dir_files/_ray_pkg_8e55505503fd8b05
(raylet) Building skyrl-gym @ file:///tmp/ray/session_2025-10-21_17-27-14_330225_3898464/runtime_resources/working_dir_files/_ray_pkg_8e55505503fd8b05/skyrl-gym
(raylet) Built skyrl-gym @ file:///tmp/ray/session_2025-10-21_17-27-14_330225_3898464/runtime_resources/working_dir_files/_ray_pkg_8e55505503fd8b05/skyrl-gym
(raylet) Built skyrl-train @ file:///tmp/ray/session_2025-10-21_17-27-14_330225_3898464/runtime_resources/working_dir_files/_ray_pkg_8e55505503fd8b05
(raylet) Installed 188 packages in 235ms
2025-10-21 17:27:21.819 | INFO | skyrl_train.utils.ppo_utils:sync_registries:542 - Synced registries to ray actor
(pid=3907559) /home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpfGZrOl/lib/python3.12/site-packages/transformers/utils/hub.py:110: FutureWarning: Using TRANSFORMERS_CACHE is deprecated and will be removed in v5 of Transformers. Use HF_HOME instead.
(pid=3907559) warnings.warn(
(raylet) Installed 188 packages in 277ms [repeated 2x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/user-guides/configure-logging.html#log-deduplication for more options.)
Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 0 examples [00:27, ? examples/s]
Error executing job with overrides: ['trainer.algorithm.advantage_estimator=grpo', "data.train_data=['/data/yarin_shaked7/SkyRL/deepcoder_train.json']", "data.val_data=['/data/yarin_shaked7/SkyRL/test_livecodebench.json']", 'trainer.policy.model.path=Qwen/Qwen3-0.6B', 'trainer.placement.colocate_all=true', 'trainer.strategy=fsdp2', 'trainer.policy.optimizer_config.max_grad_norm=0.5', 'trainer.placement.policy_num_gpus_per_node=4', 'trainer.placement.ref_num_gpus_per_node=4', 'generator.num_inference_engines=1', 'generator.inference_engine_tensor_parallel_size=4', 'trainer.policy_mini_batch_size=4', 'trainer.train_batch_size=16', 'trainer.micro_forward_batch_size_per_gpu=16', 'trainer.micro_train_batch_size_per_gpu=2', 'trainer.max_prompt_length=29000', 'generator.max_input_length=29000', 'generator.sampling_params.max_generate_length=3000', 'trainer.policy.optimizer_config.lr=1.0e-6', 'trainer.algorithm.use_kl_loss=true', 'trainer.algorithm.kl_loss_coef=0.001', 'trainer.ckpt_interval=100000', 'generator.backend=vllm', 'generator.run_engines_locally=true', 'generator.weight_sync_backend=nccl', 'generator.async_engine=true', 'generator.batched=false', 'environment.env_class=lcb', 'generator.n_samples_per_prompt=8', 'generator.gpu_memory_utilization=0.7', 'generator.sampling_params.temperature=0.6', 'generator.sampling_params.top_p=0.95', 'trainer.logger=wandb', 'trainer.project_name=skyrl', 'trainer.run_name=skyrlcode_test', 'trainer.resume_mode=null', 'trainer.ckpt_path=/data/yarin_shaked7/SkyRL/checkpoints', 'trainer.eval_batch_size=1024', 'trainer.eval_before_train=true', 'trainer.eval_interval=5']
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/dsi/yarin_shaked7/thesis/CodeExp/SkyRLExp/SkyRL/skyrl-train/skyrl_train/entrypoints/main_base.py", line 306, in
main()
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/hydra/main.py", line 94, in decorated_main
_run_hydra(
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
_run_app(
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/hydra/_internal/utils.py", line 457, in _run_app
run_and_report(
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/hydra/_internal/utils.py", line 223, in run_and_report
raise ex
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/hydra/_internal/utils.py", line 220, in run_and_report
return func()
^^^^^^
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/hydra/_internal/utils.py", line 458, in
lambda: hydra.run(
^^^^^^^^^^
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/hydra/_internal/hydra.py", line 132, in run
_ = ret.return_value
^^^^^^^^^^^^^^^^
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/hydra/core/utils.py", line 260, in return_value
raise self._return_value
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/hydra/core/utils.py", line 186, in run_job
ret.return_value = task_function(task_cfg)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dsi/yarin_shaked7/thesis/CodeExp/SkyRLExp/SkyRL/skyrl-train/skyrl_train/entrypoints/main_base.py", line 302, in main
ray.get(skyrl_entrypoint.remote(cfg))
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/ray/_private/auto_init_hook.py", line 22, in auto_init_wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/ray/_private/client_mode_hook.py", line 104, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/ray/_private/worker.py", line 2858, in get
values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpiXx6oo/lib/python3.12/site-packages/ray/_private/worker.py", line 958, in get_objects
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(DatasetGenerationError): ray::skyrl_entrypoint() (pid=3907559, ip=132.70.60.14)
^^^^^^^^^
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpfGZrOl/lib/python3.12/site-packages/datasets/packaged_modules/json/json.py", line 138, in _generate_tables
io.BytesIO(batch), read_options=paj.ReadOptions(block_size=block_size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow/_json.pyx", line 54, in pyarrow._json.ReadOptions.init
File "pyarrow/_json.pyx", line 79, in pyarrow._json.ReadOptions.block_size.set
OverflowError: value too large to convert to int32_t

The above exception was the direct cause of the following exception:

ray::skyrl_entrypoint() (pid=3907559, ip=132.70.60.14)
File "/home/dsi/yarin_shaked7/thesis/CodeExp/SkyRLExp/SkyRL/skyrl-train/skyrl_train/entrypoints/main_base.py", line 292, in skyrl_entrypoint
exp = BasePPOExp(cfg)
^^^^^^^^^^^^^^^
File "/home/dsi/yarin_shaked7/thesis/CodeExp/SkyRLExp/SkyRL/skyrl-train/skyrl_train/entrypoints/main_base.py", line 94, in init
self.train_dataset = self.get_train_dataset()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dsi/yarin_shaked7/thesis/CodeExp/SkyRLExp/SkyRL/skyrl-train/skyrl_train/entrypoints/main_base.py", line 121, in get_train_dataset
prompts_dataset = PromptDataset(
^^^^^^^^^^^^^^
File "/tmp/ray/session_2025-10-21_17-27-14_330225_3898464/runtime_resources/working_dir_files/_ray_pkg_8e55505503fd8b05/skyrl_train/dataset/dataset.py", line 28, in init
self._read_files_and_tokenize()
File "/tmp/ray/session_2025-10-21_17-27-14_330225_3898464/runtime_resources/working_dir_files/_ray_pkg_8e55505503fd8b05/skyrl_train/dataset/dataset.py", line 37, in _read_files_and_tokenize
ds = datasets.load_dataset("json", data_files=source, keep_in_memory=True)["train"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpfGZrOl/lib/python3.12/site-packages/datasets/load.py", line 1412, in load_dataset
builder_instance.download_and_prepare(
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpfGZrOl/lib/python3.12/site-packages/datasets/builder.py", line 894, in download_and_prepare
self._download_and_prepare(
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpfGZrOl/lib/python3.12/site-packages/datasets/builder.py", line 970, in _download_and_prepare
self._prepare_split(split_generator, **prepare_split_kwargs)
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpfGZrOl/lib/python3.12/site-packages/datasets/builder.py", line 1702, in _prepare_split
for job_id, done, content in self._prepare_split_single(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dsi/yarin_shaked7/.cache/uv/builds-v0/.tmpfGZrOl/lib/python3.12/site-packages/datasets/builder.py", line 1858, in _prepare_split_single
raise DatasetGenerationError("An error occurred while generating the dataset") from e
datasets.exceptions.DatasetGenerationError: An error occurred while generating the dataset