Executed run ppo.py,RuntimeError: CUDA error: device-side assert triggered occured.

[hosokawa-taiji]

Executed run ppo.py,RuntimeError: CUDA error: device-side assert triggered occured.
I did python machina/example/run_ppo.py --cuda 0 --env_name 'CartPole-v0' --rnn
(I am aware that RNN is not involved in cart-poles though)
then I got these error.

{'batch_size': 256,
 'c2d': False,
 'clip_param': 0.2,
 'cuda': 0,
 'env_name': 'CartPole-v0',
 'epoch_per_iter': 10,
 'gamma': 0.995,
 'init_kl_beta': 1,
 'kl_targ': 0.01,
 'lam': 1,
 'log': 'garbage',
 'max_epis': 1000000,
 'max_grad_norm': 10,
 'max_steps_per_iter': 10000,
 'num_parallel': 4,
 'pol_lr': 0.0003,
 'ppo_type': 'clip',
 'record': False,
 'rnn': True,
 'rnn_batch_size': 8,
 'seed': 256,
 'vf_lr': 0.0003}
2020-05-07 02:45:13.611958: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-05-07 02:45:14.846619 UTC | observation space: Box(4,)
2020-05-07 02:45:14.846752 UTC | action space: Discrete(2)
2020-05-07 02:45:22.078645 UTC | sample: 4.6384sec
2020-05-07 02:45:24.547073 UTC | Optimizing...
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [8,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [8,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [16,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [16,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [23,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [23,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [34,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [34,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [51,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [51,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [18,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [18,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [3,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [3,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [12,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [12,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [10,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [10,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [47,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [47,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [38,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [38,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [36,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [36,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [29,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [29,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [25,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [25,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [49,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [49,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [21,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [21,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [42,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [42,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [31,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [31,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [35,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [35,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [22,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [22,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [48,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [48,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [9,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [9,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [5,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [5,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [44,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [44,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [24,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [24,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [11,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [11,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [50,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [50,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [37,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [37,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [33,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [33,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [20,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [20,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [39,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [39,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [26,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [26,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [43,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [43,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [4,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [4,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [46,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [46,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [17,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [17,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [7,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [7,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [30,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [30,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [0,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [0,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [13,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [13,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [15,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [15,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [2,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [2,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [28,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [28,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [41,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [41,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [14,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [14,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [6,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [6,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [45,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [45,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [19,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [19,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [32,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [32,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [1,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [1,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [27,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [27,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [40,0,0], thread: [0,0,0] Assertion `val >= zero` failed.
/pytorch/aten/src/ATen/native/cuda/MultinomialKernel.cu:243: sampleMultinomialOnce: block: [40,0,0], thread: [1,0,0] Assertion `val >= zero` failed.
Traceback (most recent call last):
  File "machina/example/run_ppo.py", line 155, in <module>
    optim_pol=optim_pol, optim_vf=optim_vf, epoch=args.epoch_per_iter, batch_size=args.batch_size if not args.rnn else args.rnn_batch_size, max_grad_norm=args.max_grad_norm)
  File "/usr/local/lib/python3.6/dist-packages/machina/algos/ppo_clip.py", line 132, in train
    clip_param, ent_beta, max_grad_norm)
  File "/usr/local/lib/python3.6/dist-packages/machina/algos/ppo_clip.py", line 38, in update_pol
    pol_loss = lf.pg_clip(pol, batch, clip_param, ent_beta)
  File "/usr/local/lib/python3.6/dist-packages/machina/loss_functional.py", line 51, in pg_clip
    _, _, pd_params = pol(obs, h_masks=h_masks)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/machina/pols/categorical_pol.py", line 53, in forward
    ac_real = self.convert_ac_for_real(ac.detach().cpu().numpy())
RuntimeError: CUDA error: device-side assert triggered

[hosokawa-taiji]

my spec is

• OS
  NAME="Ubuntu"
  VERSION="18.04.3 LTS (Bionic Beaver)"

• CUDA
  nvcc: NVIDIA (R) Cuda compiler driver
  Copyright (c) 2005-2019 NVIDIA Corporation
  Built on Sun_Jul_28_19:07:16_PDT_2019
  Cuda compilation tools, release 10.1, V10.1.243

• NVIDIA
  NVRM version: NVIDIA UNIX x86_64 Kernel Module 418.67 Sat Apr 6 03:07:24 CDT 2019
  GCC version: Selected multilib: .;@m64

• pip freeze

absl-py==0.9.0
aiohttp==3.6.2
alabaster==0.7.12
albumentations==0.1.12
altair==4.1.0
asgiref==3.2.7
astor==0.8.1
astropy==4.0.1.post1
astunparse==1.6.3
async-timeout==3.0.1
atari-py==0.2.6
atomicwrites==1.4.0
attrs==19.3.0
audioread==2.1.8
autograd==1.3
Babel==2.8.0
backcall==0.1.0
beautifulsoup4==4.6.3
bleach==3.1.5
blis==0.4.1
bokeh==1.4.0
boto==2.49.0
boto3==1.13.1
botocore==1.16.1
Bottleneck==1.3.2
branca==0.4.0
bs4==0.0.1
CacheControl==0.12.6
cached-property==1.5.1
cachetools==3.1.1
catalogue==1.0.0
certifi==2020.4.5.1
cffi==1.14.0
chainer==6.5.0
chardet==3.0.4
click==7.1.2
cloudpickle==1.2.2
cmake==3.12.0
cmdstanpy==0.4.0
colorama==0.4.3
colorlover==0.3.0
community==1.0.0b1
contextlib2==0.5.5
convertdate==2.2.0
coverage==3.7.1
coveralls==0.5
crcmod==1.7
cufflinks==0.17.3
cupy-cuda101==6.5.0
cvxopt==1.2.5
cvxpy==1.0.31
cycler==0.10.0
cymem==2.0.3
Cython==0.29.17
daft==0.0.4
dask==2.12.0
dataclasses==0.7
datascience==0.10.6
decorator==4.4.2
defusedxml==0.6.0
descartes==1.1.0
dill==0.3.1.1
distributed==1.25.3
Django==3.0.5
dlib==19.18.0
docopt==0.6.2
docutils==0.15.2
dopamine-rl==1.0.5
earthengine-api==0.1.220
easydict==1.9
ecos==2.0.7.post1
editdistance==0.5.3
en-core-web-sm==2.2.5
entrypoints==0.3
ephem==3.7.7.1
et-xmlfile==1.0.1
fa2==0.3.5
fancyimpute==0.4.3
fastai==1.0.61
fastdtw==0.3.4
fastprogress==0.2.3
fastrlock==0.4
fbprophet==0.6
feather-format==0.4.1
featuretools==0.4.1
filelock==3.0.12
firebase-admin==4.1.0
fix-yahoo-finance==0.0.22
Flask==1.1.2
folium==0.8.3
fsspec==0.7.3
future==0.16.0
gast==0.3.3
GDAL==2.2.2
gdown==3.6.4
gensim==3.6.0
geographiclib==1.50
geopy==1.17.0
gin-config==0.3.0
glob2==0.7
google==2.0.3
google-api-core==1.16.0
google-api-python-client==1.7.12
google-auth==1.7.2
google-auth-httplib2==0.0.3
google-auth-oauthlib==0.4.1
google-cloud-bigquery==1.21.0
google-cloud-core==1.0.3
google-cloud-datastore==1.8.0
google-cloud-firestore==1.6.2
google-cloud-language==1.2.0
google-cloud-storage==1.18.1
google-cloud-translate==1.5.0
google-colab==1.0.0
google-pasta==0.2.0
google-resumable-media==0.4.1
googleapis-common-protos==1.51.0
googledrivedownloader==0.4
graphviz==0.10.1
grpcio==1.28.1
gspread==3.0.1
gspread-dataframe==3.0.6
gym==0.15.4
h5py==2.10.0
HeapDict==1.0.1
holidays==0.9.12
html5lib==1.0.1
httpimport==0.5.18
httplib2==0.17.3
httplib2shim==0.0.3
humanize==0.5.1
hyperopt==0.1.2
ideep4py==2.0.0.post3
idna==2.9
idna-ssl==1.1.0
image==1.5.31
imageio==2.4.1
imagesize==1.2.0
imbalanced-learn==0.4.3
imblearn==0.0
imgaug==0.2.9
importlib-metadata==1.6.0
imutils==0.5.3
inflect==2.1.0
intel-openmp==2020.0.133
intervaltree==2.1.0
ipykernel==4.10.1
ipython==5.5.0
ipython-genutils==0.2.0
ipython-sql==0.3.9
ipywidgets==7.5.1
itsdangerous==1.1.0
jax==0.1.64
jaxlib==0.1.45
jdcal==1.4.1
jedi==0.17.0
jieba==0.42.1
Jinja2==2.11.2
jmespath==0.9.5
joblib==0.14.1
jpeg4py==0.1.4
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.3.4
jupyter-console==5.2.0
jupyter-core==4.6.3
kaggle==1.5.6
kapre==0.1.3.1
Keras==2.3.1
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
keras-vis==0.4.1
kiwisolver==1.2.0
knnimpute==0.1.0
librosa==0.6.3
lightgbm==2.2.3
llvmlite==0.31.0
lmdb==0.98
lucid==0.3.8
LunarCalendar==0.0.9
lxml==4.2.6
machina-rl==0.2.1
Markdown==3.2.1
MarkupSafe==1.1.1
matplotlib==3.2.1
matplotlib-venn==0.11.5
missingno==0.4.2
mistune==0.8.4
mizani==0.6.0
mkl==2019.0
mlxtend==0.14.0
more-itertools==8.2.0
moviepy==0.2.3.5
mpmath==1.1.0
msgpack==1.0.0
multidict==4.7.5
multiprocess==0.70.9
multitasking==0.0.9
murmurhash==1.0.2
music21==5.5.0
natsort==5.5.0
nbconvert==5.6.1
nbformat==5.0.6
networkx==2.4
nibabel==3.0.2
nltk==3.2.5
notebook==5.2.2
np-utils==0.5.12.1
numba==0.48.0
numexpr==2.7.1
numpy==1.18.3
nvidia-ml-py3==7.352.0
oauth2client==4.1.3
oauthlib==3.1.0
okgrade==0.4.3
opencv-contrib-python==4.1.2.30
opencv-python==4.1.2.30
openpyxl==2.5.9
opt-einsum==3.2.1
osqp==0.6.1
packaging==20.3
palettable==3.3.0
pandas==1.0.3
pandas-datareader==0.8.1
pandas-gbq==0.11.0
pandas-profiling==1.4.1
pandocfilters==1.4.2
parso==0.7.0
pathlib==1.0.1
patsy==0.5.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow==7.0.0
pip-tools==4.5.1
plac==1.1.3
plotly==4.4.1
plotnine==0.6.0
pluggy==0.7.1
portpicker==1.3.1
prefetch-generator==1.0.1
preshed==3.0.2
prettytable==0.7.2
progressbar2==3.38.0
prometheus-client==0.7.1
promise==2.3
prompt-toolkit==1.0.18
protobuf==3.10.0
psutil==5.4.8
psycopg2==2.7.6.1
ptvsd==5.0.0a12
ptyprocess==0.6.0
py==1.8.1
py-spy==0.3.3
pyarrow==0.14.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycocotools==2.0.0
pycparser==2.20
pydata-google-auth==1.1.0
pydot==1.3.0
pydot-ng==2.0.0
pydotplus==2.0.2
PyDrive==1.3.1
pyemd==0.5.1
pyglet==1.3.2
Pygments==2.1.3
pygobject==3.26.1
pymc3==3.7
PyMeeus==0.3.7
pymongo==3.10.1
pymystem3==0.2.0
PyOpenGL==3.1.5
pyparsing==2.4.7
pyrsistent==0.16.0
pysndfile==1.3.8
PySocks==1.7.1
pystan==2.19.1.1
pytest==3.6.4
python-apt==1.6.5+ubuntu0.2
python-chess==0.23.11
python-dateutil==2.8.1
python-louvain==0.14
python-slugify==4.0.0
python-utils==2.4.0
pytz==2018.9
PyWavelets==1.1.1
PyYAML==3.13
pyzmq==19.0.0
qtconsole==4.7.3
QtPy==1.9.0
ray==0.8.4
redis==3.5.0
regex==2019.12.20
requests==2.23.0
requests-oauthlib==1.3.0
resampy==0.2.2
retrying==1.3.3
rpy2==3.2.7
rsa==4.0
s3fs==0.4.2
s3transfer==0.3.3
scikit-image==0.16.2
scikit-learn==0.22.2.post1
scipy==1.4.1
screen-resolution-extra==0.0.0
scs==2.1.2
seaborn==0.10.1
Send2Trash==1.5.0
setuptools-git==1.2
Shapely==1.7.0
simplegeneric==0.8.1
six==1.12.0
sklearn==0.0
sklearn-pandas==1.8.0
smart-open==2.0.0
snowballstemmer==2.0.0
sortedcontainers==2.1.0
spacy==2.2.4
Sphinx==1.8.5
sphinxcontrib-websupport==1.2.2
SQLAlchemy==1.3.16
sqlparse==0.3.1
srsly==1.0.2
statsmodels==0.10.2
sympy==1.1.1
tables==3.4.4
tabulate==0.8.7
tbb==2020.0.133
tblib==1.6.0
tensorboard==2.2.1
tensorboard-plugin-wit==1.6.0.post3
tensorboardcolab==0.0.22
tensorflow==2.2.0rc4
tensorflow-addons==0.8.3
tensorflow-datasets==2.1.0
tensorflow-estimator==2.2.0
tensorflow-gcs-config==2.1.8
tensorflow-hub==0.8.0
tensorflow-metadata==0.21.2
tensorflow-privacy==0.2.2
tensorflow-probability==0.10.0rc0
termcolor==1.1.0
terminado==0.8.3
terminaltables==3.1.0
testpath==0.4.4
text-unidecode==1.3
textblob==0.15.3
textgenrnn==1.4.1
Theano==1.0.4
thinc==7.4.0
toolz==0.10.0
torch==1.5.0+cu101
torchsummary==1.5.1
torchtext==0.3.1
torchvision==0.6.0+cu101
tornado==4.5.3
tqdm==4.38.0
traitlets==4.3.3
tweepy==3.6.0
typeguard==2.7.1
typing==3.6.6
typing-extensions==3.6.6
tzlocal==1.5.1
umap-learn==0.4.2
uritemplate==3.0.1
urllib3==1.24.3
vega-datasets==0.8.0
wasabi==0.6.0
wcwidth==0.1.9
webencodings==0.5.1
Werkzeug==1.0.1
widgetsnbextension==3.5.1
wordcloud==1.5.0
wrapt==1.12.1
xarray==0.15.1
xgboost==0.90
xkit==0.0.0
xlrd==1.1.0
xlwt==1.3.0
yarl==1.4.2
yellowbrick==0.9.1
zict==2.0.0
zipp==3.1.0

[rarilurelo]

Could you run with --cuda -1?

[hosokawa-taiji]

I ran it but got another error.

{'batch_size': 256,
 'c2d': False,
 'clip_param': 0.2,
 'cuda': -1,
 'env_name': 'CartPole-v0',
 'epoch_per_iter': 10,
 'gamma': 0.995,
 'init_kl_beta': 1,
 'kl_targ': 0.01,
 'lam': 1,
 'log': 'garbage',
 'max_epis': 1000000,
 'max_grad_norm': 10,
 'max_steps_per_iter': 10000,
 'num_parallel': 4,
 'pol_lr': 0.0003,
 'ppo_type': 'clip',
 'record': False,
 'rnn': True,
 'rnn_batch_size': 8,
 'seed': 256,
 'vf_lr': 0.0003}
2020-05-07 08:58:28.255482: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-05-07 08:58:29.510495 UTC | observation space: Box(4,)
2020-05-07 08:58:29.510618 UTC | action space: Discrete(2)
2020-05-07 08:58:34.625466 UTC | sample: 5.0639sec
2020-05-07 08:58:37.146453 UTC | Optimizing...
Traceback (most recent call last):
  File "machina/example/run_ppo.py", line 155, in <module>
    optim_pol=optim_pol, optim_vf=optim_vf, epoch=args.epoch_per_iter, batch_size=args.batch_size if not args.rnn else args.rnn_batch_size, max_grad_norm=args.max_grad_norm)
  File "/usr/local/lib/python3.6/dist-packages/machina/algos/ppo_clip.py", line 132, in train
    clip_param, ent_beta, max_grad_norm)
  File "/usr/local/lib/python3.6/dist-packages/machina/algos/ppo_clip.py", line 38, in update_pol
    pol_loss = lf.pg_clip(pol, batch, clip_param, ent_beta)
  File "/usr/local/lib/python3.6/dist-packages/machina/loss_functional.py", line 51, in pg_clip
    _, _, pd_params = pol(obs, h_masks=h_masks)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/machina/pols/categorical_pol.py", line 52, in forward
    ac = self.pd.sample(dict(pi=pi))
  File "/usr/local/lib/python3.6/dist-packages/machina/pds/categorical_pd.py", line 19, in sample
    pi_sampled = Categorical(probs=pi).sample(sample_shape)
  File "/usr/local/lib/python3.6/dist-packages/torch/distributions/categorical.py", line 106, in sample
    samples_2d = torch.multinomial(probs_2d, sample_shape.numel(), True).T
RuntimeError: invalid multinomial distribution (encountering probability entry < 0)

[rarilurelo]

I think nan would be inserted to pi. I can't understand the reason. Could you run without --rnn?

[hosokawa-taiji]

I ran it .
python machina/example/run_ppo.py --env_name 'CartPole-v0' --cuda -1
It worked fine.
But I'd like to run with --rnn.