The performances in the paper is not reproduced.

[KanghoonYoon]

Hi, I tried to reproduce the transformer hawkes process on StackOverflow fold1. However, the results of accuracy and RMSE is as below.

![image](https://user-images.githubusercontent.com/56212725/173004512-ba357b4d-244a-4f73-9ca1-9d9535f3f1df.png

)

I think I have something missing. Compared to the relased code of Self-Attentive Hawkes process, I think it is not because of scaling factor. What does make the difference between the paper and this repository?

[TendaG0]

I also find this question, I hope someone can help me, thanks a lot!

[pritamqu]

I encountered the same!

[ Epoch 100 ]
  - (Training)    loglikelihood: -0.67653, accuracy:  0.43488, RMSE:  106.19156, elapse: 0.438 min
  - (Testing)     loglikelihood: -0.65685, accuracy:  0.42897, RMSE:  107.70097, elapse: 0.045 min
  - [Info] Maximum ll: -0.65546, Maximum accuracy:  0.42900, Minimum RMSE:  107.18501

[TendaG0]

anyone can explain to me, please?

[waystogetthere]

I am still running the code, reply just to show that someone is still caring about this work.

Plus: looks like there are 2 errors in the log-likelihood function. Please refer to these 2 issues:

1. Ambiguity in calculating log likelihood #14 (comment)

2. Should event likelihood be computed using current or last hidden state? #10 (comment)

So for now I think the output does not make any sense.
I have fixed them and currently running the code.

[waystogetthere]

I have revised 2 errors and this is the result for dataset StackOverflow (data_so):

And this is the reported result:

Please refer to the 'SO' and 'StackOverflow' columns.

Only ACC matches...

[waystogetthere]

I will later upload my revised code soon. Any discussion is welcome!

[KanghoonYoon]

I think it is obvious that the transformer Hawkes process is superior to the Neural Hawkes process and RMP. However, I think the research area does not have a strict criterion for evaluating the model yet. (e.g., only the accuracy measure is reproduced). If there are something I missed, it is better for the author to revise the code and show reproducibility since I am not the only one who has a concern about the work.

[zilinjing]

I will later upload my revised code soon. Any discussion is welcome!

Hi, I am also trying to reproduce the result of this paper (especially for the mimic dataset). My event accuracy is similar but RMSE is a bit higher. Have you been able to get similar results as the original paper?

[liyuxisummer]

I will later upload my revised code soon. Any discussion is welcome!
@waystogetthere Would you mind sharing your revised code if possible? If it's convenient, you can send it to me at [DuduSummer21@163.com]. Thank you so much! 😊