deduplication: account for all punct chars in tokenization (#783) #1560
| Job | Run time |
|---|---|
| 2m 33s | |
| 4m 14s | |
| 2m 24s | |
| 4m 11s | |
| 2m 39s | |
| 4m 23s | |
| 2m 28s | |
| 2m 39s | |
| 2m 17s | |
| 3m 18s | |
| 2m 34s | |
| 33m 40s |
| Job | Run time |
|---|---|
| 2m 33s | |
| 4m 14s | |
| 2m 24s | |
| 4m 11s | |
| 2m 39s | |
| 4m 23s | |
| 2m 28s | |
| 2m 39s | |
| 2m 17s | |
| 3m 18s | |
| 2m 34s | |
| 33m 40s |