Skip to content

Commit 69ac118

Browse files
committed
Small fixes
1 parent 5d7e51b commit 69ac118

File tree

2 files changed

+6
-0
lines changed

2 files changed

+6
-0
lines changed

examples/question_answering.ipynb

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -959,6 +959,11 @@
959959
"outputs": [],
960960
"source": [
961961
"def prepare_train_features(examples):\n",
962+
" # Some of the questions have lots of whitespace on the left, which is not useful and will make the\n",
963+
" # truncation of the context fail (the tokenized question will take a lots of space). So we remove that\n",
964+
" # left whitespace\n",
965+
" examples[\"question\"] = [q.lstrip() for q in examples[\"question\"]]\n",
966+
"\n",
962967
" # Tokenize our examples with truncation and padding, but keep the overflows using a stride. This results\n",
963968
" # in one example possible giving several features when a context is long, each of those features having a\n",
964969
" # context that overlaps a bit the context of the previous feature.\n",

examples/text_classification.ipynb

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -977,6 +977,7 @@
977977
"args = TrainingArguments(\n",
978978
" \"test-glue\",\n",
979979
" evaluation_strategy = \"epoch\",\n",
980+
" save_strategy = \"epoch\",\n",
980981
" learning_rate=2e-5,\n",
981982
" per_device_train_batch_size=batch_size,\n",
982983
" per_device_eval_batch_size=batch_size,\n",

0 commit comments

Comments
 (0)