@@ -16,8 +16,8 @@ models.
16
16
17
17
### How to Initialize from Checkpoint
18
18
19
- ** Note:** TF-HUB/Savedmodel is the preferred way to distribute models as it is
20
- self-contained. Please consider using TF-HUB for finetuning tasks first.
19
+ ** Note:** TF-HUB/Kaggle- Savedmodel is the preferred way to distribute models as it is
20
+ self-contained. Please consider using TF-HUB/Kaggle for finetuning tasks first.
21
21
22
22
If you use the [ NLP training library] ( train.md ) ,
23
23
you can specify the checkpoint path link directly when launching your job. For
@@ -29,10 +29,10 @@ python3 train.py \
29
29
--params_override=task.init_checkpoint=PATH_TO_INIT_CKPT
30
30
```
31
31
32
- ### How to load TF-HUB SavedModel
32
+ ### How to load TF-HUB/Kaggle SavedModel
33
33
34
34
Finetuning tasks such as question answering (SQuAD) and sentence
35
- prediction (GLUE) support loading a model from TF-HUB. These built-in tasks
35
+ prediction (GLUE) support loading a model from TF-HUB/Kaggle . These built-in tasks
36
36
support a specific ` task.hub_module_url ` parameter. To set this parameter,
37
37
replace ` --params_override=task.init_checkpoint=... ` with
38
38
` --params_override=task.hub_module_url=TF_HUB_URL ` , like below:
@@ -54,7 +54,7 @@ in order to keep consistent with BERT paper.
54
54
55
55
### Checkpoints
56
56
57
- Model | Configuration | Training Data | Checkpoint & Vocabulary | TF-HUB SavedModels
57
+ Model | Configuration | Training Data | Checkpoint & Vocabulary | Kaggle SavedModels
58
58
---------------------------------------- | :--------------------------: | ------------: | ----------------------: | ------:
59
59
BERT-base uncased English | uncased_L-12_H-768_A-12 | Wiki + Books | [ uncased_L-12_H-768_A-12] ( https://storage.googleapis.com/tf_model_garden/nlp/bert/v3/uncased_L-12_H-768_A-12.tar.gz ) | [ ` BERT-Base, Uncased ` ] ( https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/ )
60
60
BERT-base cased English | cased_L-12_H-768_A-12 | Wiki + Books | [ cased_L-12_H-768_A-12] ( https://storage.googleapis.com/tf_model_garden/nlp/bert/v3/cased_L-12_H-768_A-12.tar.gz ) | [ ` BERT-Base, Cased ` ] ( https://tfhub.dev/tensorflow/bert_en_cased_L-12_H-768_A-12/ )
@@ -74,7 +74,7 @@ We also have pretrained BERT models with variants in both network architecture
74
74
and training methodologies. These models achieve higher downstream accuracy
75
75
scores.
76
76
77
- Model | Configuration | Training Data | TF-HUB SavedModels | Comment
77
+ Model | Configuration | Training Data | Kaggle SavedModels | Comment
78
78
-------------------------------- | :----------------------: | -----------------------: | ------------------------------------------------------------------------------------: | ------:
79
79
BERT-base talking heads + ggelu | uncased_L-12_H-768_A-12 | Wiki + Books | [ talkheads_ggelu_base] ( https://tfhub.dev/tensorflow/talkheads_ggelu_bert_en_base/1 ) | BERT-base trained with [ talking heads attention] ( https://arxiv.org/abs/2003.02436 ) and [ gated GeLU] ( https://arxiv.org/abs/2002.05202 ) .
80
80
BERT-large talking heads + ggelu | uncased_L-24_H-1024_A-16 | Wiki + Books | [ talkheads_ggelu_large] ( https://tfhub.dev/tensorflow/talkheads_ggelu_bert_en_large/1 ) | BERT-large trained with [ talking heads attention] ( https://arxiv.org/abs/2003.02436 ) and [ gated GeLU] ( https://arxiv.org/abs/2002.05202 ) .
@@ -96,12 +96,12 @@ ALBERT repository.
96
96
97
97
### Checkpoints
98
98
99
- Model | Training Data | Checkpoint & Vocabulary | TF-HUB SavedModels
99
+ Model | Training Data | Checkpoint & Vocabulary | Kaggle SavedModels
100
100
---------------------------------------- | ------------: | ----------------------: | ------:
101
- ALBERT-base English | Wiki + Books | [ ` ALBERT Base ` ] ( https://storage.googleapis.com/tf_model_garden/nlp/albert/albert_base.tar.gz ) | https://tfhub.dev/tensorflow/albert_en_base/3
102
- ALBERT-large English | Wiki + Books | [ ` ALBERT Large ` ] ( https://storage.googleapis.com/tf_model_garden/nlp/albert/albert_large.tar.gz ) | https://tfhub.dev/tensorflow/albert_en_large/3
103
- ALBERT-xlarge English | Wiki + Books | [ ` ALBERT XLarge ` ] ( https://storage.googleapis.com/tf_model_garden/nlp/albert/albert_xlarge.tar.gz ) | https://tfhub.dev/tensorflow/albert_en_xlarge/3
104
- ALBERT-xxlarge English | Wiki + Books | [ ` ALBERT XXLarge ` ] ( https://storage.googleapis.com/tf_model_garden/nlp/albert/albert_xxlarge.tar.gz ) | https://tfhub.dev/tensorflow/albert_en_xxlarge/3
101
+ ALBERT-base English | Wiki + Books | [ ` ALBERT Base ` ] ( https://storage.googleapis.com/tf_model_garden/nlp/albert/albert_base.tar.gz ) | [ albert_en_base ] ( https://tfhub.dev/tensorflow/albert_en_base/3 )
102
+ ALBERT-large English | Wiki + Books | [ ` ALBERT Large ` ] ( https://storage.googleapis.com/tf_model_garden/nlp/albert/albert_large.tar.gz ) | [ albert_en_large ] ( https://tfhub.dev/tensorflow/albert_en_large/3 )
103
+ ALBERT-xlarge English | Wiki + Books | [ ` ALBERT XLarge ` ] ( https://storage.googleapis.com/tf_model_garden/nlp/albert/albert_xlarge.tar.gz ) | [ albert_en_xlarge ] ( https://tfhub.dev/tensorflow/albert_en_xlarge/3 )
104
+ ALBERT-xxlarge English | Wiki + Books | [ ` ALBERT XXLarge ` ] ( https://storage.googleapis.com/tf_model_garden/nlp/albert/albert_xxlarge.tar.gz ) | [ albert_en_xxlarge ] ( https://tfhub.dev/tensorflow/albert_en_xxlarge/3 )
105
105
106
106
107
107
## ELECTRA
0 commit comments