Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tweak IMDB TorchText interpretation tutorial #1516

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Tweak IMDB TorchText interpretation tutorial #1516

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b0d883d
Tweak IMDB TorchText Interpretation tutorial
j-t-1 Mar 2, 2025
241d9bf
Merge branch 'master' into imdb
jjuncho Mar 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions tutorials/IMDB_TorchText_Interpret.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook loads pretrained CNN model for sentiment analysis on IMDB dataset. It makes predictions on test samples and interprets those predictions using integrated gradients method.\n",
"This notebook loads a pretrained CNN model for sentiment analysis on an IMDB dataset. It makes predictions on test samples and interprets those predictions using the Integrated Gradients method.\n",
"\n",
"The model was trained using an open source sentiment analysis tutorials described in: https://github.com/bentrevett/pytorch-sentiment-analysis/blob/master/4%20-%20Convolutional%20Sentiment%20Analysis.ipynb with the following changes:\n",
"\n",
"- TEXT: set lower=True at initialization and call build_vocab() on the entire training data including validation to avoid mismatched indices\n",
"- TEXT: set lower=True at initialization and call build_vocab() on the entire training data including validation, to avoid mismatched indices\n",
"- model: save the entire model instead of just model.state_dict()\n",
"\n",
"**Note:** Before running this tutorial, please install the spacy package, and its NLP modules for English language."
Expand Down Expand Up @@ -81,7 +81,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The dataset used for training this model can be found in: https://ai.stanford.edu/~amaas/data/sentiment/\n",
"The dataset used for training this model can be found in: https://ai.stanford.edu/~amaas/data/sentiment/.\n",
"\n",
"Redefining the model in order to be able to load it.\n"
]
Expand Down Expand Up @@ -113,31 +113,31 @@
" \n",
" def forward(self, text):\n",
" \n",
" #text = [sent len, batch size]\n",
" # text = [sent len, batch size]\n",
" \n",
" #text = text.permute(1, 0)\n",
" # text = text.permute(1, 0)\n",
" \n",
" #text = [batch size, sent len]\n",
" # text = [batch size, sent len]\n",
" \n",
" embedded = self.embedding(text)\n",
"\n",
" #embedded = [batch size, sent len, emb dim]\n",
" # embedded = [batch size, sent len, emb dim]\n",
" \n",
" embedded = embedded.unsqueeze(1)\n",
" \n",
" #embedded = [batch size, 1, sent len, emb dim]\n",
" # embedded = [batch size, 1, sent len, emb dim]\n",
" \n",
" conved = [F.relu(conv(embedded)).squeeze(3) for conv in self.convs]\n",
" \n",
" #conved_n = [batch size, n_filters, sent len - filter_sizes[n] + 1]\n",
" # conved_n = [batch size, n_filters, sent len - filter_sizes[n] + 1]\n",
" \n",
" pooled = [F.max_pool1d(conv, conv.shape[2]).squeeze(2) for conv in conved]\n",
" \n",
" #pooled_n = [batch size, n_filters]\n",
" # pooled_n = [batch size, n_filters]\n",
" \n",
" cat = self.dropout(torch.cat(pooled, dim = 1))\n",
"\n",
" #cat = [batch size, n_filters * len(filter_sizes)]\n",
" # cat = [batch size, n_filters * len(filter_sizes)]\n",
" \n",
" return self.fc(cat)"
]
Expand All @@ -148,7 +148,7 @@
"source": [
"Loads pretrained model and sets the model to eval mode.\n",
"\n",
"The model can be downloaded here: https://github.com/pytorch/captum/blob/master/tutorials/models/imdb-model-cnn-large.pt"
"The model can be downloaded here: https://github.com/pytorch/captum/blob/master/tutorials/models/imdb-model-cnn-large.pt."
]
},
{
Expand All @@ -166,7 +166,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Forward function that supports sigmoid"
"Forward function that supports sigmoid."
]
},
{
Expand Down Expand Up @@ -290,7 +290,7 @@
"metadata": {},
"source": [
"Let's create an instance of `LayerIntegratedGradients` using forward function of our model and the embedding layer.\n",
"This instance of layer integrated gradients will be used to interpret movie rating review.\n",
"This instance of Layer Integrated Gradients will be used to interpret movie rating review.\n",
"\n",
"Layer Integrated Gradients will allow us to assign an attribution score to each word/token embedding tensor in the movie review text. We will ultimately sum the attribution scores across all embedding dimensions for each word/token in order to attain a word/token level attribution score.\n",
"\n",
Expand Down Expand Up @@ -319,7 +319,7 @@
"metadata": {},
"outputs": [],
"source": [
"# accumalate couple samples in this array for visualization purposes\n",
"# accumulate couple samples in this array for visualization purposes\n",
"vis_data_records_ig = []\n",
"\n",
"def interpret_sentence(model, sentence, min_len = 7, label = 0):\n",
Expand Down