Skip to content

Commit bf6b536

Browse files
author
Vijay Vasudevan
committed
TensorFlow: Upstream changes to git.
Change 109240606 Fix typo Change 109240358 Fix bug in Concat's shape inference due to legacy scalar handling. The shape function was inadvertently converting outputs of unknown shape (rank=None) to vectors of unknown length (rank=1), due to inability to distinguish between legacy scalars and vectors, because `max(1, None)` is 1. Change 109237152 Remove numarray requirement in python_config. Change 109234003 Fix typo in elu documentation. Change 109232946 Python must now be configured via ./configure script Change 109232134 Backported fixes to the tensor comparison operators from the public Eigen repository Change 109231761 Test invalid inputs to softmax_cross_entropy_with_logits. Change 109230218 Backported fixes to the tensor comparison operators from the public Eigen repository Change 109229915 Correct comments in seq2seq to show the right input types for embedding models. (Thanks to hugman@github for bringing this up.) Change 109229118 Fix resize_images example in documentation and allow resize_images to run on a single image with partially-known shape. Change 109228940 Fix demo and node add/remove button spacing Change 109227909 Include Elu in the NN docs. Change 109227059 Adds variable_op_scope and makes variable_scope always add a name_scope. This creates an op scope for variables that makes it easy to create independent operations with a default name by making that name unique for the current scope and it allows explicit names that are not made unique. Change 109224492 Streamline yuv -> rgb conversion to be done in one pass in native code. The entire process now takes ~2ms (including the ByteBuffer.get() calls), down from 10+ ms when the arrays were being interleaved in Java prior to conversion. Also abstracting common yuv->rgb color conversion into helper method. Change 109224389 Add ability to move nodes in and out of auxiliary nodes in graph. Change 109217177 Update generated Op docs. Change 109215030 Implementation of the ELU activation function: http://arxiv.org/abs/1511.07289 Change 109209848 When GPUBFCAllocator runs out of memory, also log a summary of chunks in use by size. Change 109206569 Switched to the public version of the Eigen::sign method since it supports complex numbers. Change 109199813 Modify tensorflow.SequenceExample to support multiple-length sequences. Base CL: 109241553
1 parent fa095c5 commit bf6b536

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

57 files changed

+1923
-338
lines changed

‎configure

Lines changed: 30 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,38 @@
11
#!/bin/bash
22

3+
## Set up python-related environment settings
4+
while true; do
5+
fromuser=""
6+
if [ -z "$PYTHON_BIN_PATH" ]; then
7+
default_python_bin_path=$(which python)
8+
read -p "Please specify the location of python. [Default is $default_python_bin_path]: " PYTHON_BIN_PATH
9+
fromuser="1"
10+
if [ -z "$PYTHON_BIN_PATH" ]; then
11+
PYTHON_BIN_PATH=$default_python_bin_path
12+
fi
13+
fi
14+
if [ -e "$PYTHON_BIN_PATH" ]; then
15+
break
16+
fi
17+
echo "Invalid python path. ${PYTHON_BIN_PATH} cannot be found" 1>&2
18+
if [ -z "$fromuser" ]; then
19+
exit 1
20+
fi
21+
PYTHON_BIN_PATH=""
22+
# Retry
23+
done
24+
25+
# Invoke python_config and set up symlinks to python includes
26+
(./util/python/python_config.sh --setup "$PYTHON_BIN_PATH";) || exit -1
27+
328
## Set up Cuda-related environment settings
429

530
while [ "$TF_NEED_CUDA" == "" ]; do
6-
read -p "Do you wish to build TensorFlow with GPU support? [y/n] " INPUT
31+
read -p "Do you wish to build TensorFlow with GPU support? [y/N] " INPUT
732
case $INPUT in
8-
[Yy]* ) echo -e "GPU support will be enabled for TensorFlow\n"; TF_NEED_CUDA=1;;
9-
[Nn]* ) echo -e "No GPU support will be enabled for TensorFlow\n"; TF_NEED_CUDA=0;;
33+
[Yy]* ) echo "GPU support will be enabled for TensorFlow"; TF_NEED_CUDA=1;;
34+
[Nn]* ) echo "No GPU support will be enabled for TensorFlow"; TF_NEED_CUDA=0;;
35+
"" ) echo "No GPU support will be enabled for TensorFlow"; TF_NEED_CUDA=0;;
1036
* ) echo "Invalid selection: " $INPUT;;
1137
esac
1238
done
@@ -77,7 +103,7 @@ CUDNN_INSTALL_PATH="$CUDNN_INSTALL_PATH"
77103
EOF
78104

79105
function UnofficialSetting() {
80-
echo -e "\nWARNING: You are configuring unofficial settings in TensorFlow. Because some external libraries are not backward compatible, these settings are largely untested and unsupported. \n"
106+
echo -e "\nWARNING: You are configuring unofficial settings in TensorFlow. Because some external libraries are not backward compatible, these settings are largely untested and unsupported. \n" 1>&2
81107

82108
# Configure the compute capabilities that TensorFlow builds for.
83109
# Since Cuda toolkit is not backward-compatible, this is not guaranteed to work.

‎tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -342,6 +342,7 @@ size_t GPUBFCAllocator::AllocatedSize(void* ptr) {
342342

343343
void GPUBFCAllocator::DumpMemoryLog(size_t num_bytes) {
344344
// For each bin: tally up the total number of chunks and bytes.
345+
// Note that bins hold only free chunks.
345346
for (auto bit : bins_) {
346347
Bin* b = bit.second;
347348

@@ -389,6 +390,24 @@ void GPUBFCAllocator::DumpMemoryLog(size_t num_bytes) {
389390
LOG(INFO) << c->DebugString(true);
390391
}
391392
}
392-
}
393393

394+
// Next show the the chunks that are in use, and also summarize their
395+
// number by size.
396+
std::map<size_t, int> in_use_by_size;
397+
for (auto& it : ptr_to_chunk_map_) {
398+
const Chunk& c = *it.second;
399+
in_use_by_size[c.size]++;
400+
LOG(INFO) << "Chunk at " << it.first << " of size " << c.size;
401+
}
402+
403+
LOG(INFO) << " Summary of in-use Chunks by size: ";
404+
size_t total_bytes = 0;
405+
for (auto& it : in_use_by_size) {
406+
LOG(INFO) << it.second << " Chunks of size " << it.first << " totalling "
407+
<< strings::HumanReadableNumBytes(it.first * it.second);
408+
total_bytes += (it.first * it.second);
409+
}
410+
LOG(INFO) << "Sum Total of in-use chunks: "
411+
<< strings::HumanReadableNumBytes(total_bytes);
412+
}
394413
} // namespace tensorflow

‎tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.h

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -115,14 +115,14 @@ class GPUBFCAllocator : public VisitableAllocator {
115115
};
116116

117117
Chunk* AllocateNewChunk(size_t num_bytes);
118-
void SplitChunk(Chunk* c, size_t num_bytes);
119-
void Merge(Chunk* c1, Chunk* c2);
120-
void FreeAndMaybeCoalesce(Chunk* c);
121-
void InsertFreeChunkIntoBin(Chunk* c);
118+
void SplitChunk(Chunk* c, size_t num_bytes) EXCLUSIVE_LOCKS_REQUIRED(lock_);
119+
void Merge(Chunk* c1, Chunk* c2) EXCLUSIVE_LOCKS_REQUIRED(lock_);
120+
void FreeAndMaybeCoalesce(Chunk* c) EXCLUSIVE_LOCKS_REQUIRED(lock_);
121+
void InsertFreeChunkIntoBin(Chunk* c) EXCLUSIVE_LOCKS_REQUIRED(lock_);
122122
void RemoveFreeChunkFromBin(Chunk* c);
123-
void DeleteChunk(Chunk* c);
123+
void DeleteChunk(Chunk* c) EXCLUSIVE_LOCKS_REQUIRED(lock_);
124124

125-
void DumpMemoryLog(size_t num_bytes);
125+
void DumpMemoryLog(size_t num_bytes) EXCLUSIVE_LOCKS_REQUIRED(lock_);
126126

127127
// A Bin is a collection of similar-sized free chunks.
128128
struct Bin {
@@ -163,7 +163,7 @@ class GPUBFCAllocator : public VisitableAllocator {
163163
// Structures mutable after construction
164164
mutable mutex lock_;
165165
// Chunk * owned.
166-
std::unordered_map<void*, Chunk*> ptr_to_chunk_map_;
166+
std::unordered_map<void*, Chunk*> ptr_to_chunk_map_ GUARDED_BY(lock_);
167167

168168
// Called once on each region, ASAP.
169169
std::vector<Visitor> region_visitors_;

‎tensorflow/core/example/example.proto

Lines changed: 187 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,21 @@ import "tensorflow/core/example/feature.proto";
77

88
package tensorflow;
99

10-
// Example for a movie recommendation application:
10+
// An Example is a mostly-normalized data format for storing data for
11+
// training and inference. It contains a key-value store (features); where
12+
// each key (string) maps to a Feature message (which is oneof packed BytesList,
13+
// FloatList, or Int64List). This flexible and compact format allows the
14+
// storage of large amounts of typed data, but requires that the data shape
15+
// and use be determined by the configuration files and parsers that are used to
16+
// read and write this format. That is, the Example is mostly *not* a
17+
// self-describing format. In TensorFlow, Examples are read in row-major
18+
// format, so any configuration that describes data with rank-2 or above
19+
// should keep this in mind. For example, to store an M x N matrix of Bytes,
20+
// the BytesList must contain M*N bytes, with M rows of N contiguous values
21+
// each. That is, the BytesList value must store the matrix as:
22+
// .... row 0 .... .... row 1 .... // ........... // ... row M-1 ....
23+
//
24+
// An Example for a movie recommendation application:
1125
// features {
1226
// feature {
1327
// key: "age"
@@ -58,7 +72,7 @@ package tensorflow;
5872
// }
5973
// }
6074
//
61-
// A conformant data set obeys the following conventions:
75+
// A conformant Example data set obeys the following conventions:
6276
// - If a Feature K exists in one example with data type T, it must be of
6377
// type T in all other examples when present. It may be omitted.
6478
// - The number of instances of Feature K list data may vary across examples,
@@ -72,23 +86,182 @@ message Example {
7286
Features features = 1;
7387
};
7488

75-
// Example representing a ranking instance.
76-
message RankingExample {
77-
Features context = 1;
78-
repeated Features positive = 2;
79-
repeated Features negative = 3;
80-
};
89+
// A SequenceExample is an Example representing one or more sequences, and
90+
// some context. The context contains features which apply to the entire
91+
// example. The feature_lists contain a key, value map where each key is
92+
// associated with a repeated set of Features (a FeatureList).
93+
//
94+
// A SequenceExample for a movie recommendation application:
95+
//
96+
// context: {
97+
// feature: {
98+
// key : "locale"
99+
// value: {
100+
// bytes_list: {
101+
// value: [ "pt_BR" ]
102+
// }
103+
// }
104+
// }
105+
// feature: {
106+
// key : "age"
107+
// value: {
108+
// float_list: {
109+
// value: [ 19.0 ]
110+
// }
111+
// }
112+
// }
113+
// feature: {
114+
// key : "favorites"
115+
// value: {
116+
// bytes_list: {
117+
// value: [ "Majesty Rose", "Savannah Outen", "One Direction" ]
118+
// }
119+
// }
120+
// }
121+
// }
122+
// feature_lists: {
123+
// feature_list: {
124+
// key : "movie_ratings"
125+
// value: {
126+
// feature: {
127+
// float_list: {
128+
// value: [ 4.5 ]
129+
// }
130+
// }
131+
// feature: {
132+
// float_list: {
133+
// value: [ 5.0 ]
134+
// }
135+
// }
136+
// }
137+
// }
138+
// feature_list: {
139+
// key : "movie_names"
140+
// value: {
141+
// feature: {
142+
// bytes_list: {
143+
// value: [ "The Shawshank Redemption" ]
144+
// }
145+
// }
146+
// feature: {
147+
// bytes_list: {
148+
// value: [ "Fight Club" ]
149+
// }
150+
// }
151+
// }
152+
// }
153+
// }
154+
//
155+
// A conformant SequenceExample data set obeys the following conventions:
156+
//
157+
// Context:
158+
// - All conformant context features K must obey the same conventions as
159+
// a conformant Example's features (see above).
160+
// Feature lists:
161+
// - A FeatureList L may be missing in an example; it is up to the
162+
// parser configuration to determine if this is allowed or considered
163+
// an empty list (zero length).
164+
// - If a FeatureList L exists, it may be empty (zero length).
165+
// - If a FeatureList L is non-empty, all features within the FeatureList
166+
// must have data type T, and all features within the FeatureList must
167+
// have the same size.
168+
// - If a FeatureList L exists in one example with data type T,
169+
// it must be of type T in all other examples when present.
170+
// - If a FeatureList L exists in one example having features' sizes all S,
171+
// these sizes must be S in all other examples when present.
172+
//
173+
// Examples of conformant and non-conformant examples' FeatureLists:
174+
//
175+
// Conformant FeatureLists:
176+
// feature_lists: { feature_list: {
177+
// key: "movie_ratings"
178+
// value: { feature: { float_list: { value: [ 4.5 ] } }
179+
// feature: { float_list: { value: [ 5.0 ] } } }
180+
// } }
181+
//
182+
// Non-conformant FeatureLists (mismatched types):
183+
// feature_lists: { feature_list: {
184+
// key: "movie_ratings"
185+
// value: { feature: { float_list: { value: [ 4.5 ] } }
186+
// feature: { int64_list: { value: [ 5 ] } } }
187+
// } }
188+
//
189+
// Non-conformant FeatureLists (mismatched sizes):
190+
// feature_lists: { feature_list: {
191+
// key: "movie_ratings"
192+
// value: { feature: { float_list: { value: [ 4.5 ] } }
193+
// feature: { float_list: { value: [ 5.0, 6.0 ] } } }
194+
// } }
195+
//
196+
// Conformant pair of SequenceExample
197+
// feature_lists: { feature_list: {
198+
// key: "movie_ratings"
199+
// value: { feature: { float_list: { value: [ 4.5 ] } }
200+
// feature: { float_list: { value: [ 5.0 ] } } }
201+
// } }
202+
// and:
203+
// feature_lists: { feature_list: {
204+
// key: "movie_ratings"
205+
// value: { feature: { float_list: { value: [ 4.5 ] } }
206+
// feature: { float_list: { value: [ 5.0 ] } }
207+
// feature: { float_list: { value: [ 2.0 ] } } }
208+
// } }
209+
//
210+
// Conformant pair of SequenceExample
211+
// feature_lists: { feature_list: {
212+
// key: "movie_ratings"
213+
// value: { feature: { float_list: { value: [ 4.5 ] } }
214+
// feature: { float_list: { value: [ 5.0 ] } } }
215+
// } }
216+
// and:
217+
// feature_lists: { feature_list: {
218+
// key: "movie_ratings"
219+
// value: { }
220+
// } }
221+
//
222+
// Conditionally conformant pair of SequenceExample, the parser configuration
223+
// determines if the second feature_lists is consistent (zero-length) or
224+
// invalid (missing "movie_ratings"):
225+
// feature_lists: { feature_list: {
226+
// key: "movie_ratings"
227+
// value: { feature: { float_list: { value: [ 4.5 ] } }
228+
// feature: { float_list: { value: [ 5.0 ] } } }
229+
// } }
230+
// and:
231+
// feature_lists: { }
232+
//
233+
// Non-conformant pair of SequenceExample (mismatched types)
234+
// feature_lists: { feature_list: {
235+
// key: "movie_ratings"
236+
// value: { feature: { float_list: { value: [ 4.5 ] } }
237+
// feature: { float_list: { value: [ 5.0 ] } } }
238+
// } }
239+
// and:
240+
// feature_lists: { feature_list: {
241+
// key: "movie_ratings"
242+
// value: { feature: { int64_list: { value: [ 4 ] } }
243+
// feature: { int64_list: { value: [ 5 ] } }
244+
// feature: { int64_list: { value: [ 2 ] } } }
245+
// } }
246+
//
247+
// Non-conformant pair of SequenceExample (mismatched sizes)
248+
// feature_lists: { feature_list: {
249+
// key: "movie_ratings"
250+
// value: { feature: { float_list: { value: [ 4.5 ] } }
251+
// feature: { float_list: { value: [ 5.0 ] } } }
252+
// } }
253+
// and:
254+
// feature_lists: { feature_list: {
255+
// key: "movie_ratings"
256+
// value: { feature: { float_list: { value: [ 4.0, 5.0 ] } }
257+
// feature: { float_list: { value: [ 5.0, 3.0 ] } }
258+
// } }
81259

82-
// Example representing a sequence.
83-
// The context contains features which apply to the entire sequence.
84-
// Each element in example represents an entry in the sequence.
85260
message SequenceExample {
86261
Features context = 1;
87-
repeated Features features = 2;
262+
FeatureLists feature_lists = 2;
88263
};
89264

90-
// Example representing a list of feature maps.
91-
// The context contains features which apply to all feature maps.
92265
message InferenceExample {
93266
Features context = 1;
94267
repeated Features features = 2;

0 commit comments

Comments
 (0)