Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a keras floating point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original keras floating point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf.keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, width, height, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
49152/11490434 [..............................] - ETA: 18s
81920/11490434 [..............................] - ETA: 21s
344064/11490434 [..............................] - ETA: 6s
499712/11490434 [>.............................] - ETA: 5s
745472/11490434 [>.............................] - ETA: 4s
1261568/11490434 [==>...........................] - ETA: 2s
1867776/11490434 [===>..........................] - ETA: 2s
2891776/11490434 [======>.......................] - ETA: 1s
4390912/11490434 [==========>...................] - ETA: 0s
5980160/11490434 [==============>...............] - ETA: 0s
7553024/11490434 [==================>...........] - ETA: 0s
9224192/11490434 [=======================>......] - ETA: 0s
10797056/11490434 [===========================>..] - ETA: 0s
11490434/11490434 [==============================] - 1s 0us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard Keras CNN model.
import keras
model_keras = keras.models.Sequential([
keras.layers.Rescaling(1. / 255, input_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 1:16:47 - loss: 2.6303 - accuracy: 0.1875
21/1688 [..............................] - ETA: 4s - loss: 1.5527 - accuracy: 0.4851
42/1688 [..............................] - ETA: 4s - loss: 1.1442 - accuracy: 0.6265
63/1688 [>.............................] - ETA: 4s - loss: 0.9196 - accuracy: 0.7054
84/1688 [>.............................] - ETA: 3s - loss: 0.7894 - accuracy: 0.7530
105/1688 [>.............................] - ETA: 3s - loss: 0.7093 - accuracy: 0.7759
125/1688 [=>............................] - ETA: 3s - loss: 0.6551 - accuracy: 0.7943
146/1688 [=>............................] - ETA: 3s - loss: 0.5965 - accuracy: 0.8134
167/1688 [=>............................] - ETA: 3s - loss: 0.5601 - accuracy: 0.8252
188/1688 [==>...........................] - ETA: 3s - loss: 0.5283 - accuracy: 0.8364
209/1688 [==>...........................] - ETA: 3s - loss: 0.5016 - accuracy: 0.8448
230/1688 [===>..........................] - ETA: 3s - loss: 0.4738 - accuracy: 0.8538
251/1688 [===>..........................] - ETA: 3s - loss: 0.4555 - accuracy: 0.8592
272/1688 [===>..........................] - ETA: 3s - loss: 0.4393 - accuracy: 0.8637
293/1688 [====>.........................] - ETA: 3s - loss: 0.4207 - accuracy: 0.8692
314/1688 [====>.........................] - ETA: 3s - loss: 0.4053 - accuracy: 0.8735
335/1688 [====>.........................] - ETA: 3s - loss: 0.3916 - accuracy: 0.8776
356/1688 [=====>........................] - ETA: 3s - loss: 0.3832 - accuracy: 0.8800
377/1688 [=====>........................] - ETA: 3s - loss: 0.3727 - accuracy: 0.8832
398/1688 [======>.......................] - ETA: 3s - loss: 0.3637 - accuracy: 0.8863
419/1688 [======>.......................] - ETA: 3s - loss: 0.3543 - accuracy: 0.8895
440/1688 [======>.......................] - ETA: 3s - loss: 0.3456 - accuracy: 0.8924
461/1688 [=======>......................] - ETA: 3s - loss: 0.3355 - accuracy: 0.8958
481/1688 [=======>......................] - ETA: 2s - loss: 0.3291 - accuracy: 0.8981
501/1688 [=======>......................] - ETA: 2s - loss: 0.3218 - accuracy: 0.9001
522/1688 [========>.....................] - ETA: 2s - loss: 0.3151 - accuracy: 0.9021
543/1688 [========>.....................] - ETA: 2s - loss: 0.3092 - accuracy: 0.9037
564/1688 [=========>....................] - ETA: 2s - loss: 0.3032 - accuracy: 0.9056
585/1688 [=========>....................] - ETA: 2s - loss: 0.2988 - accuracy: 0.9070
606/1688 [=========>....................] - ETA: 2s - loss: 0.2930 - accuracy: 0.9087
627/1688 [==========>...................] - ETA: 2s - loss: 0.2882 - accuracy: 0.9106
648/1688 [==========>...................] - ETA: 2s - loss: 0.2841 - accuracy: 0.9118
669/1688 [==========>...................] - ETA: 2s - loss: 0.2785 - accuracy: 0.9137
690/1688 [===========>..................] - ETA: 2s - loss: 0.2740 - accuracy: 0.9153
711/1688 [===========>..................] - ETA: 2s - loss: 0.2699 - accuracy: 0.9164
732/1688 [============>.................] - ETA: 2s - loss: 0.2659 - accuracy: 0.9177
753/1688 [============>.................] - ETA: 2s - loss: 0.2640 - accuracy: 0.9182
773/1688 [============>.................] - ETA: 2s - loss: 0.2601 - accuracy: 0.9195
794/1688 [=============>................] - ETA: 2s - loss: 0.2561 - accuracy: 0.9208
815/1688 [=============>................] - ETA: 2s - loss: 0.2517 - accuracy: 0.9220
836/1688 [=============>................] - ETA: 2s - loss: 0.2487 - accuracy: 0.9229
857/1688 [==============>...............] - ETA: 2s - loss: 0.2445 - accuracy: 0.9242
878/1688 [==============>...............] - ETA: 1s - loss: 0.2414 - accuracy: 0.9252
900/1688 [==============>...............] - ETA: 1s - loss: 0.2381 - accuracy: 0.9263
922/1688 [===============>..............] - ETA: 1s - loss: 0.2355 - accuracy: 0.9270
944/1688 [===============>..............] - ETA: 1s - loss: 0.2334 - accuracy: 0.9278
966/1688 [================>.............] - ETA: 1s - loss: 0.2309 - accuracy: 0.9286
987/1688 [================>.............] - ETA: 1s - loss: 0.2280 - accuracy: 0.9294
1008/1688 [================>.............] - ETA: 1s - loss: 0.2249 - accuracy: 0.9304
1029/1688 [=================>............] - ETA: 1s - loss: 0.2227 - accuracy: 0.9311
1051/1688 [=================>............] - ETA: 1s - loss: 0.2200 - accuracy: 0.9318
1072/1688 [==================>...........] - ETA: 1s - loss: 0.2189 - accuracy: 0.9323
1092/1688 [==================>...........] - ETA: 1s - loss: 0.2163 - accuracy: 0.9332
1113/1688 [==================>...........] - ETA: 1s - loss: 0.2148 - accuracy: 0.9338
1134/1688 [===================>..........] - ETA: 1s - loss: 0.2124 - accuracy: 0.9344
1155/1688 [===================>..........] - ETA: 1s - loss: 0.2105 - accuracy: 0.9351
1176/1688 [===================>..........] - ETA: 1s - loss: 0.2081 - accuracy: 0.9359
1197/1688 [====================>.........] - ETA: 1s - loss: 0.2059 - accuracy: 0.9365
1219/1688 [====================>.........] - ETA: 1s - loss: 0.2036 - accuracy: 0.9372
1241/1688 [=====================>........] - ETA: 1s - loss: 0.2015 - accuracy: 0.9379
1262/1688 [=====================>........] - ETA: 1s - loss: 0.2000 - accuracy: 0.9384
1283/1688 [=====================>........] - ETA: 0s - loss: 0.1981 - accuracy: 0.9390
1304/1688 [======================>.......] - ETA: 0s - loss: 0.1969 - accuracy: 0.9395
1326/1688 [======================>.......] - ETA: 0s - loss: 0.1955 - accuracy: 0.9398
1347/1688 [======================>.......] - ETA: 0s - loss: 0.1936 - accuracy: 0.9405
1368/1688 [=======================>......] - ETA: 0s - loss: 0.1920 - accuracy: 0.9409
1389/1688 [=======================>......] - ETA: 0s - loss: 0.1906 - accuracy: 0.9414
1410/1688 [========================>.....] - ETA: 0s - loss: 0.1889 - accuracy: 0.9418
1431/1688 [========================>.....] - ETA: 0s - loss: 0.1876 - accuracy: 0.9422
1453/1688 [========================>.....] - ETA: 0s - loss: 0.1864 - accuracy: 0.9425
1473/1688 [=========================>....] - ETA: 0s - loss: 0.1853 - accuracy: 0.9430
1495/1688 [=========================>....] - ETA: 0s - loss: 0.1836 - accuracy: 0.9435
1516/1688 [=========================>....] - ETA: 0s - loss: 0.1824 - accuracy: 0.9438
1537/1688 [==========================>...] - ETA: 0s - loss: 0.1811 - accuracy: 0.9443
1558/1688 [==========================>...] - ETA: 0s - loss: 0.1798 - accuracy: 0.9446
1580/1688 [===========================>..] - ETA: 0s - loss: 0.1783 - accuracy: 0.9451
1602/1688 [===========================>..] - ETA: 0s - loss: 0.1774 - accuracy: 0.9453
1623/1688 [===========================>..] - ETA: 0s - loss: 0.1765 - accuracy: 0.9457
1644/1688 [============================>.] - ETA: 0s - loss: 0.1756 - accuracy: 0.9459
1665/1688 [============================>.] - ETA: 0s - loss: 0.1744 - accuracy: 0.9463
1686/1688 [============================>.] - ETA: 0s - loss: 0.1732 - accuracy: 0.9467
1688/1688 [==============================] - ETA: 0s - loss: 0.1731 - accuracy: 0.9467
1688/1688 [==============================] - 7s 3ms/step - loss: 0.1731 - accuracy: 0.9467 - val_loss: 0.0696 - val_accuracy: 0.9805
Epoch 2/10
1/1688 [..............................] - ETA: 5s - loss: 0.0466 - accuracy: 0.9688
22/1688 [..............................] - ETA: 3s - loss: 0.0830 - accuracy: 0.9730
42/1688 [..............................] - ETA: 4s - loss: 0.0800 - accuracy: 0.9747
63/1688 [>.............................] - ETA: 3s - loss: 0.0785 - accuracy: 0.9752
84/1688 [>.............................] - ETA: 3s - loss: 0.0851 - accuracy: 0.9721
105/1688 [>.............................] - ETA: 3s - loss: 0.0836 - accuracy: 0.9735
126/1688 [=>............................] - ETA: 3s - loss: 0.0784 - accuracy: 0.9747
147/1688 [=>............................] - ETA: 3s - loss: 0.0820 - accuracy: 0.9741
167/1688 [=>............................] - ETA: 3s - loss: 0.0806 - accuracy: 0.9742
188/1688 [==>...........................] - ETA: 3s - loss: 0.0782 - accuracy: 0.9744
209/1688 [==>...........................] - ETA: 3s - loss: 0.0746 - accuracy: 0.9756
230/1688 [===>..........................] - ETA: 3s - loss: 0.0730 - accuracy: 0.9762
251/1688 [===>..........................] - ETA: 3s - loss: 0.0738 - accuracy: 0.9763
272/1688 [===>..........................] - ETA: 3s - loss: 0.0725 - accuracy: 0.9767
293/1688 [====>.........................] - ETA: 3s - loss: 0.0728 - accuracy: 0.9765
313/1688 [====>.........................] - ETA: 3s - loss: 0.0739 - accuracy: 0.9763
334/1688 [====>.........................] - ETA: 3s - loss: 0.0741 - accuracy: 0.9762
354/1688 [=====>........................] - ETA: 3s - loss: 0.0721 - accuracy: 0.9768
375/1688 [=====>........................] - ETA: 3s - loss: 0.0725 - accuracy: 0.9768
396/1688 [======>.......................] - ETA: 3s - loss: 0.0713 - accuracy: 0.9774
416/1688 [======>.......................] - ETA: 3s - loss: 0.0719 - accuracy: 0.9776
437/1688 [======>.......................] - ETA: 3s - loss: 0.0703 - accuracy: 0.9781
459/1688 [=======>......................] - ETA: 3s - loss: 0.0700 - accuracy: 0.9779
480/1688 [=======>......................] - ETA: 2s - loss: 0.0702 - accuracy: 0.9779
501/1688 [=======>......................] - ETA: 2s - loss: 0.0690 - accuracy: 0.9784
522/1688 [========>.....................] - ETA: 2s - loss: 0.0691 - accuracy: 0.9785
543/1688 [========>.....................] - ETA: 2s - loss: 0.0692 - accuracy: 0.9785
565/1688 [=========>....................] - ETA: 2s - loss: 0.0693 - accuracy: 0.9787
586/1688 [=========>....................] - ETA: 2s - loss: 0.0692 - accuracy: 0.9788
606/1688 [=========>....................] - ETA: 2s - loss: 0.0689 - accuracy: 0.9788
626/1688 [==========>...................] - ETA: 2s - loss: 0.0692 - accuracy: 0.9784
647/1688 [==========>...................] - ETA: 2s - loss: 0.0696 - accuracy: 0.9784
668/1688 [==========>...................] - ETA: 2s - loss: 0.0698 - accuracy: 0.9784
690/1688 [===========>..................] - ETA: 2s - loss: 0.0704 - accuracy: 0.9783
710/1688 [===========>..................] - ETA: 2s - loss: 0.0704 - accuracy: 0.9783
731/1688 [===========>..................] - ETA: 2s - loss: 0.0710 - accuracy: 0.9782
752/1688 [============>.................] - ETA: 2s - loss: 0.0712 - accuracy: 0.9781
773/1688 [============>.................] - ETA: 2s - loss: 0.0715 - accuracy: 0.9780
793/1688 [=============>................] - ETA: 2s - loss: 0.0712 - accuracy: 0.9782
813/1688 [=============>................] - ETA: 2s - loss: 0.0715 - accuracy: 0.9782
834/1688 [=============>................] - ETA: 2s - loss: 0.0726 - accuracy: 0.9780
855/1688 [==============>...............] - ETA: 2s - loss: 0.0731 - accuracy: 0.9780
876/1688 [==============>...............] - ETA: 1s - loss: 0.0725 - accuracy: 0.9780
897/1688 [==============>...............] - ETA: 1s - loss: 0.0731 - accuracy: 0.9779
918/1688 [===============>..............] - ETA: 1s - loss: 0.0732 - accuracy: 0.9778
939/1688 [===============>..............] - ETA: 1s - loss: 0.0725 - accuracy: 0.9781
960/1688 [================>.............] - ETA: 1s - loss: 0.0728 - accuracy: 0.9781
982/1688 [================>.............] - ETA: 1s - loss: 0.0729 - accuracy: 0.9781
1003/1688 [================>.............] - ETA: 1s - loss: 0.0723 - accuracy: 0.9782
1024/1688 [=================>............] - ETA: 1s - loss: 0.0719 - accuracy: 0.9784
1045/1688 [=================>............] - ETA: 1s - loss: 0.0716 - accuracy: 0.9785
1066/1688 [=================>............] - ETA: 1s - loss: 0.0712 - accuracy: 0.9786
1087/1688 [==================>...........] - ETA: 1s - loss: 0.0713 - accuracy: 0.9785
1108/1688 [==================>...........] - ETA: 1s - loss: 0.0709 - accuracy: 0.9786
1129/1688 [===================>..........] - ETA: 1s - loss: 0.0715 - accuracy: 0.9785
1151/1688 [===================>..........] - ETA: 1s - loss: 0.0715 - accuracy: 0.9785
1171/1688 [===================>..........] - ETA: 1s - loss: 0.0714 - accuracy: 0.9785
1192/1688 [====================>.........] - ETA: 1s - loss: 0.0711 - accuracy: 0.9786
1213/1688 [====================>.........] - ETA: 1s - loss: 0.0707 - accuracy: 0.9787
1234/1688 [====================>.........] - ETA: 1s - loss: 0.0707 - accuracy: 0.9787
1255/1688 [=====================>........] - ETA: 1s - loss: 0.0709 - accuracy: 0.9785
1276/1688 [=====================>........] - ETA: 1s - loss: 0.0705 - accuracy: 0.9786
1297/1688 [======================>.......] - ETA: 0s - loss: 0.0708 - accuracy: 0.9787
1318/1688 [======================>.......] - ETA: 0s - loss: 0.0709 - accuracy: 0.9786
1340/1688 [======================>.......] - ETA: 0s - loss: 0.0710 - accuracy: 0.9786
1361/1688 [=======================>......] - ETA: 0s - loss: 0.0709 - accuracy: 0.9785
1382/1688 [=======================>......] - ETA: 0s - loss: 0.0707 - accuracy: 0.9786
1402/1688 [=======================>......] - ETA: 0s - loss: 0.0707 - accuracy: 0.9786
1423/1688 [========================>.....] - ETA: 0s - loss: 0.0702 - accuracy: 0.9788
1445/1688 [========================>.....] - ETA: 0s - loss: 0.0705 - accuracy: 0.9787
1467/1688 [=========================>....] - ETA: 0s - loss: 0.0708 - accuracy: 0.9786
1488/1688 [=========================>....] - ETA: 0s - loss: 0.0709 - accuracy: 0.9786
1509/1688 [=========================>....] - ETA: 0s - loss: 0.0704 - accuracy: 0.9787
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0705 - accuracy: 0.9787
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0708 - accuracy: 0.9786
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0708 - accuracy: 0.9786
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0708 - accuracy: 0.9786
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0706 - accuracy: 0.9786
1636/1688 [============================>.] - ETA: 0s - loss: 0.0701 - accuracy: 0.9787
1657/1688 [============================>.] - ETA: 0s - loss: 0.0702 - accuracy: 0.9786
1678/1688 [============================>.] - ETA: 0s - loss: 0.0703 - accuracy: 0.9786
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0703 - accuracy: 0.9786 - val_loss: 0.0672 - val_accuracy: 0.9822
Epoch 3/10
1/1688 [..............................] - ETA: 4s - loss: 0.1244 - accuracy: 0.9688
22/1688 [..............................] - ETA: 4s - loss: 0.0403 - accuracy: 0.9872
43/1688 [..............................] - ETA: 4s - loss: 0.0350 - accuracy: 0.9906
64/1688 [>.............................] - ETA: 3s - loss: 0.0484 - accuracy: 0.9858
85/1688 [>.............................] - ETA: 3s - loss: 0.0480 - accuracy: 0.9857
106/1688 [>.............................] - ETA: 3s - loss: 0.0448 - accuracy: 0.9864
127/1688 [=>............................] - ETA: 3s - loss: 0.0439 - accuracy: 0.9862
148/1688 [=>............................] - ETA: 3s - loss: 0.0497 - accuracy: 0.9846
169/1688 [==>...........................] - ETA: 3s - loss: 0.0482 - accuracy: 0.9847
190/1688 [==>...........................] - ETA: 3s - loss: 0.0509 - accuracy: 0.9842
211/1688 [==>...........................] - ETA: 3s - loss: 0.0498 - accuracy: 0.9843
232/1688 [===>..........................] - ETA: 3s - loss: 0.0508 - accuracy: 0.9845
252/1688 [===>..........................] - ETA: 3s - loss: 0.0493 - accuracy: 0.9847
273/1688 [===>..........................] - ETA: 3s - loss: 0.0482 - accuracy: 0.9852
294/1688 [====>.........................] - ETA: 3s - loss: 0.0490 - accuracy: 0.9847
315/1688 [====>.........................] - ETA: 3s - loss: 0.0487 - accuracy: 0.9846
336/1688 [====>.........................] - ETA: 3s - loss: 0.0483 - accuracy: 0.9848
357/1688 [=====>........................] - ETA: 3s - loss: 0.0480 - accuracy: 0.9847
378/1688 [=====>........................] - ETA: 3s - loss: 0.0483 - accuracy: 0.9847
398/1688 [======>.......................] - ETA: 3s - loss: 0.0473 - accuracy: 0.9851
419/1688 [======>.......................] - ETA: 3s - loss: 0.0465 - accuracy: 0.9854
440/1688 [======>.......................] - ETA: 3s - loss: 0.0473 - accuracy: 0.9849
461/1688 [=======>......................] - ETA: 3s - loss: 0.0471 - accuracy: 0.9850
482/1688 [=======>......................] - ETA: 2s - loss: 0.0472 - accuracy: 0.9846
502/1688 [=======>......................] - ETA: 2s - loss: 0.0476 - accuracy: 0.9844
523/1688 [========>.....................] - ETA: 2s - loss: 0.0481 - accuracy: 0.9843
544/1688 [========>.....................] - ETA: 2s - loss: 0.0481 - accuracy: 0.9841
565/1688 [=========>....................] - ETA: 2s - loss: 0.0487 - accuracy: 0.9840
586/1688 [=========>....................] - ETA: 2s - loss: 0.0487 - accuracy: 0.9843
607/1688 [=========>....................] - ETA: 2s - loss: 0.0481 - accuracy: 0.9844
628/1688 [==========>...................] - ETA: 2s - loss: 0.0482 - accuracy: 0.9842
649/1688 [==========>...................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9838
670/1688 [==========>...................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9837
691/1688 [===========>..................] - ETA: 2s - loss: 0.0488 - accuracy: 0.9838
711/1688 [===========>..................] - ETA: 2s - loss: 0.0493 - accuracy: 0.9836
733/1688 [============>.................] - ETA: 2s - loss: 0.0489 - accuracy: 0.9838
754/1688 [============>.................] - ETA: 2s - loss: 0.0486 - accuracy: 0.9839
775/1688 [============>.................] - ETA: 2s - loss: 0.0483 - accuracy: 0.9840
795/1688 [=============>................] - ETA: 2s - loss: 0.0486 - accuracy: 0.9840
816/1688 [=============>................] - ETA: 2s - loss: 0.0497 - accuracy: 0.9836
838/1688 [=============>................] - ETA: 2s - loss: 0.0496 - accuracy: 0.9838
860/1688 [==============>...............] - ETA: 2s - loss: 0.0501 - accuracy: 0.9838
881/1688 [==============>...............] - ETA: 1s - loss: 0.0505 - accuracy: 0.9837
902/1688 [===============>..............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9838
922/1688 [===============>..............] - ETA: 1s - loss: 0.0497 - accuracy: 0.9838
944/1688 [===============>..............] - ETA: 1s - loss: 0.0497 - accuracy: 0.9838
965/1688 [================>.............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9840
985/1688 [================>.............] - ETA: 1s - loss: 0.0492 - accuracy: 0.9840
1006/1688 [================>.............] - ETA: 1s - loss: 0.0497 - accuracy: 0.9838
1027/1688 [=================>............] - ETA: 1s - loss: 0.0498 - accuracy: 0.9837
1048/1688 [=================>............] - ETA: 1s - loss: 0.0496 - accuracy: 0.9837
1068/1688 [=================>............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9836
1089/1688 [==================>...........] - ETA: 1s - loss: 0.0507 - accuracy: 0.9834
1110/1688 [==================>...........] - ETA: 1s - loss: 0.0506 - accuracy: 0.9834
1132/1688 [===================>..........] - ETA: 1s - loss: 0.0510 - accuracy: 0.9832
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0508 - accuracy: 0.9833
1175/1688 [===================>..........] - ETA: 1s - loss: 0.0510 - accuracy: 0.9833
1196/1688 [====================>.........] - ETA: 1s - loss: 0.0507 - accuracy: 0.9834
1218/1688 [====================>.........] - ETA: 1s - loss: 0.0504 - accuracy: 0.9835
1240/1688 [=====================>........] - ETA: 1s - loss: 0.0503 - accuracy: 0.9834
1262/1688 [=====================>........] - ETA: 1s - loss: 0.0503 - accuracy: 0.9835
1284/1688 [=====================>........] - ETA: 0s - loss: 0.0503 - accuracy: 0.9835
1307/1688 [======================>.......] - ETA: 0s - loss: 0.0508 - accuracy: 0.9834
1328/1688 [======================>.......] - ETA: 0s - loss: 0.0503 - accuracy: 0.9835
1349/1688 [======================>.......] - ETA: 0s - loss: 0.0504 - accuracy: 0.9836
1369/1688 [=======================>......] - ETA: 0s - loss: 0.0501 - accuracy: 0.9836
1389/1688 [=======================>......] - ETA: 0s - loss: 0.0500 - accuracy: 0.9836
1411/1688 [========================>.....] - ETA: 0s - loss: 0.0502 - accuracy: 0.9835
1432/1688 [========================>.....] - ETA: 0s - loss: 0.0506 - accuracy: 0.9834
1453/1688 [========================>.....] - ETA: 0s - loss: 0.0508 - accuracy: 0.9833
1474/1688 [=========================>....] - ETA: 0s - loss: 0.0511 - accuracy: 0.9832
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0512 - accuracy: 0.9831
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0512 - accuracy: 0.9831
1537/1688 [==========================>...] - ETA: 0s - loss: 0.0514 - accuracy: 0.9831
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0519 - accuracy: 0.9830
1581/1688 [===========================>..] - ETA: 0s - loss: 0.0518 - accuracy: 0.9830
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0521 - accuracy: 0.9829
1623/1688 [===========================>..] - ETA: 0s - loss: 0.0525 - accuracy: 0.9828
1644/1688 [============================>.] - ETA: 0s - loss: 0.0528 - accuracy: 0.9828
1665/1688 [============================>.] - ETA: 0s - loss: 0.0530 - accuracy: 0.9827
1687/1688 [============================>.] - ETA: 0s - loss: 0.0530 - accuracy: 0.9826
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0530 - accuracy: 0.9826 - val_loss: 0.0548 - val_accuracy: 0.9862
Epoch 4/10
1/1688 [..............................] - ETA: 4s - loss: 0.0050 - accuracy: 1.0000
22/1688 [..............................] - ETA: 4s - loss: 0.0430 - accuracy: 0.9886
43/1688 [..............................] - ETA: 4s - loss: 0.0386 - accuracy: 0.9862
64/1688 [>.............................] - ETA: 4s - loss: 0.0352 - accuracy: 0.9873
85/1688 [>.............................] - ETA: 3s - loss: 0.0352 - accuracy: 0.9875
107/1688 [>.............................] - ETA: 3s - loss: 0.0327 - accuracy: 0.9889
128/1688 [=>............................] - ETA: 3s - loss: 0.0328 - accuracy: 0.9890
149/1688 [=>............................] - ETA: 3s - loss: 0.0318 - accuracy: 0.9889
170/1688 [==>...........................] - ETA: 3s - loss: 0.0320 - accuracy: 0.9888
191/1688 [==>...........................] - ETA: 3s - loss: 0.0327 - accuracy: 0.9885
212/1688 [==>...........................] - ETA: 3s - loss: 0.0338 - accuracy: 0.9884
233/1688 [===>..........................] - ETA: 3s - loss: 0.0352 - accuracy: 0.9874
254/1688 [===>..........................] - ETA: 3s - loss: 0.0340 - accuracy: 0.9877
275/1688 [===>..........................] - ETA: 3s - loss: 0.0337 - accuracy: 0.9881
297/1688 [====>.........................] - ETA: 3s - loss: 0.0330 - accuracy: 0.9882
318/1688 [====>.........................] - ETA: 3s - loss: 0.0330 - accuracy: 0.9882
338/1688 [=====>........................] - ETA: 3s - loss: 0.0330 - accuracy: 0.9884
359/1688 [=====>........................] - ETA: 3s - loss: 0.0351 - accuracy: 0.9878
381/1688 [=====>........................] - ETA: 3s - loss: 0.0357 - accuracy: 0.9878
402/1688 [======>.......................] - ETA: 3s - loss: 0.0354 - accuracy: 0.9880
423/1688 [======>.......................] - ETA: 3s - loss: 0.0356 - accuracy: 0.9881
443/1688 [======>.......................] - ETA: 3s - loss: 0.0350 - accuracy: 0.9883
464/1688 [=======>......................] - ETA: 2s - loss: 0.0357 - accuracy: 0.9881
485/1688 [=======>......................] - ETA: 2s - loss: 0.0359 - accuracy: 0.9880
506/1688 [=======>......................] - ETA: 2s - loss: 0.0359 - accuracy: 0.9881
527/1688 [========>.....................] - ETA: 2s - loss: 0.0367 - accuracy: 0.9878
548/1688 [========>.....................] - ETA: 2s - loss: 0.0371 - accuracy: 0.9877
568/1688 [=========>....................] - ETA: 2s - loss: 0.0364 - accuracy: 0.9878
589/1688 [=========>....................] - ETA: 2s - loss: 0.0371 - accuracy: 0.9877
610/1688 [=========>....................] - ETA: 2s - loss: 0.0374 - accuracy: 0.9875
630/1688 [==========>...................] - ETA: 2s - loss: 0.0382 - accuracy: 0.9873
651/1688 [==========>...................] - ETA: 2s - loss: 0.0381 - accuracy: 0.9872
672/1688 [==========>...................] - ETA: 2s - loss: 0.0384 - accuracy: 0.9873
692/1688 [===========>..................] - ETA: 2s - loss: 0.0391 - accuracy: 0.9869
713/1688 [===========>..................] - ETA: 2s - loss: 0.0391 - accuracy: 0.9869
734/1688 [============>.................] - ETA: 2s - loss: 0.0389 - accuracy: 0.9870
756/1688 [============>.................] - ETA: 2s - loss: 0.0387 - accuracy: 0.9869
777/1688 [============>.................] - ETA: 2s - loss: 0.0392 - accuracy: 0.9868
799/1688 [=============>................] - ETA: 2s - loss: 0.0394 - accuracy: 0.9867
820/1688 [=============>................] - ETA: 2s - loss: 0.0392 - accuracy: 0.9867
841/1688 [=============>................] - ETA: 2s - loss: 0.0395 - accuracy: 0.9866
862/1688 [==============>...............] - ETA: 2s - loss: 0.0393 - accuracy: 0.9866
883/1688 [==============>...............] - ETA: 1s - loss: 0.0397 - accuracy: 0.9865
903/1688 [===============>..............] - ETA: 1s - loss: 0.0398 - accuracy: 0.9864
924/1688 [===============>..............] - ETA: 1s - loss: 0.0400 - accuracy: 0.9864
945/1688 [===============>..............] - ETA: 1s - loss: 0.0400 - accuracy: 0.9864
966/1688 [================>.............] - ETA: 1s - loss: 0.0402 - accuracy: 0.9863
987/1688 [================>.............] - ETA: 1s - loss: 0.0406 - accuracy: 0.9862
1008/1688 [================>.............] - ETA: 1s - loss: 0.0403 - accuracy: 0.9863
1029/1688 [=================>............] - ETA: 1s - loss: 0.0404 - accuracy: 0.9862
1050/1688 [=================>............] - ETA: 1s - loss: 0.0404 - accuracy: 0.9863
1071/1688 [==================>...........] - ETA: 1s - loss: 0.0409 - accuracy: 0.9861
1092/1688 [==================>...........] - ETA: 1s - loss: 0.0409 - accuracy: 0.9861
1112/1688 [==================>...........] - ETA: 1s - loss: 0.0407 - accuracy: 0.9862
1132/1688 [===================>..........] - ETA: 1s - loss: 0.0408 - accuracy: 0.9861
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0411 - accuracy: 0.9861
1174/1688 [===================>..........] - ETA: 1s - loss: 0.0413 - accuracy: 0.9861
1195/1688 [====================>.........] - ETA: 1s - loss: 0.0418 - accuracy: 0.9859
1215/1688 [====================>.........] - ETA: 1s - loss: 0.0417 - accuracy: 0.9860
1236/1688 [====================>.........] - ETA: 1s - loss: 0.0418 - accuracy: 0.9860
1257/1688 [=====================>........] - ETA: 1s - loss: 0.0422 - accuracy: 0.9859
1278/1688 [=====================>........] - ETA: 1s - loss: 0.0420 - accuracy: 0.9860
1299/1688 [======================>.......] - ETA: 0s - loss: 0.0419 - accuracy: 0.9860
1319/1688 [======================>.......] - ETA: 0s - loss: 0.0421 - accuracy: 0.9859
1339/1688 [======================>.......] - ETA: 0s - loss: 0.0420 - accuracy: 0.9860
1360/1688 [=======================>......] - ETA: 0s - loss: 0.0421 - accuracy: 0.9858
1380/1688 [=======================>......] - ETA: 0s - loss: 0.0426 - accuracy: 0.9858
1401/1688 [=======================>......] - ETA: 0s - loss: 0.0427 - accuracy: 0.9858
1422/1688 [========================>.....] - ETA: 0s - loss: 0.0423 - accuracy: 0.9860
1444/1688 [========================>.....] - ETA: 0s - loss: 0.0420 - accuracy: 0.9861
1465/1688 [=========================>....] - ETA: 0s - loss: 0.0420 - accuracy: 0.9861
1486/1688 [=========================>....] - ETA: 0s - loss: 0.0417 - accuracy: 0.9862
1507/1688 [=========================>....] - ETA: 0s - loss: 0.0416 - accuracy: 0.9862
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0420 - accuracy: 0.9862
1550/1688 [==========================>...] - ETA: 0s - loss: 0.0423 - accuracy: 0.9862
1571/1688 [==========================>...] - ETA: 0s - loss: 0.0423 - accuracy: 0.9862
1591/1688 [===========================>..] - ETA: 0s - loss: 0.0424 - accuracy: 0.9863
1612/1688 [===========================>..] - ETA: 0s - loss: 0.0422 - accuracy: 0.9863
1632/1688 [============================>.] - ETA: 0s - loss: 0.0420 - accuracy: 0.9863
1653/1688 [============================>.] - ETA: 0s - loss: 0.0419 - accuracy: 0.9864
1674/1688 [============================>.] - ETA: 0s - loss: 0.0419 - accuracy: 0.9864
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0419 - accuracy: 0.9864 - val_loss: 0.0609 - val_accuracy: 0.9830
Epoch 5/10
1/1688 [..............................] - ETA: 4s - loss: 0.0050 - accuracy: 1.0000
22/1688 [..............................] - ETA: 4s - loss: 0.0380 - accuracy: 0.9858
43/1688 [..............................] - ETA: 4s - loss: 0.0404 - accuracy: 0.9876
64/1688 [>.............................] - ETA: 3s - loss: 0.0399 - accuracy: 0.9883
86/1688 [>.............................] - ETA: 3s - loss: 0.0365 - accuracy: 0.9884
107/1688 [>.............................] - ETA: 3s - loss: 0.0363 - accuracy: 0.9892
128/1688 [=>............................] - ETA: 3s - loss: 0.0370 - accuracy: 0.9893
149/1688 [=>............................] - ETA: 3s - loss: 0.0354 - accuracy: 0.9901
170/1688 [==>...........................] - ETA: 3s - loss: 0.0357 - accuracy: 0.9901
191/1688 [==>...........................] - ETA: 3s - loss: 0.0348 - accuracy: 0.9903
212/1688 [==>...........................] - ETA: 3s - loss: 0.0341 - accuracy: 0.9904
233/1688 [===>..........................] - ETA: 3s - loss: 0.0343 - accuracy: 0.9903
254/1688 [===>..........................] - ETA: 3s - loss: 0.0338 - accuracy: 0.9904
274/1688 [===>..........................] - ETA: 3s - loss: 0.0337 - accuracy: 0.9903
296/1688 [====>.........................] - ETA: 3s - loss: 0.0337 - accuracy: 0.9900
318/1688 [====>.........................] - ETA: 3s - loss: 0.0333 - accuracy: 0.9901
339/1688 [=====>........................] - ETA: 3s - loss: 0.0343 - accuracy: 0.9899
360/1688 [=====>........................] - ETA: 3s - loss: 0.0339 - accuracy: 0.9898
381/1688 [=====>........................] - ETA: 3s - loss: 0.0338 - accuracy: 0.9896
403/1688 [======>.......................] - ETA: 3s - loss: 0.0338 - accuracy: 0.9897
423/1688 [======>.......................] - ETA: 3s - loss: 0.0341 - accuracy: 0.9895
445/1688 [======>.......................] - ETA: 3s - loss: 0.0340 - accuracy: 0.9895
466/1688 [=======>......................] - ETA: 2s - loss: 0.0338 - accuracy: 0.9896
487/1688 [=======>......................] - ETA: 2s - loss: 0.0343 - accuracy: 0.9896
508/1688 [========>.....................] - ETA: 2s - loss: 0.0338 - accuracy: 0.9898
529/1688 [========>.....................] - ETA: 2s - loss: 0.0341 - accuracy: 0.9894
550/1688 [========>.....................] - ETA: 2s - loss: 0.0341 - accuracy: 0.9894
572/1688 [=========>....................] - ETA: 2s - loss: 0.0342 - accuracy: 0.9893
594/1688 [=========>....................] - ETA: 2s - loss: 0.0340 - accuracy: 0.9894
615/1688 [=========>....................] - ETA: 2s - loss: 0.0337 - accuracy: 0.9895
636/1688 [==========>...................] - ETA: 2s - loss: 0.0338 - accuracy: 0.9895
657/1688 [==========>...................] - ETA: 2s - loss: 0.0338 - accuracy: 0.9896
678/1688 [===========>..................] - ETA: 2s - loss: 0.0332 - accuracy: 0.9896
699/1688 [===========>..................] - ETA: 2s - loss: 0.0337 - accuracy: 0.9893
720/1688 [===========>..................] - ETA: 2s - loss: 0.0338 - accuracy: 0.9892
741/1688 [============>.................] - ETA: 2s - loss: 0.0341 - accuracy: 0.9891
762/1688 [============>.................] - ETA: 2s - loss: 0.0340 - accuracy: 0.9891
783/1688 [============>.................] - ETA: 2s - loss: 0.0337 - accuracy: 0.9892
804/1688 [=============>................] - ETA: 2s - loss: 0.0337 - accuracy: 0.9893
825/1688 [=============>................] - ETA: 2s - loss: 0.0340 - accuracy: 0.9892
847/1688 [==============>...............] - ETA: 2s - loss: 0.0339 - accuracy: 0.9892
868/1688 [==============>...............] - ETA: 1s - loss: 0.0337 - accuracy: 0.9893
888/1688 [==============>...............] - ETA: 1s - loss: 0.0340 - accuracy: 0.9891
909/1688 [===============>..............] - ETA: 1s - loss: 0.0337 - accuracy: 0.9892
930/1688 [===============>..............] - ETA: 1s - loss: 0.0339 - accuracy: 0.9892
951/1688 [===============>..............] - ETA: 1s - loss: 0.0342 - accuracy: 0.9890
972/1688 [================>.............] - ETA: 1s - loss: 0.0343 - accuracy: 0.9890
993/1688 [================>.............] - ETA: 1s - loss: 0.0346 - accuracy: 0.9889
1014/1688 [=================>............] - ETA: 1s - loss: 0.0345 - accuracy: 0.9888
1035/1688 [=================>............] - ETA: 1s - loss: 0.0346 - accuracy: 0.9888
1055/1688 [=================>............] - ETA: 1s - loss: 0.0344 - accuracy: 0.9888
1076/1688 [==================>...........] - ETA: 1s - loss: 0.0345 - accuracy: 0.9888
1097/1688 [==================>...........] - ETA: 1s - loss: 0.0343 - accuracy: 0.9888
1117/1688 [==================>...........] - ETA: 1s - loss: 0.0341 - accuracy: 0.9889
1139/1688 [===================>..........] - ETA: 1s - loss: 0.0343 - accuracy: 0.9888
1161/1688 [===================>..........] - ETA: 1s - loss: 0.0341 - accuracy: 0.9888
1182/1688 [====================>.........] - ETA: 1s - loss: 0.0343 - accuracy: 0.9887
1203/1688 [====================>.........] - ETA: 1s - loss: 0.0344 - accuracy: 0.9887
1223/1688 [====================>.........] - ETA: 1s - loss: 0.0344 - accuracy: 0.9887
1244/1688 [=====================>........] - ETA: 1s - loss: 0.0344 - accuracy: 0.9887
1264/1688 [=====================>........] - ETA: 1s - loss: 0.0345 - accuracy: 0.9887
1282/1688 [=====================>........] - ETA: 0s - loss: 0.0346 - accuracy: 0.9887
1301/1688 [======================>.......] - ETA: 0s - loss: 0.0344 - accuracy: 0.9888
1321/1688 [======================>.......] - ETA: 0s - loss: 0.0343 - accuracy: 0.9888
1341/1688 [======================>.......] - ETA: 0s - loss: 0.0343 - accuracy: 0.9888
1361/1688 [=======================>......] - ETA: 0s - loss: 0.0342 - accuracy: 0.9888
1381/1688 [=======================>......] - ETA: 0s - loss: 0.0342 - accuracy: 0.9888
1402/1688 [=======================>......] - ETA: 0s - loss: 0.0342 - accuracy: 0.9889
1423/1688 [========================>.....] - ETA: 0s - loss: 0.0342 - accuracy: 0.9889
1445/1688 [========================>.....] - ETA: 0s - loss: 0.0341 - accuracy: 0.9889
1466/1688 [=========================>....] - ETA: 0s - loss: 0.0343 - accuracy: 0.9889
1487/1688 [=========================>....] - ETA: 0s - loss: 0.0342 - accuracy: 0.9888
1509/1688 [=========================>....] - ETA: 0s - loss: 0.0343 - accuracy: 0.9889
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0343 - accuracy: 0.9888
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0345 - accuracy: 0.9888
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0345 - accuracy: 0.9887
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0347 - accuracy: 0.9887
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0349 - accuracy: 0.9886
1634/1688 [============================>.] - ETA: 0s - loss: 0.0352 - accuracy: 0.9885
1655/1688 [============================>.] - ETA: 0s - loss: 0.0355 - accuracy: 0.9884
1676/1688 [============================>.] - ETA: 0s - loss: 0.0356 - accuracy: 0.9884
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0355 - accuracy: 0.9884 - val_loss: 0.0549 - val_accuracy: 0.9858
Epoch 6/10
1/1688 [..............................] - ETA: 4s - loss: 0.0702 - accuracy: 0.9375
22/1688 [..............................] - ETA: 3s - loss: 0.0207 - accuracy: 0.9901
43/1688 [..............................] - ETA: 4s - loss: 0.0214 - accuracy: 0.9906
64/1688 [>.............................] - ETA: 3s - loss: 0.0279 - accuracy: 0.9902
85/1688 [>.............................] - ETA: 3s - loss: 0.0258 - accuracy: 0.9904
106/1688 [>.............................] - ETA: 3s - loss: 0.0252 - accuracy: 0.9906
127/1688 [=>............................] - ETA: 3s - loss: 0.0243 - accuracy: 0.9906
148/1688 [=>............................] - ETA: 3s - loss: 0.0241 - accuracy: 0.9907
169/1688 [==>...........................] - ETA: 3s - loss: 0.0230 - accuracy: 0.9909
190/1688 [==>...........................] - ETA: 3s - loss: 0.0232 - accuracy: 0.9910
211/1688 [==>...........................] - ETA: 3s - loss: 0.0227 - accuracy: 0.9913
232/1688 [===>..........................] - ETA: 3s - loss: 0.0234 - accuracy: 0.9915
253/1688 [===>..........................] - ETA: 3s - loss: 0.0236 - accuracy: 0.9918
274/1688 [===>..........................] - ETA: 3s - loss: 0.0236 - accuracy: 0.9919
295/1688 [====>.........................] - ETA: 3s - loss: 0.0232 - accuracy: 0.9918
316/1688 [====>.........................] - ETA: 3s - loss: 0.0245 - accuracy: 0.9916
337/1688 [====>.........................] - ETA: 3s - loss: 0.0252 - accuracy: 0.9915
358/1688 [=====>........................] - ETA: 3s - loss: 0.0250 - accuracy: 0.9916
378/1688 [=====>........................] - ETA: 3s - loss: 0.0242 - accuracy: 0.9919
399/1688 [======>.......................] - ETA: 3s - loss: 0.0237 - accuracy: 0.9922
420/1688 [======>.......................] - ETA: 3s - loss: 0.0239 - accuracy: 0.9920
441/1688 [======>.......................] - ETA: 3s - loss: 0.0239 - accuracy: 0.9920
462/1688 [=======>......................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9920
483/1688 [=======>......................] - ETA: 2s - loss: 0.0238 - accuracy: 0.9920
504/1688 [=======>......................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9921
525/1688 [========>.....................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9921
546/1688 [========>.....................] - ETA: 2s - loss: 0.0243 - accuracy: 0.9920
566/1688 [=========>....................] - ETA: 2s - loss: 0.0249 - accuracy: 0.9919
587/1688 [=========>....................] - ETA: 2s - loss: 0.0250 - accuracy: 0.9919
608/1688 [=========>....................] - ETA: 2s - loss: 0.0248 - accuracy: 0.9918
629/1688 [==========>...................] - ETA: 2s - loss: 0.0246 - accuracy: 0.9919
650/1688 [==========>...................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9919
671/1688 [==========>...................] - ETA: 2s - loss: 0.0249 - accuracy: 0.9918
692/1688 [===========>..................] - ETA: 2s - loss: 0.0248 - accuracy: 0.9919
713/1688 [===========>..................] - ETA: 2s - loss: 0.0252 - accuracy: 0.9918
734/1688 [============>.................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9916
755/1688 [============>.................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9918
776/1688 [============>.................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9917
797/1688 [=============>................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9916
819/1688 [=============>................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9916
840/1688 [=============>................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9916
861/1688 [==============>...............] - ETA: 2s - loss: 0.0254 - accuracy: 0.9916
882/1688 [==============>...............] - ETA: 1s - loss: 0.0254 - accuracy: 0.9915
903/1688 [===============>..............] - ETA: 1s - loss: 0.0254 - accuracy: 0.9915
925/1688 [===============>..............] - ETA: 1s - loss: 0.0255 - accuracy: 0.9914
947/1688 [===============>..............] - ETA: 1s - loss: 0.0257 - accuracy: 0.9913
969/1688 [================>.............] - ETA: 1s - loss: 0.0258 - accuracy: 0.9913
990/1688 [================>.............] - ETA: 1s - loss: 0.0262 - accuracy: 0.9912
1010/1688 [================>.............] - ETA: 1s - loss: 0.0263 - accuracy: 0.9912
1031/1688 [=================>............] - ETA: 1s - loss: 0.0261 - accuracy: 0.9913
1052/1688 [=================>............] - ETA: 1s - loss: 0.0261 - accuracy: 0.9913
1073/1688 [==================>...........] - ETA: 1s - loss: 0.0262 - accuracy: 0.9913
1095/1688 [==================>...........] - ETA: 1s - loss: 0.0270 - accuracy: 0.9910
1115/1688 [==================>...........] - ETA: 1s - loss: 0.0270 - accuracy: 0.9910
1136/1688 [===================>..........] - ETA: 1s - loss: 0.0272 - accuracy: 0.9909
1156/1688 [===================>..........] - ETA: 1s - loss: 0.0270 - accuracy: 0.9910
1178/1688 [===================>..........] - ETA: 1s - loss: 0.0270 - accuracy: 0.9910
1199/1688 [====================>.........] - ETA: 1s - loss: 0.0276 - accuracy: 0.9908
1220/1688 [====================>.........] - ETA: 1s - loss: 0.0277 - accuracy: 0.9908
1241/1688 [=====================>........] - ETA: 1s - loss: 0.0279 - accuracy: 0.9907
1261/1688 [=====================>........] - ETA: 1s - loss: 0.0280 - accuracy: 0.9907
1282/1688 [=====================>........] - ETA: 0s - loss: 0.0282 - accuracy: 0.9906
1302/1688 [======================>.......] - ETA: 0s - loss: 0.0283 - accuracy: 0.9905
1322/1688 [======================>.......] - ETA: 0s - loss: 0.0287 - accuracy: 0.9904
1343/1688 [======================>.......] - ETA: 0s - loss: 0.0289 - accuracy: 0.9903
1364/1688 [=======================>......] - ETA: 0s - loss: 0.0290 - accuracy: 0.9902
1386/1688 [=======================>......] - ETA: 0s - loss: 0.0287 - accuracy: 0.9903
1407/1688 [========================>.....] - ETA: 0s - loss: 0.0288 - accuracy: 0.9903
1429/1688 [========================>.....] - ETA: 0s - loss: 0.0288 - accuracy: 0.9903
1450/1688 [========================>.....] - ETA: 0s - loss: 0.0288 - accuracy: 0.9903
1471/1688 [=========================>....] - ETA: 0s - loss: 0.0288 - accuracy: 0.9903
1493/1688 [=========================>....] - ETA: 0s - loss: 0.0287 - accuracy: 0.9903
1514/1688 [=========================>....] - ETA: 0s - loss: 0.0288 - accuracy: 0.9903
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0290 - accuracy: 0.9902
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0291 - accuracy: 0.9902
1577/1688 [===========================>..] - ETA: 0s - loss: 0.0294 - accuracy: 0.9902
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0294 - accuracy: 0.9902
1619/1688 [===========================>..] - ETA: 0s - loss: 0.0296 - accuracy: 0.9901
1640/1688 [============================>.] - ETA: 0s - loss: 0.0298 - accuracy: 0.9900
1661/1688 [============================>.] - ETA: 0s - loss: 0.0297 - accuracy: 0.9900
1682/1688 [============================>.] - ETA: 0s - loss: 0.0299 - accuracy: 0.9899
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0300 - accuracy: 0.9899 - val_loss: 0.0647 - val_accuracy: 0.9852
Epoch 7/10
1/1688 [..............................] - ETA: 5s - loss: 0.0029 - accuracy: 1.0000
19/1688 [..............................] - ETA: 4s - loss: 0.0235 - accuracy: 0.9951
40/1688 [..............................] - ETA: 4s - loss: 0.0302 - accuracy: 0.9906
60/1688 [>.............................] - ETA: 4s - loss: 0.0246 - accuracy: 0.9927
81/1688 [>.............................] - ETA: 4s - loss: 0.0252 - accuracy: 0.9923
102/1688 [>.............................] - ETA: 4s - loss: 0.0238 - accuracy: 0.9930
123/1688 [=>............................] - ETA: 3s - loss: 0.0212 - accuracy: 0.9939
144/1688 [=>............................] - ETA: 3s - loss: 0.0207 - accuracy: 0.9939
165/1688 [=>............................] - ETA: 3s - loss: 0.0200 - accuracy: 0.9937
186/1688 [==>...........................] - ETA: 3s - loss: 0.0213 - accuracy: 0.9936
207/1688 [==>...........................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9938
227/1688 [===>..........................] - ETA: 3s - loss: 0.0193 - accuracy: 0.9941
248/1688 [===>..........................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9933
268/1688 [===>..........................] - ETA: 3s - loss: 0.0209 - accuracy: 0.9930
289/1688 [====>.........................] - ETA: 3s - loss: 0.0206 - accuracy: 0.9931
309/1688 [====>.........................] - ETA: 3s - loss: 0.0216 - accuracy: 0.9925
330/1688 [====>.........................] - ETA: 3s - loss: 0.0215 - accuracy: 0.9925
350/1688 [=====>........................] - ETA: 3s - loss: 0.0212 - accuracy: 0.9927
371/1688 [=====>........................] - ETA: 3s - loss: 0.0212 - accuracy: 0.9927
392/1688 [=====>........................] - ETA: 3s - loss: 0.0206 - accuracy: 0.9929
413/1688 [======>.......................] - ETA: 3s - loss: 0.0213 - accuracy: 0.9928
434/1688 [======>.......................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9930
456/1688 [=======>......................] - ETA: 3s - loss: 0.0210 - accuracy: 0.9928
477/1688 [=======>......................] - ETA: 3s - loss: 0.0218 - accuracy: 0.9925
498/1688 [=======>......................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9923
519/1688 [========>.....................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9924
540/1688 [========>.....................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9925
562/1688 [========>.....................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9922
583/1688 [=========>....................] - ETA: 2s - loss: 0.0227 - accuracy: 0.9923
604/1688 [=========>....................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9921
625/1688 [==========>...................] - ETA: 2s - loss: 0.0236 - accuracy: 0.9918
645/1688 [==========>...................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9918
666/1688 [==========>...................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9917
687/1688 [===========>..................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9918
708/1688 [===========>..................] - ETA: 2s - loss: 0.0238 - accuracy: 0.9919
729/1688 [===========>..................] - ETA: 2s - loss: 0.0235 - accuracy: 0.9921
750/1688 [============>.................] - ETA: 2s - loss: 0.0234 - accuracy: 0.9921
771/1688 [============>.................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9921
792/1688 [=============>................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9921
813/1688 [=============>................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9922
834/1688 [=============>................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9922
855/1688 [==============>...............] - ETA: 2s - loss: 0.0227 - accuracy: 0.9921
876/1688 [==============>...............] - ETA: 1s - loss: 0.0229 - accuracy: 0.9921
898/1688 [==============>...............] - ETA: 1s - loss: 0.0228 - accuracy: 0.9921
919/1688 [===============>..............] - ETA: 1s - loss: 0.0230 - accuracy: 0.9920
940/1688 [===============>..............] - ETA: 1s - loss: 0.0228 - accuracy: 0.9921
961/1688 [================>.............] - ETA: 1s - loss: 0.0227 - accuracy: 0.9921
983/1688 [================>.............] - ETA: 1s - loss: 0.0225 - accuracy: 0.9921
1004/1688 [================>.............] - ETA: 1s - loss: 0.0224 - accuracy: 0.9921
1025/1688 [=================>............] - ETA: 1s - loss: 0.0225 - accuracy: 0.9921
1046/1688 [=================>............] - ETA: 1s - loss: 0.0224 - accuracy: 0.9920
1067/1688 [=================>............] - ETA: 1s - loss: 0.0224 - accuracy: 0.9920
1088/1688 [==================>...........] - ETA: 1s - loss: 0.0225 - accuracy: 0.9920
1109/1688 [==================>...........] - ETA: 1s - loss: 0.0227 - accuracy: 0.9919
1131/1688 [===================>..........] - ETA: 1s - loss: 0.0225 - accuracy: 0.9920
1152/1688 [===================>..........] - ETA: 1s - loss: 0.0225 - accuracy: 0.9919
1174/1688 [===================>..........] - ETA: 1s - loss: 0.0229 - accuracy: 0.9918
1195/1688 [====================>.........] - ETA: 1s - loss: 0.0228 - accuracy: 0.9918
1216/1688 [====================>.........] - ETA: 1s - loss: 0.0228 - accuracy: 0.9918
1237/1688 [====================>.........] - ETA: 1s - loss: 0.0227 - accuracy: 0.9918
1258/1688 [=====================>........] - ETA: 1s - loss: 0.0228 - accuracy: 0.9919
1278/1688 [=====================>........] - ETA: 1s - loss: 0.0228 - accuracy: 0.9919
1299/1688 [======================>.......] - ETA: 0s - loss: 0.0229 - accuracy: 0.9919
1320/1688 [======================>.......] - ETA: 0s - loss: 0.0227 - accuracy: 0.9920
1340/1688 [======================>.......] - ETA: 0s - loss: 0.0227 - accuracy: 0.9920
1361/1688 [=======================>......] - ETA: 0s - loss: 0.0226 - accuracy: 0.9921
1382/1688 [=======================>......] - ETA: 0s - loss: 0.0226 - accuracy: 0.9921
1403/1688 [=======================>......] - ETA: 0s - loss: 0.0229 - accuracy: 0.9920
1424/1688 [========================>.....] - ETA: 0s - loss: 0.0231 - accuracy: 0.9920
1445/1688 [========================>.....] - ETA: 0s - loss: 0.0232 - accuracy: 0.9920
1466/1688 [=========================>....] - ETA: 0s - loss: 0.0232 - accuracy: 0.9919
1488/1688 [=========================>....] - ETA: 0s - loss: 0.0231 - accuracy: 0.9920
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0232 - accuracy: 0.9920
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0233 - accuracy: 0.9920
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0235 - accuracy: 0.9919
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0236 - accuracy: 0.9918
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0239 - accuracy: 0.9918
1615/1688 [===========================>..] - ETA: 0s - loss: 0.0244 - accuracy: 0.9917
1637/1688 [============================>.] - ETA: 0s - loss: 0.0246 - accuracy: 0.9917
1658/1688 [============================>.] - ETA: 0s - loss: 0.0246 - accuracy: 0.9916
1678/1688 [============================>.] - ETA: 0s - loss: 0.0248 - accuracy: 0.9916
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0248 - accuracy: 0.9915 - val_loss: 0.0586 - val_accuracy: 0.9847
Epoch 8/10
1/1688 [..............................] - ETA: 4s - loss: 0.0028 - accuracy: 1.0000
22/1688 [..............................] - ETA: 3s - loss: 0.0156 - accuracy: 0.9972
42/1688 [..............................] - ETA: 4s - loss: 0.0167 - accuracy: 0.9948
63/1688 [>.............................] - ETA: 3s - loss: 0.0157 - accuracy: 0.9950
83/1688 [>.............................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9955
104/1688 [>.............................] - ETA: 3s - loss: 0.0143 - accuracy: 0.9958
125/1688 [=>............................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9952
146/1688 [=>............................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9957
167/1688 [=>............................] - ETA: 3s - loss: 0.0139 - accuracy: 0.9955
188/1688 [==>...........................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9952
208/1688 [==>...........................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9950
229/1688 [===>..........................] - ETA: 3s - loss: 0.0162 - accuracy: 0.9944
250/1688 [===>..........................] - ETA: 3s - loss: 0.0175 - accuracy: 0.9937
271/1688 [===>..........................] - ETA: 3s - loss: 0.0174 - accuracy: 0.9938
292/1688 [====>.........................] - ETA: 3s - loss: 0.0181 - accuracy: 0.9935
313/1688 [====>.........................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9934
334/1688 [====>.........................] - ETA: 3s - loss: 0.0191 - accuracy: 0.9931
355/1688 [=====>........................] - ETA: 3s - loss: 0.0187 - accuracy: 0.9932
377/1688 [=====>........................] - ETA: 3s - loss: 0.0185 - accuracy: 0.9934
398/1688 [======>.......................] - ETA: 3s - loss: 0.0185 - accuracy: 0.9932
419/1688 [======>.......................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9934
439/1688 [======>.......................] - ETA: 3s - loss: 0.0185 - accuracy: 0.9934
461/1688 [=======>......................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9936
483/1688 [=======>......................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9935
504/1688 [=======>......................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9936
525/1688 [========>.....................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9936
546/1688 [========>.....................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9938
567/1688 [=========>....................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9938
588/1688 [=========>....................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9939
608/1688 [=========>....................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9938
629/1688 [==========>...................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9939
650/1688 [==========>...................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9939
671/1688 [==========>...................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9939
692/1688 [===========>..................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9939
713/1688 [===========>..................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9939
734/1688 [============>.................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9940
755/1688 [============>.................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9940
776/1688 [============>.................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9941
797/1688 [=============>................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9940
818/1688 [=============>................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9940
839/1688 [=============>................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9939
860/1688 [==============>...............] - ETA: 2s - loss: 0.0183 - accuracy: 0.9939
881/1688 [==============>...............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9939
902/1688 [===============>..............] - ETA: 1s - loss: 0.0187 - accuracy: 0.9937
923/1688 [===============>..............] - ETA: 1s - loss: 0.0189 - accuracy: 0.9936
944/1688 [===============>..............] - ETA: 1s - loss: 0.0189 - accuracy: 0.9936
966/1688 [================>.............] - ETA: 1s - loss: 0.0190 - accuracy: 0.9936
987/1688 [================>.............] - ETA: 1s - loss: 0.0192 - accuracy: 0.9936
1008/1688 [================>.............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9935
1028/1688 [=================>............] - ETA: 1s - loss: 0.0194 - accuracy: 0.9934
1049/1688 [=================>............] - ETA: 1s - loss: 0.0195 - accuracy: 0.9933
1070/1688 [==================>...........] - ETA: 1s - loss: 0.0197 - accuracy: 0.9932
1091/1688 [==================>...........] - ETA: 1s - loss: 0.0197 - accuracy: 0.9932
1112/1688 [==================>...........] - ETA: 1s - loss: 0.0196 - accuracy: 0.9933
1134/1688 [===================>..........] - ETA: 1s - loss: 0.0197 - accuracy: 0.9932
1155/1688 [===================>..........] - ETA: 1s - loss: 0.0196 - accuracy: 0.9933
1176/1688 [===================>..........] - ETA: 1s - loss: 0.0200 - accuracy: 0.9932
1197/1688 [====================>.........] - ETA: 1s - loss: 0.0202 - accuracy: 0.9931
1218/1688 [====================>.........] - ETA: 1s - loss: 0.0201 - accuracy: 0.9932
1238/1688 [=====================>........] - ETA: 1s - loss: 0.0201 - accuracy: 0.9932
1259/1688 [=====================>........] - ETA: 1s - loss: 0.0203 - accuracy: 0.9931
1280/1688 [=====================>........] - ETA: 0s - loss: 0.0204 - accuracy: 0.9930
1301/1688 [======================>.......] - ETA: 0s - loss: 0.0205 - accuracy: 0.9930
1321/1688 [======================>.......] - ETA: 0s - loss: 0.0205 - accuracy: 0.9930
1342/1688 [======================>.......] - ETA: 0s - loss: 0.0205 - accuracy: 0.9930
1363/1688 [=======================>......] - ETA: 0s - loss: 0.0204 - accuracy: 0.9930
1384/1688 [=======================>......] - ETA: 0s - loss: 0.0206 - accuracy: 0.9930
1405/1688 [=======================>......] - ETA: 0s - loss: 0.0211 - accuracy: 0.9929
1426/1688 [========================>.....] - ETA: 0s - loss: 0.0214 - accuracy: 0.9928
1447/1688 [========================>.....] - ETA: 0s - loss: 0.0215 - accuracy: 0.9927
1468/1688 [=========================>....] - ETA: 0s - loss: 0.0217 - accuracy: 0.9926
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0218 - accuracy: 0.9927
1509/1688 [=========================>....] - ETA: 0s - loss: 0.0217 - accuracy: 0.9927
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0217 - accuracy: 0.9926
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0217 - accuracy: 0.9926
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0218 - accuracy: 0.9926
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0218 - accuracy: 0.9926
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0218 - accuracy: 0.9926
1633/1688 [============================>.] - ETA: 0s - loss: 0.0218 - accuracy: 0.9926
1654/1688 [============================>.] - ETA: 0s - loss: 0.0217 - accuracy: 0.9927
1675/1688 [============================>.] - ETA: 0s - loss: 0.0219 - accuracy: 0.9926
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0218 - accuracy: 0.9926 - val_loss: 0.0594 - val_accuracy: 0.9855
Epoch 9/10
1/1688 [..............................] - ETA: 4s - loss: 6.4394e-04 - accuracy: 1.0000
22/1688 [..............................] - ETA: 4s - loss: 0.0150 - accuracy: 0.9943
43/1688 [..............................] - ETA: 4s - loss: 0.0126 - accuracy: 0.9964
64/1688 [>.............................] - ETA: 3s - loss: 0.0117 - accuracy: 0.9961
85/1688 [>.............................] - ETA: 3s - loss: 0.0110 - accuracy: 0.9967
106/1688 [>.............................] - ETA: 3s - loss: 0.0108 - accuracy: 0.9968
127/1688 [=>............................] - ETA: 3s - loss: 0.0120 - accuracy: 0.9968
148/1688 [=>............................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9966
169/1688 [==>...........................] - ETA: 3s - loss: 0.0148 - accuracy: 0.9963
190/1688 [==>...........................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9962
211/1688 [==>...........................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9960
231/1688 [===>..........................] - ETA: 3s - loss: 0.0156 - accuracy: 0.9954
252/1688 [===>..........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9954
273/1688 [===>..........................] - ETA: 3s - loss: 0.0149 - accuracy: 0.9954
294/1688 [====>.........................] - ETA: 3s - loss: 0.0149 - accuracy: 0.9953
315/1688 [====>.........................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9953
336/1688 [====>.........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9952
356/1688 [=====>........................] - ETA: 3s - loss: 0.0156 - accuracy: 0.9946
377/1688 [=====>........................] - ETA: 3s - loss: 0.0157 - accuracy: 0.9947
398/1688 [======>.......................] - ETA: 3s - loss: 0.0164 - accuracy: 0.9944
419/1688 [======>.......................] - ETA: 3s - loss: 0.0162 - accuracy: 0.9946
440/1688 [======>.......................] - ETA: 3s - loss: 0.0160 - accuracy: 0.9946
461/1688 [=======>......................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9944
482/1688 [=======>......................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9943
503/1688 [=======>......................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9943
524/1688 [========>.....................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9945
545/1688 [========>.....................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9943
566/1688 [=========>....................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9944
587/1688 [=========>....................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9945
608/1688 [=========>....................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9944
629/1688 [==========>...................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9943
650/1688 [==========>...................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9941
671/1688 [==========>...................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9941
692/1688 [===========>..................] - ETA: 2s - loss: 0.0171 - accuracy: 0.9942
713/1688 [===========>..................] - ETA: 2s - loss: 0.0172 - accuracy: 0.9943
734/1688 [============>.................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9943
755/1688 [============>.................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9944
776/1688 [============>.................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9944
798/1688 [=============>................] - ETA: 2s - loss: 0.0172 - accuracy: 0.9942
819/1688 [=============>................] - ETA: 2s - loss: 0.0176 - accuracy: 0.9941
840/1688 [=============>................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9940
861/1688 [==============>...............] - ETA: 2s - loss: 0.0176 - accuracy: 0.9941
881/1688 [==============>...............] - ETA: 1s - loss: 0.0177 - accuracy: 0.9940
901/1688 [===============>..............] - ETA: 1s - loss: 0.0177 - accuracy: 0.9941
922/1688 [===============>..............] - ETA: 1s - loss: 0.0177 - accuracy: 0.9940
943/1688 [===============>..............] - ETA: 1s - loss: 0.0178 - accuracy: 0.9940
963/1688 [================>.............] - ETA: 1s - loss: 0.0176 - accuracy: 0.9940
985/1688 [================>.............] - ETA: 1s - loss: 0.0179 - accuracy: 0.9940
1006/1688 [================>.............] - ETA: 1s - loss: 0.0179 - accuracy: 0.9940
1027/1688 [=================>............] - ETA: 1s - loss: 0.0177 - accuracy: 0.9941
1048/1688 [=================>............] - ETA: 1s - loss: 0.0177 - accuracy: 0.9941
1069/1688 [=================>............] - ETA: 1s - loss: 0.0176 - accuracy: 0.9942
1091/1688 [==================>...........] - ETA: 1s - loss: 0.0177 - accuracy: 0.9941
1112/1688 [==================>...........] - ETA: 1s - loss: 0.0177 - accuracy: 0.9941
1133/1688 [===================>..........] - ETA: 1s - loss: 0.0177 - accuracy: 0.9940
1154/1688 [===================>..........] - ETA: 1s - loss: 0.0176 - accuracy: 0.9941
1175/1688 [===================>..........] - ETA: 1s - loss: 0.0176 - accuracy: 0.9941
1196/1688 [====================>.........] - ETA: 1s - loss: 0.0176 - accuracy: 0.9941
1217/1688 [====================>.........] - ETA: 1s - loss: 0.0176 - accuracy: 0.9940
1238/1688 [=====================>........] - ETA: 1s - loss: 0.0176 - accuracy: 0.9940
1259/1688 [=====================>........] - ETA: 1s - loss: 0.0175 - accuracy: 0.9940
1281/1688 [=====================>........] - ETA: 0s - loss: 0.0177 - accuracy: 0.9940
1302/1688 [======================>.......] - ETA: 0s - loss: 0.0177 - accuracy: 0.9940
1323/1688 [======================>.......] - ETA: 0s - loss: 0.0179 - accuracy: 0.9940
1345/1688 [======================>.......] - ETA: 0s - loss: 0.0177 - accuracy: 0.9941
1367/1688 [=======================>......] - ETA: 0s - loss: 0.0180 - accuracy: 0.9940
1388/1688 [=======================>......] - ETA: 0s - loss: 0.0181 - accuracy: 0.9939
1408/1688 [========================>.....] - ETA: 0s - loss: 0.0181 - accuracy: 0.9939
1428/1688 [========================>.....] - ETA: 0s - loss: 0.0182 - accuracy: 0.9939
1449/1688 [========================>.....] - ETA: 0s - loss: 0.0184 - accuracy: 0.9938
1470/1688 [=========================>....] - ETA: 0s - loss: 0.0184 - accuracy: 0.9938
1491/1688 [=========================>....] - ETA: 0s - loss: 0.0184 - accuracy: 0.9938
1512/1688 [=========================>....] - ETA: 0s - loss: 0.0187 - accuracy: 0.9937
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0188 - accuracy: 0.9937
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0186 - accuracy: 0.9937
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0187 - accuracy: 0.9937
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0188 - accuracy: 0.9937
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0188 - accuracy: 0.9937
1638/1688 [============================>.] - ETA: 0s - loss: 0.0188 - accuracy: 0.9937
1659/1688 [============================>.] - ETA: 0s - loss: 0.0187 - accuracy: 0.9937
1681/1688 [============================>.] - ETA: 0s - loss: 0.0187 - accuracy: 0.9937
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0187 - accuracy: 0.9937 - val_loss: 0.0601 - val_accuracy: 0.9863
Epoch 10/10
1/1688 [..............................] - ETA: 4s - loss: 0.0101 - accuracy: 1.0000
22/1688 [..............................] - ETA: 3s - loss: 0.0118 - accuracy: 0.9957
43/1688 [..............................] - ETA: 3s - loss: 0.0122 - accuracy: 0.9949
64/1688 [>.............................] - ETA: 3s - loss: 0.0144 - accuracy: 0.9946
85/1688 [>.............................] - ETA: 3s - loss: 0.0137 - accuracy: 0.9949
106/1688 [>.............................] - ETA: 3s - loss: 0.0130 - accuracy: 0.9950
127/1688 [=>............................] - ETA: 3s - loss: 0.0139 - accuracy: 0.9946
148/1688 [=>............................] - ETA: 3s - loss: 0.0131 - accuracy: 0.9945
169/1688 [==>...........................] - ETA: 3s - loss: 0.0143 - accuracy: 0.9941
190/1688 [==>...........................] - ETA: 3s - loss: 0.0160 - accuracy: 0.9937
211/1688 [==>...........................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9941
232/1688 [===>..........................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9941
254/1688 [===>..........................] - ETA: 3s - loss: 0.0154 - accuracy: 0.9945
275/1688 [===>..........................] - ETA: 3s - loss: 0.0163 - accuracy: 0.9942
296/1688 [====>.........................] - ETA: 3s - loss: 0.0161 - accuracy: 0.9942
317/1688 [====>.........................] - ETA: 3s - loss: 0.0157 - accuracy: 0.9944
338/1688 [=====>........................] - ETA: 3s - loss: 0.0154 - accuracy: 0.9946
359/1688 [=====>........................] - ETA: 3s - loss: 0.0150 - accuracy: 0.9949
380/1688 [=====>........................] - ETA: 3s - loss: 0.0144 - accuracy: 0.9951
401/1688 [======>.......................] - ETA: 3s - loss: 0.0138 - accuracy: 0.9954
423/1688 [======>.......................] - ETA: 3s - loss: 0.0136 - accuracy: 0.9956
444/1688 [======>.......................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9957
465/1688 [=======>......................] - ETA: 2s - loss: 0.0128 - accuracy: 0.9959
486/1688 [=======>......................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9959
507/1688 [========>.....................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9960
528/1688 [========>.....................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9959
549/1688 [========>.....................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9960
570/1688 [=========>....................] - ETA: 2s - loss: 0.0123 - accuracy: 0.9961
591/1688 [=========>....................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9960
612/1688 [=========>....................] - ETA: 2s - loss: 0.0122 - accuracy: 0.9961
633/1688 [==========>...................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9961
654/1688 [==========>...................] - ETA: 2s - loss: 0.0122 - accuracy: 0.9961
675/1688 [==========>...................] - ETA: 2s - loss: 0.0121 - accuracy: 0.9962
696/1688 [===========>..................] - ETA: 2s - loss: 0.0120 - accuracy: 0.9962
717/1688 [===========>..................] - ETA: 2s - loss: 0.0122 - accuracy: 0.9961
738/1688 [============>.................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9960
758/1688 [============>.................] - ETA: 2s - loss: 0.0122 - accuracy: 0.9961
779/1688 [============>.................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9959
800/1688 [=============>................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9958
821/1688 [=============>................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9958
842/1688 [=============>................] - ETA: 2s - loss: 0.0128 - accuracy: 0.9958
863/1688 [==============>...............] - ETA: 2s - loss: 0.0128 - accuracy: 0.9958
884/1688 [==============>...............] - ETA: 1s - loss: 0.0133 - accuracy: 0.9957
906/1688 [===============>..............] - ETA: 1s - loss: 0.0133 - accuracy: 0.9956
927/1688 [===============>..............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9956
948/1688 [===============>..............] - ETA: 1s - loss: 0.0131 - accuracy: 0.9957
969/1688 [================>.............] - ETA: 1s - loss: 0.0130 - accuracy: 0.9957
990/1688 [================>.............] - ETA: 1s - loss: 0.0131 - accuracy: 0.9958
1011/1688 [================>.............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9958
1033/1688 [=================>............] - ETA: 1s - loss: 0.0132 - accuracy: 0.9958
1054/1688 [=================>............] - ETA: 1s - loss: 0.0131 - accuracy: 0.9958
1075/1688 [==================>...........] - ETA: 1s - loss: 0.0131 - accuracy: 0.9958
1096/1688 [==================>...........] - ETA: 1s - loss: 0.0129 - accuracy: 0.9958
1117/1688 [==================>...........] - ETA: 1s - loss: 0.0130 - accuracy: 0.9959
1138/1688 [===================>..........] - ETA: 1s - loss: 0.0129 - accuracy: 0.9959
1159/1688 [===================>..........] - ETA: 1s - loss: 0.0130 - accuracy: 0.9958
1180/1688 [===================>..........] - ETA: 1s - loss: 0.0131 - accuracy: 0.9958
1202/1688 [====================>.........] - ETA: 1s - loss: 0.0134 - accuracy: 0.9957
1223/1688 [====================>.........] - ETA: 1s - loss: 0.0139 - accuracy: 0.9955
1244/1688 [=====================>........] - ETA: 1s - loss: 0.0141 - accuracy: 0.9954
1266/1688 [=====================>........] - ETA: 1s - loss: 0.0143 - accuracy: 0.9954
1287/1688 [=====================>........] - ETA: 0s - loss: 0.0143 - accuracy: 0.9953
1308/1688 [======================>.......] - ETA: 0s - loss: 0.0144 - accuracy: 0.9953
1329/1688 [======================>.......] - ETA: 0s - loss: 0.0148 - accuracy: 0.9953
1350/1688 [======================>.......] - ETA: 0s - loss: 0.0149 - accuracy: 0.9952
1372/1688 [=======================>......] - ETA: 0s - loss: 0.0152 - accuracy: 0.9951
1394/1688 [=======================>......] - ETA: 0s - loss: 0.0154 - accuracy: 0.9950
1415/1688 [========================>.....] - ETA: 0s - loss: 0.0155 - accuracy: 0.9950
1436/1688 [========================>.....] - ETA: 0s - loss: 0.0157 - accuracy: 0.9949
1457/1688 [========================>.....] - ETA: 0s - loss: 0.0159 - accuracy: 0.9948
1478/1688 [=========================>....] - ETA: 0s - loss: 0.0164 - accuracy: 0.9947
1499/1688 [=========================>....] - ETA: 0s - loss: 0.0166 - accuracy: 0.9946
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0166 - accuracy: 0.9946
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0167 - accuracy: 0.9946
1561/1688 [==========================>...] - ETA: 0s - loss: 0.0168 - accuracy: 0.9945
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0170 - accuracy: 0.9944
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0170 - accuracy: 0.9944
1625/1688 [===========================>..] - ETA: 0s - loss: 0.0169 - accuracy: 0.9944
1646/1688 [============================>.] - ETA: 0s - loss: 0.0170 - accuracy: 0.9945
1667/1688 [============================>.] - ETA: 0s - loss: 0.0172 - accuracy: 0.9944
1688/1688 [==============================] - ETA: 0s - loss: 0.0173 - accuracy: 0.9944
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0173 - accuracy: 0.9944 - val_loss: 0.0895 - val_accuracy: 0.9795
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.9768000245094299
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bits integer activations and weights. Therefore, the floating point Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating point Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a Keras model with custom QuantizeML quantized layers substituted for the original Keras layers.
All Keras API functions can be applied on this new model: summary(), compile(), fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:488: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate. "
1/1024 [..............................] - ETA: 3:49
52/1024 [>.............................] - ETA: 0s
103/1024 [==>...........................] - ETA: 0s
154/1024 [===>..........................] - ETA: 0s
205/1024 [=====>........................] - ETA: 0s
257/1024 [======>.......................] - ETA: 0s
308/1024 [========>.....................] - ETA: 0s
359/1024 [=========>....................] - ETA: 0s
411/1024 [===========>..................] - ETA: 0s
462/1024 [============>.................] - ETA: 0s
513/1024 [==============>...............] - ETA: 0s
565/1024 [===============>..............] - ETA: 0s
616/1024 [=================>............] - ETA: 0s
666/1024 [==================>...........] - ETA: 0s
718/1024 [====================>.........] - ETA: 0s
769/1024 [=====================>........] - ETA: 0s
820/1024 [=======================>......] - ETA: 0s
872/1024 [========================>.....] - ETA: 0s
924/1024 [==========================>...] - ETA: 0s
975/1024 [===========================>..] - ETA: 0s
1024/1024 [==============================] - 1s 985us/step
model_quantized.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling_input (InputLaye [(None, 28, 28, 1)] 0
r)
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.9664999842643738
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 2s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.9675999879837036
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.9623000025749207
2.4. Model fine tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 2:15:25 - loss: 0.0770 - accuracy: 0.9688
9/1688 [..............................] - ETA: 10s - loss: 0.0742 - accuracy: 0.9653
17/1688 [..............................] - ETA: 10s - loss: 0.0653 - accuracy: 0.9688
25/1688 [..............................] - ETA: 10s - loss: 0.0614 - accuracy: 0.9700
34/1688 [..............................] - ETA: 10s - loss: 0.0648 - accuracy: 0.9724
42/1688 [..............................] - ETA: 10s - loss: 0.0557 - accuracy: 0.9762
50/1688 [..............................] - ETA: 10s - loss: 0.0535 - accuracy: 0.9775
58/1688 [>.............................] - ETA: 10s - loss: 0.0558 - accuracy: 0.9763
67/1688 [>.............................] - ETA: 10s - loss: 0.0534 - accuracy: 0.9767
75/1688 [>.............................] - ETA: 10s - loss: 0.0505 - accuracy: 0.9783
83/1688 [>.............................] - ETA: 10s - loss: 0.0473 - accuracy: 0.9800
92/1688 [>.............................] - ETA: 10s - loss: 0.0491 - accuracy: 0.9796
100/1688 [>.............................] - ETA: 10s - loss: 0.0465 - accuracy: 0.9809
109/1688 [>.............................] - ETA: 9s - loss: 0.0452 - accuracy: 0.9819
118/1688 [=>............................] - ETA: 9s - loss: 0.0441 - accuracy: 0.9828
127/1688 [=>............................] - ETA: 9s - loss: 0.0427 - accuracy: 0.9835
135/1688 [=>............................] - ETA: 9s - loss: 0.0418 - accuracy: 0.9840
143/1688 [=>............................] - ETA: 9s - loss: 0.0411 - accuracy: 0.9843
151/1688 [=>............................] - ETA: 9s - loss: 0.0396 - accuracy: 0.9851
160/1688 [=>............................] - ETA: 9s - loss: 0.0392 - accuracy: 0.9855
168/1688 [=>............................] - ETA: 9s - loss: 0.0376 - accuracy: 0.9862
176/1688 [==>...........................] - ETA: 9s - loss: 0.0364 - accuracy: 0.9867
185/1688 [==>...........................] - ETA: 9s - loss: 0.0360 - accuracy: 0.9870
193/1688 [==>...........................] - ETA: 9s - loss: 0.0353 - accuracy: 0.9875
202/1688 [==>...........................] - ETA: 9s - loss: 0.0348 - accuracy: 0.9879
210/1688 [==>...........................] - ETA: 9s - loss: 0.0341 - accuracy: 0.9881
218/1688 [==>...........................] - ETA: 9s - loss: 0.0334 - accuracy: 0.9884
226/1688 [===>..........................] - ETA: 9s - loss: 0.0325 - accuracy: 0.9888
234/1688 [===>..........................] - ETA: 9s - loss: 0.0323 - accuracy: 0.9888
243/1688 [===>..........................] - ETA: 9s - loss: 0.0325 - accuracy: 0.9887
251/1688 [===>..........................] - ETA: 9s - loss: 0.0320 - accuracy: 0.9888
259/1688 [===>..........................] - ETA: 9s - loss: 0.0320 - accuracy: 0.9887
267/1688 [===>..........................] - ETA: 8s - loss: 0.0320 - accuracy: 0.9886
275/1688 [===>..........................] - ETA: 8s - loss: 0.0316 - accuracy: 0.9890
283/1688 [====>.........................] - ETA: 8s - loss: 0.0313 - accuracy: 0.9892
291/1688 [====>.........................] - ETA: 8s - loss: 0.0310 - accuracy: 0.9893
299/1688 [====>.........................] - ETA: 8s - loss: 0.0305 - accuracy: 0.9894
307/1688 [====>.........................] - ETA: 8s - loss: 0.0305 - accuracy: 0.9894
316/1688 [====>.........................] - ETA: 8s - loss: 0.0304 - accuracy: 0.9895
324/1688 [====>.........................] - ETA: 8s - loss: 0.0299 - accuracy: 0.9898
332/1688 [====>.........................] - ETA: 8s - loss: 0.0301 - accuracy: 0.9896
341/1688 [=====>........................] - ETA: 8s - loss: 0.0303 - accuracy: 0.9897
350/1688 [=====>........................] - ETA: 8s - loss: 0.0302 - accuracy: 0.9897
358/1688 [=====>........................] - ETA: 8s - loss: 0.0306 - accuracy: 0.9895
367/1688 [=====>........................] - ETA: 8s - loss: 0.0303 - accuracy: 0.9895
375/1688 [=====>........................] - ETA: 8s - loss: 0.0300 - accuracy: 0.9897
383/1688 [=====>........................] - ETA: 8s - loss: 0.0296 - accuracy: 0.9898
391/1688 [=====>........................] - ETA: 8s - loss: 0.0297 - accuracy: 0.9898
399/1688 [======>.......................] - ETA: 8s - loss: 0.0294 - accuracy: 0.9899
407/1688 [======>.......................] - ETA: 8s - loss: 0.0290 - accuracy: 0.9901
415/1688 [======>.......................] - ETA: 8s - loss: 0.0289 - accuracy: 0.9900
423/1688 [======>.......................] - ETA: 8s - loss: 0.0287 - accuracy: 0.9901
432/1688 [======>.......................] - ETA: 7s - loss: 0.0284 - accuracy: 0.9903
441/1688 [======>.......................] - ETA: 7s - loss: 0.0280 - accuracy: 0.9904
449/1688 [======>.......................] - ETA: 7s - loss: 0.0278 - accuracy: 0.9906
457/1688 [=======>......................] - ETA: 7s - loss: 0.0275 - accuracy: 0.9908
466/1688 [=======>......................] - ETA: 7s - loss: 0.0272 - accuracy: 0.9909
474/1688 [=======>......................] - ETA: 7s - loss: 0.0271 - accuracy: 0.9909
482/1688 [=======>......................] - ETA: 7s - loss: 0.0272 - accuracy: 0.9909
490/1688 [=======>......................] - ETA: 7s - loss: 0.0272 - accuracy: 0.9909
498/1688 [=======>......................] - ETA: 7s - loss: 0.0271 - accuracy: 0.9909
507/1688 [========>.....................] - ETA: 7s - loss: 0.0270 - accuracy: 0.9909
515/1688 [========>.....................] - ETA: 7s - loss: 0.0268 - accuracy: 0.9911
523/1688 [========>.....................] - ETA: 7s - loss: 0.0267 - accuracy: 0.9911
531/1688 [========>.....................] - ETA: 7s - loss: 0.0264 - accuracy: 0.9912
539/1688 [========>.....................] - ETA: 7s - loss: 0.0262 - accuracy: 0.9913
548/1688 [========>.....................] - ETA: 7s - loss: 0.0263 - accuracy: 0.9912
556/1688 [========>.....................] - ETA: 7s - loss: 0.0262 - accuracy: 0.9912
564/1688 [=========>....................] - ETA: 7s - loss: 0.0260 - accuracy: 0.9914
572/1688 [=========>....................] - ETA: 7s - loss: 0.0262 - accuracy: 0.9913
580/1688 [=========>....................] - ETA: 7s - loss: 0.0261 - accuracy: 0.9914
589/1688 [=========>....................] - ETA: 6s - loss: 0.0259 - accuracy: 0.9915
597/1688 [=========>....................] - ETA: 6s - loss: 0.0259 - accuracy: 0.9915
605/1688 [=========>....................] - ETA: 6s - loss: 0.0257 - accuracy: 0.9916
613/1688 [=========>....................] - ETA: 6s - loss: 0.0255 - accuracy: 0.9917
622/1688 [==========>...................] - ETA: 6s - loss: 0.0255 - accuracy: 0.9917
631/1688 [==========>...................] - ETA: 6s - loss: 0.0252 - accuracy: 0.9918
639/1688 [==========>...................] - ETA: 6s - loss: 0.0251 - accuracy: 0.9918
647/1688 [==========>...................] - ETA: 6s - loss: 0.0249 - accuracy: 0.9919
655/1688 [==========>...................] - ETA: 6s - loss: 0.0250 - accuracy: 0.9919
663/1688 [==========>...................] - ETA: 6s - loss: 0.0248 - accuracy: 0.9920
672/1688 [==========>...................] - ETA: 6s - loss: 0.0249 - accuracy: 0.9920
681/1688 [===========>..................] - ETA: 6s - loss: 0.0248 - accuracy: 0.9921
690/1688 [===========>..................] - ETA: 6s - loss: 0.0248 - accuracy: 0.9920
698/1688 [===========>..................] - ETA: 6s - loss: 0.0247 - accuracy: 0.9921
706/1688 [===========>..................] - ETA: 6s - loss: 0.0245 - accuracy: 0.9922
715/1688 [===========>..................] - ETA: 6s - loss: 0.0244 - accuracy: 0.9923
723/1688 [===========>..................] - ETA: 6s - loss: 0.0243 - accuracy: 0.9923
731/1688 [===========>..................] - ETA: 6s - loss: 0.0242 - accuracy: 0.9923
739/1688 [============>.................] - ETA: 6s - loss: 0.0242 - accuracy: 0.9923
748/1688 [============>.................] - ETA: 5s - loss: 0.0241 - accuracy: 0.9923
756/1688 [============>.................] - ETA: 5s - loss: 0.0240 - accuracy: 0.9923
764/1688 [============>.................] - ETA: 5s - loss: 0.0240 - accuracy: 0.9924
773/1688 [============>.................] - ETA: 5s - loss: 0.0239 - accuracy: 0.9924
782/1688 [============>.................] - ETA: 5s - loss: 0.0239 - accuracy: 0.9924
790/1688 [=============>................] - ETA: 5s - loss: 0.0238 - accuracy: 0.9924
798/1688 [=============>................] - ETA: 5s - loss: 0.0237 - accuracy: 0.9925
806/1688 [=============>................] - ETA: 5s - loss: 0.0235 - accuracy: 0.9926
814/1688 [=============>................] - ETA: 5s - loss: 0.0236 - accuracy: 0.9925
822/1688 [=============>................] - ETA: 5s - loss: 0.0237 - accuracy: 0.9925
831/1688 [=============>................] - ETA: 5s - loss: 0.0237 - accuracy: 0.9925
839/1688 [=============>................] - ETA: 5s - loss: 0.0237 - accuracy: 0.9924
848/1688 [==============>...............] - ETA: 5s - loss: 0.0238 - accuracy: 0.9924
856/1688 [==============>...............] - ETA: 5s - loss: 0.0238 - accuracy: 0.9924
865/1688 [==============>...............] - ETA: 5s - loss: 0.0238 - accuracy: 0.9924
874/1688 [==============>...............] - ETA: 5s - loss: 0.0237 - accuracy: 0.9925
882/1688 [==============>...............] - ETA: 5s - loss: 0.0236 - accuracy: 0.9925
890/1688 [==============>...............] - ETA: 5s - loss: 0.0237 - accuracy: 0.9925
898/1688 [==============>...............] - ETA: 5s - loss: 0.0235 - accuracy: 0.9925
906/1688 [===============>..............] - ETA: 4s - loss: 0.0234 - accuracy: 0.9926
914/1688 [===============>..............] - ETA: 4s - loss: 0.0234 - accuracy: 0.9926
923/1688 [===============>..............] - ETA: 4s - loss: 0.0234 - accuracy: 0.9926
932/1688 [===============>..............] - ETA: 4s - loss: 0.0234 - accuracy: 0.9926
940/1688 [===============>..............] - ETA: 4s - loss: 0.0234 - accuracy: 0.9926
948/1688 [===============>..............] - ETA: 4s - loss: 0.0233 - accuracy: 0.9926
956/1688 [===============>..............] - ETA: 4s - loss: 0.0232 - accuracy: 0.9926
964/1688 [================>.............] - ETA: 4s - loss: 0.0233 - accuracy: 0.9926
973/1688 [================>.............] - ETA: 4s - loss: 0.0231 - accuracy: 0.9926
981/1688 [================>.............] - ETA: 4s - loss: 0.0231 - accuracy: 0.9927
989/1688 [================>.............] - ETA: 4s - loss: 0.0230 - accuracy: 0.9927
997/1688 [================>.............] - ETA: 4s - loss: 0.0230 - accuracy: 0.9927
1005/1688 [================>.............] - ETA: 4s - loss: 0.0229 - accuracy: 0.9928
1013/1688 [=================>............] - ETA: 4s - loss: 0.0228 - accuracy: 0.9928
1022/1688 [=================>............] - ETA: 4s - loss: 0.0227 - accuracy: 0.9928
1030/1688 [=================>............] - ETA: 4s - loss: 0.0228 - accuracy: 0.9927
1038/1688 [=================>............] - ETA: 4s - loss: 0.0227 - accuracy: 0.9927
1046/1688 [=================>............] - ETA: 4s - loss: 0.0227 - accuracy: 0.9928
1054/1688 [=================>............] - ETA: 4s - loss: 0.0226 - accuracy: 0.9928
1062/1688 [=================>............] - ETA: 3s - loss: 0.0227 - accuracy: 0.9928
1070/1688 [==================>...........] - ETA: 3s - loss: 0.0228 - accuracy: 0.9927
1078/1688 [==================>...........] - ETA: 3s - loss: 0.0227 - accuracy: 0.9928
1086/1688 [==================>...........] - ETA: 3s - loss: 0.0228 - accuracy: 0.9927
1094/1688 [==================>...........] - ETA: 3s - loss: 0.0227 - accuracy: 0.9927
1102/1688 [==================>...........] - ETA: 3s - loss: 0.0227 - accuracy: 0.9927
1111/1688 [==================>...........] - ETA: 3s - loss: 0.0228 - accuracy: 0.9927
1119/1688 [==================>...........] - ETA: 3s - loss: 0.0227 - accuracy: 0.9927
1127/1688 [===================>..........] - ETA: 3s - loss: 0.0226 - accuracy: 0.9928
1135/1688 [===================>..........] - ETA: 3s - loss: 0.0226 - accuracy: 0.9928
1144/1688 [===================>..........] - ETA: 3s - loss: 0.0225 - accuracy: 0.9928
1152/1688 [===================>..........] - ETA: 3s - loss: 0.0226 - accuracy: 0.9928
1160/1688 [===================>..........] - ETA: 3s - loss: 0.0225 - accuracy: 0.9928
1168/1688 [===================>..........] - ETA: 3s - loss: 0.0224 - accuracy: 0.9928
1176/1688 [===================>..........] - ETA: 3s - loss: 0.0224 - accuracy: 0.9928
1185/1688 [====================>.........] - ETA: 3s - loss: 0.0223 - accuracy: 0.9929
1193/1688 [====================>.........] - ETA: 3s - loss: 0.0225 - accuracy: 0.9928
1202/1688 [====================>.........] - ETA: 3s - loss: 0.0224 - accuracy: 0.9928
1210/1688 [====================>.........] - ETA: 3s - loss: 0.0224 - accuracy: 0.9928
1218/1688 [====================>.........] - ETA: 2s - loss: 0.0224 - accuracy: 0.9929
1226/1688 [====================>.........] - ETA: 2s - loss: 0.0224 - accuracy: 0.9929
1234/1688 [====================>.........] - ETA: 2s - loss: 0.0223 - accuracy: 0.9929
1243/1688 [=====================>........] - ETA: 2s - loss: 0.0223 - accuracy: 0.9929
1251/1688 [=====================>........] - ETA: 2s - loss: 0.0223 - accuracy: 0.9929
1259/1688 [=====================>........] - ETA: 2s - loss: 0.0222 - accuracy: 0.9929
1268/1688 [=====================>........] - ETA: 2s - loss: 0.0221 - accuracy: 0.9930
1277/1688 [=====================>........] - ETA: 2s - loss: 0.0221 - accuracy: 0.9929
1285/1688 [=====================>........] - ETA: 2s - loss: 0.0221 - accuracy: 0.9929
1293/1688 [=====================>........] - ETA: 2s - loss: 0.0220 - accuracy: 0.9929
1301/1688 [======================>.......] - ETA: 2s - loss: 0.0220 - accuracy: 0.9930
1309/1688 [======================>.......] - ETA: 2s - loss: 0.0219 - accuracy: 0.9930
1317/1688 [======================>.......] - ETA: 2s - loss: 0.0219 - accuracy: 0.9930
1326/1688 [======================>.......] - ETA: 2s - loss: 0.0218 - accuracy: 0.9931
1335/1688 [======================>.......] - ETA: 2s - loss: 0.0217 - accuracy: 0.9931
1343/1688 [======================>.......] - ETA: 2s - loss: 0.0216 - accuracy: 0.9931
1351/1688 [=======================>......] - ETA: 2s - loss: 0.0216 - accuracy: 0.9932
1360/1688 [=======================>......] - ETA: 2s - loss: 0.0216 - accuracy: 0.9932
1369/1688 [=======================>......] - ETA: 2s - loss: 0.0215 - accuracy: 0.9932
1377/1688 [=======================>......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9932
1385/1688 [=======================>......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9932
1393/1688 [=======================>......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9932
1401/1688 [=======================>......] - ETA: 1s - loss: 0.0215 - accuracy: 0.9932
1409/1688 [========================>.....] - ETA: 1s - loss: 0.0215 - accuracy: 0.9932
1417/1688 [========================>.....] - ETA: 1s - loss: 0.0214 - accuracy: 0.9932
1425/1688 [========================>.....] - ETA: 1s - loss: 0.0214 - accuracy: 0.9932
1433/1688 [========================>.....] - ETA: 1s - loss: 0.0213 - accuracy: 0.9933
1441/1688 [========================>.....] - ETA: 1s - loss: 0.0213 - accuracy: 0.9933
1449/1688 [========================>.....] - ETA: 1s - loss: 0.0212 - accuracy: 0.9933
1458/1688 [========================>.....] - ETA: 1s - loss: 0.0212 - accuracy: 0.9933
1466/1688 [=========================>....] - ETA: 1s - loss: 0.0212 - accuracy: 0.9934
1474/1688 [=========================>....] - ETA: 1s - loss: 0.0211 - accuracy: 0.9934
1482/1688 [=========================>....] - ETA: 1s - loss: 0.0211 - accuracy: 0.9934
1490/1688 [=========================>....] - ETA: 1s - loss: 0.0211 - accuracy: 0.9934
1499/1688 [=========================>....] - ETA: 1s - loss: 0.0210 - accuracy: 0.9935
1508/1688 [=========================>....] - ETA: 1s - loss: 0.0210 - accuracy: 0.9934
1516/1688 [=========================>....] - ETA: 1s - loss: 0.0209 - accuracy: 0.9934
1525/1688 [==========================>...] - ETA: 1s - loss: 0.0209 - accuracy: 0.9934
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0209 - accuracy: 0.9934
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0209 - accuracy: 0.9935
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0208 - accuracy: 0.9935
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0208 - accuracy: 0.9935
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0207 - accuracy: 0.9935
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0207 - accuracy: 0.9935
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0207 - accuracy: 0.9936
1590/1688 [===========================>..] - ETA: 0s - loss: 0.0207 - accuracy: 0.9936
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0207 - accuracy: 0.9935
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0208 - accuracy: 0.9935
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0208 - accuracy: 0.9935
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0208 - accuracy: 0.9935
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0208 - accuracy: 0.9935
1639/1688 [============================>.] - ETA: 0s - loss: 0.0208 - accuracy: 0.9935
1648/1688 [============================>.] - ETA: 0s - loss: 0.0207 - accuracy: 0.9936
1657/1688 [============================>.] - ETA: 0s - loss: 0.0206 - accuracy: 0.9936
1665/1688 [============================>.] - ETA: 0s - loss: 0.0206 - accuracy: 0.9936
1673/1688 [============================>.] - ETA: 0s - loss: 0.0206 - accuracy: 0.9936
1681/1688 [============================>.] - ETA: 0s - loss: 0.0206 - accuracy: 0.9936
1688/1688 [==============================] - 18s 8ms/step - loss: 0.0206 - accuracy: 0.9936 - val_loss: 0.0495 - val_accuracy: 0.9863
Epoch 2/5
1/1688 [..............................] - ETA: 10s - loss: 0.0059 - accuracy: 1.0000
9/1688 [..............................] - ETA: 11s - loss: 0.0078 - accuracy: 1.0000
17/1688 [..............................] - ETA: 10s - loss: 0.0168 - accuracy: 0.9945
26/1688 [..............................] - ETA: 10s - loss: 0.0142 - accuracy: 0.9952
34/1688 [..............................] - ETA: 10s - loss: 0.0139 - accuracy: 0.9954
43/1688 [..............................] - ETA: 10s - loss: 0.0158 - accuracy: 0.9949
51/1688 [..............................] - ETA: 10s - loss: 0.0163 - accuracy: 0.9951
59/1688 [>.............................] - ETA: 10s - loss: 0.0151 - accuracy: 0.9958
67/1688 [>.............................] - ETA: 10s - loss: 0.0139 - accuracy: 0.9963
76/1688 [>.............................] - ETA: 10s - loss: 0.0142 - accuracy: 0.9967
85/1688 [>.............................] - ETA: 10s - loss: 0.0142 - accuracy: 0.9963
93/1688 [>.............................] - ETA: 10s - loss: 0.0150 - accuracy: 0.9963
102/1688 [>.............................] - ETA: 10s - loss: 0.0150 - accuracy: 0.9963
110/1688 [>.............................] - ETA: 10s - loss: 0.0147 - accuracy: 0.9966
119/1688 [=>............................] - ETA: 9s - loss: 0.0145 - accuracy: 0.9966
127/1688 [=>............................] - ETA: 9s - loss: 0.0141 - accuracy: 0.9968
136/1688 [=>............................] - ETA: 9s - loss: 0.0134 - accuracy: 0.9970
144/1688 [=>............................] - ETA: 9s - loss: 0.0137 - accuracy: 0.9967
152/1688 [=>............................] - ETA: 9s - loss: 0.0133 - accuracy: 0.9969
160/1688 [=>............................] - ETA: 9s - loss: 0.0141 - accuracy: 0.9969
168/1688 [=>............................] - ETA: 9s - loss: 0.0138 - accuracy: 0.9970
177/1688 [==>...........................] - ETA: 9s - loss: 0.0134 - accuracy: 0.9972
185/1688 [==>...........................] - ETA: 9s - loss: 0.0134 - accuracy: 0.9971
193/1688 [==>...........................] - ETA: 9s - loss: 0.0134 - accuracy: 0.9971
201/1688 [==>...........................] - ETA: 9s - loss: 0.0132 - accuracy: 0.9972
209/1688 [==>...........................] - ETA: 9s - loss: 0.0131 - accuracy: 0.9973
217/1688 [==>...........................] - ETA: 9s - loss: 0.0130 - accuracy: 0.9973
226/1688 [===>..........................] - ETA: 9s - loss: 0.0132 - accuracy: 0.9971
234/1688 [===>..........................] - ETA: 9s - loss: 0.0131 - accuracy: 0.9972
243/1688 [===>..........................] - ETA: 9s - loss: 0.0129 - accuracy: 0.9973
251/1688 [===>..........................] - ETA: 9s - loss: 0.0128 - accuracy: 0.9973
259/1688 [===>..........................] - ETA: 9s - loss: 0.0133 - accuracy: 0.9972
267/1688 [===>..........................] - ETA: 9s - loss: 0.0133 - accuracy: 0.9972
275/1688 [===>..........................] - ETA: 8s - loss: 0.0133 - accuracy: 0.9973
283/1688 [====>.........................] - ETA: 8s - loss: 0.0136 - accuracy: 0.9971
291/1688 [====>.........................] - ETA: 8s - loss: 0.0134 - accuracy: 0.9972
299/1688 [====>.........................] - ETA: 8s - loss: 0.0134 - accuracy: 0.9972
307/1688 [====>.........................] - ETA: 8s - loss: 0.0135 - accuracy: 0.9973
315/1688 [====>.........................] - ETA: 8s - loss: 0.0137 - accuracy: 0.9971
323/1688 [====>.........................] - ETA: 8s - loss: 0.0136 - accuracy: 0.9972
331/1688 [====>.........................] - ETA: 8s - loss: 0.0137 - accuracy: 0.9971
340/1688 [=====>........................] - ETA: 8s - loss: 0.0139 - accuracy: 0.9970
348/1688 [=====>........................] - ETA: 8s - loss: 0.0138 - accuracy: 0.9970
356/1688 [=====>........................] - ETA: 8s - loss: 0.0143 - accuracy: 0.9968
364/1688 [=====>........................] - ETA: 8s - loss: 0.0144 - accuracy: 0.9967
372/1688 [=====>........................] - ETA: 8s - loss: 0.0144 - accuracy: 0.9967
380/1688 [=====>........................] - ETA: 8s - loss: 0.0144 - accuracy: 0.9966
388/1688 [=====>........................] - ETA: 8s - loss: 0.0143 - accuracy: 0.9967
396/1688 [======>.......................] - ETA: 8s - loss: 0.0144 - accuracy: 0.9967
404/1688 [======>.......................] - ETA: 8s - loss: 0.0143 - accuracy: 0.9968
412/1688 [======>.......................] - ETA: 8s - loss: 0.0143 - accuracy: 0.9968
420/1688 [======>.......................] - ETA: 8s - loss: 0.0142 - accuracy: 0.9969
428/1688 [======>.......................] - ETA: 8s - loss: 0.0140 - accuracy: 0.9969
436/1688 [======>.......................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9970
444/1688 [======>.......................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9970
452/1688 [=======>......................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9970
461/1688 [=======>......................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9971
469/1688 [=======>......................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9971
478/1688 [=======>......................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9971
487/1688 [=======>......................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9970
495/1688 [=======>......................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9970
503/1688 [=======>......................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9970
511/1688 [========>.....................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9969
519/1688 [========>.....................] - ETA: 7s - loss: 0.0139 - accuracy: 0.9968
528/1688 [========>.....................] - ETA: 7s - loss: 0.0139 - accuracy: 0.9967
536/1688 [========>.....................] - ETA: 7s - loss: 0.0141 - accuracy: 0.9967
544/1688 [========>.....................] - ETA: 7s - loss: 0.0141 - accuracy: 0.9967
552/1688 [========>.....................] - ETA: 7s - loss: 0.0141 - accuracy: 0.9966
560/1688 [========>.....................] - ETA: 7s - loss: 0.0144 - accuracy: 0.9965
568/1688 [=========>....................] - ETA: 7s - loss: 0.0143 - accuracy: 0.9965
576/1688 [=========>....................] - ETA: 7s - loss: 0.0143 - accuracy: 0.9965
584/1688 [=========>....................] - ETA: 7s - loss: 0.0146 - accuracy: 0.9964
592/1688 [=========>....................] - ETA: 6s - loss: 0.0145 - accuracy: 0.9964
600/1688 [=========>....................] - ETA: 6s - loss: 0.0146 - accuracy: 0.9964
608/1688 [=========>....................] - ETA: 6s - loss: 0.0145 - accuracy: 0.9965
616/1688 [=========>....................] - ETA: 6s - loss: 0.0144 - accuracy: 0.9964
624/1688 [==========>...................] - ETA: 6s - loss: 0.0145 - accuracy: 0.9964
632/1688 [==========>...................] - ETA: 6s - loss: 0.0144 - accuracy: 0.9965
640/1688 [==========>...................] - ETA: 6s - loss: 0.0144 - accuracy: 0.9965
648/1688 [==========>...................] - ETA: 6s - loss: 0.0143 - accuracy: 0.9965
657/1688 [==========>...................] - ETA: 6s - loss: 0.0142 - accuracy: 0.9966
665/1688 [==========>...................] - ETA: 6s - loss: 0.0141 - accuracy: 0.9966
673/1688 [==========>...................] - ETA: 6s - loss: 0.0141 - accuracy: 0.9966
681/1688 [===========>..................] - ETA: 6s - loss: 0.0140 - accuracy: 0.9966
689/1688 [===========>..................] - ETA: 6s - loss: 0.0141 - accuracy: 0.9966
697/1688 [===========>..................] - ETA: 6s - loss: 0.0140 - accuracy: 0.9966
705/1688 [===========>..................] - ETA: 6s - loss: 0.0141 - accuracy: 0.9966
713/1688 [===========>..................] - ETA: 6s - loss: 0.0140 - accuracy: 0.9966
721/1688 [===========>..................] - ETA: 6s - loss: 0.0140 - accuracy: 0.9966
729/1688 [===========>..................] - ETA: 6s - loss: 0.0139 - accuracy: 0.9967
737/1688 [============>.................] - ETA: 6s - loss: 0.0139 - accuracy: 0.9966
745/1688 [============>.................] - ETA: 6s - loss: 0.0139 - accuracy: 0.9966
753/1688 [============>.................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9966
761/1688 [============>.................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9966
769/1688 [============>.................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9966
777/1688 [============>.................] - ETA: 5s - loss: 0.0139 - accuracy: 0.9966
785/1688 [============>.................] - ETA: 5s - loss: 0.0140 - accuracy: 0.9966
793/1688 [=============>................] - ETA: 5s - loss: 0.0141 - accuracy: 0.9965
802/1688 [=============>................] - ETA: 5s - loss: 0.0141 - accuracy: 0.9965
810/1688 [=============>................] - ETA: 5s - loss: 0.0141 - accuracy: 0.9964
818/1688 [=============>................] - ETA: 5s - loss: 0.0141 - accuracy: 0.9964
826/1688 [=============>................] - ETA: 5s - loss: 0.0141 - accuracy: 0.9964
835/1688 [=============>................] - ETA: 5s - loss: 0.0140 - accuracy: 0.9964
843/1688 [=============>................] - ETA: 5s - loss: 0.0140 - accuracy: 0.9964
851/1688 [==============>...............] - ETA: 5s - loss: 0.0141 - accuracy: 0.9964
860/1688 [==============>...............] - ETA: 5s - loss: 0.0140 - accuracy: 0.9964
868/1688 [==============>...............] - ETA: 5s - loss: 0.0141 - accuracy: 0.9964
877/1688 [==============>...............] - ETA: 5s - loss: 0.0143 - accuracy: 0.9964
885/1688 [==============>...............] - ETA: 5s - loss: 0.0143 - accuracy: 0.9964
894/1688 [==============>...............] - ETA: 5s - loss: 0.0143 - accuracy: 0.9964
903/1688 [===============>..............] - ETA: 5s - loss: 0.0142 - accuracy: 0.9964
911/1688 [===============>..............] - ETA: 4s - loss: 0.0143 - accuracy: 0.9964
919/1688 [===============>..............] - ETA: 4s - loss: 0.0143 - accuracy: 0.9964
927/1688 [===============>..............] - ETA: 4s - loss: 0.0143 - accuracy: 0.9964
936/1688 [===============>..............] - ETA: 4s - loss: 0.0142 - accuracy: 0.9964
944/1688 [===============>..............] - ETA: 4s - loss: 0.0142 - accuracy: 0.9964
953/1688 [===============>..............] - ETA: 4s - loss: 0.0143 - accuracy: 0.9964
961/1688 [================>.............] - ETA: 4s - loss: 0.0143 - accuracy: 0.9964
969/1688 [================>.............] - ETA: 4s - loss: 0.0142 - accuracy: 0.9964
978/1688 [================>.............] - ETA: 4s - loss: 0.0142 - accuracy: 0.9964
986/1688 [================>.............] - ETA: 4s - loss: 0.0142 - accuracy: 0.9964
994/1688 [================>.............] - ETA: 4s - loss: 0.0142 - accuracy: 0.9964
1002/1688 [================>.............] - ETA: 4s - loss: 0.0143 - accuracy: 0.9963
1011/1688 [================>.............] - ETA: 4s - loss: 0.0142 - accuracy: 0.9964
1020/1688 [=================>............] - ETA: 4s - loss: 0.0142 - accuracy: 0.9963
1028/1688 [=================>............] - ETA: 4s - loss: 0.0142 - accuracy: 0.9963
1037/1688 [=================>............] - ETA: 4s - loss: 0.0144 - accuracy: 0.9963
1045/1688 [=================>............] - ETA: 4s - loss: 0.0143 - accuracy: 0.9963
1053/1688 [=================>............] - ETA: 4s - loss: 0.0144 - accuracy: 0.9962
1061/1688 [=================>............] - ETA: 3s - loss: 0.0145 - accuracy: 0.9962
1069/1688 [=================>............] - ETA: 3s - loss: 0.0145 - accuracy: 0.9962
1077/1688 [==================>...........] - ETA: 3s - loss: 0.0145 - accuracy: 0.9962
1085/1688 [==================>...........] - ETA: 3s - loss: 0.0145 - accuracy: 0.9962
1094/1688 [==================>...........] - ETA: 3s - loss: 0.0145 - accuracy: 0.9961
1103/1688 [==================>...........] - ETA: 3s - loss: 0.0146 - accuracy: 0.9961
1111/1688 [==================>...........] - ETA: 3s - loss: 0.0145 - accuracy: 0.9961
1119/1688 [==================>...........] - ETA: 3s - loss: 0.0145 - accuracy: 0.9961
1127/1688 [===================>..........] - ETA: 3s - loss: 0.0145 - accuracy: 0.9961
1136/1688 [===================>..........] - ETA: 3s - loss: 0.0146 - accuracy: 0.9961
1144/1688 [===================>..........] - ETA: 3s - loss: 0.0145 - accuracy: 0.9961
1152/1688 [===================>..........] - ETA: 3s - loss: 0.0146 - accuracy: 0.9961
1160/1688 [===================>..........] - ETA: 3s - loss: 0.0146 - accuracy: 0.9961
1169/1688 [===================>..........] - ETA: 3s - loss: 0.0146 - accuracy: 0.9960
1177/1688 [===================>..........] - ETA: 3s - loss: 0.0146 - accuracy: 0.9960
1185/1688 [====================>.........] - ETA: 3s - loss: 0.0146 - accuracy: 0.9960
1194/1688 [====================>.........] - ETA: 3s - loss: 0.0147 - accuracy: 0.9960
1203/1688 [====================>.........] - ETA: 3s - loss: 0.0146 - accuracy: 0.9961
1211/1688 [====================>.........] - ETA: 3s - loss: 0.0146 - accuracy: 0.9960
1219/1688 [====================>.........] - ETA: 2s - loss: 0.0147 - accuracy: 0.9959
1227/1688 [====================>.........] - ETA: 2s - loss: 0.0147 - accuracy: 0.9959
1235/1688 [====================>.........] - ETA: 2s - loss: 0.0147 - accuracy: 0.9959
1243/1688 [=====================>........] - ETA: 2s - loss: 0.0146 - accuracy: 0.9959
1252/1688 [=====================>........] - ETA: 2s - loss: 0.0146 - accuracy: 0.9960
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0146 - accuracy: 0.9960
1269/1688 [=====================>........] - ETA: 2s - loss: 0.0146 - accuracy: 0.9960
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0146 - accuracy: 0.9959
1286/1688 [=====================>........] - ETA: 2s - loss: 0.0146 - accuracy: 0.9959
1294/1688 [=====================>........] - ETA: 2s - loss: 0.0145 - accuracy: 0.9959
1303/1688 [======================>.......] - ETA: 2s - loss: 0.0146 - accuracy: 0.9959
1312/1688 [======================>.......] - ETA: 2s - loss: 0.0146 - accuracy: 0.9959
1321/1688 [======================>.......] - ETA: 2s - loss: 0.0145 - accuracy: 0.9959
1330/1688 [======================>.......] - ETA: 2s - loss: 0.0145 - accuracy: 0.9959
1338/1688 [======================>.......] - ETA: 2s - loss: 0.0145 - accuracy: 0.9959
1346/1688 [======================>.......] - ETA: 2s - loss: 0.0145 - accuracy: 0.9960
1355/1688 [=======================>......] - ETA: 2s - loss: 0.0145 - accuracy: 0.9959
1364/1688 [=======================>......] - ETA: 2s - loss: 0.0145 - accuracy: 0.9959
1372/1688 [=======================>......] - ETA: 2s - loss: 0.0145 - accuracy: 0.9959
1380/1688 [=======================>......] - ETA: 1s - loss: 0.0145 - accuracy: 0.9959
1388/1688 [=======================>......] - ETA: 1s - loss: 0.0145 - accuracy: 0.9959
1396/1688 [=======================>......] - ETA: 1s - loss: 0.0145 - accuracy: 0.9959
1404/1688 [=======================>......] - ETA: 1s - loss: 0.0145 - accuracy: 0.9959
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0145 - accuracy: 0.9959
1420/1688 [========================>.....] - ETA: 1s - loss: 0.0145 - accuracy: 0.9959
1428/1688 [========================>.....] - ETA: 1s - loss: 0.0145 - accuracy: 0.9959
1436/1688 [========================>.....] - ETA: 1s - loss: 0.0145 - accuracy: 0.9960
1444/1688 [========================>.....] - ETA: 1s - loss: 0.0144 - accuracy: 0.9960
1452/1688 [========================>.....] - ETA: 1s - loss: 0.0144 - accuracy: 0.9960
1460/1688 [========================>.....] - ETA: 1s - loss: 0.0143 - accuracy: 0.9960
1468/1688 [=========================>....] - ETA: 1s - loss: 0.0143 - accuracy: 0.9960
1476/1688 [=========================>....] - ETA: 1s - loss: 0.0143 - accuracy: 0.9960
1484/1688 [=========================>....] - ETA: 1s - loss: 0.0142 - accuracy: 0.9961
1492/1688 [=========================>....] - ETA: 1s - loss: 0.0142 - accuracy: 0.9961
1500/1688 [=========================>....] - ETA: 1s - loss: 0.0141 - accuracy: 0.9961
1508/1688 [=========================>....] - ETA: 1s - loss: 0.0142 - accuracy: 0.9961
1516/1688 [=========================>....] - ETA: 1s - loss: 0.0142 - accuracy: 0.9961
1524/1688 [==========================>...] - ETA: 1s - loss: 0.0141 - accuracy: 0.9961
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0141 - accuracy: 0.9961
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0141 - accuracy: 0.9961
1550/1688 [==========================>...] - ETA: 0s - loss: 0.0141 - accuracy: 0.9961
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0141 - accuracy: 0.9961
1566/1688 [==========================>...] - ETA: 0s - loss: 0.0140 - accuracy: 0.9961
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1625/1688 [===========================>..] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1633/1688 [============================>.] - ETA: 0s - loss: 0.0141 - accuracy: 0.9961
1641/1688 [============================>.] - ETA: 0s - loss: 0.0141 - accuracy: 0.9961
1649/1688 [============================>.] - ETA: 0s - loss: 0.0141 - accuracy: 0.9961
1657/1688 [============================>.] - ETA: 0s - loss: 0.0141 - accuracy: 0.9961
1665/1688 [============================>.] - ETA: 0s - loss: 0.0140 - accuracy: 0.9961
1673/1688 [============================>.] - ETA: 0s - loss: 0.0140 - accuracy: 0.9961
1681/1688 [============================>.] - ETA: 0s - loss: 0.0140 - accuracy: 0.9961
1688/1688 [==============================] - 12s 7ms/step - loss: 0.0140 - accuracy: 0.9961 - val_loss: 0.0486 - val_accuracy: 0.9872
Epoch 3/5
1/1688 [..............................] - ETA: 11s - loss: 0.0011 - accuracy: 1.0000
9/1688 [..............................] - ETA: 11s - loss: 0.0080 - accuracy: 0.9965
17/1688 [..............................] - ETA: 10s - loss: 0.0094 - accuracy: 0.9982
25/1688 [..............................] - ETA: 10s - loss: 0.0083 - accuracy: 0.9987
34/1688 [..............................] - ETA: 10s - loss: 0.0114 - accuracy: 0.9982
42/1688 [..............................] - ETA: 10s - loss: 0.0115 - accuracy: 0.9978
50/1688 [..............................] - ETA: 10s - loss: 0.0114 - accuracy: 0.9981
58/1688 [>.............................] - ETA: 10s - loss: 0.0114 - accuracy: 0.9978
66/1688 [>.............................] - ETA: 10s - loss: 0.0120 - accuracy: 0.9976
74/1688 [>.............................] - ETA: 10s - loss: 0.0128 - accuracy: 0.9975
83/1688 [>.............................] - ETA: 10s - loss: 0.0132 - accuracy: 0.9974
91/1688 [>.............................] - ETA: 10s - loss: 0.0126 - accuracy: 0.9976
99/1688 [>.............................] - ETA: 10s - loss: 0.0124 - accuracy: 0.9978
107/1688 [>.............................] - ETA: 10s - loss: 0.0119 - accuracy: 0.9980
115/1688 [=>............................] - ETA: 9s - loss: 0.0113 - accuracy: 0.9981
123/1688 [=>............................] - ETA: 9s - loss: 0.0115 - accuracy: 0.9980
131/1688 [=>............................] - ETA: 9s - loss: 0.0116 - accuracy: 0.9979
140/1688 [=>............................] - ETA: 9s - loss: 0.0116 - accuracy: 0.9980
148/1688 [=>............................] - ETA: 9s - loss: 0.0118 - accuracy: 0.9979
156/1688 [=>............................] - ETA: 9s - loss: 0.0118 - accuracy: 0.9980
164/1688 [=>............................] - ETA: 9s - loss: 0.0117 - accuracy: 0.9979
172/1688 [==>...........................] - ETA: 9s - loss: 0.0115 - accuracy: 0.9980
180/1688 [==>...........................] - ETA: 9s - loss: 0.0114 - accuracy: 0.9979
189/1688 [==>...........................] - ETA: 9s - loss: 0.0110 - accuracy: 0.9980
197/1688 [==>...........................] - ETA: 9s - loss: 0.0111 - accuracy: 0.9981
205/1688 [==>...........................] - ETA: 9s - loss: 0.0109 - accuracy: 0.9982
213/1688 [==>...........................] - ETA: 9s - loss: 0.0106 - accuracy: 0.9982
221/1688 [==>...........................] - ETA: 9s - loss: 0.0107 - accuracy: 0.9982
229/1688 [===>..........................] - ETA: 9s - loss: 0.0107 - accuracy: 0.9982
237/1688 [===>..........................] - ETA: 9s - loss: 0.0110 - accuracy: 0.9980
245/1688 [===>..........................] - ETA: 9s - loss: 0.0110 - accuracy: 0.9980
253/1688 [===>..........................] - ETA: 9s - loss: 0.0110 - accuracy: 0.9979
261/1688 [===>..........................] - ETA: 9s - loss: 0.0109 - accuracy: 0.9978
269/1688 [===>..........................] - ETA: 9s - loss: 0.0113 - accuracy: 0.9978
277/1688 [===>..........................] - ETA: 8s - loss: 0.0112 - accuracy: 0.9977
286/1688 [====>.........................] - ETA: 8s - loss: 0.0114 - accuracy: 0.9977
294/1688 [====>.........................] - ETA: 8s - loss: 0.0112 - accuracy: 0.9978
302/1688 [====>.........................] - ETA: 8s - loss: 0.0112 - accuracy: 0.9977
310/1688 [====>.........................] - ETA: 8s - loss: 0.0112 - accuracy: 0.9978
319/1688 [====>.........................] - ETA: 8s - loss: 0.0113 - accuracy: 0.9976
327/1688 [====>.........................] - ETA: 8s - loss: 0.0112 - accuracy: 0.9976
335/1688 [====>.........................] - ETA: 8s - loss: 0.0111 - accuracy: 0.9977
343/1688 [=====>........................] - ETA: 8s - loss: 0.0111 - accuracy: 0.9976
351/1688 [=====>........................] - ETA: 8s - loss: 0.0112 - accuracy: 0.9974
359/1688 [=====>........................] - ETA: 8s - loss: 0.0114 - accuracy: 0.9974
367/1688 [=====>........................] - ETA: 8s - loss: 0.0113 - accuracy: 0.9974
376/1688 [=====>........................] - ETA: 8s - loss: 0.0112 - accuracy: 0.9975
384/1688 [=====>........................] - ETA: 8s - loss: 0.0111 - accuracy: 0.9975
393/1688 [=====>........................] - ETA: 8s - loss: 0.0110 - accuracy: 0.9975
401/1688 [======>.......................] - ETA: 8s - loss: 0.0109 - accuracy: 0.9976
409/1688 [======>.......................] - ETA: 8s - loss: 0.0112 - accuracy: 0.9976
417/1688 [======>.......................] - ETA: 8s - loss: 0.0112 - accuracy: 0.9976
425/1688 [======>.......................] - ETA: 8s - loss: 0.0111 - accuracy: 0.9976
434/1688 [======>.......................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9977
443/1688 [======>.......................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9977
452/1688 [=======>......................] - ETA: 7s - loss: 0.0111 - accuracy: 0.9977
460/1688 [=======>......................] - ETA: 7s - loss: 0.0111 - accuracy: 0.9976
468/1688 [=======>......................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9976
476/1688 [=======>......................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9976
484/1688 [=======>......................] - ETA: 7s - loss: 0.0114 - accuracy: 0.9975
492/1688 [=======>......................] - ETA: 7s - loss: 0.0114 - accuracy: 0.9976
500/1688 [=======>......................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9976
508/1688 [========>.....................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9977
516/1688 [========>.....................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9976
524/1688 [========>.....................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9977
532/1688 [========>.....................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9976
541/1688 [========>.....................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9976
550/1688 [========>.....................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9977
558/1688 [========>.....................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9976
566/1688 [=========>....................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9977
574/1688 [=========>....................] - ETA: 7s - loss: 0.0111 - accuracy: 0.9977
582/1688 [=========>....................] - ETA: 7s - loss: 0.0111 - accuracy: 0.9977
590/1688 [=========>....................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9978
599/1688 [=========>....................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9978
607/1688 [=========>....................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9978
615/1688 [=========>....................] - ETA: 6s - loss: 0.0113 - accuracy: 0.9978
623/1688 [==========>...................] - ETA: 6s - loss: 0.0113 - accuracy: 0.9977
631/1688 [==========>...................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9978
639/1688 [==========>...................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9978
648/1688 [==========>...................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9977
657/1688 [==========>...................] - ETA: 6s - loss: 0.0111 - accuracy: 0.9977
665/1688 [==========>...................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9977
674/1688 [==========>...................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9977
682/1688 [===========>..................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9978
691/1688 [===========>..................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9977
699/1688 [===========>..................] - ETA: 6s - loss: 0.0109 - accuracy: 0.9978
707/1688 [===========>..................] - ETA: 6s - loss: 0.0109 - accuracy: 0.9978
715/1688 [===========>..................] - ETA: 6s - loss: 0.0109 - accuracy: 0.9978
724/1688 [===========>..................] - ETA: 6s - loss: 0.0108 - accuracy: 0.9978
732/1688 [============>.................] - ETA: 6s - loss: 0.0109 - accuracy: 0.9977
741/1688 [============>.................] - ETA: 6s - loss: 0.0108 - accuracy: 0.9977
749/1688 [============>.................] - ETA: 5s - loss: 0.0109 - accuracy: 0.9977
757/1688 [============>.................] - ETA: 5s - loss: 0.0109 - accuracy: 0.9977
765/1688 [============>.................] - ETA: 5s - loss: 0.0110 - accuracy: 0.9977
773/1688 [============>.................] - ETA: 5s - loss: 0.0110 - accuracy: 0.9977
781/1688 [============>.................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9976
790/1688 [=============>................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9975
799/1688 [=============>................] - ETA: 5s - loss: 0.0111 - accuracy: 0.9975
807/1688 [=============>................] - ETA: 5s - loss: 0.0113 - accuracy: 0.9975
815/1688 [=============>................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9975
823/1688 [=============>................] - ETA: 5s - loss: 0.0113 - accuracy: 0.9975
831/1688 [=============>................] - ETA: 5s - loss: 0.0114 - accuracy: 0.9974
839/1688 [=============>................] - ETA: 5s - loss: 0.0114 - accuracy: 0.9974
847/1688 [==============>...............] - ETA: 5s - loss: 0.0114 - accuracy: 0.9973
855/1688 [==============>...............] - ETA: 5s - loss: 0.0115 - accuracy: 0.9973
863/1688 [==============>...............] - ETA: 5s - loss: 0.0115 - accuracy: 0.9974
872/1688 [==============>...............] - ETA: 5s - loss: 0.0115 - accuracy: 0.9973
881/1688 [==============>...............] - ETA: 5s - loss: 0.0114 - accuracy: 0.9973
889/1688 [==============>...............] - ETA: 5s - loss: 0.0115 - accuracy: 0.9973
898/1688 [==============>...............] - ETA: 5s - loss: 0.0115 - accuracy: 0.9973
906/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9973
914/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9973
922/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9973
930/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9973
938/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9973
947/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9973
955/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9973
964/1688 [================>.............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9972
973/1688 [================>.............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9972
981/1688 [================>.............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9971
989/1688 [================>.............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9972
997/1688 [================>.............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9971
1006/1688 [================>.............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9971
1014/1688 [=================>............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9971
1022/1688 [=================>............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9971
1030/1688 [=================>............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9971
1038/1688 [=================>............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9971
1046/1688 [=================>............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9971
1054/1688 [=================>............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9971
1062/1688 [=================>............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9971
1071/1688 [==================>...........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9971
1079/1688 [==================>...........] - ETA: 3s - loss: 0.0115 - accuracy: 0.9971
1087/1688 [==================>...........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9971
1095/1688 [==================>...........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9971
1103/1688 [==================>...........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9971
1111/1688 [==================>...........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9971
1119/1688 [==================>...........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9971
1127/1688 [===================>..........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9971
1135/1688 [===================>..........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9972
1143/1688 [===================>..........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9972
1151/1688 [===================>..........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9971
1160/1688 [===================>..........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9971
1169/1688 [===================>..........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9971
1177/1688 [===================>..........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9972
1186/1688 [====================>.........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9972
1195/1688 [====================>.........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9971
1203/1688 [====================>.........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9971
1211/1688 [====================>.........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9972
1219/1688 [====================>.........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9972
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1236/1688 [====================>.........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9972
1244/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1252/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1260/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1268/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1276/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1284/1688 [=====================>........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1293/1688 [=====================>........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1301/1688 [======================>.......] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1309/1688 [======================>.......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1317/1688 [======================>.......] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1326/1688 [======================>.......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1334/1688 [======================>.......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1342/1688 [======================>.......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1350/1688 [======================>.......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1358/1688 [=======================>......] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1366/1688 [=======================>......] - ETA: 2s - loss: 0.0118 - accuracy: 0.9970
1374/1688 [=======================>......] - ETA: 2s - loss: 0.0118 - accuracy: 0.9970
1382/1688 [=======================>......] - ETA: 1s - loss: 0.0118 - accuracy: 0.9970
1390/1688 [=======================>......] - ETA: 1s - loss: 0.0118 - accuracy: 0.9970
1398/1688 [=======================>......] - ETA: 1s - loss: 0.0117 - accuracy: 0.9970
1406/1688 [=======================>......] - ETA: 1s - loss: 0.0117 - accuracy: 0.9970
1414/1688 [========================>.....] - ETA: 1s - loss: 0.0117 - accuracy: 0.9970
1422/1688 [========================>.....] - ETA: 1s - loss: 0.0117 - accuracy: 0.9970
1431/1688 [========================>.....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9970
1439/1688 [========================>.....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9969
1447/1688 [========================>.....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9969
1456/1688 [========================>.....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9970
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9969
1472/1688 [=========================>....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9970
1480/1688 [=========================>....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9969
1488/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9969
1496/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9969
1504/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9969
1512/1688 [=========================>....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9969
1520/1688 [==========================>...] - ETA: 1s - loss: 0.0121 - accuracy: 0.9969
1528/1688 [==========================>...] - ETA: 1s - loss: 0.0121 - accuracy: 0.9969
1537/1688 [==========================>...] - ETA: 0s - loss: 0.0121 - accuracy: 0.9969
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0121 - accuracy: 0.9968
1554/1688 [==========================>...] - ETA: 0s - loss: 0.0120 - accuracy: 0.9969
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0120 - accuracy: 0.9969
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0120 - accuracy: 0.9969
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9969
1588/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9969
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9968
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9969
1612/1688 [===========================>..] - ETA: 0s - loss: 0.0120 - accuracy: 0.9969
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9969
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0121 - accuracy: 0.9968
1637/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9968
1645/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9968
1654/1688 [============================>.] - ETA: 0s - loss: 0.0122 - accuracy: 0.9968
1662/1688 [============================>.] - ETA: 0s - loss: 0.0122 - accuracy: 0.9968
1670/1688 [============================>.] - ETA: 0s - loss: 0.0122 - accuracy: 0.9967
1678/1688 [============================>.] - ETA: 0s - loss: 0.0122 - accuracy: 0.9967
1686/1688 [============================>.] - ETA: 0s - loss: 0.0123 - accuracy: 0.9967
1688/1688 [==============================] - 12s 7ms/step - loss: 0.0123 - accuracy: 0.9967 - val_loss: 0.0481 - val_accuracy: 0.9883
Epoch 4/5
1/1688 [..............................] - ETA: 10s - loss: 0.0156 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0074 - accuracy: 1.0000
18/1688 [..............................] - ETA: 10s - loss: 0.0134 - accuracy: 0.9965
26/1688 [..............................] - ETA: 10s - loss: 0.0159 - accuracy: 0.9940
35/1688 [..............................] - ETA: 10s - loss: 0.0150 - accuracy: 0.9946
43/1688 [..............................] - ETA: 10s - loss: 0.0133 - accuracy: 0.9956
52/1688 [..............................] - ETA: 10s - loss: 0.0133 - accuracy: 0.9958
60/1688 [>.............................] - ETA: 10s - loss: 0.0129 - accuracy: 0.9958
68/1688 [>.............................] - ETA: 10s - loss: 0.0144 - accuracy: 0.9949
76/1688 [>.............................] - ETA: 10s - loss: 0.0140 - accuracy: 0.9955
84/1688 [>.............................] - ETA: 10s - loss: 0.0131 - accuracy: 0.9959
92/1688 [>.............................] - ETA: 10s - loss: 0.0128 - accuracy: 0.9959
100/1688 [>.............................] - ETA: 10s - loss: 0.0132 - accuracy: 0.9956
108/1688 [>.............................] - ETA: 10s - loss: 0.0128 - accuracy: 0.9959
116/1688 [=>............................] - ETA: 9s - loss: 0.0126 - accuracy: 0.9960
124/1688 [=>............................] - ETA: 9s - loss: 0.0120 - accuracy: 0.9962
132/1688 [=>............................] - ETA: 9s - loss: 0.0115 - accuracy: 0.9964
140/1688 [=>............................] - ETA: 9s - loss: 0.0111 - accuracy: 0.9967
149/1688 [=>............................] - ETA: 9s - loss: 0.0111 - accuracy: 0.9966
157/1688 [=>............................] - ETA: 9s - loss: 0.0111 - accuracy: 0.9966
165/1688 [=>............................] - ETA: 9s - loss: 0.0111 - accuracy: 0.9966
173/1688 [==>...........................] - ETA: 9s - loss: 0.0111 - accuracy: 0.9966
181/1688 [==>...........................] - ETA: 9s - loss: 0.0109 - accuracy: 0.9967
189/1688 [==>...........................] - ETA: 9s - loss: 0.0108 - accuracy: 0.9969
197/1688 [==>...........................] - ETA: 9s - loss: 0.0107 - accuracy: 0.9970
205/1688 [==>...........................] - ETA: 9s - loss: 0.0104 - accuracy: 0.9971
213/1688 [==>...........................] - ETA: 9s - loss: 0.0108 - accuracy: 0.9968
221/1688 [==>...........................] - ETA: 9s - loss: 0.0107 - accuracy: 0.9967
230/1688 [===>..........................] - ETA: 9s - loss: 0.0106 - accuracy: 0.9967
238/1688 [===>..........................] - ETA: 9s - loss: 0.0103 - accuracy: 0.9968
246/1688 [===>..........................] - ETA: 9s - loss: 0.0102 - accuracy: 0.9970
254/1688 [===>..........................] - ETA: 9s - loss: 0.0101 - accuracy: 0.9970
262/1688 [===>..........................] - ETA: 9s - loss: 0.0100 - accuracy: 0.9970
270/1688 [===>..........................] - ETA: 9s - loss: 0.0099 - accuracy: 0.9971
278/1688 [===>..........................] - ETA: 9s - loss: 0.0100 - accuracy: 0.9972
287/1688 [====>.........................] - ETA: 8s - loss: 0.0100 - accuracy: 0.9972
295/1688 [====>.........................] - ETA: 8s - loss: 0.0101 - accuracy: 0.9971
304/1688 [====>.........................] - ETA: 8s - loss: 0.0101 - accuracy: 0.9969
312/1688 [====>.........................] - ETA: 8s - loss: 0.0102 - accuracy: 0.9969
321/1688 [====>.........................] - ETA: 8s - loss: 0.0102 - accuracy: 0.9970
329/1688 [====>.........................] - ETA: 8s - loss: 0.0105 - accuracy: 0.9968
338/1688 [=====>........................] - ETA: 8s - loss: 0.0106 - accuracy: 0.9968
346/1688 [=====>........................] - ETA: 8s - loss: 0.0105 - accuracy: 0.9968
354/1688 [=====>........................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9969
362/1688 [=====>........................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9970
370/1688 [=====>........................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9970
378/1688 [=====>........................] - ETA: 8s - loss: 0.0105 - accuracy: 0.9969
386/1688 [=====>........................] - ETA: 8s - loss: 0.0105 - accuracy: 0.9969
394/1688 [======>.......................] - ETA: 8s - loss: 0.0106 - accuracy: 0.9968
403/1688 [======>.......................] - ETA: 8s - loss: 0.0108 - accuracy: 0.9968
411/1688 [======>.......................] - ETA: 8s - loss: 0.0110 - accuracy: 0.9966
420/1688 [======>.......................] - ETA: 8s - loss: 0.0111 - accuracy: 0.9966
428/1688 [======>.......................] - ETA: 8s - loss: 0.0114 - accuracy: 0.9965
436/1688 [======>.......................] - ETA: 7s - loss: 0.0114 - accuracy: 0.9965
445/1688 [======>.......................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9965
453/1688 [=======>......................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9966
461/1688 [=======>......................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9966
469/1688 [=======>......................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9966
477/1688 [=======>......................] - ETA: 7s - loss: 0.0114 - accuracy: 0.9966
485/1688 [=======>......................] - ETA: 7s - loss: 0.0114 - accuracy: 0.9966
493/1688 [=======>......................] - ETA: 7s - loss: 0.0114 - accuracy: 0.9966
501/1688 [=======>......................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9966
510/1688 [========>.....................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9966
518/1688 [========>.....................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9966
526/1688 [========>.....................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9966
534/1688 [========>.....................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9967
542/1688 [========>.....................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9967
550/1688 [========>.....................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9967
559/1688 [========>.....................] - ETA: 7s - loss: 0.0111 - accuracy: 0.9968
567/1688 [=========>....................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9968
575/1688 [=========>....................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9968
584/1688 [=========>....................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9968
592/1688 [=========>....................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9968
600/1688 [=========>....................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9968
609/1688 [=========>....................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9968
618/1688 [=========>....................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9968
626/1688 [==========>...................] - ETA: 6s - loss: 0.0111 - accuracy: 0.9968
635/1688 [==========>...................] - ETA: 6s - loss: 0.0111 - accuracy: 0.9968
643/1688 [==========>...................] - ETA: 6s - loss: 0.0111 - accuracy: 0.9968
651/1688 [==========>...................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9968
659/1688 [==========>...................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9969
667/1688 [==========>...................] - ETA: 6s - loss: 0.0111 - accuracy: 0.9969
675/1688 [==========>...................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9968
684/1688 [===========>..................] - ETA: 6s - loss: 0.0111 - accuracy: 0.9968
692/1688 [===========>..................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9969
701/1688 [===========>..................] - ETA: 6s - loss: 0.0111 - accuracy: 0.9969
710/1688 [===========>..................] - ETA: 6s - loss: 0.0111 - accuracy: 0.9968
719/1688 [===========>..................] - ETA: 6s - loss: 0.0110 - accuracy: 0.9969
727/1688 [===========>..................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9969
735/1688 [============>.................] - ETA: 6s - loss: 0.0112 - accuracy: 0.9969
743/1688 [============>.................] - ETA: 6s - loss: 0.0113 - accuracy: 0.9968
751/1688 [============>.................] - ETA: 5s - loss: 0.0113 - accuracy: 0.9969
759/1688 [============>.................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9969
767/1688 [============>.................] - ETA: 5s - loss: 0.0113 - accuracy: 0.9969
775/1688 [============>.................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9969
783/1688 [============>.................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9969
792/1688 [=============>................] - ETA: 5s - loss: 0.0114 - accuracy: 0.9968
800/1688 [=============>................] - ETA: 5s - loss: 0.0114 - accuracy: 0.9968
809/1688 [=============>................] - ETA: 5s - loss: 0.0114 - accuracy: 0.9968
818/1688 [=============>................] - ETA: 5s - loss: 0.0113 - accuracy: 0.9969
826/1688 [=============>................] - ETA: 5s - loss: 0.0114 - accuracy: 0.9968
835/1688 [=============>................] - ETA: 5s - loss: 0.0114 - accuracy: 0.9969
843/1688 [=============>................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9968
851/1688 [==============>...............] - ETA: 5s - loss: 0.0114 - accuracy: 0.9968
860/1688 [==============>...............] - ETA: 5s - loss: 0.0114 - accuracy: 0.9968
868/1688 [==============>...............] - ETA: 5s - loss: 0.0114 - accuracy: 0.9968
876/1688 [==============>...............] - ETA: 5s - loss: 0.0115 - accuracy: 0.9967
884/1688 [==============>...............] - ETA: 5s - loss: 0.0115 - accuracy: 0.9967
893/1688 [==============>...............] - ETA: 5s - loss: 0.0115 - accuracy: 0.9967
901/1688 [===============>..............] - ETA: 5s - loss: 0.0115 - accuracy: 0.9967
909/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9967
917/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9968
925/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9968
933/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9968
941/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9968
950/1688 [===============>..............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9967
958/1688 [================>.............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9968
966/1688 [================>.............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9968
974/1688 [================>.............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9968
982/1688 [================>.............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9968
990/1688 [================>.............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9968
998/1688 [================>.............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9968
1007/1688 [================>.............] - ETA: 4s - loss: 0.0114 - accuracy: 0.9968
1016/1688 [=================>............] - ETA: 4s - loss: 0.0115 - accuracy: 0.9967
1024/1688 [=================>............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9967
1032/1688 [=================>............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9967
1040/1688 [=================>............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9966
1048/1688 [=================>............] - ETA: 4s - loss: 0.0116 - accuracy: 0.9967
1056/1688 [=================>............] - ETA: 4s - loss: 0.0117 - accuracy: 0.9967
1065/1688 [=================>............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9967
1073/1688 [==================>...........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9967
1081/1688 [==================>...........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9966
1089/1688 [==================>...........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9966
1097/1688 [==================>...........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9966
1105/1688 [==================>...........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9966
1113/1688 [==================>...........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9966
1121/1688 [==================>...........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9966
1129/1688 [===================>..........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9967
1137/1688 [===================>..........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9966
1145/1688 [===================>..........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9967
1153/1688 [===================>..........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9967
1162/1688 [===================>..........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9967
1170/1688 [===================>..........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9967
1178/1688 [===================>..........] - ETA: 3s - loss: 0.0115 - accuracy: 0.9967
1186/1688 [====================>.........] - ETA: 3s - loss: 0.0115 - accuracy: 0.9967
1194/1688 [====================>.........] - ETA: 3s - loss: 0.0115 - accuracy: 0.9967
1203/1688 [====================>.........] - ETA: 3s - loss: 0.0115 - accuracy: 0.9968
1212/1688 [====================>.........] - ETA: 3s - loss: 0.0115 - accuracy: 0.9968
1220/1688 [====================>.........] - ETA: 2s - loss: 0.0115 - accuracy: 0.9968
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0114 - accuracy: 0.9968
1236/1688 [====================>.........] - ETA: 2s - loss: 0.0115 - accuracy: 0.9968
1245/1688 [=====================>........] - ETA: 2s - loss: 0.0115 - accuracy: 0.9967
1253/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9967
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0115 - accuracy: 0.9967
1269/1688 [=====================>........] - ETA: 2s - loss: 0.0115 - accuracy: 0.9967
1277/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9967
1286/1688 [=====================>........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9967
1295/1688 [======================>.......] - ETA: 2s - loss: 0.0117 - accuracy: 0.9967
1303/1688 [======================>.......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9967
1311/1688 [======================>.......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9968
1319/1688 [======================>.......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9968
1328/1688 [======================>.......] - ETA: 2s - loss: 0.0117 - accuracy: 0.9967
1336/1688 [======================>.......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9967
1344/1688 [======================>.......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9967
1352/1688 [=======================>......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9967
1361/1688 [=======================>......] - ETA: 2s - loss: 0.0117 - accuracy: 0.9967
1370/1688 [=======================>......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9967
1379/1688 [=======================>......] - ETA: 1s - loss: 0.0116 - accuracy: 0.9967
1387/1688 [=======================>......] - ETA: 1s - loss: 0.0116 - accuracy: 0.9967
1395/1688 [=======================>......] - ETA: 1s - loss: 0.0117 - accuracy: 0.9967
1403/1688 [=======================>......] - ETA: 1s - loss: 0.0117 - accuracy: 0.9967
1411/1688 [========================>.....] - ETA: 1s - loss: 0.0117 - accuracy: 0.9967
1419/1688 [========================>.....] - ETA: 1s - loss: 0.0117 - accuracy: 0.9967
1427/1688 [========================>.....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9968
1435/1688 [========================>.....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9968
1444/1688 [========================>.....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9968
1452/1688 [========================>.....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9968
1461/1688 [========================>.....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9968
1469/1688 [=========================>....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9968
1477/1688 [=========================>....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9968
1485/1688 [=========================>....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9968
1493/1688 [=========================>....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9968
1502/1688 [=========================>....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9968
1510/1688 [=========================>....] - ETA: 1s - loss: 0.0115 - accuracy: 0.9968
1518/1688 [=========================>....] - ETA: 1s - loss: 0.0115 - accuracy: 0.9968
1526/1688 [==========================>...] - ETA: 1s - loss: 0.0115 - accuracy: 0.9968
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0115 - accuracy: 0.9968
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0115 - accuracy: 0.9968
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0115 - accuracy: 0.9968
1560/1688 [==========================>...] - ETA: 0s - loss: 0.0115 - accuracy: 0.9968
1568/1688 [==========================>...] - ETA: 0s - loss: 0.0114 - accuracy: 0.9968
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0114 - accuracy: 0.9968
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0115 - accuracy: 0.9968
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0116 - accuracy: 0.9968
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0116 - accuracy: 0.9968
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0116 - accuracy: 0.9968
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0115 - accuracy: 0.9968
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0116 - accuracy: 0.9968
1634/1688 [============================>.] - ETA: 0s - loss: 0.0116 - accuracy: 0.9968
1643/1688 [============================>.] - ETA: 0s - loss: 0.0116 - accuracy: 0.9968
1651/1688 [============================>.] - ETA: 0s - loss: 0.0116 - accuracy: 0.9968
1660/1688 [============================>.] - ETA: 0s - loss: 0.0116 - accuracy: 0.9968
1668/1688 [============================>.] - ETA: 0s - loss: 0.0116 - accuracy: 0.9968
1676/1688 [============================>.] - ETA: 0s - loss: 0.0115 - accuracy: 0.9968
1684/1688 [============================>.] - ETA: 0s - loss: 0.0115 - accuracy: 0.9968
1688/1688 [==============================] - 12s 7ms/step - loss: 0.0115 - accuracy: 0.9969 - val_loss: 0.0505 - val_accuracy: 0.9875
Epoch 5/5
1/1688 [..............................] - ETA: 11s - loss: 9.5942e-04 - accuracy: 1.0000
10/1688 [..............................] - ETA: 10s - loss: 0.0168 - accuracy: 0.9906
18/1688 [..............................] - ETA: 10s - loss: 0.0122 - accuracy: 0.9948
26/1688 [..............................] - ETA: 10s - loss: 0.0105 - accuracy: 0.9964
34/1688 [..............................] - ETA: 10s - loss: 0.0095 - accuracy: 0.9972
42/1688 [..............................] - ETA: 10s - loss: 0.0098 - accuracy: 0.9963
50/1688 [..............................] - ETA: 10s - loss: 0.0101 - accuracy: 0.9962
59/1688 [>.............................] - ETA: 10s - loss: 0.0093 - accuracy: 0.9968
67/1688 [>.............................] - ETA: 10s - loss: 0.0091 - accuracy: 0.9972
75/1688 [>.............................] - ETA: 10s - loss: 0.0094 - accuracy: 0.9971
83/1688 [>.............................] - ETA: 10s - loss: 0.0098 - accuracy: 0.9970
91/1688 [>.............................] - ETA: 10s - loss: 0.0120 - accuracy: 0.9959
99/1688 [>.............................] - ETA: 10s - loss: 0.0114 - accuracy: 0.9962
107/1688 [>.............................] - ETA: 10s - loss: 0.0106 - accuracy: 0.9965
115/1688 [=>............................] - ETA: 10s - loss: 0.0106 - accuracy: 0.9967
123/1688 [=>............................] - ETA: 10s - loss: 0.0104 - accuracy: 0.9970
132/1688 [=>............................] - ETA: 9s - loss: 0.0102 - accuracy: 0.9969
140/1688 [=>............................] - ETA: 9s - loss: 0.0099 - accuracy: 0.9971
148/1688 [=>............................] - ETA: 9s - loss: 0.0097 - accuracy: 0.9973
156/1688 [=>............................] - ETA: 9s - loss: 0.0098 - accuracy: 0.9972
164/1688 [=>............................] - ETA: 9s - loss: 0.0098 - accuracy: 0.9973
172/1688 [==>...........................] - ETA: 9s - loss: 0.0098 - accuracy: 0.9973
180/1688 [==>...........................] - ETA: 9s - loss: 0.0107 - accuracy: 0.9969
189/1688 [==>...........................] - ETA: 9s - loss: 0.0110 - accuracy: 0.9967
198/1688 [==>...........................] - ETA: 9s - loss: 0.0108 - accuracy: 0.9968
206/1688 [==>...........................] - ETA: 9s - loss: 0.0107 - accuracy: 0.9970
214/1688 [==>...........................] - ETA: 9s - loss: 0.0107 - accuracy: 0.9969
222/1688 [==>...........................] - ETA: 9s - loss: 0.0106 - accuracy: 0.9970
230/1688 [===>..........................] - ETA: 9s - loss: 0.0106 - accuracy: 0.9969
238/1688 [===>..........................] - ETA: 9s - loss: 0.0105 - accuracy: 0.9970
247/1688 [===>..........................] - ETA: 9s - loss: 0.0108 - accuracy: 0.9968
255/1688 [===>..........................] - ETA: 9s - loss: 0.0106 - accuracy: 0.9969
263/1688 [===>..........................] - ETA: 9s - loss: 0.0105 - accuracy: 0.9970
271/1688 [===>..........................] - ETA: 9s - loss: 0.0106 - accuracy: 0.9969
280/1688 [===>..........................] - ETA: 9s - loss: 0.0105 - accuracy: 0.9969
288/1688 [====>.........................] - ETA: 8s - loss: 0.0104 - accuracy: 0.9969
297/1688 [====>.........................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9968
306/1688 [====>.........................] - ETA: 8s - loss: 0.0105 - accuracy: 0.9968
314/1688 [====>.........................] - ETA: 8s - loss: 0.0105 - accuracy: 0.9967
322/1688 [====>.........................] - ETA: 8s - loss: 0.0104 - accuracy: 0.9968
330/1688 [====>.........................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9969
338/1688 [=====>........................] - ETA: 8s - loss: 0.0102 - accuracy: 0.9969
346/1688 [=====>........................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9969
354/1688 [=====>........................] - ETA: 8s - loss: 0.0102 - accuracy: 0.9970
362/1688 [=====>........................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9970
370/1688 [=====>........................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9970
378/1688 [=====>........................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9970
386/1688 [=====>........................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9971
395/1688 [======>.......................] - ETA: 8s - loss: 0.0102 - accuracy: 0.9972
403/1688 [======>.......................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9972
411/1688 [======>.......................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9973
419/1688 [======>.......................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9973
427/1688 [======>.......................] - ETA: 8s - loss: 0.0103 - accuracy: 0.9972
436/1688 [======>.......................] - ETA: 7s - loss: 0.0102 - accuracy: 0.9973
444/1688 [======>.......................] - ETA: 7s - loss: 0.0102 - accuracy: 0.9973
452/1688 [=======>......................] - ETA: 7s - loss: 0.0101 - accuracy: 0.9973
460/1688 [=======>......................] - ETA: 7s - loss: 0.0103 - accuracy: 0.9972
469/1688 [=======>......................] - ETA: 7s - loss: 0.0103 - accuracy: 0.9973
478/1688 [=======>......................] - ETA: 7s - loss: 0.0102 - accuracy: 0.9973
487/1688 [=======>......................] - ETA: 7s - loss: 0.0105 - accuracy: 0.9970
495/1688 [=======>......................] - ETA: 7s - loss: 0.0104 - accuracy: 0.9971
503/1688 [=======>......................] - ETA: 7s - loss: 0.0104 - accuracy: 0.9971
511/1688 [========>.....................] - ETA: 7s - loss: 0.0105 - accuracy: 0.9970
519/1688 [========>.....................] - ETA: 7s - loss: 0.0106 - accuracy: 0.9970
528/1688 [========>.....................] - ETA: 7s - loss: 0.0108 - accuracy: 0.9970
536/1688 [========>.....................] - ETA: 7s - loss: 0.0106 - accuracy: 0.9970
544/1688 [========>.....................] - ETA: 7s - loss: 0.0106 - accuracy: 0.9971
552/1688 [========>.....................] - ETA: 7s - loss: 0.0106 - accuracy: 0.9971
560/1688 [========>.....................] - ETA: 7s - loss: 0.0107 - accuracy: 0.9970
568/1688 [=========>....................] - ETA: 7s - loss: 0.0107 - accuracy: 0.9971
576/1688 [=========>....................] - ETA: 7s - loss: 0.0107 - accuracy: 0.9971
584/1688 [=========>....................] - ETA: 7s - loss: 0.0108 - accuracy: 0.9971
592/1688 [=========>....................] - ETA: 7s - loss: 0.0107 - accuracy: 0.9971
600/1688 [=========>....................] - ETA: 6s - loss: 0.0107 - accuracy: 0.9971
609/1688 [=========>....................] - ETA: 6s - loss: 0.0106 - accuracy: 0.9971
617/1688 [=========>....................] - ETA: 6s - loss: 0.0105 - accuracy: 0.9972
625/1688 [==========>...................] - ETA: 6s - loss: 0.0106 - accuracy: 0.9972
634/1688 [==========>...................] - ETA: 6s - loss: 0.0105 - accuracy: 0.9972
642/1688 [==========>...................] - ETA: 6s - loss: 0.0105 - accuracy: 0.9973
650/1688 [==========>...................] - ETA: 6s - loss: 0.0105 - accuracy: 0.9973
658/1688 [==========>...................] - ETA: 6s - loss: 0.0106 - accuracy: 0.9972
666/1688 [==========>...................] - ETA: 6s - loss: 0.0106 - accuracy: 0.9972
674/1688 [==========>...................] - ETA: 6s - loss: 0.0105 - accuracy: 0.9972
683/1688 [===========>..................] - ETA: 6s - loss: 0.0105 - accuracy: 0.9973
692/1688 [===========>..................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9973
700/1688 [===========>..................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9973
709/1688 [===========>..................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9973
717/1688 [===========>..................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9973
725/1688 [===========>..................] - ETA: 6s - loss: 0.0104 - accuracy: 0.9973
734/1688 [============>.................] - ETA: 6s - loss: 0.0103 - accuracy: 0.9974
742/1688 [============>.................] - ETA: 6s - loss: 0.0103 - accuracy: 0.9973
751/1688 [============>.................] - ETA: 5s - loss: 0.0104 - accuracy: 0.9973
759/1688 [============>.................] - ETA: 5s - loss: 0.0104 - accuracy: 0.9974
767/1688 [============>.................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9974
776/1688 [============>.................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9974
785/1688 [============>.................] - ETA: 5s - loss: 0.0103 - accuracy: 0.9975
794/1688 [=============>................] - ETA: 5s - loss: 0.0102 - accuracy: 0.9975
803/1688 [=============>................] - ETA: 5s - loss: 0.0106 - accuracy: 0.9974
811/1688 [=============>................] - ETA: 5s - loss: 0.0106 - accuracy: 0.9974
819/1688 [=============>................] - ETA: 5s - loss: 0.0106 - accuracy: 0.9974
827/1688 [=============>................] - ETA: 5s - loss: 0.0106 - accuracy: 0.9974
835/1688 [=============>................] - ETA: 5s - loss: 0.0106 - accuracy: 0.9974
844/1688 [==============>...............] - ETA: 5s - loss: 0.0106 - accuracy: 0.9974
852/1688 [==============>...............] - ETA: 5s - loss: 0.0106 - accuracy: 0.9974
860/1688 [==============>...............] - ETA: 5s - loss: 0.0106 - accuracy: 0.9974
868/1688 [==============>...............] - ETA: 5s - loss: 0.0108 - accuracy: 0.9973
877/1688 [==============>...............] - ETA: 5s - loss: 0.0107 - accuracy: 0.9974
885/1688 [==============>...............] - ETA: 5s - loss: 0.0108 - accuracy: 0.9973
894/1688 [==============>...............] - ETA: 5s - loss: 0.0108 - accuracy: 0.9973
902/1688 [===============>..............] - ETA: 5s - loss: 0.0108 - accuracy: 0.9973
910/1688 [===============>..............] - ETA: 4s - loss: 0.0107 - accuracy: 0.9973
918/1688 [===============>..............] - ETA: 4s - loss: 0.0107 - accuracy: 0.9973
926/1688 [===============>..............] - ETA: 4s - loss: 0.0107 - accuracy: 0.9973
934/1688 [===============>..............] - ETA: 4s - loss: 0.0106 - accuracy: 0.9974
943/1688 [===============>..............] - ETA: 4s - loss: 0.0106 - accuracy: 0.9973
952/1688 [===============>..............] - ETA: 4s - loss: 0.0105 - accuracy: 0.9974
961/1688 [================>.............] - ETA: 4s - loss: 0.0106 - accuracy: 0.9974
969/1688 [================>.............] - ETA: 4s - loss: 0.0106 - accuracy: 0.9974
977/1688 [================>.............] - ETA: 4s - loss: 0.0105 - accuracy: 0.9974
985/1688 [================>.............] - ETA: 4s - loss: 0.0105 - accuracy: 0.9974
994/1688 [================>.............] - ETA: 4s - loss: 0.0105 - accuracy: 0.9974
1002/1688 [================>.............] - ETA: 4s - loss: 0.0104 - accuracy: 0.9974
1010/1688 [================>.............] - ETA: 4s - loss: 0.0104 - accuracy: 0.9974
1018/1688 [=================>............] - ETA: 4s - loss: 0.0106 - accuracy: 0.9974
1026/1688 [=================>............] - ETA: 4s - loss: 0.0106 - accuracy: 0.9974
1034/1688 [=================>............] - ETA: 4s - loss: 0.0107 - accuracy: 0.9973
1042/1688 [=================>............] - ETA: 4s - loss: 0.0107 - accuracy: 0.9973
1050/1688 [=================>............] - ETA: 4s - loss: 0.0106 - accuracy: 0.9973
1058/1688 [=================>............] - ETA: 4s - loss: 0.0105 - accuracy: 0.9973
1066/1688 [=================>............] - ETA: 3s - loss: 0.0106 - accuracy: 0.9973
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0107 - accuracy: 0.9973
1083/1688 [==================>...........] - ETA: 3s - loss: 0.0107 - accuracy: 0.9973
1091/1688 [==================>...........] - ETA: 3s - loss: 0.0106 - accuracy: 0.9973
1099/1688 [==================>...........] - ETA: 3s - loss: 0.0106 - accuracy: 0.9973
1108/1688 [==================>...........] - ETA: 3s - loss: 0.0107 - accuracy: 0.9973
1116/1688 [==================>...........] - ETA: 3s - loss: 0.0107 - accuracy: 0.9973
1125/1688 [==================>...........] - ETA: 3s - loss: 0.0107 - accuracy: 0.9973
1133/1688 [===================>..........] - ETA: 3s - loss: 0.0106 - accuracy: 0.9973
1141/1688 [===================>..........] - ETA: 3s - loss: 0.0107 - accuracy: 0.9973
1149/1688 [===================>..........] - ETA: 3s - loss: 0.0107 - accuracy: 0.9973
1157/1688 [===================>..........] - ETA: 3s - loss: 0.0108 - accuracy: 0.9972
1165/1688 [===================>..........] - ETA: 3s - loss: 0.0108 - accuracy: 0.9972
1173/1688 [===================>..........] - ETA: 3s - loss: 0.0108 - accuracy: 0.9972
1181/1688 [===================>..........] - ETA: 3s - loss: 0.0107 - accuracy: 0.9972
1189/1688 [====================>.........] - ETA: 3s - loss: 0.0107 - accuracy: 0.9972
1197/1688 [====================>.........] - ETA: 3s - loss: 0.0107 - accuracy: 0.9972
1205/1688 [====================>.........] - ETA: 3s - loss: 0.0107 - accuracy: 0.9973
1213/1688 [====================>.........] - ETA: 3s - loss: 0.0108 - accuracy: 0.9972
1222/1688 [====================>.........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1238/1688 [=====================>........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9971
1246/1688 [=====================>........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9971
1254/1688 [=====================>........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1262/1688 [=====================>........] - ETA: 2s - loss: 0.0110 - accuracy: 0.9972
1271/1688 [=====================>........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1279/1688 [=====================>........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1287/1688 [=====================>........] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1296/1688 [======================>.......] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1304/1688 [======================>.......] - ETA: 2s - loss: 0.0109 - accuracy: 0.9972
1313/1688 [======================>.......] - ETA: 2s - loss: 0.0108 - accuracy: 0.9972
1321/1688 [======================>.......] - ETA: 2s - loss: 0.0108 - accuracy: 0.9972
1329/1688 [======================>.......] - ETA: 2s - loss: 0.0108 - accuracy: 0.9972
1337/1688 [======================>.......] - ETA: 2s - loss: 0.0108 - accuracy: 0.9972
1345/1688 [======================>.......] - ETA: 2s - loss: 0.0108 - accuracy: 0.9972
1353/1688 [=======================>......] - ETA: 2s - loss: 0.0107 - accuracy: 0.9972
1361/1688 [=======================>......] - ETA: 2s - loss: 0.0107 - accuracy: 0.9972
1369/1688 [=======================>......] - ETA: 2s - loss: 0.0107 - accuracy: 0.9972
1378/1688 [=======================>......] - ETA: 1s - loss: 0.0107 - accuracy: 0.9972
1386/1688 [=======================>......] - ETA: 1s - loss: 0.0107 - accuracy: 0.9972
1395/1688 [=======================>......] - ETA: 1s - loss: 0.0108 - accuracy: 0.9972
1403/1688 [=======================>......] - ETA: 1s - loss: 0.0108 - accuracy: 0.9972
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9972
1420/1688 [========================>.....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9972
1428/1688 [========================>.....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9972
1436/1688 [========================>.....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9972
1445/1688 [========================>.....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9972
1453/1688 [========================>.....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9972
1461/1688 [========================>.....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9972
1470/1688 [=========================>....] - ETA: 1s - loss: 0.0107 - accuracy: 0.9972
1478/1688 [=========================>....] - ETA: 1s - loss: 0.0107 - accuracy: 0.9972
1486/1688 [=========================>....] - ETA: 1s - loss: 0.0107 - accuracy: 0.9972
1494/1688 [=========================>....] - ETA: 1s - loss: 0.0107 - accuracy: 0.9972
1502/1688 [=========================>....] - ETA: 1s - loss: 0.0107 - accuracy: 0.9972
1511/1688 [=========================>....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9972
1519/1688 [=========================>....] - ETA: 1s - loss: 0.0107 - accuracy: 0.9972
1527/1688 [==========================>...] - ETA: 1s - loss: 0.0108 - accuracy: 0.9972
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0108 - accuracy: 0.9971
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0108 - accuracy: 0.9971
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0107 - accuracy: 0.9971
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1616/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1632/1688 [============================>.] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1640/1688 [============================>.] - ETA: 0s - loss: 0.0106 - accuracy: 0.9972
1648/1688 [============================>.] - ETA: 0s - loss: 0.0106 - accuracy: 0.9972
1656/1688 [============================>.] - ETA: 0s - loss: 0.0106 - accuracy: 0.9972
1664/1688 [============================>.] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1673/1688 [============================>.] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1681/1688 [============================>.] - ETA: 0s - loss: 0.0107 - accuracy: 0.9972
1688/1688 [==============================] - 12s 7ms/step - loss: 0.0107 - accuracy: 0.9972 - val_loss: 0.0513 - val_accuracy: 0.9870
<keras.src.callbacks.History object at 0x71471fae4e90>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine tuning:', score)
Test accuracy after fine tuning: 0.9847999811172485
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test.astype(np.int32))
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.986299991607666
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[-10.563311 -6.432473 -5.1664925 -2.7509713 -9.338767 -4.8781767
-22.582956 6.6830773 -5.4157724 -1.2905043]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 24.960 seconds)