Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a TF-Keras floating-point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original TF-Keras floating-point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf_keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import tensorflow as tf
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from tf_keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, height, width, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
49152/11490434 [..............................] - ETA: 27s
81920/11490434 [..............................] - ETA: 28s
147456/11490434 [..............................] - ETA: 21s
212992/11490434 [..............................] - ETA: 17s
319488/11490434 [..............................] - ETA: 13s
450560/11490434 [>.............................] - ETA: 10s
688128/11490434 [>.............................] - ETA: 7s
974848/11490434 [=>............................] - ETA: 5s
1556480/11490434 [===>..........................] - ETA: 3s
2375680/11490434 [=====>........................] - ETA: 2s
2891776/11490434 [======>.......................] - ETA: 2s
3383296/11490434 [=======>......................] - ETA: 1s
4120576/11490434 [=========>....................] - ETA: 1s
4767744/11490434 [===========>..................] - ETA: 1s
5816320/11490434 [==============>...............] - ETA: 0s
6406144/11490434 [===============>..............] - ETA: 0s
7127040/11490434 [=================>............] - ETA: 0s
7716864/11490434 [===================>..........] - ETA: 0s
8306688/11490434 [====================>.........] - ETA: 0s
9043968/11490434 [======================>.......] - ETA: 0s
9633792/11490434 [========================>.....] - ETA: 0s
10256384/11490434 [=========================>....] - ETA: 0s
11108352/11490434 [============================>.] - ETA: 0s
11490434/11490434 [==============================] - 1s 0us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard TF-Keras CNN model.
import tf_keras as keras
model_keras = keras.models.Sequential([
keras.layers.Input(shape=(28, 28, 1), name="input", dtype=tf.uint8),
keras.layers.Rescaling(1. / 255),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from tf_keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 1:15:40 - loss: 2.7102 - accuracy: 0.0938
23/1688 [..............................] - ETA: 3s - loss: 1.3850 - accuracy: 0.5625
45/1688 [..............................] - ETA: 3s - loss: 0.9938 - accuracy: 0.6924
67/1688 [>.............................] - ETA: 3s - loss: 0.8075 - accuracy: 0.7509
89/1688 [>.............................] - ETA: 3s - loss: 0.7063 - accuracy: 0.7848
112/1688 [>.............................] - ETA: 3s - loss: 0.6309 - accuracy: 0.8080
135/1688 [=>............................] - ETA: 3s - loss: 0.5829 - accuracy: 0.8248
157/1688 [=>............................] - ETA: 3s - loss: 0.5411 - accuracy: 0.8366
179/1688 [==>...........................] - ETA: 3s - loss: 0.5156 - accuracy: 0.8439
202/1688 [==>...........................] - ETA: 3s - loss: 0.4911 - accuracy: 0.8519
224/1688 [==>...........................] - ETA: 3s - loss: 0.4665 - accuracy: 0.8598
244/1688 [===>..........................] - ETA: 3s - loss: 0.4481 - accuracy: 0.8660
266/1688 [===>..........................] - ETA: 3s - loss: 0.4269 - accuracy: 0.8722
288/1688 [====>.........................] - ETA: 3s - loss: 0.4110 - accuracy: 0.8773
310/1688 [====>.........................] - ETA: 3s - loss: 0.3970 - accuracy: 0.8817
332/1688 [====>.........................] - ETA: 3s - loss: 0.3815 - accuracy: 0.8861
354/1688 [=====>........................] - ETA: 3s - loss: 0.3721 - accuracy: 0.8892
377/1688 [=====>........................] - ETA: 3s - loss: 0.3582 - accuracy: 0.8937
400/1688 [======>.......................] - ETA: 2s - loss: 0.3494 - accuracy: 0.8968
423/1688 [======>.......................] - ETA: 2s - loss: 0.3379 - accuracy: 0.9006
444/1688 [======>.......................] - ETA: 2s - loss: 0.3291 - accuracy: 0.9032
466/1688 [=======>......................] - ETA: 2s - loss: 0.3204 - accuracy: 0.9058
489/1688 [=======>......................] - ETA: 2s - loss: 0.3103 - accuracy: 0.9087
512/1688 [========>.....................] - ETA: 2s - loss: 0.3040 - accuracy: 0.9106
535/1688 [========>.....................] - ETA: 2s - loss: 0.2968 - accuracy: 0.9126
558/1688 [========>.....................] - ETA: 2s - loss: 0.2909 - accuracy: 0.9144
580/1688 [=========>....................] - ETA: 2s - loss: 0.2852 - accuracy: 0.9159
602/1688 [=========>....................] - ETA: 2s - loss: 0.2812 - accuracy: 0.9172
625/1688 [==========>...................] - ETA: 2s - loss: 0.2758 - accuracy: 0.9186
647/1688 [==========>...................] - ETA: 2s - loss: 0.2710 - accuracy: 0.9200
669/1688 [==========>...................] - ETA: 2s - loss: 0.2669 - accuracy: 0.9212
692/1688 [===========>..................] - ETA: 2s - loss: 0.2621 - accuracy: 0.9227
714/1688 [===========>..................] - ETA: 2s - loss: 0.2584 - accuracy: 0.9238
735/1688 [============>.................] - ETA: 2s - loss: 0.2557 - accuracy: 0.9245
758/1688 [============>.................] - ETA: 2s - loss: 0.2520 - accuracy: 0.9254
781/1688 [============>.................] - ETA: 2s - loss: 0.2482 - accuracy: 0.9267
804/1688 [=============>................] - ETA: 2s - loss: 0.2433 - accuracy: 0.9279
827/1688 [=============>................] - ETA: 1s - loss: 0.2387 - accuracy: 0.9290
850/1688 [==============>...............] - ETA: 1s - loss: 0.2345 - accuracy: 0.9301
872/1688 [==============>...............] - ETA: 1s - loss: 0.2320 - accuracy: 0.9308
895/1688 [==============>...............] - ETA: 1s - loss: 0.2294 - accuracy: 0.9316
918/1688 [===============>..............] - ETA: 1s - loss: 0.2266 - accuracy: 0.9324
941/1688 [===============>..............] - ETA: 1s - loss: 0.2237 - accuracy: 0.9333
964/1688 [================>.............] - ETA: 1s - loss: 0.2210 - accuracy: 0.9341
987/1688 [================>.............] - ETA: 1s - loss: 0.2187 - accuracy: 0.9348
1011/1688 [================>.............] - ETA: 1s - loss: 0.2162 - accuracy: 0.9354
1033/1688 [=================>............] - ETA: 1s - loss: 0.2138 - accuracy: 0.9360
1054/1688 [=================>............] - ETA: 1s - loss: 0.2125 - accuracy: 0.9362
1077/1688 [==================>...........] - ETA: 1s - loss: 0.2105 - accuracy: 0.9368
1099/1688 [==================>...........] - ETA: 1s - loss: 0.2086 - accuracy: 0.9374
1121/1688 [==================>...........] - ETA: 1s - loss: 0.2071 - accuracy: 0.9380
1144/1688 [===================>..........] - ETA: 1s - loss: 0.2052 - accuracy: 0.9386
1167/1688 [===================>..........] - ETA: 1s - loss: 0.2040 - accuracy: 0.9389
1190/1688 [====================>.........] - ETA: 1s - loss: 0.2017 - accuracy: 0.9395
1213/1688 [====================>.........] - ETA: 1s - loss: 0.1998 - accuracy: 0.9401
1236/1688 [====================>.........] - ETA: 1s - loss: 0.1978 - accuracy: 0.9406
1259/1688 [=====================>........] - ETA: 0s - loss: 0.1958 - accuracy: 0.9412
1281/1688 [=====================>........] - ETA: 0s - loss: 0.1937 - accuracy: 0.9417
1305/1688 [======================>.......] - ETA: 0s - loss: 0.1914 - accuracy: 0.9424
1327/1688 [======================>.......] - ETA: 0s - loss: 0.1896 - accuracy: 0.9428
1349/1688 [======================>.......] - ETA: 0s - loss: 0.1884 - accuracy: 0.9432
1372/1688 [=======================>......] - ETA: 0s - loss: 0.1870 - accuracy: 0.9436
1396/1688 [=======================>......] - ETA: 0s - loss: 0.1861 - accuracy: 0.9439
1419/1688 [========================>.....] - ETA: 0s - loss: 0.1842 - accuracy: 0.9444
1442/1688 [========================>.....] - ETA: 0s - loss: 0.1826 - accuracy: 0.9449
1464/1688 [=========================>....] - ETA: 0s - loss: 0.1810 - accuracy: 0.9453
1486/1688 [=========================>....] - ETA: 0s - loss: 0.1798 - accuracy: 0.9457
1509/1688 [=========================>....] - ETA: 0s - loss: 0.1783 - accuracy: 0.9462
1531/1688 [==========================>...] - ETA: 0s - loss: 0.1768 - accuracy: 0.9466
1554/1688 [==========================>...] - ETA: 0s - loss: 0.1754 - accuracy: 0.9471
1577/1688 [===========================>..] - ETA: 0s - loss: 0.1739 - accuracy: 0.9476
1599/1688 [===========================>..] - ETA: 0s - loss: 0.1725 - accuracy: 0.9479
1621/1688 [===========================>..] - ETA: 0s - loss: 0.1715 - accuracy: 0.9482
1643/1688 [============================>.] - ETA: 0s - loss: 0.1703 - accuracy: 0.9486
1665/1688 [============================>.] - ETA: 0s - loss: 0.1690 - accuracy: 0.9490
1688/1688 [==============================] - ETA: 0s - loss: 0.1685 - accuracy: 0.9492
1688/1688 [==============================] - 7s 3ms/step - loss: 0.1685 - accuracy: 0.9492 - val_loss: 0.0873 - val_accuracy: 0.9748
Epoch 2/10
1/1688 [..............................] - ETA: 4s - loss: 0.0181 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0613 - accuracy: 0.9810
46/1688 [..............................] - ETA: 3s - loss: 0.0588 - accuracy: 0.9817
69/1688 [>.............................] - ETA: 3s - loss: 0.0625 - accuracy: 0.9810
91/1688 [>.............................] - ETA: 3s - loss: 0.0656 - accuracy: 0.9801
113/1688 [=>............................] - ETA: 3s - loss: 0.0633 - accuracy: 0.9795
136/1688 [=>............................] - ETA: 3s - loss: 0.0637 - accuracy: 0.9800
159/1688 [=>............................] - ETA: 3s - loss: 0.0654 - accuracy: 0.9801
182/1688 [==>...........................] - ETA: 3s - loss: 0.0649 - accuracy: 0.9804
204/1688 [==>...........................] - ETA: 3s - loss: 0.0648 - accuracy: 0.9802
226/1688 [===>..........................] - ETA: 3s - loss: 0.0677 - accuracy: 0.9791
249/1688 [===>..........................] - ETA: 3s - loss: 0.0663 - accuracy: 0.9794
272/1688 [===>..........................] - ETA: 3s - loss: 0.0696 - accuracy: 0.9785
294/1688 [====>.........................] - ETA: 3s - loss: 0.0717 - accuracy: 0.9785
316/1688 [====>.........................] - ETA: 3s - loss: 0.0702 - accuracy: 0.9789
338/1688 [=====>........................] - ETA: 3s - loss: 0.0702 - accuracy: 0.9791
361/1688 [=====>........................] - ETA: 3s - loss: 0.0683 - accuracy: 0.9798
384/1688 [=====>........................] - ETA: 2s - loss: 0.0663 - accuracy: 0.9801
406/1688 [======>.......................] - ETA: 2s - loss: 0.0672 - accuracy: 0.9795
429/1688 [======>.......................] - ETA: 2s - loss: 0.0678 - accuracy: 0.9795
452/1688 [=======>......................] - ETA: 2s - loss: 0.0680 - accuracy: 0.9794
474/1688 [=======>......................] - ETA: 2s - loss: 0.0678 - accuracy: 0.9794
496/1688 [=======>......................] - ETA: 2s - loss: 0.0668 - accuracy: 0.9798
518/1688 [========>.....................] - ETA: 2s - loss: 0.0677 - accuracy: 0.9797
540/1688 [========>.....................] - ETA: 2s - loss: 0.0684 - accuracy: 0.9795
562/1688 [========>.....................] - ETA: 2s - loss: 0.0692 - accuracy: 0.9793
585/1688 [=========>....................] - ETA: 2s - loss: 0.0686 - accuracy: 0.9795
607/1688 [=========>....................] - ETA: 2s - loss: 0.0697 - accuracy: 0.9794
630/1688 [==========>...................] - ETA: 2s - loss: 0.0688 - accuracy: 0.9797
653/1688 [==========>...................] - ETA: 2s - loss: 0.0679 - accuracy: 0.9799
676/1688 [===========>..................] - ETA: 2s - loss: 0.0675 - accuracy: 0.9799
698/1688 [===========>..................] - ETA: 2s - loss: 0.0674 - accuracy: 0.9801
720/1688 [===========>..................] - ETA: 2s - loss: 0.0672 - accuracy: 0.9800
742/1688 [============>.................] - ETA: 2s - loss: 0.0670 - accuracy: 0.9799
764/1688 [============>.................] - ETA: 2s - loss: 0.0672 - accuracy: 0.9798
785/1688 [============>.................] - ETA: 2s - loss: 0.0670 - accuracy: 0.9799
809/1688 [=============>................] - ETA: 2s - loss: 0.0672 - accuracy: 0.9798
831/1688 [=============>................] - ETA: 1s - loss: 0.0667 - accuracy: 0.9800
854/1688 [==============>...............] - ETA: 1s - loss: 0.0662 - accuracy: 0.9800
876/1688 [==============>...............] - ETA: 1s - loss: 0.0673 - accuracy: 0.9797
899/1688 [==============>...............] - ETA: 1s - loss: 0.0672 - accuracy: 0.9797
921/1688 [===============>..............] - ETA: 1s - loss: 0.0673 - accuracy: 0.9798
944/1688 [===============>..............] - ETA: 1s - loss: 0.0671 - accuracy: 0.9798
966/1688 [================>.............] - ETA: 1s - loss: 0.0670 - accuracy: 0.9798
989/1688 [================>.............] - ETA: 1s - loss: 0.0668 - accuracy: 0.9798
1012/1688 [================>.............] - ETA: 1s - loss: 0.0662 - accuracy: 0.9800
1034/1688 [=================>............] - ETA: 1s - loss: 0.0672 - accuracy: 0.9797
1056/1688 [=================>............] - ETA: 1s - loss: 0.0669 - accuracy: 0.9796
1078/1688 [==================>...........] - ETA: 1s - loss: 0.0667 - accuracy: 0.9797
1100/1688 [==================>...........] - ETA: 1s - loss: 0.0666 - accuracy: 0.9798
1122/1688 [==================>...........] - ETA: 1s - loss: 0.0668 - accuracy: 0.9797
1145/1688 [===================>..........] - ETA: 1s - loss: 0.0669 - accuracy: 0.9798
1168/1688 [===================>..........] - ETA: 1s - loss: 0.0671 - accuracy: 0.9797
1191/1688 [====================>.........] - ETA: 1s - loss: 0.0677 - accuracy: 0.9794
1214/1688 [====================>.........] - ETA: 1s - loss: 0.0676 - accuracy: 0.9794
1236/1688 [====================>.........] - ETA: 1s - loss: 0.0674 - accuracy: 0.9795
1259/1688 [=====================>........] - ETA: 0s - loss: 0.0672 - accuracy: 0.9794
1282/1688 [=====================>........] - ETA: 0s - loss: 0.0673 - accuracy: 0.9793
1305/1688 [======================>.......] - ETA: 0s - loss: 0.0674 - accuracy: 0.9792
1327/1688 [======================>.......] - ETA: 0s - loss: 0.0683 - accuracy: 0.9790
1350/1688 [======================>.......] - ETA: 0s - loss: 0.0681 - accuracy: 0.9790
1373/1688 [=======================>......] - ETA: 0s - loss: 0.0686 - accuracy: 0.9789
1395/1688 [=======================>......] - ETA: 0s - loss: 0.0686 - accuracy: 0.9790
1418/1688 [========================>.....] - ETA: 0s - loss: 0.0683 - accuracy: 0.9791
1441/1688 [========================>.....] - ETA: 0s - loss: 0.0681 - accuracy: 0.9792
1463/1688 [=========================>....] - ETA: 0s - loss: 0.0684 - accuracy: 0.9792
1486/1688 [=========================>....] - ETA: 0s - loss: 0.0688 - accuracy: 0.9790
1509/1688 [=========================>....] - ETA: 0s - loss: 0.0685 - accuracy: 0.9791
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0687 - accuracy: 0.9790
1553/1688 [==========================>...] - ETA: 0s - loss: 0.0690 - accuracy: 0.9790
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0691 - accuracy: 0.9790
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0691 - accuracy: 0.9791
1621/1688 [===========================>..] - ETA: 0s - loss: 0.0698 - accuracy: 0.9789
1643/1688 [============================>.] - ETA: 0s - loss: 0.0698 - accuracy: 0.9788
1666/1688 [============================>.] - ETA: 0s - loss: 0.0698 - accuracy: 0.9788
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0694 - accuracy: 0.9789 - val_loss: 0.0649 - val_accuracy: 0.9823
Epoch 3/10
1/1688 [..............................] - ETA: 4s - loss: 0.0209 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0382 - accuracy: 0.9878
45/1688 [..............................] - ETA: 3s - loss: 0.0454 - accuracy: 0.9840
67/1688 [>.............................] - ETA: 3s - loss: 0.0413 - accuracy: 0.9865
89/1688 [>.............................] - ETA: 3s - loss: 0.0440 - accuracy: 0.9860
112/1688 [>.............................] - ETA: 3s - loss: 0.0509 - accuracy: 0.9849
135/1688 [=>............................] - ETA: 3s - loss: 0.0485 - accuracy: 0.9856
158/1688 [=>............................] - ETA: 3s - loss: 0.0461 - accuracy: 0.9862
181/1688 [==>...........................] - ETA: 3s - loss: 0.0461 - accuracy: 0.9865
204/1688 [==>...........................] - ETA: 3s - loss: 0.0461 - accuracy: 0.9870
228/1688 [===>..........................] - ETA: 3s - loss: 0.0445 - accuracy: 0.9875
251/1688 [===>..........................] - ETA: 3s - loss: 0.0461 - accuracy: 0.9868
274/1688 [===>..........................] - ETA: 3s - loss: 0.0451 - accuracy: 0.9872
297/1688 [====>.........................] - ETA: 3s - loss: 0.0441 - accuracy: 0.9873
319/1688 [====>.........................] - ETA: 3s - loss: 0.0454 - accuracy: 0.9872
342/1688 [=====>........................] - ETA: 3s - loss: 0.0453 - accuracy: 0.9872
365/1688 [=====>........................] - ETA: 2s - loss: 0.0451 - accuracy: 0.9868
387/1688 [=====>........................] - ETA: 2s - loss: 0.0446 - accuracy: 0.9868
410/1688 [======>.......................] - ETA: 2s - loss: 0.0441 - accuracy: 0.9868
433/1688 [======>.......................] - ETA: 2s - loss: 0.0446 - accuracy: 0.9866
456/1688 [=======>......................] - ETA: 2s - loss: 0.0449 - accuracy: 0.9865
478/1688 [=======>......................] - ETA: 2s - loss: 0.0459 - accuracy: 0.9863
500/1688 [=======>......................] - ETA: 2s - loss: 0.0463 - accuracy: 0.9859
522/1688 [========>.....................] - ETA: 2s - loss: 0.0476 - accuracy: 0.9856
544/1688 [========>.....................] - ETA: 2s - loss: 0.0475 - accuracy: 0.9854
566/1688 [=========>....................] - ETA: 2s - loss: 0.0473 - accuracy: 0.9853
589/1688 [=========>....................] - ETA: 2s - loss: 0.0468 - accuracy: 0.9853
611/1688 [=========>....................] - ETA: 2s - loss: 0.0476 - accuracy: 0.9851
633/1688 [==========>...................] - ETA: 2s - loss: 0.0472 - accuracy: 0.9852
655/1688 [==========>...................] - ETA: 2s - loss: 0.0480 - accuracy: 0.9850
678/1688 [===========>..................] - ETA: 2s - loss: 0.0478 - accuracy: 0.9851
701/1688 [===========>..................] - ETA: 2s - loss: 0.0488 - accuracy: 0.9847
725/1688 [===========>..................] - ETA: 2s - loss: 0.0491 - accuracy: 0.9846
747/1688 [============>.................] - ETA: 2s - loss: 0.0488 - accuracy: 0.9846
770/1688 [============>.................] - ETA: 2s - loss: 0.0484 - accuracy: 0.9846
793/1688 [=============>................] - ETA: 2s - loss: 0.0483 - accuracy: 0.9846
815/1688 [=============>................] - ETA: 1s - loss: 0.0488 - accuracy: 0.9844
837/1688 [=============>................] - ETA: 1s - loss: 0.0495 - accuracy: 0.9841
861/1688 [==============>...............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9841
883/1688 [==============>...............] - ETA: 1s - loss: 0.0495 - accuracy: 0.9842
906/1688 [===============>..............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9843
929/1688 [===============>..............] - ETA: 1s - loss: 0.0498 - accuracy: 0.9842
951/1688 [===============>..............] - ETA: 1s - loss: 0.0495 - accuracy: 0.9842
973/1688 [================>.............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9842
996/1688 [================>.............] - ETA: 1s - loss: 0.0491 - accuracy: 0.9842
1019/1688 [=================>............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9841
1042/1688 [=================>............] - ETA: 1s - loss: 0.0492 - accuracy: 0.9840
1064/1688 [=================>............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9839
1086/1688 [==================>...........] - ETA: 1s - loss: 0.0505 - accuracy: 0.9839
1109/1688 [==================>...........] - ETA: 1s - loss: 0.0506 - accuracy: 0.9838
1132/1688 [===================>..........] - ETA: 1s - loss: 0.0502 - accuracy: 0.9840
1155/1688 [===================>..........] - ETA: 1s - loss: 0.0507 - accuracy: 0.9838
1178/1688 [===================>..........] - ETA: 1s - loss: 0.0507 - accuracy: 0.9838
1201/1688 [====================>.........] - ETA: 1s - loss: 0.0507 - accuracy: 0.9839
1223/1688 [====================>.........] - ETA: 1s - loss: 0.0508 - accuracy: 0.9839
1245/1688 [=====================>........] - ETA: 1s - loss: 0.0513 - accuracy: 0.9837
1268/1688 [=====================>........] - ETA: 0s - loss: 0.0511 - accuracy: 0.9837
1291/1688 [=====================>........] - ETA: 0s - loss: 0.0509 - accuracy: 0.9837
1314/1688 [======================>.......] - ETA: 0s - loss: 0.0510 - accuracy: 0.9838
1337/1688 [======================>.......] - ETA: 0s - loss: 0.0510 - accuracy: 0.9838
1360/1688 [=======================>......] - ETA: 0s - loss: 0.0507 - accuracy: 0.9839
1383/1688 [=======================>......] - ETA: 0s - loss: 0.0508 - accuracy: 0.9839
1405/1688 [=======================>......] - ETA: 0s - loss: 0.0508 - accuracy: 0.9838
1428/1688 [========================>.....] - ETA: 0s - loss: 0.0511 - accuracy: 0.9837
1451/1688 [========================>.....] - ETA: 0s - loss: 0.0509 - accuracy: 0.9837
1474/1688 [=========================>....] - ETA: 0s - loss: 0.0512 - accuracy: 0.9836
1497/1688 [=========================>....] - ETA: 0s - loss: 0.0510 - accuracy: 0.9837
1520/1688 [==========================>...] - ETA: 0s - loss: 0.0509 - accuracy: 0.9837
1542/1688 [==========================>...] - ETA: 0s - loss: 0.0512 - accuracy: 0.9836
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0513 - accuracy: 0.9836
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0514 - accuracy: 0.9835
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0514 - accuracy: 0.9835
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0514 - accuracy: 0.9834
1653/1688 [============================>.] - ETA: 0s - loss: 0.0512 - accuracy: 0.9836
1675/1688 [============================>.] - ETA: 0s - loss: 0.0509 - accuracy: 0.9836
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0509 - accuracy: 0.9836 - val_loss: 0.0751 - val_accuracy: 0.9817
Epoch 4/10
1/1688 [..............................] - ETA: 4s - loss: 0.1718 - accuracy: 0.9688
24/1688 [..............................] - ETA: 3s - loss: 0.0347 - accuracy: 0.9909
46/1688 [..............................] - ETA: 3s - loss: 0.0373 - accuracy: 0.9905
68/1688 [>.............................] - ETA: 3s - loss: 0.0308 - accuracy: 0.9917
90/1688 [>.............................] - ETA: 3s - loss: 0.0305 - accuracy: 0.9913
112/1688 [>.............................] - ETA: 3s - loss: 0.0291 - accuracy: 0.9911
135/1688 [=>............................] - ETA: 3s - loss: 0.0290 - accuracy: 0.9905
158/1688 [=>............................] - ETA: 3s - loss: 0.0289 - accuracy: 0.9905
181/1688 [==>...........................] - ETA: 3s - loss: 0.0324 - accuracy: 0.9896
203/1688 [==>...........................] - ETA: 3s - loss: 0.0323 - accuracy: 0.9895
226/1688 [===>..........................] - ETA: 3s - loss: 0.0329 - accuracy: 0.9892
248/1688 [===>..........................] - ETA: 3s - loss: 0.0316 - accuracy: 0.9898
270/1688 [===>..........................] - ETA: 3s - loss: 0.0320 - accuracy: 0.9897
292/1688 [====>.........................] - ETA: 3s - loss: 0.0337 - accuracy: 0.9891
313/1688 [====>.........................] - ETA: 3s - loss: 0.0330 - accuracy: 0.9894
335/1688 [====>.........................] - ETA: 3s - loss: 0.0329 - accuracy: 0.9896
358/1688 [=====>........................] - ETA: 3s - loss: 0.0329 - accuracy: 0.9894
381/1688 [=====>........................] - ETA: 2s - loss: 0.0333 - accuracy: 0.9893
403/1688 [======>.......................] - ETA: 2s - loss: 0.0346 - accuracy: 0.9888
425/1688 [======>.......................] - ETA: 2s - loss: 0.0357 - accuracy: 0.9885
447/1688 [======>.......................] - ETA: 2s - loss: 0.0356 - accuracy: 0.9886
469/1688 [=======>......................] - ETA: 2s - loss: 0.0350 - accuracy: 0.9886
491/1688 [=======>......................] - ETA: 2s - loss: 0.0348 - accuracy: 0.9886
513/1688 [========>.....................] - ETA: 2s - loss: 0.0354 - accuracy: 0.9884
535/1688 [========>.....................] - ETA: 2s - loss: 0.0358 - accuracy: 0.9881
557/1688 [========>.....................] - ETA: 2s - loss: 0.0366 - accuracy: 0.9879
580/1688 [=========>....................] - ETA: 2s - loss: 0.0366 - accuracy: 0.9879
603/1688 [=========>....................] - ETA: 2s - loss: 0.0363 - accuracy: 0.9879
626/1688 [==========>...................] - ETA: 2s - loss: 0.0367 - accuracy: 0.9879
649/1688 [==========>...................] - ETA: 2s - loss: 0.0369 - accuracy: 0.9879
671/1688 [==========>...................] - ETA: 2s - loss: 0.0370 - accuracy: 0.9879
694/1688 [===========>..................] - ETA: 2s - loss: 0.0370 - accuracy: 0.9880
716/1688 [===========>..................] - ETA: 2s - loss: 0.0376 - accuracy: 0.9877
738/1688 [============>.................] - ETA: 2s - loss: 0.0374 - accuracy: 0.9877
761/1688 [============>.................] - ETA: 2s - loss: 0.0382 - accuracy: 0.9876
784/1688 [============>.................] - ETA: 2s - loss: 0.0384 - accuracy: 0.9876
806/1688 [=============>................] - ETA: 2s - loss: 0.0381 - accuracy: 0.9877
828/1688 [=============>................] - ETA: 1s - loss: 0.0381 - accuracy: 0.9877
850/1688 [==============>...............] - ETA: 1s - loss: 0.0391 - accuracy: 0.9874
872/1688 [==============>...............] - ETA: 1s - loss: 0.0388 - accuracy: 0.9874
894/1688 [==============>...............] - ETA: 1s - loss: 0.0389 - accuracy: 0.9874
917/1688 [===============>..............] - ETA: 1s - loss: 0.0387 - accuracy: 0.9874
940/1688 [===============>..............] - ETA: 1s - loss: 0.0391 - accuracy: 0.9873
963/1688 [================>.............] - ETA: 1s - loss: 0.0392 - accuracy: 0.9872
985/1688 [================>.............] - ETA: 1s - loss: 0.0395 - accuracy: 0.9873
1008/1688 [================>.............] - ETA: 1s - loss: 0.0396 - accuracy: 0.9872
1030/1688 [=================>............] - ETA: 1s - loss: 0.0397 - accuracy: 0.9873
1052/1688 [=================>............] - ETA: 1s - loss: 0.0403 - accuracy: 0.9872
1075/1688 [==================>...........] - ETA: 1s - loss: 0.0412 - accuracy: 0.9870
1097/1688 [==================>...........] - ETA: 1s - loss: 0.0413 - accuracy: 0.9870
1119/1688 [==================>...........] - ETA: 1s - loss: 0.0420 - accuracy: 0.9866
1142/1688 [===================>..........] - ETA: 1s - loss: 0.0421 - accuracy: 0.9865
1165/1688 [===================>..........] - ETA: 1s - loss: 0.0421 - accuracy: 0.9865
1187/1688 [====================>.........] - ETA: 1s - loss: 0.0420 - accuracy: 0.9866
1211/1688 [====================>.........] - ETA: 1s - loss: 0.0422 - accuracy: 0.9865
1233/1688 [====================>.........] - ETA: 1s - loss: 0.0420 - accuracy: 0.9866
1256/1688 [=====================>........] - ETA: 0s - loss: 0.0424 - accuracy: 0.9864
1278/1688 [=====================>........] - ETA: 0s - loss: 0.0423 - accuracy: 0.9865
1300/1688 [======================>.......] - ETA: 0s - loss: 0.0429 - accuracy: 0.9862
1323/1688 [======================>.......] - ETA: 0s - loss: 0.0432 - accuracy: 0.9861
1346/1688 [======================>.......] - ETA: 0s - loss: 0.0434 - accuracy: 0.9860
1369/1688 [=======================>......] - ETA: 0s - loss: 0.0433 - accuracy: 0.9860
1392/1688 [=======================>......] - ETA: 0s - loss: 0.0432 - accuracy: 0.9860
1414/1688 [========================>.....] - ETA: 0s - loss: 0.0432 - accuracy: 0.9861
1437/1688 [========================>.....] - ETA: 0s - loss: 0.0429 - accuracy: 0.9862
1459/1688 [========================>.....] - ETA: 0s - loss: 0.0429 - accuracy: 0.9862
1481/1688 [=========================>....] - ETA: 0s - loss: 0.0427 - accuracy: 0.9863
1504/1688 [=========================>....] - ETA: 0s - loss: 0.0426 - accuracy: 0.9863
1526/1688 [==========================>...] - ETA: 0s - loss: 0.0428 - accuracy: 0.9862
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0426 - accuracy: 0.9862
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0425 - accuracy: 0.9863
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0427 - accuracy: 0.9862
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0426 - accuracy: 0.9862
1640/1688 [============================>.] - ETA: 0s - loss: 0.0426 - accuracy: 0.9862
1662/1688 [============================>.] - ETA: 0s - loss: 0.0425 - accuracy: 0.9862
1684/1688 [============================>.] - ETA: 0s - loss: 0.0424 - accuracy: 0.9862
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0424 - accuracy: 0.9862 - val_loss: 0.0492 - val_accuracy: 0.9870
Epoch 5/10
1/1688 [..............................] - ETA: 4s - loss: 0.0039 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0120 - accuracy: 0.9987
47/1688 [..............................] - ETA: 3s - loss: 0.0337 - accuracy: 0.9894
70/1688 [>.............................] - ETA: 3s - loss: 0.0300 - accuracy: 0.9902
92/1688 [>.............................] - ETA: 3s - loss: 0.0318 - accuracy: 0.9898
115/1688 [=>............................] - ETA: 3s - loss: 0.0310 - accuracy: 0.9902
138/1688 [=>............................] - ETA: 3s - loss: 0.0287 - accuracy: 0.9909
160/1688 [=>............................] - ETA: 3s - loss: 0.0268 - accuracy: 0.9914
183/1688 [==>...........................] - ETA: 3s - loss: 0.0257 - accuracy: 0.9913
206/1688 [==>...........................] - ETA: 3s - loss: 0.0248 - accuracy: 0.9918
229/1688 [===>..........................] - ETA: 3s - loss: 0.0259 - accuracy: 0.9911
252/1688 [===>..........................] - ETA: 3s - loss: 0.0269 - accuracy: 0.9906
275/1688 [===>..........................] - ETA: 3s - loss: 0.0275 - accuracy: 0.9900
297/1688 [====>.........................] - ETA: 3s - loss: 0.0280 - accuracy: 0.9901
320/1688 [====>.........................] - ETA: 3s - loss: 0.0271 - accuracy: 0.9904
342/1688 [=====>........................] - ETA: 3s - loss: 0.0272 - accuracy: 0.9903
365/1688 [=====>........................] - ETA: 2s - loss: 0.0271 - accuracy: 0.9907
388/1688 [=====>........................] - ETA: 2s - loss: 0.0274 - accuracy: 0.9906
411/1688 [======>.......................] - ETA: 2s - loss: 0.0275 - accuracy: 0.9906
434/1688 [======>.......................] - ETA: 2s - loss: 0.0276 - accuracy: 0.9906
456/1688 [=======>......................] - ETA: 2s - loss: 0.0268 - accuracy: 0.9910
479/1688 [=======>......................] - ETA: 2s - loss: 0.0266 - accuracy: 0.9910
501/1688 [=======>......................] - ETA: 2s - loss: 0.0262 - accuracy: 0.9912
523/1688 [========>.....................] - ETA: 2s - loss: 0.0269 - accuracy: 0.9912
545/1688 [========>.....................] - ETA: 2s - loss: 0.0270 - accuracy: 0.9911
568/1688 [=========>....................] - ETA: 2s - loss: 0.0279 - accuracy: 0.9907
590/1688 [=========>....................] - ETA: 2s - loss: 0.0279 - accuracy: 0.9907
611/1688 [=========>....................] - ETA: 2s - loss: 0.0277 - accuracy: 0.9907
634/1688 [==========>...................] - ETA: 2s - loss: 0.0273 - accuracy: 0.9908
657/1688 [==========>...................] - ETA: 2s - loss: 0.0273 - accuracy: 0.9909
679/1688 [===========>..................] - ETA: 2s - loss: 0.0275 - accuracy: 0.9907
701/1688 [===========>..................] - ETA: 2s - loss: 0.0277 - accuracy: 0.9906
723/1688 [===========>..................] - ETA: 2s - loss: 0.0283 - accuracy: 0.9904
745/1688 [============>.................] - ETA: 2s - loss: 0.0281 - accuracy: 0.9905
767/1688 [============>.................] - ETA: 2s - loss: 0.0284 - accuracy: 0.9905
790/1688 [=============>................] - ETA: 2s - loss: 0.0285 - accuracy: 0.9905
814/1688 [=============>................] - ETA: 1s - loss: 0.0285 - accuracy: 0.9906
838/1688 [=============>................] - ETA: 1s - loss: 0.0288 - accuracy: 0.9906
861/1688 [==============>...............] - ETA: 1s - loss: 0.0288 - accuracy: 0.9905
883/1688 [==============>...............] - ETA: 1s - loss: 0.0290 - accuracy: 0.9904
905/1688 [===============>..............] - ETA: 1s - loss: 0.0292 - accuracy: 0.9904
926/1688 [===============>..............] - ETA: 1s - loss: 0.0296 - accuracy: 0.9903
949/1688 [===============>..............] - ETA: 1s - loss: 0.0293 - accuracy: 0.9904
971/1688 [================>.............] - ETA: 1s - loss: 0.0292 - accuracy: 0.9904
993/1688 [================>.............] - ETA: 1s - loss: 0.0292 - accuracy: 0.9904
1015/1688 [=================>............] - ETA: 1s - loss: 0.0294 - accuracy: 0.9903
1038/1688 [=================>............] - ETA: 1s - loss: 0.0299 - accuracy: 0.9902
1061/1688 [=================>............] - ETA: 1s - loss: 0.0301 - accuracy: 0.9902
1083/1688 [==================>...........] - ETA: 1s - loss: 0.0306 - accuracy: 0.9900
1105/1688 [==================>...........] - ETA: 1s - loss: 0.0304 - accuracy: 0.9901
1127/1688 [===================>..........] - ETA: 1s - loss: 0.0305 - accuracy: 0.9901
1149/1688 [===================>..........] - ETA: 1s - loss: 0.0308 - accuracy: 0.9900
1172/1688 [===================>..........] - ETA: 1s - loss: 0.0309 - accuracy: 0.9899
1194/1688 [====================>.........] - ETA: 1s - loss: 0.0308 - accuracy: 0.9900
1216/1688 [====================>.........] - ETA: 1s - loss: 0.0308 - accuracy: 0.9900
1239/1688 [=====================>........] - ETA: 1s - loss: 0.0311 - accuracy: 0.9899
1261/1688 [=====================>........] - ETA: 0s - loss: 0.0311 - accuracy: 0.9899
1284/1688 [=====================>........] - ETA: 0s - loss: 0.0312 - accuracy: 0.9900
1306/1688 [======================>.......] - ETA: 0s - loss: 0.0314 - accuracy: 0.9899
1328/1688 [======================>.......] - ETA: 0s - loss: 0.0314 - accuracy: 0.9899
1350/1688 [======================>.......] - ETA: 0s - loss: 0.0319 - accuracy: 0.9897
1372/1688 [=======================>......] - ETA: 0s - loss: 0.0320 - accuracy: 0.9896
1394/1688 [=======================>......] - ETA: 0s - loss: 0.0321 - accuracy: 0.9896
1416/1688 [========================>.....] - ETA: 0s - loss: 0.0323 - accuracy: 0.9896
1439/1688 [========================>.....] - ETA: 0s - loss: 0.0324 - accuracy: 0.9896
1462/1688 [========================>.....] - ETA: 0s - loss: 0.0324 - accuracy: 0.9895
1484/1688 [=========================>....] - ETA: 0s - loss: 0.0326 - accuracy: 0.9894
1506/1688 [=========================>....] - ETA: 0s - loss: 0.0328 - accuracy: 0.9895
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0332 - accuracy: 0.9893
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0331 - accuracy: 0.9894
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0333 - accuracy: 0.9893
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0332 - accuracy: 0.9894
1619/1688 [===========================>..] - ETA: 0s - loss: 0.0333 - accuracy: 0.9894
1642/1688 [============================>.] - ETA: 0s - loss: 0.0335 - accuracy: 0.9893
1664/1688 [============================>.] - ETA: 0s - loss: 0.0334 - accuracy: 0.9894
1686/1688 [============================>.] - ETA: 0s - loss: 0.0333 - accuracy: 0.9894
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0333 - accuracy: 0.9894 - val_loss: 0.0661 - val_accuracy: 0.9850
Epoch 6/10
1/1688 [..............................] - ETA: 3s - loss: 0.0038 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0261 - accuracy: 0.9918
46/1688 [..............................] - ETA: 3s - loss: 0.0315 - accuracy: 0.9932
69/1688 [>.............................] - ETA: 3s - loss: 0.0342 - accuracy: 0.9909
91/1688 [>.............................] - ETA: 3s - loss: 0.0338 - accuracy: 0.9900
114/1688 [=>............................] - ETA: 3s - loss: 0.0294 - accuracy: 0.9918
137/1688 [=>............................] - ETA: 3s - loss: 0.0275 - accuracy: 0.9922
159/1688 [=>............................] - ETA: 3s - loss: 0.0260 - accuracy: 0.9927
181/1688 [==>...........................] - ETA: 3s - loss: 0.0245 - accuracy: 0.9933
204/1688 [==>...........................] - ETA: 3s - loss: 0.0236 - accuracy: 0.9933
226/1688 [===>..........................] - ETA: 3s - loss: 0.0223 - accuracy: 0.9935
248/1688 [===>..........................] - ETA: 3s - loss: 0.0230 - accuracy: 0.9927
271/1688 [===>..........................] - ETA: 3s - loss: 0.0236 - accuracy: 0.9926
293/1688 [====>.........................] - ETA: 3s - loss: 0.0237 - accuracy: 0.9925
316/1688 [====>.........................] - ETA: 3s - loss: 0.0241 - accuracy: 0.9924
339/1688 [=====>........................] - ETA: 3s - loss: 0.0236 - accuracy: 0.9923
361/1688 [=====>........................] - ETA: 3s - loss: 0.0241 - accuracy: 0.9922
384/1688 [=====>........................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9918
407/1688 [======>.......................] - ETA: 2s - loss: 0.0243 - accuracy: 0.9919
429/1688 [======>.......................] - ETA: 2s - loss: 0.0248 - accuracy: 0.9916
451/1688 [=======>......................] - ETA: 2s - loss: 0.0241 - accuracy: 0.9919
474/1688 [=======>......................] - ETA: 2s - loss: 0.0238 - accuracy: 0.9921
497/1688 [=======>......................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9919
519/1688 [========>.....................] - ETA: 2s - loss: 0.0237 - accuracy: 0.9918
541/1688 [========>.....................] - ETA: 2s - loss: 0.0236 - accuracy: 0.9919
564/1688 [=========>....................] - ETA: 2s - loss: 0.0236 - accuracy: 0.9918
587/1688 [=========>....................] - ETA: 2s - loss: 0.0239 - accuracy: 0.9916
609/1688 [=========>....................] - ETA: 2s - loss: 0.0235 - accuracy: 0.9916
631/1688 [==========>...................] - ETA: 2s - loss: 0.0236 - accuracy: 0.9917
654/1688 [==========>...................] - ETA: 2s - loss: 0.0244 - accuracy: 0.9915
677/1688 [===========>..................] - ETA: 2s - loss: 0.0244 - accuracy: 0.9915
699/1688 [===========>..................] - ETA: 2s - loss: 0.0246 - accuracy: 0.9915
722/1688 [===========>..................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9916
745/1688 [============>.................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9915
767/1688 [============>.................] - ETA: 2s - loss: 0.0246 - accuracy: 0.9916
789/1688 [=============>................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9916
812/1688 [=============>................] - ETA: 1s - loss: 0.0246 - accuracy: 0.9917
834/1688 [=============>................] - ETA: 1s - loss: 0.0243 - accuracy: 0.9918
856/1688 [==============>...............] - ETA: 1s - loss: 0.0242 - accuracy: 0.9919
879/1688 [==============>...............] - ETA: 1s - loss: 0.0243 - accuracy: 0.9919
901/1688 [===============>..............] - ETA: 1s - loss: 0.0246 - accuracy: 0.9917
923/1688 [===============>..............] - ETA: 1s - loss: 0.0248 - accuracy: 0.9917
946/1688 [===============>..............] - ETA: 1s - loss: 0.0250 - accuracy: 0.9917
970/1688 [================>.............] - ETA: 1s - loss: 0.0253 - accuracy: 0.9916
993/1688 [================>.............] - ETA: 1s - loss: 0.0257 - accuracy: 0.9915
1017/1688 [=================>............] - ETA: 1s - loss: 0.0258 - accuracy: 0.9914
1040/1688 [=================>............] - ETA: 1s - loss: 0.0257 - accuracy: 0.9915
1062/1688 [=================>............] - ETA: 1s - loss: 0.0261 - accuracy: 0.9914
1085/1688 [==================>...........] - ETA: 1s - loss: 0.0261 - accuracy: 0.9914
1109/1688 [==================>...........] - ETA: 1s - loss: 0.0261 - accuracy: 0.9915
1131/1688 [===================>..........] - ETA: 1s - loss: 0.0262 - accuracy: 0.9914
1154/1688 [===================>..........] - ETA: 1s - loss: 0.0263 - accuracy: 0.9913
1178/1688 [===================>..........] - ETA: 1s - loss: 0.0263 - accuracy: 0.9913
1201/1688 [====================>.........] - ETA: 1s - loss: 0.0260 - accuracy: 0.9914
1223/1688 [====================>.........] - ETA: 1s - loss: 0.0261 - accuracy: 0.9914
1245/1688 [=====================>........] - ETA: 1s - loss: 0.0263 - accuracy: 0.9912
1267/1688 [=====================>........] - ETA: 0s - loss: 0.0268 - accuracy: 0.9911
1290/1688 [=====================>........] - ETA: 0s - loss: 0.0267 - accuracy: 0.9912
1312/1688 [======================>.......] - ETA: 0s - loss: 0.0264 - accuracy: 0.9912
1335/1688 [======================>.......] - ETA: 0s - loss: 0.0262 - accuracy: 0.9913
1357/1688 [=======================>......] - ETA: 0s - loss: 0.0262 - accuracy: 0.9913
1379/1688 [=======================>......] - ETA: 0s - loss: 0.0265 - accuracy: 0.9912
1402/1688 [=======================>......] - ETA: 0s - loss: 0.0265 - accuracy: 0.9912
1425/1688 [========================>.....] - ETA: 0s - loss: 0.0268 - accuracy: 0.9910
1448/1688 [========================>.....] - ETA: 0s - loss: 0.0270 - accuracy: 0.9910
1471/1688 [=========================>....] - ETA: 0s - loss: 0.0271 - accuracy: 0.9908
1493/1688 [=========================>....] - ETA: 0s - loss: 0.0271 - accuracy: 0.9909
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0270 - accuracy: 0.9909
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0268 - accuracy: 0.9909
1561/1688 [==========================>...] - ETA: 0s - loss: 0.0268 - accuracy: 0.9910
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0269 - accuracy: 0.9910
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0269 - accuracy: 0.9910
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0272 - accuracy: 0.9909
1652/1688 [============================>.] - ETA: 0s - loss: 0.0274 - accuracy: 0.9908
1675/1688 [============================>.] - ETA: 0s - loss: 0.0278 - accuracy: 0.9908
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0278 - accuracy: 0.9908 - val_loss: 0.0557 - val_accuracy: 0.9850
Epoch 7/10
1/1688 [..............................] - ETA: 4s - loss: 0.0339 - accuracy: 0.9688
24/1688 [..............................] - ETA: 3s - loss: 0.0215 - accuracy: 0.9896
46/1688 [..............................] - ETA: 3s - loss: 0.0218 - accuracy: 0.9898
69/1688 [>.............................] - ETA: 3s - loss: 0.0247 - accuracy: 0.9896
91/1688 [>.............................] - ETA: 3s - loss: 0.0214 - accuracy: 0.9911
114/1688 [=>............................] - ETA: 3s - loss: 0.0193 - accuracy: 0.9921
137/1688 [=>............................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9916
160/1688 [=>............................] - ETA: 3s - loss: 0.0198 - accuracy: 0.9922
183/1688 [==>...........................] - ETA: 3s - loss: 0.0198 - accuracy: 0.9925
207/1688 [==>...........................] - ETA: 3s - loss: 0.0192 - accuracy: 0.9928
229/1688 [===>..........................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9929
251/1688 [===>..........................] - ETA: 3s - loss: 0.0199 - accuracy: 0.9929
273/1688 [===>..........................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9930
295/1688 [====>.........................] - ETA: 3s - loss: 0.0190 - accuracy: 0.9932
319/1688 [====>.........................] - ETA: 3s - loss: 0.0190 - accuracy: 0.9935
341/1688 [=====>........................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9934
364/1688 [=====>........................] - ETA: 2s - loss: 0.0204 - accuracy: 0.9929
387/1688 [=====>........................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9931
409/1688 [======>.......................] - ETA: 2s - loss: 0.0198 - accuracy: 0.9931
431/1688 [======>.......................] - ETA: 2s - loss: 0.0199 - accuracy: 0.9929
453/1688 [=======>......................] - ETA: 2s - loss: 0.0196 - accuracy: 0.9930
477/1688 [=======>......................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9929
499/1688 [=======>......................] - ETA: 2s - loss: 0.0196 - accuracy: 0.9930
522/1688 [========>.....................] - ETA: 2s - loss: 0.0202 - accuracy: 0.9928
544/1688 [========>.....................] - ETA: 2s - loss: 0.0200 - accuracy: 0.9929
567/1688 [=========>....................] - ETA: 2s - loss: 0.0196 - accuracy: 0.9930
590/1688 [=========>....................] - ETA: 2s - loss: 0.0206 - accuracy: 0.9927
613/1688 [=========>....................] - ETA: 2s - loss: 0.0206 - accuracy: 0.9928
636/1688 [==========>...................] - ETA: 2s - loss: 0.0202 - accuracy: 0.9930
659/1688 [==========>...................] - ETA: 2s - loss: 0.0205 - accuracy: 0.9929
682/1688 [===========>..................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9929
704/1688 [===========>..................] - ETA: 2s - loss: 0.0206 - accuracy: 0.9929
727/1688 [===========>..................] - ETA: 2s - loss: 0.0207 - accuracy: 0.9929
750/1688 [============>.................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9930
772/1688 [============>.................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9929
795/1688 [=============>................] - ETA: 2s - loss: 0.0213 - accuracy: 0.9927
818/1688 [=============>................] - ETA: 1s - loss: 0.0217 - accuracy: 0.9927
840/1688 [=============>................] - ETA: 1s - loss: 0.0222 - accuracy: 0.9926
862/1688 [==============>...............] - ETA: 1s - loss: 0.0223 - accuracy: 0.9926
885/1688 [==============>...............] - ETA: 1s - loss: 0.0221 - accuracy: 0.9926
907/1688 [===============>..............] - ETA: 1s - loss: 0.0223 - accuracy: 0.9926
929/1688 [===============>..............] - ETA: 1s - loss: 0.0224 - accuracy: 0.9925
951/1688 [===============>..............] - ETA: 1s - loss: 0.0224 - accuracy: 0.9925
974/1688 [================>.............] - ETA: 1s - loss: 0.0224 - accuracy: 0.9926
996/1688 [================>.............] - ETA: 1s - loss: 0.0224 - accuracy: 0.9926
1018/1688 [=================>............] - ETA: 1s - loss: 0.0223 - accuracy: 0.9927
1041/1688 [=================>............] - ETA: 1s - loss: 0.0227 - accuracy: 0.9925
1063/1688 [=================>............] - ETA: 1s - loss: 0.0228 - accuracy: 0.9924
1086/1688 [==================>...........] - ETA: 1s - loss: 0.0228 - accuracy: 0.9923
1108/1688 [==================>...........] - ETA: 1s - loss: 0.0233 - accuracy: 0.9922
1131/1688 [===================>..........] - ETA: 1s - loss: 0.0235 - accuracy: 0.9922
1154/1688 [===================>..........] - ETA: 1s - loss: 0.0234 - accuracy: 0.9922
1177/1688 [===================>..........] - ETA: 1s - loss: 0.0237 - accuracy: 0.9921
1200/1688 [====================>.........] - ETA: 1s - loss: 0.0238 - accuracy: 0.9920
1223/1688 [====================>.........] - ETA: 1s - loss: 0.0237 - accuracy: 0.9921
1246/1688 [=====================>........] - ETA: 1s - loss: 0.0235 - accuracy: 0.9921
1269/1688 [=====================>........] - ETA: 0s - loss: 0.0233 - accuracy: 0.9921
1291/1688 [=====================>........] - ETA: 0s - loss: 0.0235 - accuracy: 0.9921
1313/1688 [======================>.......] - ETA: 0s - loss: 0.0234 - accuracy: 0.9921
1336/1688 [======================>.......] - ETA: 0s - loss: 0.0237 - accuracy: 0.9920
1358/1688 [=======================>......] - ETA: 0s - loss: 0.0237 - accuracy: 0.9920
1380/1688 [=======================>......] - ETA: 0s - loss: 0.0238 - accuracy: 0.9921
1402/1688 [=======================>......] - ETA: 0s - loss: 0.0237 - accuracy: 0.9921
1425/1688 [========================>.....] - ETA: 0s - loss: 0.0237 - accuracy: 0.9921
1448/1688 [========================>.....] - ETA: 0s - loss: 0.0237 - accuracy: 0.9920
1472/1688 [=========================>....] - ETA: 0s - loss: 0.0237 - accuracy: 0.9920
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0239 - accuracy: 0.9920
1517/1688 [=========================>....] - ETA: 0s - loss: 0.0238 - accuracy: 0.9920
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0238 - accuracy: 0.9920
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0242 - accuracy: 0.9920
1586/1688 [===========================>..] - ETA: 0s - loss: 0.0244 - accuracy: 0.9919
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0244 - accuracy: 0.9919
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0244 - accuracy: 0.9919
1653/1688 [============================>.] - ETA: 0s - loss: 0.0242 - accuracy: 0.9919
1676/1688 [============================>.] - ETA: 0s - loss: 0.0242 - accuracy: 0.9919
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0241 - accuracy: 0.9919 - val_loss: 0.0639 - val_accuracy: 0.9835
Epoch 8/10
1/1688 [..............................] - ETA: 3s - loss: 0.0018 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0319 - accuracy: 0.9878
45/1688 [..............................] - ETA: 3s - loss: 0.0269 - accuracy: 0.9889
68/1688 [>.............................] - ETA: 3s - loss: 0.0216 - accuracy: 0.9913
91/1688 [>.............................] - ETA: 3s - loss: 0.0206 - accuracy: 0.9918
114/1688 [=>............................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9931
137/1688 [=>............................] - ETA: 3s - loss: 0.0181 - accuracy: 0.9936
159/1688 [=>............................] - ETA: 3s - loss: 0.0173 - accuracy: 0.9939
181/1688 [==>...........................] - ETA: 3s - loss: 0.0162 - accuracy: 0.9946
203/1688 [==>...........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9949
225/1688 [==>...........................] - ETA: 3s - loss: 0.0159 - accuracy: 0.9947
248/1688 [===>..........................] - ETA: 3s - loss: 0.0159 - accuracy: 0.9946
270/1688 [===>..........................] - ETA: 3s - loss: 0.0164 - accuracy: 0.9944
293/1688 [====>.........................] - ETA: 3s - loss: 0.0178 - accuracy: 0.9943
316/1688 [====>.........................] - ETA: 3s - loss: 0.0187 - accuracy: 0.9940
338/1688 [=====>........................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9941
360/1688 [=====>........................] - ETA: 3s - loss: 0.0183 - accuracy: 0.9941
382/1688 [=====>........................] - ETA: 2s - loss: 0.0176 - accuracy: 0.9944
404/1688 [======>.......................] - ETA: 2s - loss: 0.0175 - accuracy: 0.9944
427/1688 [======>.......................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9941
450/1688 [======>.......................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9941
473/1688 [=======>......................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9941
496/1688 [=======>......................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9943
518/1688 [========>.....................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9941
541/1688 [========>.....................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9942
564/1688 [=========>....................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9943
586/1688 [=========>....................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9943
609/1688 [=========>....................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9943
631/1688 [==========>...................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9943
654/1688 [==========>...................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9941
676/1688 [===========>..................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9941
698/1688 [===========>..................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9940
721/1688 [===========>..................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9939
744/1688 [============>.................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9937
767/1688 [============>.................] - ETA: 2s - loss: 0.0188 - accuracy: 0.9936
789/1688 [=============>................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9935
811/1688 [=============>................] - ETA: 1s - loss: 0.0189 - accuracy: 0.9937
834/1688 [=============>................] - ETA: 1s - loss: 0.0189 - accuracy: 0.9937
856/1688 [==============>...............] - ETA: 1s - loss: 0.0187 - accuracy: 0.9938
878/1688 [==============>...............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9939
901/1688 [===============>..............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9939
923/1688 [===============>..............] - ETA: 1s - loss: 0.0184 - accuracy: 0.9939
945/1688 [===============>..............] - ETA: 1s - loss: 0.0186 - accuracy: 0.9939
968/1688 [================>.............] - ETA: 1s - loss: 0.0183 - accuracy: 0.9940
990/1688 [================>.............] - ETA: 1s - loss: 0.0185 - accuracy: 0.9940
1012/1688 [================>.............] - ETA: 1s - loss: 0.0186 - accuracy: 0.9939
1034/1688 [=================>............] - ETA: 1s - loss: 0.0186 - accuracy: 0.9938
1057/1688 [=================>............] - ETA: 1s - loss: 0.0185 - accuracy: 0.9939
1080/1688 [==================>...........] - ETA: 1s - loss: 0.0183 - accuracy: 0.9940
1103/1688 [==================>...........] - ETA: 1s - loss: 0.0182 - accuracy: 0.9940
1125/1688 [==================>...........] - ETA: 1s - loss: 0.0181 - accuracy: 0.9941
1148/1688 [===================>..........] - ETA: 1s - loss: 0.0183 - accuracy: 0.9940
1171/1688 [===================>..........] - ETA: 1s - loss: 0.0186 - accuracy: 0.9939
1194/1688 [====================>.........] - ETA: 1s - loss: 0.0186 - accuracy: 0.9940
1217/1688 [====================>.........] - ETA: 1s - loss: 0.0186 - accuracy: 0.9939
1240/1688 [=====================>........] - ETA: 1s - loss: 0.0189 - accuracy: 0.9939
1263/1688 [=====================>........] - ETA: 0s - loss: 0.0188 - accuracy: 0.9939
1286/1688 [=====================>........] - ETA: 0s - loss: 0.0189 - accuracy: 0.9939
1309/1688 [======================>.......] - ETA: 0s - loss: 0.0189 - accuracy: 0.9939
1332/1688 [======================>.......] - ETA: 0s - loss: 0.0190 - accuracy: 0.9939
1355/1688 [=======================>......] - ETA: 0s - loss: 0.0194 - accuracy: 0.9937
1377/1688 [=======================>......] - ETA: 0s - loss: 0.0195 - accuracy: 0.9937
1399/1688 [=======================>......] - ETA: 0s - loss: 0.0198 - accuracy: 0.9935
1422/1688 [========================>.....] - ETA: 0s - loss: 0.0202 - accuracy: 0.9934
1444/1688 [========================>.....] - ETA: 0s - loss: 0.0204 - accuracy: 0.9933
1466/1688 [=========================>....] - ETA: 0s - loss: 0.0204 - accuracy: 0.9932
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0205 - accuracy: 0.9932
1512/1688 [=========================>....] - ETA: 0s - loss: 0.0206 - accuracy: 0.9931
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0206 - accuracy: 0.9931
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0206 - accuracy: 0.9930
1579/1688 [===========================>..] - ETA: 0s - loss: 0.0206 - accuracy: 0.9931
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0206 - accuracy: 0.9930
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0208 - accuracy: 0.9929
1646/1688 [============================>.] - ETA: 0s - loss: 0.0209 - accuracy: 0.9928
1669/1688 [============================>.] - ETA: 0s - loss: 0.0210 - accuracy: 0.9928
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0210 - accuracy: 0.9928 - val_loss: 0.0782 - val_accuracy: 0.9823
Epoch 9/10
1/1688 [..............................] - ETA: 4s - loss: 0.0117 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0228 - accuracy: 0.9948
47/1688 [..............................] - ETA: 3s - loss: 0.0189 - accuracy: 0.9953
70/1688 [>.............................] - ETA: 3s - loss: 0.0179 - accuracy: 0.9960
94/1688 [>.............................] - ETA: 3s - loss: 0.0159 - accuracy: 0.9963
117/1688 [=>............................] - ETA: 3s - loss: 0.0160 - accuracy: 0.9960
140/1688 [=>............................] - ETA: 3s - loss: 0.0172 - accuracy: 0.9958
163/1688 [=>............................] - ETA: 3s - loss: 0.0165 - accuracy: 0.9958
186/1688 [==>...........................] - ETA: 3s - loss: 0.0160 - accuracy: 0.9958
210/1688 [==>...........................] - ETA: 3s - loss: 0.0150 - accuracy: 0.9960
232/1688 [===>..........................] - ETA: 3s - loss: 0.0148 - accuracy: 0.9960
255/1688 [===>..........................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9961
278/1688 [===>..........................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9960
301/1688 [====>.........................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9955
323/1688 [====>.........................] - ETA: 3s - loss: 0.0148 - accuracy: 0.9956
346/1688 [=====>........................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9958
369/1688 [=====>........................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9961
392/1688 [=====>........................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9959
414/1688 [======>.......................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9958
436/1688 [======>.......................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9956
459/1688 [=======>......................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9954
482/1688 [=======>......................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9951
504/1688 [=======>......................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9949
526/1688 [========>.....................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9951
548/1688 [========>.....................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9952
571/1688 [=========>....................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9952
594/1688 [=========>....................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9952
616/1688 [=========>....................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9952
638/1688 [==========>...................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9952
661/1688 [==========>...................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9951
683/1688 [===========>..................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9952
706/1688 [===========>..................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9951
729/1688 [===========>..................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9952
752/1688 [============>.................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9952
775/1688 [============>.................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9952
797/1688 [=============>................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9952
819/1688 [=============>................] - ETA: 1s - loss: 0.0150 - accuracy: 0.9952
842/1688 [=============>................] - ETA: 1s - loss: 0.0149 - accuracy: 0.9952
864/1688 [==============>...............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9952
886/1688 [==============>...............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9952
909/1688 [===============>..............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9952
932/1688 [===============>..............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9952
954/1688 [===============>..............] - ETA: 1s - loss: 0.0155 - accuracy: 0.9951
976/1688 [================>.............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9950
999/1688 [================>.............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9951
1022/1688 [=================>............] - ETA: 1s - loss: 0.0155 - accuracy: 0.9950
1044/1688 [=================>............] - ETA: 1s - loss: 0.0158 - accuracy: 0.9950
1066/1688 [=================>............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9949
1089/1688 [==================>...........] - ETA: 1s - loss: 0.0159 - accuracy: 0.9949
1112/1688 [==================>...........] - ETA: 1s - loss: 0.0157 - accuracy: 0.9950
1134/1688 [===================>..........] - ETA: 1s - loss: 0.0155 - accuracy: 0.9950
1157/1688 [===================>..........] - ETA: 1s - loss: 0.0154 - accuracy: 0.9951
1179/1688 [===================>..........] - ETA: 1s - loss: 0.0155 - accuracy: 0.9951
1202/1688 [====================>.........] - ETA: 1s - loss: 0.0157 - accuracy: 0.9951
1225/1688 [====================>.........] - ETA: 1s - loss: 0.0160 - accuracy: 0.9949
1247/1688 [=====================>........] - ETA: 0s - loss: 0.0162 - accuracy: 0.9949
1269/1688 [=====================>........] - ETA: 0s - loss: 0.0163 - accuracy: 0.9949
1291/1688 [=====================>........] - ETA: 0s - loss: 0.0162 - accuracy: 0.9950
1312/1688 [======================>.......] - ETA: 0s - loss: 0.0163 - accuracy: 0.9950
1335/1688 [======================>.......] - ETA: 0s - loss: 0.0163 - accuracy: 0.9949
1358/1688 [=======================>......] - ETA: 0s - loss: 0.0163 - accuracy: 0.9949
1380/1688 [=======================>......] - ETA: 0s - loss: 0.0164 - accuracy: 0.9949
1402/1688 [=======================>......] - ETA: 0s - loss: 0.0165 - accuracy: 0.9949
1425/1688 [========================>.....] - ETA: 0s - loss: 0.0165 - accuracy: 0.9949
1448/1688 [========================>.....] - ETA: 0s - loss: 0.0166 - accuracy: 0.9948
1471/1688 [=========================>....] - ETA: 0s - loss: 0.0167 - accuracy: 0.9948
1493/1688 [=========================>....] - ETA: 0s - loss: 0.0169 - accuracy: 0.9948
1515/1688 [=========================>....] - ETA: 0s - loss: 0.0170 - accuracy: 0.9948
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0173 - accuracy: 0.9946
1561/1688 [==========================>...] - ETA: 0s - loss: 0.0173 - accuracy: 0.9946
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0174 - accuracy: 0.9945
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0175 - accuracy: 0.9945
1632/1688 [============================>.] - ETA: 0s - loss: 0.0175 - accuracy: 0.9945
1655/1688 [============================>.] - ETA: 0s - loss: 0.0175 - accuracy: 0.9944
1678/1688 [============================>.] - ETA: 0s - loss: 0.0176 - accuracy: 0.9944
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0176 - accuracy: 0.9943 - val_loss: 0.0646 - val_accuracy: 0.9855
Epoch 10/10
1/1688 [..............................] - ETA: 4s - loss: 0.2058 - accuracy: 0.9688
23/1688 [..............................] - ETA: 3s - loss: 0.0240 - accuracy: 0.9932
45/1688 [..............................] - ETA: 3s - loss: 0.0217 - accuracy: 0.9931
67/1688 [>.............................] - ETA: 3s - loss: 0.0190 - accuracy: 0.9930
90/1688 [>.............................] - ETA: 3s - loss: 0.0189 - accuracy: 0.9941
113/1688 [=>............................] - ETA: 3s - loss: 0.0179 - accuracy: 0.9939
135/1688 [=>............................] - ETA: 3s - loss: 0.0167 - accuracy: 0.9940
158/1688 [=>............................] - ETA: 3s - loss: 0.0156 - accuracy: 0.9947
180/1688 [==>...........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9944
202/1688 [==>...........................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9947
224/1688 [==>...........................] - ETA: 3s - loss: 0.0143 - accuracy: 0.9950
247/1688 [===>..........................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9952
269/1688 [===>..........................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9956
292/1688 [====>.........................] - ETA: 3s - loss: 0.0140 - accuracy: 0.9953
314/1688 [====>.........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9947
336/1688 [====>.........................] - ETA: 3s - loss: 0.0151 - accuracy: 0.9947
359/1688 [=====>........................] - ETA: 3s - loss: 0.0155 - accuracy: 0.9945
381/1688 [=====>........................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9947
403/1688 [======>.......................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9948
425/1688 [======>.......................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9946
448/1688 [======>.......................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9946
471/1688 [=======>......................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9947
494/1688 [=======>......................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9947
518/1688 [========>.....................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9948
541/1688 [========>.....................] - ETA: 2s - loss: 0.0167 - accuracy: 0.9946
564/1688 [=========>....................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9947
586/1688 [=========>....................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9948
609/1688 [=========>....................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9949
632/1688 [==========>...................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9951
654/1688 [==========>...................] - ETA: 2s - loss: 0.0156 - accuracy: 0.9951
677/1688 [===========>..................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9951
700/1688 [===========>..................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9952
723/1688 [===========>..................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9953
746/1688 [============>.................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9953
768/1688 [============>.................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9954
790/1688 [=============>................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9954
813/1688 [=============>................] - ETA: 1s - loss: 0.0146 - accuracy: 0.9953
835/1688 [=============>................] - ETA: 1s - loss: 0.0148 - accuracy: 0.9953
857/1688 [==============>...............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9954
879/1688 [==============>...............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9954
902/1688 [===============>..............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9955
925/1688 [===============>..............] - ETA: 1s - loss: 0.0140 - accuracy: 0.9956
948/1688 [===============>..............] - ETA: 1s - loss: 0.0138 - accuracy: 0.9957
970/1688 [================>.............] - ETA: 1s - loss: 0.0138 - accuracy: 0.9957
993/1688 [================>.............] - ETA: 1s - loss: 0.0140 - accuracy: 0.9956
1016/1688 [=================>............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9956
1038/1688 [=================>............] - ETA: 1s - loss: 0.0141 - accuracy: 0.9956
1060/1688 [=================>............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9955
1082/1688 [==================>...........] - ETA: 1s - loss: 0.0144 - accuracy: 0.9954
1104/1688 [==================>...........] - ETA: 1s - loss: 0.0145 - accuracy: 0.9954
1127/1688 [===================>..........] - ETA: 1s - loss: 0.0144 - accuracy: 0.9954
1149/1688 [===================>..........] - ETA: 1s - loss: 0.0148 - accuracy: 0.9953
1172/1688 [===================>..........] - ETA: 1s - loss: 0.0148 - accuracy: 0.9952
1195/1688 [====================>.........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9952
1217/1688 [====================>.........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9952
1240/1688 [=====================>........] - ETA: 1s - loss: 0.0148 - accuracy: 0.9952
1262/1688 [=====================>........] - ETA: 0s - loss: 0.0150 - accuracy: 0.9950
1284/1688 [=====================>........] - ETA: 0s - loss: 0.0152 - accuracy: 0.9950
1307/1688 [======================>.......] - ETA: 0s - loss: 0.0152 - accuracy: 0.9949
1329/1688 [======================>.......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9950
1351/1688 [=======================>......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9950
1373/1688 [=======================>......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9950
1395/1688 [=======================>......] - ETA: 0s - loss: 0.0150 - accuracy: 0.9950
1417/1688 [========================>.....] - ETA: 0s - loss: 0.0151 - accuracy: 0.9950
1439/1688 [========================>.....] - ETA: 0s - loss: 0.0151 - accuracy: 0.9950
1462/1688 [========================>.....] - ETA: 0s - loss: 0.0152 - accuracy: 0.9950
1485/1688 [=========================>....] - ETA: 0s - loss: 0.0153 - accuracy: 0.9949
1507/1688 [=========================>....] - ETA: 0s - loss: 0.0156 - accuracy: 0.9949
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0156 - accuracy: 0.9949
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0156 - accuracy: 0.9949
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0156 - accuracy: 0.9949
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0158 - accuracy: 0.9948
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0160 - accuracy: 0.9948
1641/1688 [============================>.] - ETA: 0s - loss: 0.0163 - accuracy: 0.9947
1663/1688 [============================>.] - ETA: 0s - loss: 0.0165 - accuracy: 0.9946
1686/1688 [============================>.] - ETA: 0s - loss: 0.0168 - accuracy: 0.9945
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0168 - accuracy: 0.9945 - val_loss: 0.0654 - val_accuracy: 0.9843
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.9811999797821045
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bit integer activations and weights. Therefore, the floating-point TF-Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a TF-Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating-point TF-Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a TF-Keras model with custom QuantizeML quantized layers substituted for the original TF-Keras layers.
All TF-Keras API functions can be applied on this new model: summary(), compile(),
fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:577: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples. Continuing execution.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate. "
1/1024 [..............................] - ETA: 2:33
51/1024 [>.............................] - ETA: 0s
103/1024 [==>...........................] - ETA: 0s
155/1024 [===>..........................] - ETA: 0s
207/1024 [=====>........................] - ETA: 0s
259/1024 [======>.......................] - ETA: 0s
311/1024 [========>.....................] - ETA: 0s
362/1024 [=========>....................] - ETA: 0s
413/1024 [===========>..................] - ETA: 0s
465/1024 [============>.................] - ETA: 0s
517/1024 [==============>...............] - ETA: 0s
568/1024 [===============>..............] - ETA: 0s
619/1024 [=================>............] - ETA: 0s
672/1024 [==================>...........] - ETA: 0s
723/1024 [====================>.........] - ETA: 0s
776/1024 [=====================>........] - ETA: 0s
827/1024 [=======================>......] - ETA: 0s
878/1024 [========================>.....] - ETA: 0s
929/1024 [==========================>...] - ETA: 0s
980/1024 [===========================>..] - ETA: 0s
1024/1024 [==============================] - 1s 981us/step
model_quantized.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input (InputLayer) [(None, 28, 28, 1)] 0
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.9706000089645386
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.9797000288963318
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating-point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.9778000116348267
2.4. Model fine-tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine-tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine-tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 1:51:30 - loss: 0.0217 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0211 - accuracy: 0.9931
18/1688 [..............................] - ETA: 10s - loss: 0.0200 - accuracy: 0.9913
27/1688 [..............................] - ETA: 10s - loss: 0.0187 - accuracy: 0.9919
36/1688 [..............................] - ETA: 10s - loss: 0.0244 - accuracy: 0.9878
45/1688 [..............................] - ETA: 10s - loss: 0.0315 - accuracy: 0.9854
54/1688 [..............................] - ETA: 9s - loss: 0.0286 - accuracy: 0.9867
63/1688 [>.............................] - ETA: 9s - loss: 0.0259 - accuracy: 0.9876
72/1688 [>.............................] - ETA: 9s - loss: 0.0232 - accuracy: 0.9887
80/1688 [>.............................] - ETA: 9s - loss: 0.0229 - accuracy: 0.9891
89/1688 [>.............................] - ETA: 9s - loss: 0.0257 - accuracy: 0.9891
98/1688 [>.............................] - ETA: 9s - loss: 0.0243 - accuracy: 0.9898
107/1688 [>.............................] - ETA: 9s - loss: 0.0241 - accuracy: 0.9901
116/1688 [=>............................] - ETA: 9s - loss: 0.0234 - accuracy: 0.9906
125/1688 [=>............................] - ETA: 9s - loss: 0.0231 - accuracy: 0.9905
134/1688 [=>............................] - ETA: 9s - loss: 0.0217 - accuracy: 0.9911
143/1688 [=>............................] - ETA: 9s - loss: 0.0214 - accuracy: 0.9913
152/1688 [=>............................] - ETA: 9s - loss: 0.0213 - accuracy: 0.9914
160/1688 [=>............................] - ETA: 9s - loss: 0.0206 - accuracy: 0.9916
169/1688 [==>...........................] - ETA: 9s - loss: 0.0213 - accuracy: 0.9917
177/1688 [==>...........................] - ETA: 9s - loss: 0.0217 - accuracy: 0.9913
186/1688 [==>...........................] - ETA: 9s - loss: 0.0214 - accuracy: 0.9914
195/1688 [==>...........................] - ETA: 9s - loss: 0.0207 - accuracy: 0.9918
203/1688 [==>...........................] - ETA: 9s - loss: 0.0201 - accuracy: 0.9921
212/1688 [==>...........................] - ETA: 9s - loss: 0.0205 - accuracy: 0.9919
221/1688 [==>...........................] - ETA: 8s - loss: 0.0199 - accuracy: 0.9922
230/1688 [===>..........................] - ETA: 8s - loss: 0.0199 - accuracy: 0.9924
239/1688 [===>..........................] - ETA: 8s - loss: 0.0192 - accuracy: 0.9927
248/1688 [===>..........................] - ETA: 8s - loss: 0.0191 - accuracy: 0.9927
257/1688 [===>..........................] - ETA: 8s - loss: 0.0186 - accuracy: 0.9929
266/1688 [===>..........................] - ETA: 8s - loss: 0.0183 - accuracy: 0.9931
275/1688 [===>..........................] - ETA: 8s - loss: 0.0179 - accuracy: 0.9933
284/1688 [====>.........................] - ETA: 8s - loss: 0.0178 - accuracy: 0.9934
293/1688 [====>.........................] - ETA: 8s - loss: 0.0176 - accuracy: 0.9935
302/1688 [====>.........................] - ETA: 8s - loss: 0.0176 - accuracy: 0.9936
311/1688 [====>.........................] - ETA: 8s - loss: 0.0173 - accuracy: 0.9938
320/1688 [====>.........................] - ETA: 8s - loss: 0.0169 - accuracy: 0.9939
328/1688 [====>.........................] - ETA: 8s - loss: 0.0166 - accuracy: 0.9940
337/1688 [====>.........................] - ETA: 8s - loss: 0.0165 - accuracy: 0.9940
346/1688 [=====>........................] - ETA: 8s - loss: 0.0163 - accuracy: 0.9941
355/1688 [=====>........................] - ETA: 8s - loss: 0.0164 - accuracy: 0.9941
364/1688 [=====>........................] - ETA: 8s - loss: 0.0164 - accuracy: 0.9942
373/1688 [=====>........................] - ETA: 8s - loss: 0.0161 - accuracy: 0.9943
381/1688 [=====>........................] - ETA: 8s - loss: 0.0159 - accuracy: 0.9943
389/1688 [=====>........................] - ETA: 7s - loss: 0.0157 - accuracy: 0.9944
398/1688 [======>.......................] - ETA: 7s - loss: 0.0155 - accuracy: 0.9945
407/1688 [======>.......................] - ETA: 7s - loss: 0.0153 - accuracy: 0.9946
416/1688 [======>.......................] - ETA: 7s - loss: 0.0152 - accuracy: 0.9947
425/1688 [======>.......................] - ETA: 7s - loss: 0.0149 - accuracy: 0.9948
434/1688 [======>.......................] - ETA: 7s - loss: 0.0148 - accuracy: 0.9948
443/1688 [======>.......................] - ETA: 7s - loss: 0.0146 - accuracy: 0.9949
452/1688 [=======>......................] - ETA: 7s - loss: 0.0145 - accuracy: 0.9950
461/1688 [=======>......................] - ETA: 7s - loss: 0.0144 - accuracy: 0.9951
470/1688 [=======>......................] - ETA: 7s - loss: 0.0142 - accuracy: 0.9951
479/1688 [=======>......................] - ETA: 7s - loss: 0.0144 - accuracy: 0.9950
488/1688 [=======>......................] - ETA: 7s - loss: 0.0144 - accuracy: 0.9950
497/1688 [=======>......................] - ETA: 7s - loss: 0.0142 - accuracy: 0.9951
506/1688 [=======>......................] - ETA: 7s - loss: 0.0141 - accuracy: 0.9951
515/1688 [========>.....................] - ETA: 7s - loss: 0.0139 - accuracy: 0.9952
524/1688 [========>.....................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9953
533/1688 [========>.....................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9953
542/1688 [========>.....................] - ETA: 6s - loss: 0.0137 - accuracy: 0.9953
551/1688 [========>.....................] - ETA: 6s - loss: 0.0137 - accuracy: 0.9953
560/1688 [========>.....................] - ETA: 6s - loss: 0.0136 - accuracy: 0.9954
569/1688 [=========>....................] - ETA: 6s - loss: 0.0135 - accuracy: 0.9954
578/1688 [=========>....................] - ETA: 6s - loss: 0.0135 - accuracy: 0.9955
587/1688 [=========>....................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9955
596/1688 [=========>....................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9955
605/1688 [=========>....................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9955
613/1688 [=========>....................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9955
622/1688 [==========>...................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9955
631/1688 [==========>...................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9954
640/1688 [==========>...................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9954
649/1688 [==========>...................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9954
657/1688 [==========>...................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9954
665/1688 [==========>...................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9954
674/1688 [==========>...................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9955
683/1688 [===========>..................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9955
692/1688 [===========>..................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9954
701/1688 [===========>..................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9954
710/1688 [===========>..................] - ETA: 5s - loss: 0.0131 - accuracy: 0.9954
719/1688 [===========>..................] - ETA: 5s - loss: 0.0130 - accuracy: 0.9954
728/1688 [===========>..................] - ETA: 5s - loss: 0.0130 - accuracy: 0.9954
737/1688 [============>.................] - ETA: 5s - loss: 0.0131 - accuracy: 0.9954
746/1688 [============>.................] - ETA: 5s - loss: 0.0132 - accuracy: 0.9954
755/1688 [============>.................] - ETA: 5s - loss: 0.0131 - accuracy: 0.9954
764/1688 [============>.................] - ETA: 5s - loss: 0.0131 - accuracy: 0.9954
773/1688 [============>.................] - ETA: 5s - loss: 0.0130 - accuracy: 0.9954
782/1688 [============>.................] - ETA: 5s - loss: 0.0129 - accuracy: 0.9955
791/1688 [=============>................] - ETA: 5s - loss: 0.0128 - accuracy: 0.9955
799/1688 [=============>................] - ETA: 5s - loss: 0.0127 - accuracy: 0.9955
807/1688 [=============>................] - ETA: 5s - loss: 0.0127 - accuracy: 0.9956
816/1688 [=============>................] - ETA: 5s - loss: 0.0127 - accuracy: 0.9956
825/1688 [=============>................] - ETA: 5s - loss: 0.0127 - accuracy: 0.9955
834/1688 [=============>................] - ETA: 5s - loss: 0.0127 - accuracy: 0.9955
843/1688 [=============>................] - ETA: 5s - loss: 0.0126 - accuracy: 0.9956
852/1688 [==============>...............] - ETA: 5s - loss: 0.0125 - accuracy: 0.9956
861/1688 [==============>...............] - ETA: 5s - loss: 0.0124 - accuracy: 0.9956
870/1688 [==============>...............] - ETA: 4s - loss: 0.0124 - accuracy: 0.9957
879/1688 [==============>...............] - ETA: 4s - loss: 0.0126 - accuracy: 0.9956
888/1688 [==============>...............] - ETA: 4s - loss: 0.0125 - accuracy: 0.9956
896/1688 [==============>...............] - ETA: 4s - loss: 0.0125 - accuracy: 0.9956
905/1688 [===============>..............] - ETA: 4s - loss: 0.0125 - accuracy: 0.9956
914/1688 [===============>..............] - ETA: 4s - loss: 0.0124 - accuracy: 0.9956
923/1688 [===============>..............] - ETA: 4s - loss: 0.0123 - accuracy: 0.9957
932/1688 [===============>..............] - ETA: 4s - loss: 0.0123 - accuracy: 0.9957
941/1688 [===============>..............] - ETA: 4s - loss: 0.0122 - accuracy: 0.9957
950/1688 [===============>..............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9958
959/1688 [================>.............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9958
968/1688 [================>.............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9958
977/1688 [================>.............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9958
986/1688 [================>.............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9958
995/1688 [================>.............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9958
1004/1688 [================>.............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9958
1013/1688 [=================>............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9958
1021/1688 [=================>............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9959
1030/1688 [=================>............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9959
1039/1688 [=================>............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9959
1048/1688 [=================>............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9959
1057/1688 [=================>............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9959
1066/1688 [=================>............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9960
1075/1688 [==================>...........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9960
1084/1688 [==================>...........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9960
1093/1688 [==================>...........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9960
1102/1688 [==================>...........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9960
1111/1688 [==================>...........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9960
1120/1688 [==================>...........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9961
1129/1688 [===================>..........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9960
1138/1688 [===================>..........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9960
1147/1688 [===================>..........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9960
1156/1688 [===================>..........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9960
1165/1688 [===================>..........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9961
1174/1688 [===================>..........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9960
1183/1688 [====================>.........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9960
1192/1688 [====================>.........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9960
1201/1688 [====================>.........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9960
1210/1688 [====================>.........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9960
1219/1688 [====================>.........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9961
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9960
1237/1688 [====================>.........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9960
1246/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9960
1255/1688 [=====================>........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9960
1264/1688 [=====================>........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9961
1273/1688 [=====================>........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9961
1282/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9961
1291/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9960
1300/1688 [======================>.......] - ETA: 2s - loss: 0.0119 - accuracy: 0.9961
1309/1688 [======================>.......] - ETA: 2s - loss: 0.0118 - accuracy: 0.9961
1318/1688 [======================>.......] - ETA: 2s - loss: 0.0118 - accuracy: 0.9961
1327/1688 [======================>.......] - ETA: 2s - loss: 0.0118 - accuracy: 0.9961
1336/1688 [======================>.......] - ETA: 2s - loss: 0.0118 - accuracy: 0.9961
1345/1688 [======================>.......] - ETA: 2s - loss: 0.0119 - accuracy: 0.9961
1354/1688 [=======================>......] - ETA: 2s - loss: 0.0118 - accuracy: 0.9961
1363/1688 [=======================>......] - ETA: 1s - loss: 0.0119 - accuracy: 0.9960
1372/1688 [=======================>......] - ETA: 1s - loss: 0.0118 - accuracy: 0.9961
1381/1688 [=======================>......] - ETA: 1s - loss: 0.0119 - accuracy: 0.9960
1390/1688 [=======================>......] - ETA: 1s - loss: 0.0120 - accuracy: 0.9960
1399/1688 [=======================>......] - ETA: 1s - loss: 0.0120 - accuracy: 0.9960
1408/1688 [========================>.....] - ETA: 1s - loss: 0.0120 - accuracy: 0.9959
1417/1688 [========================>.....] - ETA: 1s - loss: 0.0120 - accuracy: 0.9960
1426/1688 [========================>.....] - ETA: 1s - loss: 0.0120 - accuracy: 0.9960
1435/1688 [========================>.....] - ETA: 1s - loss: 0.0120 - accuracy: 0.9960
1443/1688 [========================>.....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9960
1452/1688 [========================>.....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9960
1461/1688 [========================>.....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9960
1470/1688 [=========================>....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9960
1479/1688 [=========================>....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9960
1488/1688 [=========================>....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9961
1497/1688 [=========================>....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9960
1506/1688 [=========================>....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9960
1515/1688 [=========================>....] - ETA: 1s - loss: 0.0117 - accuracy: 0.9961
1524/1688 [==========================>...] - ETA: 0s - loss: 0.0117 - accuracy: 0.9961
1532/1688 [==========================>...] - ETA: 0s - loss: 0.0117 - accuracy: 0.9961
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0117 - accuracy: 0.9961
1550/1688 [==========================>...] - ETA: 0s - loss: 0.0117 - accuracy: 0.9961
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0116 - accuracy: 0.9961
1568/1688 [==========================>...] - ETA: 0s - loss: 0.0116 - accuracy: 0.9961
1577/1688 [===========================>..] - ETA: 0s - loss: 0.0116 - accuracy: 0.9961
1586/1688 [===========================>..] - ETA: 0s - loss: 0.0115 - accuracy: 0.9961
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0115 - accuracy: 0.9961
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0115 - accuracy: 0.9961
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0115 - accuracy: 0.9961
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0115 - accuracy: 0.9961
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0115 - accuracy: 0.9961
1640/1688 [============================>.] - ETA: 0s - loss: 0.0114 - accuracy: 0.9962
1649/1688 [============================>.] - ETA: 0s - loss: 0.0114 - accuracy: 0.9962
1658/1688 [============================>.] - ETA: 0s - loss: 0.0114 - accuracy: 0.9961
1667/1688 [============================>.] - ETA: 0s - loss: 0.0114 - accuracy: 0.9961
1676/1688 [============================>.] - ETA: 0s - loss: 0.0114 - accuracy: 0.9961
1685/1688 [============================>.] - ETA: 0s - loss: 0.0114 - accuracy: 0.9962
1688/1688 [==============================] - 16s 7ms/step - loss: 0.0113 - accuracy: 0.9962 - val_loss: 0.0535 - val_accuracy: 0.9892
Epoch 2/5
1/1688 [..............................] - ETA: 11s - loss: 0.0058 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0031 - accuracy: 1.0000
18/1688 [..............................] - ETA: 10s - loss: 0.0035 - accuracy: 1.0000
27/1688 [..............................] - ETA: 10s - loss: 0.0053 - accuracy: 0.9988
36/1688 [..............................] - ETA: 10s - loss: 0.0048 - accuracy: 0.9991
45/1688 [..............................] - ETA: 10s - loss: 0.0052 - accuracy: 0.9993
54/1688 [..............................] - ETA: 9s - loss: 0.0056 - accuracy: 0.9988
63/1688 [>.............................] - ETA: 9s - loss: 0.0058 - accuracy: 0.9985
72/1688 [>.............................] - ETA: 9s - loss: 0.0060 - accuracy: 0.9987
80/1688 [>.............................] - ETA: 9s - loss: 0.0066 - accuracy: 0.9984
89/1688 [>.............................] - ETA: 9s - loss: 0.0063 - accuracy: 0.9986
98/1688 [>.............................] - ETA: 9s - loss: 0.0064 - accuracy: 0.9984
107/1688 [>.............................] - ETA: 9s - loss: 0.0063 - accuracy: 0.9985
116/1688 [=>............................] - ETA: 9s - loss: 0.0068 - accuracy: 0.9984
125/1688 [=>............................] - ETA: 9s - loss: 0.0066 - accuracy: 0.9985
134/1688 [=>............................] - ETA: 9s - loss: 0.0064 - accuracy: 0.9986
143/1688 [=>............................] - ETA: 9s - loss: 0.0066 - accuracy: 0.9985
152/1688 [=>............................] - ETA: 9s - loss: 0.0072 - accuracy: 0.9984
161/1688 [=>............................] - ETA: 9s - loss: 0.0071 - accuracy: 0.9984
170/1688 [==>...........................] - ETA: 9s - loss: 0.0070 - accuracy: 0.9985
179/1688 [==>...........................] - ETA: 9s - loss: 0.0069 - accuracy: 0.9986
188/1688 [==>...........................] - ETA: 9s - loss: 0.0069 - accuracy: 0.9985
197/1688 [==>...........................] - ETA: 9s - loss: 0.0068 - accuracy: 0.9986
206/1688 [==>...........................] - ETA: 8s - loss: 0.0076 - accuracy: 0.9983
215/1688 [==>...........................] - ETA: 8s - loss: 0.0077 - accuracy: 0.9983
224/1688 [==>...........................] - ETA: 8s - loss: 0.0076 - accuracy: 0.9983
233/1688 [===>..........................] - ETA: 8s - loss: 0.0076 - accuracy: 0.9983
242/1688 [===>..........................] - ETA: 8s - loss: 0.0074 - accuracy: 0.9983
250/1688 [===>..........................] - ETA: 8s - loss: 0.0073 - accuracy: 0.9984
259/1688 [===>..........................] - ETA: 8s - loss: 0.0073 - accuracy: 0.9984
268/1688 [===>..........................] - ETA: 8s - loss: 0.0071 - accuracy: 0.9985
277/1688 [===>..........................] - ETA: 8s - loss: 0.0071 - accuracy: 0.9985
286/1688 [====>.........................] - ETA: 8s - loss: 0.0074 - accuracy: 0.9985
295/1688 [====>.........................] - ETA: 8s - loss: 0.0073 - accuracy: 0.9985
303/1688 [====>.........................] - ETA: 8s - loss: 0.0073 - accuracy: 0.9986
312/1688 [====>.........................] - ETA: 8s - loss: 0.0074 - accuracy: 0.9984
321/1688 [====>.........................] - ETA: 8s - loss: 0.0075 - accuracy: 0.9983
330/1688 [====>.........................] - ETA: 8s - loss: 0.0076 - accuracy: 0.9982
338/1688 [=====>........................] - ETA: 8s - loss: 0.0076 - accuracy: 0.9982
347/1688 [=====>........................] - ETA: 8s - loss: 0.0075 - accuracy: 0.9983
356/1688 [=====>........................] - ETA: 8s - loss: 0.0073 - accuracy: 0.9983
365/1688 [=====>........................] - ETA: 8s - loss: 0.0075 - accuracy: 0.9982
374/1688 [=====>........................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9982
383/1688 [=====>........................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9983
392/1688 [=====>........................] - ETA: 7s - loss: 0.0077 - accuracy: 0.9982
401/1688 [======>.......................] - ETA: 7s - loss: 0.0077 - accuracy: 0.9982
410/1688 [======>.......................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9982
419/1688 [======>.......................] - ETA: 7s - loss: 0.0075 - accuracy: 0.9983
428/1688 [======>.......................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9983
437/1688 [======>.......................] - ETA: 7s - loss: 0.0073 - accuracy: 0.9984
446/1688 [======>.......................] - ETA: 7s - loss: 0.0073 - accuracy: 0.9983
455/1688 [=======>......................] - ETA: 7s - loss: 0.0072 - accuracy: 0.9984
464/1688 [=======>......................] - ETA: 7s - loss: 0.0071 - accuracy: 0.9984
473/1688 [=======>......................] - ETA: 7s - loss: 0.0072 - accuracy: 0.9983
482/1688 [=======>......................] - ETA: 7s - loss: 0.0071 - accuracy: 0.9984
491/1688 [=======>......................] - ETA: 7s - loss: 0.0071 - accuracy: 0.9983
500/1688 [=======>......................] - ETA: 7s - loss: 0.0070 - accuracy: 0.9984
509/1688 [========>.....................] - ETA: 7s - loss: 0.0072 - accuracy: 0.9983
517/1688 [========>.....................] - ETA: 7s - loss: 0.0072 - accuracy: 0.9982
526/1688 [========>.....................] - ETA: 7s - loss: 0.0072 - accuracy: 0.9982
535/1688 [========>.....................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9982
544/1688 [========>.....................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9982
553/1688 [========>.....................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9981
562/1688 [========>.....................] - ETA: 6s - loss: 0.0072 - accuracy: 0.9982
570/1688 [=========>....................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9981
579/1688 [=========>....................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9980
588/1688 [=========>....................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9980
597/1688 [=========>....................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9980
606/1688 [=========>....................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9980
615/1688 [=========>....................] - ETA: 6s - loss: 0.0075 - accuracy: 0.9980
624/1688 [==========>...................] - ETA: 6s - loss: 0.0074 - accuracy: 0.9980
633/1688 [==========>...................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9981
642/1688 [==========>...................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9981
651/1688 [==========>...................] - ETA: 6s - loss: 0.0072 - accuracy: 0.9981
660/1688 [==========>...................] - ETA: 6s - loss: 0.0072 - accuracy: 0.9981
669/1688 [==========>...................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9980
678/1688 [===========>..................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9980
686/1688 [===========>..................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9980
695/1688 [===========>..................] - ETA: 6s - loss: 0.0073 - accuracy: 0.9981
704/1688 [===========>..................] - ETA: 5s - loss: 0.0073 - accuracy: 0.9981
712/1688 [===========>..................] - ETA: 5s - loss: 0.0073 - accuracy: 0.9981
721/1688 [===========>..................] - ETA: 5s - loss: 0.0073 - accuracy: 0.9981
730/1688 [===========>..................] - ETA: 5s - loss: 0.0072 - accuracy: 0.9981
739/1688 [============>.................] - ETA: 5s - loss: 0.0073 - accuracy: 0.9981
748/1688 [============>.................] - ETA: 5s - loss: 0.0073 - accuracy: 0.9980
757/1688 [============>.................] - ETA: 5s - loss: 0.0073 - accuracy: 0.9981
766/1688 [============>.................] - ETA: 5s - loss: 0.0072 - accuracy: 0.9981
775/1688 [============>.................] - ETA: 5s - loss: 0.0074 - accuracy: 0.9980
784/1688 [============>.................] - ETA: 5s - loss: 0.0073 - accuracy: 0.9980
793/1688 [=============>................] - ETA: 5s - loss: 0.0074 - accuracy: 0.9980
802/1688 [=============>................] - ETA: 5s - loss: 0.0074 - accuracy: 0.9980
810/1688 [=============>................] - ETA: 5s - loss: 0.0073 - accuracy: 0.9980
819/1688 [=============>................] - ETA: 5s - loss: 0.0073 - accuracy: 0.9981
828/1688 [=============>................] - ETA: 5s - loss: 0.0073 - accuracy: 0.9981
837/1688 [=============>................] - ETA: 5s - loss: 0.0073 - accuracy: 0.9981
846/1688 [==============>...............] - ETA: 5s - loss: 0.0073 - accuracy: 0.9981
855/1688 [==============>...............] - ETA: 5s - loss: 0.0073 - accuracy: 0.9980
864/1688 [==============>...............] - ETA: 4s - loss: 0.0073 - accuracy: 0.9980
873/1688 [==============>...............] - ETA: 4s - loss: 0.0072 - accuracy: 0.9981
882/1688 [==============>...............] - ETA: 4s - loss: 0.0072 - accuracy: 0.9981
891/1688 [==============>...............] - ETA: 4s - loss: 0.0072 - accuracy: 0.9981
900/1688 [==============>...............] - ETA: 4s - loss: 0.0072 - accuracy: 0.9981
909/1688 [===============>..............] - ETA: 4s - loss: 0.0072 - accuracy: 0.9981
918/1688 [===============>..............] - ETA: 4s - loss: 0.0072 - accuracy: 0.9981
927/1688 [===============>..............] - ETA: 4s - loss: 0.0072 - accuracy: 0.9981
936/1688 [===============>..............] - ETA: 4s - loss: 0.0071 - accuracy: 0.9981
944/1688 [===============>..............] - ETA: 4s - loss: 0.0072 - accuracy: 0.9981
953/1688 [===============>..............] - ETA: 4s - loss: 0.0072 - accuracy: 0.9981
961/1688 [================>.............] - ETA: 4s - loss: 0.0071 - accuracy: 0.9981
970/1688 [================>.............] - ETA: 4s - loss: 0.0071 - accuracy: 0.9981
979/1688 [================>.............] - ETA: 4s - loss: 0.0071 - accuracy: 0.9981
988/1688 [================>.............] - ETA: 4s - loss: 0.0071 - accuracy: 0.9981
997/1688 [================>.............] - ETA: 4s - loss: 0.0071 - accuracy: 0.9981
1006/1688 [================>.............] - ETA: 4s - loss: 0.0071 - accuracy: 0.9980
1015/1688 [=================>............] - ETA: 4s - loss: 0.0071 - accuracy: 0.9981
1024/1688 [=================>............] - ETA: 4s - loss: 0.0071 - accuracy: 0.9981
1033/1688 [=================>............] - ETA: 3s - loss: 0.0071 - accuracy: 0.9981
1042/1688 [=================>............] - ETA: 3s - loss: 0.0071 - accuracy: 0.9981
1051/1688 [=================>............] - ETA: 3s - loss: 0.0071 - accuracy: 0.9981
1060/1688 [=================>............] - ETA: 3s - loss: 0.0071 - accuracy: 0.9981
1069/1688 [=================>............] - ETA: 3s - loss: 0.0071 - accuracy: 0.9980
1078/1688 [==================>...........] - ETA: 3s - loss: 0.0071 - accuracy: 0.9980
1087/1688 [==================>...........] - ETA: 3s - loss: 0.0072 - accuracy: 0.9980
1096/1688 [==================>...........] - ETA: 3s - loss: 0.0072 - accuracy: 0.9980
1105/1688 [==================>...........] - ETA: 3s - loss: 0.0072 - accuracy: 0.9980
1114/1688 [==================>...........] - ETA: 3s - loss: 0.0071 - accuracy: 0.9981
1123/1688 [==================>...........] - ETA: 3s - loss: 0.0071 - accuracy: 0.9981
1132/1688 [===================>..........] - ETA: 3s - loss: 0.0071 - accuracy: 0.9981
1141/1688 [===================>..........] - ETA: 3s - loss: 0.0071 - accuracy: 0.9981
1150/1688 [===================>..........] - ETA: 3s - loss: 0.0071 - accuracy: 0.9981
1159/1688 [===================>..........] - ETA: 3s - loss: 0.0072 - accuracy: 0.9980
1168/1688 [===================>..........] - ETA: 3s - loss: 0.0072 - accuracy: 0.9980
1177/1688 [===================>..........] - ETA: 3s - loss: 0.0072 - accuracy: 0.9980
1186/1688 [====================>.........] - ETA: 3s - loss: 0.0072 - accuracy: 0.9980
1195/1688 [====================>.........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1204/1688 [====================>.........] - ETA: 2s - loss: 0.0071 - accuracy: 0.9981
1213/1688 [====================>.........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1222/1688 [====================>.........] - ETA: 2s - loss: 0.0071 - accuracy: 0.9981
1231/1688 [====================>.........] - ETA: 2s - loss: 0.0071 - accuracy: 0.9981
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0071 - accuracy: 0.9981
1248/1688 [=====================>........] - ETA: 2s - loss: 0.0070 - accuracy: 0.9981
1257/1688 [=====================>........] - ETA: 2s - loss: 0.0071 - accuracy: 0.9981
1266/1688 [=====================>........] - ETA: 2s - loss: 0.0071 - accuracy: 0.9980
1274/1688 [=====================>........] - ETA: 2s - loss: 0.0071 - accuracy: 0.9981
1283/1688 [=====================>........] - ETA: 2s - loss: 0.0071 - accuracy: 0.9980
1292/1688 [=====================>........] - ETA: 2s - loss: 0.0071 - accuracy: 0.9980
1300/1688 [======================>.......] - ETA: 2s - loss: 0.0071 - accuracy: 0.9980
1308/1688 [======================>.......] - ETA: 2s - loss: 0.0071 - accuracy: 0.9980
1317/1688 [======================>.......] - ETA: 2s - loss: 0.0071 - accuracy: 0.9981
1326/1688 [======================>.......] - ETA: 2s - loss: 0.0070 - accuracy: 0.9981
1335/1688 [======================>.......] - ETA: 2s - loss: 0.0070 - accuracy: 0.9981
1344/1688 [======================>.......] - ETA: 2s - loss: 0.0070 - accuracy: 0.9980
1353/1688 [=======================>......] - ETA: 2s - loss: 0.0070 - accuracy: 0.9981
1362/1688 [=======================>......] - ETA: 1s - loss: 0.0070 - accuracy: 0.9981
1371/1688 [=======================>......] - ETA: 1s - loss: 0.0070 - accuracy: 0.9981
1380/1688 [=======================>......] - ETA: 1s - loss: 0.0070 - accuracy: 0.9980
1389/1688 [=======================>......] - ETA: 1s - loss: 0.0070 - accuracy: 0.9980
1398/1688 [=======================>......] - ETA: 1s - loss: 0.0070 - accuracy: 0.9981
1407/1688 [========================>.....] - ETA: 1s - loss: 0.0070 - accuracy: 0.9981
1416/1688 [========================>.....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1424/1688 [========================>.....] - ETA: 1s - loss: 0.0070 - accuracy: 0.9981
1433/1688 [========================>.....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1442/1688 [========================>.....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1451/1688 [========================>.....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1460/1688 [========================>.....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1469/1688 [=========================>....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1478/1688 [=========================>....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1487/1688 [=========================>....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1496/1688 [=========================>....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1505/1688 [=========================>....] - ETA: 1s - loss: 0.0068 - accuracy: 0.9981
1513/1688 [=========================>....] - ETA: 1s - loss: 0.0068 - accuracy: 0.9981
1522/1688 [==========================>...] - ETA: 1s - loss: 0.0069 - accuracy: 0.9980
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0069 - accuracy: 0.9981
1612/1688 [===========================>..] - ETA: 0s - loss: 0.0068 - accuracy: 0.9981
1621/1688 [===========================>..] - ETA: 0s - loss: 0.0068 - accuracy: 0.9981
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0068 - accuracy: 0.9980
1638/1688 [============================>.] - ETA: 0s - loss: 0.0068 - accuracy: 0.9980
1647/1688 [============================>.] - ETA: 0s - loss: 0.0068 - accuracy: 0.9980
1656/1688 [============================>.] - ETA: 0s - loss: 0.0068 - accuracy: 0.9980
1664/1688 [============================>.] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1673/1688 [============================>.] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1682/1688 [============================>.] - ETA: 0s - loss: 0.0068 - accuracy: 0.9980
1688/1688 [==============================] - 11s 6ms/step - loss: 0.0068 - accuracy: 0.9980 - val_loss: 0.0529 - val_accuracy: 0.9887
Epoch 3/5
1/1688 [..............................] - ETA: 10s - loss: 0.0227 - accuracy: 1.0000
10/1688 [..............................] - ETA: 10s - loss: 0.0072 - accuracy: 0.9969
19/1688 [..............................] - ETA: 10s - loss: 0.0063 - accuracy: 0.9984
28/1688 [..............................] - ETA: 10s - loss: 0.0066 - accuracy: 0.9989
37/1688 [..............................] - ETA: 9s - loss: 0.0063 - accuracy: 0.9992
46/1688 [..............................] - ETA: 9s - loss: 0.0054 - accuracy: 0.9993
55/1688 [..............................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9994
64/1688 [>.............................] - ETA: 9s - loss: 0.0052 - accuracy: 0.9995
73/1688 [>.............................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9996
82/1688 [>.............................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9989
91/1688 [>.............................] - ETA: 9s - loss: 0.0054 - accuracy: 0.9986
100/1688 [>.............................] - ETA: 9s - loss: 0.0052 - accuracy: 0.9987
109/1688 [>.............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9989
118/1688 [=>............................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9989
127/1688 [=>............................] - ETA: 9s - loss: 0.0052 - accuracy: 0.9988
136/1688 [=>............................] - ETA: 9s - loss: 0.0052 - accuracy: 0.9989
145/1688 [=>............................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9989
154/1688 [=>............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9988
163/1688 [=>............................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9988
172/1688 [==>...........................] - ETA: 9s - loss: 0.0054 - accuracy: 0.9987
181/1688 [==>...........................] - ETA: 9s - loss: 0.0058 - accuracy: 0.9984
190/1688 [==>...........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9985
198/1688 [==>...........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9984
207/1688 [==>...........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9983
216/1688 [==>...........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9984
225/1688 [==>...........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9983
234/1688 [===>..........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9984
243/1688 [===>..........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9983
252/1688 [===>..........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9984
261/1688 [===>..........................] - ETA: 8s - loss: 0.0058 - accuracy: 0.9984
270/1688 [===>..........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9985
279/1688 [===>..........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9985
288/1688 [====>.........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9986
297/1688 [====>.........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9985
306/1688 [====>.........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9986
315/1688 [====>.........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9986
323/1688 [====>.........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9986
332/1688 [====>.........................] - ETA: 8s - loss: 0.0055 - accuracy: 0.9987
341/1688 [=====>........................] - ETA: 8s - loss: 0.0055 - accuracy: 0.9987
350/1688 [=====>........................] - ETA: 8s - loss: 0.0055 - accuracy: 0.9987
359/1688 [=====>........................] - ETA: 7s - loss: 0.0057 - accuracy: 0.9987
368/1688 [=====>........................] - ETA: 7s - loss: 0.0058 - accuracy: 0.9986
377/1688 [=====>........................] - ETA: 7s - loss: 0.0058 - accuracy: 0.9987
386/1688 [=====>........................] - ETA: 7s - loss: 0.0057 - accuracy: 0.9987
394/1688 [======>.......................] - ETA: 7s - loss: 0.0058 - accuracy: 0.9987
403/1688 [======>.......................] - ETA: 7s - loss: 0.0057 - accuracy: 0.9987
412/1688 [======>.......................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9987
421/1688 [======>.......................] - ETA: 7s - loss: 0.0057 - accuracy: 0.9987
430/1688 [======>.......................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9987
439/1688 [======>.......................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9987
448/1688 [======>.......................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9987
457/1688 [=======>......................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9988
466/1688 [=======>......................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9987
475/1688 [=======>......................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9987
484/1688 [=======>......................] - ETA: 7s - loss: 0.0057 - accuracy: 0.9986
493/1688 [=======>......................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9987
502/1688 [=======>......................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9987
511/1688 [========>.....................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9987
520/1688 [========>.....................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9987
528/1688 [========>.....................] - ETA: 6s - loss: 0.0056 - accuracy: 0.9987
537/1688 [========>.....................] - ETA: 6s - loss: 0.0056 - accuracy: 0.9987
546/1688 [========>.....................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9987
555/1688 [========>.....................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9988
564/1688 [=========>....................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9988
573/1688 [=========>....................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9987
582/1688 [=========>....................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9988
591/1688 [=========>....................] - ETA: 6s - loss: 0.0055 - accuracy: 0.9988
600/1688 [=========>....................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9988
609/1688 [=========>....................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9988
618/1688 [=========>....................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9988
627/1688 [==========>...................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9988
636/1688 [==========>...................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9988
645/1688 [==========>...................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9988
654/1688 [==========>...................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9989
663/1688 [==========>...................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9989
672/1688 [==========>...................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9989
681/1688 [===========>..................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9989
690/1688 [===========>..................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9989
699/1688 [===========>..................] - ETA: 5s - loss: 0.0053 - accuracy: 0.9989
708/1688 [===========>..................] - ETA: 5s - loss: 0.0053 - accuracy: 0.9989
717/1688 [===========>..................] - ETA: 5s - loss: 0.0053 - accuracy: 0.9989
726/1688 [===========>..................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9988
735/1688 [============>.................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9988
743/1688 [============>.................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
752/1688 [============>.................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9988
761/1688 [============>.................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
770/1688 [============>.................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
779/1688 [============>.................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
788/1688 [=============>................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
797/1688 [=============>................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
806/1688 [=============>................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
815/1688 [=============>................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
824/1688 [=============>................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
833/1688 [=============>................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
842/1688 [=============>................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9987
851/1688 [==============>...............] - ETA: 5s - loss: 0.0055 - accuracy: 0.9986
860/1688 [==============>...............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9986
869/1688 [==============>...............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9986
878/1688 [==============>...............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9986
887/1688 [==============>...............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
896/1688 [==============>...............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
905/1688 [===============>..............] - ETA: 4s - loss: 0.0056 - accuracy: 0.9987
914/1688 [===============>..............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
923/1688 [===============>..............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
932/1688 [===============>..............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
940/1688 [===============>..............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
949/1688 [===============>..............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
958/1688 [================>.............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
967/1688 [================>.............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
976/1688 [================>.............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
985/1688 [================>.............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
994/1688 [================>.............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
1003/1688 [================>.............] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
1012/1688 [================>.............] - ETA: 4s - loss: 0.0056 - accuracy: 0.9986
1020/1688 [=================>............] - ETA: 4s - loss: 0.0056 - accuracy: 0.9986
1029/1688 [=================>............] - ETA: 3s - loss: 0.0057 - accuracy: 0.9986
1038/1688 [=================>............] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1047/1688 [=================>............] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1056/1688 [=================>............] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1065/1688 [=================>............] - ETA: 3s - loss: 0.0056 - accuracy: 0.9987
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1083/1688 [==================>...........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1092/1688 [==================>...........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1101/1688 [==================>...........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1110/1688 [==================>...........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1119/1688 [==================>...........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1128/1688 [===================>..........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1137/1688 [===================>..........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9987
1146/1688 [===================>..........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1155/1688 [===================>..........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1164/1688 [===================>..........] - ETA: 3s - loss: 0.0056 - accuracy: 0.9987
1173/1688 [===================>..........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9987
1181/1688 [===================>..........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9987
1190/1688 [====================>.........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9987
1199/1688 [====================>.........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9986
1208/1688 [====================>.........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9987
1217/1688 [====================>.........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9986
1225/1688 [====================>.........] - ETA: 2s - loss: 0.0058 - accuracy: 0.9986
1234/1688 [====================>.........] - ETA: 2s - loss: 0.0058 - accuracy: 0.9986
1243/1688 [=====================>........] - ETA: 2s - loss: 0.0058 - accuracy: 0.9986
1252/1688 [=====================>........] - ETA: 2s - loss: 0.0058 - accuracy: 0.9986
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9986
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9986
1279/1688 [=====================>........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9986
1287/1688 [=====================>........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9986
1296/1688 [======================>.......] - ETA: 2s - loss: 0.0057 - accuracy: 0.9986
1305/1688 [======================>.......] - ETA: 2s - loss: 0.0057 - accuracy: 0.9987
1314/1688 [======================>.......] - ETA: 2s - loss: 0.0056 - accuracy: 0.9987
1323/1688 [======================>.......] - ETA: 2s - loss: 0.0056 - accuracy: 0.9987
1332/1688 [======================>.......] - ETA: 2s - loss: 0.0056 - accuracy: 0.9987
1341/1688 [======================>.......] - ETA: 2s - loss: 0.0056 - accuracy: 0.9987
1349/1688 [======================>.......] - ETA: 2s - loss: 0.0055 - accuracy: 0.9987
1357/1688 [=======================>......] - ETA: 2s - loss: 0.0055 - accuracy: 0.9987
1366/1688 [=======================>......] - ETA: 1s - loss: 0.0055 - accuracy: 0.9987
1375/1688 [=======================>......] - ETA: 1s - loss: 0.0055 - accuracy: 0.9987
1384/1688 [=======================>......] - ETA: 1s - loss: 0.0055 - accuracy: 0.9987
1393/1688 [=======================>......] - ETA: 1s - loss: 0.0055 - accuracy: 0.9987
1402/1688 [=======================>......] - ETA: 1s - loss: 0.0055 - accuracy: 0.9987
1411/1688 [========================>.....] - ETA: 1s - loss: 0.0055 - accuracy: 0.9987
1420/1688 [========================>.....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9987
1429/1688 [========================>.....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9987
1438/1688 [========================>.....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9987
1447/1688 [========================>.....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9987
1456/1688 [========================>.....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9987
1465/1688 [=========================>....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9987
1474/1688 [=========================>....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9987
1483/1688 [=========================>....] - ETA: 1s - loss: 0.0057 - accuracy: 0.9986
1492/1688 [=========================>....] - ETA: 1s - loss: 0.0057 - accuracy: 0.9986
1501/1688 [=========================>....] - ETA: 1s - loss: 0.0058 - accuracy: 0.9986
1510/1688 [=========================>....] - ETA: 1s - loss: 0.0057 - accuracy: 0.9986
1519/1688 [=========================>....] - ETA: 1s - loss: 0.0057 - accuracy: 0.9986
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0057 - accuracy: 0.9986
1537/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9986
1546/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9985
1554/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9985
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9985
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0058 - accuracy: 0.9985
1581/1688 [===========================>..] - ETA: 0s - loss: 0.0057 - accuracy: 0.9985
1590/1688 [===========================>..] - ETA: 0s - loss: 0.0058 - accuracy: 0.9985
1599/1688 [===========================>..] - ETA: 0s - loss: 0.0058 - accuracy: 0.9985
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0057 - accuracy: 0.9985
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0057 - accuracy: 0.9985
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0058 - accuracy: 0.9985
1634/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9985
1643/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9985
1652/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9985
1661/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9985
1670/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9984
1679/1688 [============================>.] - ETA: 0s - loss: 0.0058 - accuracy: 0.9985
1688/1688 [==============================] - ETA: 0s - loss: 0.0058 - accuracy: 0.9984
1688/1688 [==============================] - 11s 6ms/step - loss: 0.0058 - accuracy: 0.9984 - val_loss: 0.0586 - val_accuracy: 0.9892
Epoch 4/5
1/1688 [..............................] - ETA: 11s - loss: 2.1857e-04 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0025 - accuracy: 1.0000
18/1688 [..............................] - ETA: 10s - loss: 0.0028 - accuracy: 1.0000
27/1688 [..............................] - ETA: 10s - loss: 0.0056 - accuracy: 0.9988
36/1688 [..............................] - ETA: 10s - loss: 0.0054 - accuracy: 0.9991
45/1688 [..............................] - ETA: 9s - loss: 0.0059 - accuracy: 0.9986
54/1688 [..............................] - ETA: 9s - loss: 0.0055 - accuracy: 0.9988
63/1688 [>.............................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9990
72/1688 [>.............................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9991
81/1688 [>.............................] - ETA: 9s - loss: 0.0046 - accuracy: 0.9992
89/1688 [>.............................] - ETA: 9s - loss: 0.0045 - accuracy: 0.9993
98/1688 [>.............................] - ETA: 9s - loss: 0.0045 - accuracy: 0.9990
107/1688 [>.............................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9988
116/1688 [=>............................] - ETA: 9s - loss: 0.0047 - accuracy: 0.9989
125/1688 [=>............................] - ETA: 9s - loss: 0.0047 - accuracy: 0.9990
134/1688 [=>............................] - ETA: 9s - loss: 0.0046 - accuracy: 0.9991
143/1688 [=>............................] - ETA: 9s - loss: 0.0046 - accuracy: 0.9991
152/1688 [=>............................] - ETA: 9s - loss: 0.0046 - accuracy: 0.9992
161/1688 [=>............................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9990
170/1688 [==>...........................] - ETA: 9s - loss: 0.0047 - accuracy: 0.9991
179/1688 [==>...........................] - ETA: 9s - loss: 0.0046 - accuracy: 0.9991
188/1688 [==>...........................] - ETA: 9s - loss: 0.0045 - accuracy: 0.9992
197/1688 [==>...........................] - ETA: 9s - loss: 0.0046 - accuracy: 0.9992
206/1688 [==>...........................] - ETA: 8s - loss: 0.0045 - accuracy: 0.9992
215/1688 [==>...........................] - ETA: 8s - loss: 0.0045 - accuracy: 0.9993
224/1688 [==>...........................] - ETA: 8s - loss: 0.0044 - accuracy: 0.9993
233/1688 [===>..........................] - ETA: 8s - loss: 0.0044 - accuracy: 0.9993
242/1688 [===>..........................] - ETA: 8s - loss: 0.0044 - accuracy: 0.9992
251/1688 [===>..........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9993
260/1688 [===>..........................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9993
269/1688 [===>..........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9993
277/1688 [===>..........................] - ETA: 8s - loss: 0.0044 - accuracy: 0.9991
286/1688 [====>.........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9991
294/1688 [====>.........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9991
303/1688 [====>.........................] - ETA: 8s - loss: 0.0044 - accuracy: 0.9991
312/1688 [====>.........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9991
321/1688 [====>.........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9991
330/1688 [====>.........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9991
339/1688 [=====>........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9992
348/1688 [=====>........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9991
357/1688 [=====>........................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9991
366/1688 [=====>........................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9991
375/1688 [=====>........................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9990
384/1688 [=====>........................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9989
393/1688 [=====>........................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9990
402/1688 [======>.......................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9990
411/1688 [======>.......................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9989
419/1688 [======>.......................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9990
428/1688 [======>.......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9990
436/1688 [======>.......................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9989
445/1688 [======>.......................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9989
454/1688 [=======>......................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9989
463/1688 [=======>......................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9989
472/1688 [=======>......................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9989
480/1688 [=======>......................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9990
489/1688 [=======>......................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9990
498/1688 [=======>......................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9990
507/1688 [========>.....................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9990
515/1688 [========>.....................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9990
524/1688 [========>.....................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9990
533/1688 [========>.....................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9991
542/1688 [========>.....................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9991
551/1688 [========>.....................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9991
560/1688 [========>.....................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9991
569/1688 [=========>....................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9991
577/1688 [=========>....................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9991
586/1688 [=========>....................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9991
595/1688 [=========>....................] - ETA: 6s - loss: 0.0043 - accuracy: 0.9991
604/1688 [=========>....................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9991
613/1688 [=========>....................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9991
622/1688 [==========>...................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9990
631/1688 [==========>...................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9991
640/1688 [==========>...................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9991
649/1688 [==========>...................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9991
658/1688 [==========>...................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9991
667/1688 [==========>...................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9991
676/1688 [===========>..................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9991
685/1688 [===========>..................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9991
694/1688 [===========>..................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9991
703/1688 [===========>..................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
712/1688 [===========>..................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
721/1688 [===========>..................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
730/1688 [===========>..................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
739/1688 [============>.................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9991
747/1688 [============>.................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9990
756/1688 [============>.................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9990
765/1688 [============>.................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9990
774/1688 [============>.................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9990
783/1688 [============>.................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9990
792/1688 [=============>................] - ETA: 5s - loss: 0.0047 - accuracy: 0.9990
801/1688 [=============>................] - ETA: 5s - loss: 0.0047 - accuracy: 0.9990
810/1688 [=============>................] - ETA: 5s - loss: 0.0047 - accuracy: 0.9990
819/1688 [=============>................] - ETA: 5s - loss: 0.0047 - accuracy: 0.9990
827/1688 [=============>................] - ETA: 5s - loss: 0.0047 - accuracy: 0.9990
835/1688 [=============>................] - ETA: 5s - loss: 0.0047 - accuracy: 0.9990
844/1688 [==============>...............] - ETA: 5s - loss: 0.0048 - accuracy: 0.9990
852/1688 [==============>...............] - ETA: 5s - loss: 0.0048 - accuracy: 0.9990
861/1688 [==============>...............] - ETA: 5s - loss: 0.0048 - accuracy: 0.9990
870/1688 [==============>...............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
879/1688 [==============>...............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
888/1688 [==============>...............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
897/1688 [==============>...............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
905/1688 [===============>..............] - ETA: 4s - loss: 0.0048 - accuracy: 0.9990
914/1688 [===============>..............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
923/1688 [===============>..............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
931/1688 [===============>..............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
940/1688 [===============>..............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
949/1688 [===============>..............] - ETA: 4s - loss: 0.0047 - accuracy: 0.9990
958/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
967/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
976/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
985/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
993/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
1002/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
1011/1688 [================>.............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
1019/1688 [=================>............] - ETA: 4s - loss: 0.0046 - accuracy: 0.9991
1028/1688 [=================>............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1036/1688 [=================>............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1045/1688 [=================>............] - ETA: 3s - loss: 0.0047 - accuracy: 0.9990
1054/1688 [=================>............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1063/1688 [=================>............] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1072/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1080/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1088/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1097/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1106/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1115/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1124/1688 [==================>...........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1133/1688 [===================>..........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1141/1688 [===================>..........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1150/1688 [===================>..........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1159/1688 [===================>..........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1168/1688 [===================>..........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1177/1688 [===================>..........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1186/1688 [====================>.........] - ETA: 3s - loss: 0.0046 - accuracy: 0.9991
1195/1688 [====================>.........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1204/1688 [====================>.........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1213/1688 [====================>.........] - ETA: 2s - loss: 0.0046 - accuracy: 0.9990
1222/1688 [====================>.........] - ETA: 2s - loss: 0.0046 - accuracy: 0.9990
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1247/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1255/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1263/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1272/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1290/1688 [=====================>........] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1299/1688 [======================>.......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9990
1308/1688 [======================>.......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9990
1317/1688 [======================>.......] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1326/1688 [======================>.......] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1335/1688 [======================>.......] - ETA: 2s - loss: 0.0047 - accuracy: 0.9990
1344/1688 [======================>.......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9990
1353/1688 [=======================>......] - ETA: 2s - loss: 0.0046 - accuracy: 0.9990
1362/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1371/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1380/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1388/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1397/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1406/1688 [=======================>......] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1415/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1424/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1433/1688 [========================>.....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1442/1688 [========================>.....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9990
1450/1688 [========================>.....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9991
1459/1688 [========================>.....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1468/1688 [=========================>....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1477/1688 [=========================>....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9990
1486/1688 [=========================>....] - ETA: 1s - loss: 0.0046 - accuracy: 0.9991
1495/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9991
1504/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9991
1513/1688 [=========================>....] - ETA: 1s - loss: 0.0045 - accuracy: 0.9991
1522/1688 [==========================>...] - ETA: 1s - loss: 0.0045 - accuracy: 0.9991
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0045 - accuracy: 0.9991
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0045 - accuracy: 0.9991
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0045 - accuracy: 0.9991
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0046 - accuracy: 0.9991
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0046 - accuracy: 0.9991
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0046 - accuracy: 0.9991
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0046 - accuracy: 0.9991
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0046 - accuracy: 0.9991
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0047 - accuracy: 0.9990
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0047 - accuracy: 0.9990
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0047 - accuracy: 0.9990
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0047 - accuracy: 0.9990
1638/1688 [============================>.] - ETA: 0s - loss: 0.0047 - accuracy: 0.9990
1647/1688 [============================>.] - ETA: 0s - loss: 0.0047 - accuracy: 0.9990
1656/1688 [============================>.] - ETA: 0s - loss: 0.0047 - accuracy: 0.9990
1665/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1674/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1683/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9991
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0046 - accuracy: 0.9991 - val_loss: 0.0551 - val_accuracy: 0.9893
Epoch 5/5
1/1688 [..............................] - ETA: 10s - loss: 4.5975e-04 - accuracy: 1.0000
9/1688 [..............................] - ETA: 11s - loss: 0.0017 - accuracy: 1.0000
18/1688 [..............................] - ETA: 10s - loss: 0.0037 - accuracy: 1.0000
27/1688 [..............................] - ETA: 10s - loss: 0.0041 - accuracy: 1.0000
36/1688 [..............................] - ETA: 10s - loss: 0.0034 - accuracy: 1.0000
45/1688 [..............................] - ETA: 10s - loss: 0.0041 - accuracy: 0.9993
54/1688 [..............................] - ETA: 9s - loss: 0.0038 - accuracy: 0.9994
63/1688 [>.............................] - ETA: 9s - loss: 0.0036 - accuracy: 0.9995
72/1688 [>.............................] - ETA: 9s - loss: 0.0034 - accuracy: 0.9996
81/1688 [>.............................] - ETA: 9s - loss: 0.0037 - accuracy: 0.9992
90/1688 [>.............................] - ETA: 9s - loss: 0.0036 - accuracy: 0.9993
99/1688 [>.............................] - ETA: 9s - loss: 0.0037 - accuracy: 0.9994
108/1688 [>.............................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9994
117/1688 [=>............................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9995
126/1688 [=>............................] - ETA: 9s - loss: 0.0038 - accuracy: 0.9995
135/1688 [=>............................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9993
144/1688 [=>............................] - ETA: 9s - loss: 0.0038 - accuracy: 0.9993
153/1688 [=>............................] - ETA: 9s - loss: 0.0037 - accuracy: 0.9994
162/1688 [=>............................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9992
171/1688 [==>...........................] - ETA: 9s - loss: 0.0038 - accuracy: 0.9993
180/1688 [==>...........................] - ETA: 9s - loss: 0.0037 - accuracy: 0.9993
188/1688 [==>...........................] - ETA: 9s - loss: 0.0036 - accuracy: 0.9993
197/1688 [==>...........................] - ETA: 9s - loss: 0.0036 - accuracy: 0.9994
206/1688 [==>...........................] - ETA: 8s - loss: 0.0036 - accuracy: 0.9994
215/1688 [==>...........................] - ETA: 8s - loss: 0.0039 - accuracy: 0.9993
224/1688 [==>...........................] - ETA: 8s - loss: 0.0038 - accuracy: 0.9993
233/1688 [===>..........................] - ETA: 8s - loss: 0.0037 - accuracy: 0.9993
242/1688 [===>..........................] - ETA: 8s - loss: 0.0037 - accuracy: 0.9994
251/1688 [===>..........................] - ETA: 8s - loss: 0.0037 - accuracy: 0.9994
260/1688 [===>..........................] - ETA: 8s - loss: 0.0036 - accuracy: 0.9994
269/1688 [===>..........................] - ETA: 8s - loss: 0.0035 - accuracy: 0.9994
278/1688 [===>..........................] - ETA: 8s - loss: 0.0035 - accuracy: 0.9994
286/1688 [====>.........................] - ETA: 8s - loss: 0.0035 - accuracy: 0.9995
294/1688 [====>.........................] - ETA: 8s - loss: 0.0035 - accuracy: 0.9995
302/1688 [====>.........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9995
311/1688 [====>.........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9995
320/1688 [====>.........................] - ETA: 8s - loss: 0.0033 - accuracy: 0.9995
329/1688 [====>.........................] - ETA: 8s - loss: 0.0032 - accuracy: 0.9995
338/1688 [=====>........................] - ETA: 8s - loss: 0.0032 - accuracy: 0.9995
347/1688 [=====>........................] - ETA: 8s - loss: 0.0032 - accuracy: 0.9995
355/1688 [=====>........................] - ETA: 8s - loss: 0.0032 - accuracy: 0.9996
364/1688 [=====>........................] - ETA: 8s - loss: 0.0032 - accuracy: 0.9996
373/1688 [=====>........................] - ETA: 7s - loss: 0.0032 - accuracy: 0.9996
382/1688 [=====>........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9995
391/1688 [=====>........................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9995
400/1688 [======>.......................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9995
408/1688 [======>.......................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9995
417/1688 [======>.......................] - ETA: 7s - loss: 0.0032 - accuracy: 0.9996
426/1688 [======>.......................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9995
434/1688 [======>.......................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9995
443/1688 [======>.......................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9995
452/1688 [=======>......................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9994
461/1688 [=======>......................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9993
470/1688 [=======>......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9993
479/1688 [=======>......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9993
488/1688 [=======>......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9993
496/1688 [=======>......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9993
505/1688 [=======>......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9993
514/1688 [========>.....................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9993
523/1688 [========>.....................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9993
532/1688 [========>.....................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9993
541/1688 [========>.....................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9993
550/1688 [========>.....................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9993
559/1688 [========>.....................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9993
568/1688 [=========>....................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9993
577/1688 [=========>....................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
585/1688 [=========>....................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9992
593/1688 [=========>....................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9992
602/1688 [=========>....................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9992
611/1688 [=========>....................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9991
620/1688 [==========>...................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9991
629/1688 [==========>...................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9992
637/1688 [==========>...................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9992
646/1688 [==========>...................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9991
655/1688 [==========>...................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9990
664/1688 [==========>...................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9991
673/1688 [==========>...................] - ETA: 6s - loss: 0.0042 - accuracy: 0.9991
682/1688 [===========>..................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9991
691/1688 [===========>..................] - ETA: 6s - loss: 0.0041 - accuracy: 0.9991
700/1688 [===========>..................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9991
709/1688 [===========>..................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9991
718/1688 [===========>..................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
727/1688 [===========>..................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
736/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9992
745/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9992
754/1688 [============>.................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
763/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
772/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
781/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9992
790/1688 [=============>................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9992
799/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
808/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
817/1688 [=============>................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9992
826/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9992
835/1688 [=============>................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9992
843/1688 [=============>................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9992
852/1688 [==============>...............] - ETA: 5s - loss: 0.0040 - accuracy: 0.9992
861/1688 [==============>...............] - ETA: 5s - loss: 0.0040 - accuracy: 0.9992
870/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
879/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
888/1688 [==============>...............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9992
897/1688 [==============>...............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9992
906/1688 [===============>..............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9992
915/1688 [===============>..............] - ETA: 4s - loss: 0.0042 - accuracy: 0.9991
924/1688 [===============>..............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9992
932/1688 [===============>..............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9992
941/1688 [===============>..............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9992
950/1688 [===============>..............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
959/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
968/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
977/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
985/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
994/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
1003/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
1012/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
1021/1688 [=================>............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
1030/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9992
1039/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9992
1047/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9993
1056/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9993
1065/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9992
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9992
1083/1688 [==================>...........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9992
1092/1688 [==================>...........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9993
1101/1688 [==================>...........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9993
1110/1688 [==================>...........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9992
1119/1688 [==================>...........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9992
1128/1688 [===================>..........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9993
1137/1688 [===================>..........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9993
1145/1688 [===================>..........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9993
1154/1688 [===================>..........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9993
1163/1688 [===================>..........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9993
1172/1688 [===================>..........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9993
1181/1688 [===================>..........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9993
1190/1688 [====================>.........] - ETA: 3s - loss: 0.0040 - accuracy: 0.9992
1199/1688 [====================>.........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9992
1208/1688 [====================>.........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9992
1217/1688 [====================>.........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9992
1226/1688 [====================>.........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9992
1235/1688 [====================>.........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9992
1244/1688 [=====================>........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9992
1252/1688 [=====================>........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9992
1260/1688 [=====================>........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9992
1269/1688 [=====================>........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9992
1277/1688 [=====================>........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9992
1286/1688 [=====================>........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9992
1295/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9993
1304/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9993
1313/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9993
1322/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9993
1331/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9993
1340/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9993
1349/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9993
1358/1688 [=======================>......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9993
1367/1688 [=======================>......] - ETA: 1s - loss: 0.0041 - accuracy: 0.9993
1376/1688 [=======================>......] - ETA: 1s - loss: 0.0041 - accuracy: 0.9993
1385/1688 [=======================>......] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1394/1688 [=======================>......] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1403/1688 [=======================>......] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1420/1688 [========================>.....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1429/1688 [========================>.....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1438/1688 [========================>.....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1447/1688 [========================>.....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1456/1688 [========================>.....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1465/1688 [=========================>....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1474/1688 [=========================>....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1483/1688 [=========================>....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1492/1688 [=========================>....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1501/1688 [=========================>....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1510/1688 [=========================>....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1519/1688 [=========================>....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9993
1527/1688 [==========================>...] - ETA: 0s - loss: 0.0041 - accuracy: 0.9993
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0041 - accuracy: 0.9993
1544/1688 [==========================>...] - ETA: 0s - loss: 0.0042 - accuracy: 0.9993
1553/1688 [==========================>...] - ETA: 0s - loss: 0.0042 - accuracy: 0.9993
1562/1688 [==========================>...] - ETA: 0s - loss: 0.0041 - accuracy: 0.9993
1571/1688 [==========================>...] - ETA: 0s - loss: 0.0042 - accuracy: 0.9993
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0042 - accuracy: 0.9993
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0042 - accuracy: 0.9993
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0041 - accuracy: 0.9993
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0042 - accuracy: 0.9993
1616/1688 [===========================>..] - ETA: 0s - loss: 0.0042 - accuracy: 0.9993
1625/1688 [===========================>..] - ETA: 0s - loss: 0.0041 - accuracy: 0.9993
1634/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9993
1643/1688 [============================>.] - ETA: 0s - loss: 0.0041 - accuracy: 0.9993
1652/1688 [============================>.] - ETA: 0s - loss: 0.0041 - accuracy: 0.9993
1661/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9993
1670/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9993
1679/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9993
1688/1688 [==============================] - ETA: 0s - loss: 0.0042 - accuracy: 0.9992
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0042 - accuracy: 0.9992 - val_loss: 0.0577 - val_accuracy: 0.9887
<tf_keras.src.callbacks.History object at 0x7dfed53261d0>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine-tuning:', score)
Test accuracy after fine-tuning: 0.9872000217437744
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with TF-Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test.astype(np.int32))
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.984499990940094
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[-14.591123 -9.837732 -5.383876 -0.2440347 -20.134254 -7.0385084
-25.732773 11.545014 -9.120562 -1.8651428]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 13.778 seconds)