Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a TF-Keras floating-point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original TF-Keras floating-point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf_keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import tensorflow as tf
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from tf_keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, height, width, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
32768/11490434 [..............................] - ETA: 23s
49152/11490434 [..............................] - ETA: 29s
81920/11490434 [..............................] - ETA: 29s
163840/11490434 [..............................] - ETA: 19s
245760/11490434 [..............................] - ETA: 15s
393216/11490434 [>.............................] - ETA: 11s
524288/11490434 [>.............................] - ETA: 9s
901120/11490434 [=>............................] - ETA: 5s
1343488/11490434 [==>...........................] - ETA: 4s
1851392/11490434 [===>..........................] - ETA: 3s
2891776/11490434 [======>.......................] - ETA: 1s
4104192/11490434 [=========>....................] - ETA: 1s
5881856/11490434 [==============>...............] - ETA: 0s
7634944/11490434 [==================>...........] - ETA: 0s
9420800/11490434 [=======================>......] - ETA: 0s
10944512/11490434 [===========================>..] - ETA: 0s
11490434/11490434 [==============================] - 1s 0us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard TF-Keras CNN model.
import tf_keras as keras
model_keras = keras.models.Sequential([
keras.layers.Input(shape=(28, 28, 1), name="input", dtype=tf.uint8),
keras.layers.Rescaling(1. / 255),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from tf_keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 1:09:12 - loss: 2.8717 - accuracy: 0.0938
22/1688 [..............................] - ETA: 4s - loss: 1.4577 - accuracy: 0.5384
43/1688 [..............................] - ETA: 3s - loss: 1.0636 - accuracy: 0.6628
65/1688 [>.............................] - ETA: 3s - loss: 0.8581 - accuracy: 0.7260
87/1688 [>.............................] - ETA: 3s - loss: 0.7381 - accuracy: 0.7654
110/1688 [>.............................] - ETA: 3s - loss: 0.6567 - accuracy: 0.7926
132/1688 [=>............................] - ETA: 3s - loss: 0.5977 - accuracy: 0.8118
154/1688 [=>............................] - ETA: 3s - loss: 0.5522 - accuracy: 0.8269
176/1688 [==>...........................] - ETA: 3s - loss: 0.5140 - accuracy: 0.8395
199/1688 [==>...........................] - ETA: 3s - loss: 0.4825 - accuracy: 0.8505
221/1688 [==>...........................] - ETA: 3s - loss: 0.4655 - accuracy: 0.8562
244/1688 [===>..........................] - ETA: 3s - loss: 0.4465 - accuracy: 0.8632
267/1688 [===>..........................] - ETA: 3s - loss: 0.4233 - accuracy: 0.8703
289/1688 [====>.........................] - ETA: 3s - loss: 0.4080 - accuracy: 0.8749
311/1688 [====>.........................] - ETA: 3s - loss: 0.3932 - accuracy: 0.8795
333/1688 [====>.........................] - ETA: 3s - loss: 0.3790 - accuracy: 0.8838
355/1688 [=====>........................] - ETA: 3s - loss: 0.3677 - accuracy: 0.8876
377/1688 [=====>........................] - ETA: 3s - loss: 0.3589 - accuracy: 0.8903
399/1688 [======>.......................] - ETA: 2s - loss: 0.3498 - accuracy: 0.8929
422/1688 [======>.......................] - ETA: 2s - loss: 0.3385 - accuracy: 0.8965
444/1688 [======>.......................] - ETA: 2s - loss: 0.3309 - accuracy: 0.8990
466/1688 [=======>......................] - ETA: 2s - loss: 0.3222 - accuracy: 0.9015
488/1688 [=======>......................] - ETA: 2s - loss: 0.3138 - accuracy: 0.9043
510/1688 [========>.....................] - ETA: 2s - loss: 0.3076 - accuracy: 0.9063
532/1688 [========>.....................] - ETA: 2s - loss: 0.3008 - accuracy: 0.9085
554/1688 [========>.....................] - ETA: 2s - loss: 0.2935 - accuracy: 0.9108
576/1688 [=========>....................] - ETA: 2s - loss: 0.2885 - accuracy: 0.9122
599/1688 [=========>....................] - ETA: 2s - loss: 0.2849 - accuracy: 0.9137
622/1688 [==========>...................] - ETA: 2s - loss: 0.2792 - accuracy: 0.9148
645/1688 [==========>...................] - ETA: 2s - loss: 0.2739 - accuracy: 0.9163
668/1688 [==========>...................] - ETA: 2s - loss: 0.2685 - accuracy: 0.9177
691/1688 [===========>..................] - ETA: 2s - loss: 0.2632 - accuracy: 0.9192
713/1688 [===========>..................] - ETA: 2s - loss: 0.2585 - accuracy: 0.9206
736/1688 [============>.................] - ETA: 2s - loss: 0.2559 - accuracy: 0.9217
758/1688 [============>.................] - ETA: 2s - loss: 0.2514 - accuracy: 0.9230
781/1688 [============>.................] - ETA: 2s - loss: 0.2472 - accuracy: 0.9244
803/1688 [=============>................] - ETA: 2s - loss: 0.2438 - accuracy: 0.9255
826/1688 [=============>................] - ETA: 1s - loss: 0.2400 - accuracy: 0.9266
849/1688 [==============>...............] - ETA: 1s - loss: 0.2362 - accuracy: 0.9279
871/1688 [==============>...............] - ETA: 1s - loss: 0.2339 - accuracy: 0.9289
894/1688 [==============>...............] - ETA: 1s - loss: 0.2303 - accuracy: 0.9298
917/1688 [===============>..............] - ETA: 1s - loss: 0.2277 - accuracy: 0.9307
939/1688 [===============>..............] - ETA: 1s - loss: 0.2253 - accuracy: 0.9312
961/1688 [================>.............] - ETA: 1s - loss: 0.2222 - accuracy: 0.9322
983/1688 [================>.............] - ETA: 1s - loss: 0.2194 - accuracy: 0.9331
1006/1688 [================>.............] - ETA: 1s - loss: 0.2162 - accuracy: 0.9341
1030/1688 [=================>............] - ETA: 1s - loss: 0.2133 - accuracy: 0.9349
1052/1688 [=================>............] - ETA: 1s - loss: 0.2108 - accuracy: 0.9356
1074/1688 [==================>...........] - ETA: 1s - loss: 0.2087 - accuracy: 0.9362
1096/1688 [==================>...........] - ETA: 1s - loss: 0.2060 - accuracy: 0.9370
1118/1688 [==================>...........] - ETA: 1s - loss: 0.2036 - accuracy: 0.9377
1140/1688 [===================>..........] - ETA: 1s - loss: 0.2012 - accuracy: 0.9385
1162/1688 [===================>..........] - ETA: 1s - loss: 0.1991 - accuracy: 0.9392
1184/1688 [====================>.........] - ETA: 1s - loss: 0.1968 - accuracy: 0.9398
1206/1688 [====================>.........] - ETA: 1s - loss: 0.1948 - accuracy: 0.9405
1229/1688 [====================>.........] - ETA: 1s - loss: 0.1922 - accuracy: 0.9412
1251/1688 [=====================>........] - ETA: 0s - loss: 0.1904 - accuracy: 0.9419
1274/1688 [=====================>........] - ETA: 0s - loss: 0.1884 - accuracy: 0.9425
1297/1688 [======================>.......] - ETA: 0s - loss: 0.1868 - accuracy: 0.9430
1319/1688 [======================>.......] - ETA: 0s - loss: 0.1852 - accuracy: 0.9436
1342/1688 [======================>.......] - ETA: 0s - loss: 0.1843 - accuracy: 0.9440
1364/1688 [=======================>......] - ETA: 0s - loss: 0.1829 - accuracy: 0.9444
1387/1688 [=======================>......] - ETA: 0s - loss: 0.1812 - accuracy: 0.9450
1410/1688 [========================>.....] - ETA: 0s - loss: 0.1797 - accuracy: 0.9454
1432/1688 [========================>.....] - ETA: 0s - loss: 0.1785 - accuracy: 0.9458
1454/1688 [========================>.....] - ETA: 0s - loss: 0.1773 - accuracy: 0.9462
1477/1688 [=========================>....] - ETA: 0s - loss: 0.1758 - accuracy: 0.9467
1500/1688 [=========================>....] - ETA: 0s - loss: 0.1749 - accuracy: 0.9470
1523/1688 [==========================>...] - ETA: 0s - loss: 0.1738 - accuracy: 0.9473
1546/1688 [==========================>...] - ETA: 0s - loss: 0.1728 - accuracy: 0.9476
1569/1688 [==========================>...] - ETA: 0s - loss: 0.1715 - accuracy: 0.9480
1591/1688 [===========================>..] - ETA: 0s - loss: 0.1704 - accuracy: 0.9484
1614/1688 [===========================>..] - ETA: 0s - loss: 0.1691 - accuracy: 0.9487
1637/1688 [============================>.] - ETA: 0s - loss: 0.1681 - accuracy: 0.9491
1659/1688 [============================>.] - ETA: 0s - loss: 0.1669 - accuracy: 0.9496
1682/1688 [============================>.] - ETA: 0s - loss: 0.1658 - accuracy: 0.9499
1688/1688 [==============================] - ETA: 0s - loss: 0.1655 - accuracy: 0.9501
1688/1688 [==============================] - 7s 3ms/step - loss: 0.1655 - accuracy: 0.9501 - val_loss: 0.0712 - val_accuracy: 0.9807
Epoch 2/10
1/1688 [..............................] - ETA: 4s - loss: 0.0422 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0375 - accuracy: 0.9932
46/1688 [..............................] - ETA: 3s - loss: 0.0452 - accuracy: 0.9857
68/1688 [>.............................] - ETA: 3s - loss: 0.0489 - accuracy: 0.9844
90/1688 [>.............................] - ETA: 3s - loss: 0.0517 - accuracy: 0.9837
112/1688 [>.............................] - ETA: 3s - loss: 0.0515 - accuracy: 0.9847
134/1688 [=>............................] - ETA: 3s - loss: 0.0563 - accuracy: 0.9837
157/1688 [=>............................] - ETA: 3s - loss: 0.0582 - accuracy: 0.9837
180/1688 [==>...........................] - ETA: 3s - loss: 0.0591 - accuracy: 0.9833
203/1688 [==>...........................] - ETA: 3s - loss: 0.0605 - accuracy: 0.9817
226/1688 [===>..........................] - ETA: 3s - loss: 0.0606 - accuracy: 0.9812
248/1688 [===>..........................] - ETA: 3s - loss: 0.0605 - accuracy: 0.9816
271/1688 [===>..........................] - ETA: 3s - loss: 0.0612 - accuracy: 0.9814
294/1688 [====>.........................] - ETA: 3s - loss: 0.0623 - accuracy: 0.9811
317/1688 [====>.........................] - ETA: 3s - loss: 0.0619 - accuracy: 0.9812
339/1688 [=====>........................] - ETA: 3s - loss: 0.0623 - accuracy: 0.9807
361/1688 [=====>........................] - ETA: 3s - loss: 0.0615 - accuracy: 0.9808
384/1688 [=====>........................] - ETA: 2s - loss: 0.0616 - accuracy: 0.9805
406/1688 [======>.......................] - ETA: 2s - loss: 0.0617 - accuracy: 0.9804
429/1688 [======>.......................] - ETA: 2s - loss: 0.0638 - accuracy: 0.9799
453/1688 [=======>......................] - ETA: 2s - loss: 0.0648 - accuracy: 0.9799
475/1688 [=======>......................] - ETA: 2s - loss: 0.0650 - accuracy: 0.9797
498/1688 [=======>......................] - ETA: 2s - loss: 0.0644 - accuracy: 0.9797
520/1688 [========>.....................] - ETA: 2s - loss: 0.0637 - accuracy: 0.9798
543/1688 [========>.....................] - ETA: 2s - loss: 0.0646 - accuracy: 0.9796
566/1688 [=========>....................] - ETA: 2s - loss: 0.0654 - accuracy: 0.9794
588/1688 [=========>....................] - ETA: 2s - loss: 0.0650 - accuracy: 0.9794
610/1688 [=========>....................] - ETA: 2s - loss: 0.0654 - accuracy: 0.9795
632/1688 [==========>...................] - ETA: 2s - loss: 0.0648 - accuracy: 0.9796
654/1688 [==========>...................] - ETA: 2s - loss: 0.0648 - accuracy: 0.9796
677/1688 [===========>..................] - ETA: 2s - loss: 0.0641 - accuracy: 0.9796
699/1688 [===========>..................] - ETA: 2s - loss: 0.0644 - accuracy: 0.9795
722/1688 [===========>..................] - ETA: 2s - loss: 0.0645 - accuracy: 0.9794
745/1688 [============>.................] - ETA: 2s - loss: 0.0667 - accuracy: 0.9788
768/1688 [============>.................] - ETA: 2s - loss: 0.0681 - accuracy: 0.9786
790/1688 [=============>................] - ETA: 2s - loss: 0.0689 - accuracy: 0.9784
813/1688 [=============>................] - ETA: 1s - loss: 0.0689 - accuracy: 0.9783
835/1688 [=============>................] - ETA: 1s - loss: 0.0691 - accuracy: 0.9781
857/1688 [==============>...............] - ETA: 1s - loss: 0.0696 - accuracy: 0.9781
880/1688 [==============>...............] - ETA: 1s - loss: 0.0704 - accuracy: 0.9778
903/1688 [===============>..............] - ETA: 1s - loss: 0.0707 - accuracy: 0.9777
926/1688 [===============>..............] - ETA: 1s - loss: 0.0705 - accuracy: 0.9777
948/1688 [===============>..............] - ETA: 1s - loss: 0.0699 - accuracy: 0.9779
970/1688 [================>.............] - ETA: 1s - loss: 0.0697 - accuracy: 0.9779
993/1688 [================>.............] - ETA: 1s - loss: 0.0691 - accuracy: 0.9782
1016/1688 [=================>............] - ETA: 1s - loss: 0.0691 - accuracy: 0.9782
1038/1688 [=================>............] - ETA: 1s - loss: 0.0698 - accuracy: 0.9781
1060/1688 [=================>............] - ETA: 1s - loss: 0.0697 - accuracy: 0.9781
1082/1688 [==================>...........] - ETA: 1s - loss: 0.0691 - accuracy: 0.9783
1104/1688 [==================>...........] - ETA: 1s - loss: 0.0694 - accuracy: 0.9783
1127/1688 [===================>..........] - ETA: 1s - loss: 0.0689 - accuracy: 0.9784
1149/1688 [===================>..........] - ETA: 1s - loss: 0.0693 - accuracy: 0.9783
1171/1688 [===================>..........] - ETA: 1s - loss: 0.0690 - accuracy: 0.9783
1193/1688 [====================>.........] - ETA: 1s - loss: 0.0692 - accuracy: 0.9782
1215/1688 [====================>.........] - ETA: 1s - loss: 0.0693 - accuracy: 0.9781
1237/1688 [====================>.........] - ETA: 1s - loss: 0.0695 - accuracy: 0.9780
1260/1688 [=====================>........] - ETA: 0s - loss: 0.0696 - accuracy: 0.9781
1282/1688 [=====================>........] - ETA: 0s - loss: 0.0699 - accuracy: 0.9779
1304/1688 [======================>.......] - ETA: 0s - loss: 0.0698 - accuracy: 0.9779
1326/1688 [======================>.......] - ETA: 0s - loss: 0.0694 - accuracy: 0.9780
1349/1688 [======================>.......] - ETA: 0s - loss: 0.0689 - accuracy: 0.9782
1371/1688 [=======================>......] - ETA: 0s - loss: 0.0689 - accuracy: 0.9781
1393/1688 [=======================>......] - ETA: 0s - loss: 0.0687 - accuracy: 0.9782
1416/1688 [========================>.....] - ETA: 0s - loss: 0.0687 - accuracy: 0.9782
1439/1688 [========================>.....] - ETA: 0s - loss: 0.0690 - accuracy: 0.9782
1462/1688 [========================>.....] - ETA: 0s - loss: 0.0689 - accuracy: 0.9782
1484/1688 [=========================>....] - ETA: 0s - loss: 0.0685 - accuracy: 0.9784
1507/1688 [=========================>....] - ETA: 0s - loss: 0.0680 - accuracy: 0.9785
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0681 - accuracy: 0.9785
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0677 - accuracy: 0.9786
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0674 - accuracy: 0.9787
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0673 - accuracy: 0.9787
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0676 - accuracy: 0.9786
1640/1688 [============================>.] - ETA: 0s - loss: 0.0673 - accuracy: 0.9787
1662/1688 [============================>.] - ETA: 0s - loss: 0.0673 - accuracy: 0.9788
1681/1688 [============================>.] - ETA: 0s - loss: 0.0673 - accuracy: 0.9788
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0672 - accuracy: 0.9788 - val_loss: 0.0620 - val_accuracy: 0.9842
Epoch 3/10
1/1688 [..............................] - ETA: 3s - loss: 0.0077 - accuracy: 1.0000
25/1688 [..............................] - ETA: 3s - loss: 0.0414 - accuracy: 0.9850
47/1688 [..............................] - ETA: 3s - loss: 0.0461 - accuracy: 0.9860
71/1688 [>.............................] - ETA: 3s - loss: 0.0419 - accuracy: 0.9877
93/1688 [>.............................] - ETA: 3s - loss: 0.0484 - accuracy: 0.9869
116/1688 [=>............................] - ETA: 3s - loss: 0.0479 - accuracy: 0.9871
139/1688 [=>............................] - ETA: 3s - loss: 0.0488 - accuracy: 0.9867
162/1688 [=>............................] - ETA: 3s - loss: 0.0532 - accuracy: 0.9859
185/1688 [==>...........................] - ETA: 3s - loss: 0.0524 - accuracy: 0.9858
207/1688 [==>...........................] - ETA: 3s - loss: 0.0545 - accuracy: 0.9851
229/1688 [===>..........................] - ETA: 3s - loss: 0.0550 - accuracy: 0.9846
252/1688 [===>..........................] - ETA: 3s - loss: 0.0544 - accuracy: 0.9844
274/1688 [===>..........................] - ETA: 3s - loss: 0.0524 - accuracy: 0.9848
297/1688 [====>.........................] - ETA: 3s - loss: 0.0522 - accuracy: 0.9846
320/1688 [====>.........................] - ETA: 3s - loss: 0.0540 - accuracy: 0.9845
341/1688 [=====>........................] - ETA: 3s - loss: 0.0534 - accuracy: 0.9842
363/1688 [=====>........................] - ETA: 2s - loss: 0.0521 - accuracy: 0.9844
385/1688 [=====>........................] - ETA: 2s - loss: 0.0515 - accuracy: 0.9845
408/1688 [======>.......................] - ETA: 2s - loss: 0.0522 - accuracy: 0.9843
430/1688 [======>.......................] - ETA: 2s - loss: 0.0525 - accuracy: 0.9842
453/1688 [=======>......................] - ETA: 2s - loss: 0.0521 - accuracy: 0.9842
475/1688 [=======>......................] - ETA: 2s - loss: 0.0519 - accuracy: 0.9844
497/1688 [=======>......................] - ETA: 2s - loss: 0.0510 - accuracy: 0.9845
519/1688 [========>.....................] - ETA: 2s - loss: 0.0505 - accuracy: 0.9846
541/1688 [========>.....................] - ETA: 2s - loss: 0.0497 - accuracy: 0.9849
564/1688 [=========>....................] - ETA: 2s - loss: 0.0499 - accuracy: 0.9848
586/1688 [=========>....................] - ETA: 2s - loss: 0.0493 - accuracy: 0.9850
609/1688 [=========>....................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9849
631/1688 [==========>...................] - ETA: 2s - loss: 0.0488 - accuracy: 0.9847
653/1688 [==========>...................] - ETA: 2s - loss: 0.0480 - accuracy: 0.9851
675/1688 [==========>...................] - ETA: 2s - loss: 0.0483 - accuracy: 0.9850
697/1688 [===========>..................] - ETA: 2s - loss: 0.0486 - accuracy: 0.9847
720/1688 [===========>..................] - ETA: 2s - loss: 0.0485 - accuracy: 0.9846
742/1688 [============>.................] - ETA: 2s - loss: 0.0483 - accuracy: 0.9846
764/1688 [============>.................] - ETA: 2s - loss: 0.0483 - accuracy: 0.9845
787/1688 [============>.................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9842
810/1688 [=============>................] - ETA: 1s - loss: 0.0483 - accuracy: 0.9844
832/1688 [=============>................] - ETA: 1s - loss: 0.0484 - accuracy: 0.9845
855/1688 [==============>...............] - ETA: 1s - loss: 0.0485 - accuracy: 0.9844
877/1688 [==============>...............] - ETA: 1s - loss: 0.0490 - accuracy: 0.9842
899/1688 [==============>...............] - ETA: 1s - loss: 0.0489 - accuracy: 0.9843
922/1688 [===============>..............] - ETA: 1s - loss: 0.0489 - accuracy: 0.9843
944/1688 [===============>..............] - ETA: 1s - loss: 0.0490 - accuracy: 0.9843
966/1688 [================>.............] - ETA: 1s - loss: 0.0492 - accuracy: 0.9843
989/1688 [================>.............] - ETA: 1s - loss: 0.0498 - accuracy: 0.9840
1011/1688 [================>.............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9839
1034/1688 [=================>............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9839
1058/1688 [=================>............] - ETA: 1s - loss: 0.0499 - accuracy: 0.9840
1081/1688 [==================>...........] - ETA: 1s - loss: 0.0502 - accuracy: 0.9839
1103/1688 [==================>...........] - ETA: 1s - loss: 0.0502 - accuracy: 0.9838
1126/1688 [===================>..........] - ETA: 1s - loss: 0.0500 - accuracy: 0.9838
1149/1688 [===================>..........] - ETA: 1s - loss: 0.0495 - accuracy: 0.9840
1171/1688 [===================>..........] - ETA: 1s - loss: 0.0494 - accuracy: 0.9841
1194/1688 [====================>.........] - ETA: 1s - loss: 0.0493 - accuracy: 0.9842
1216/1688 [====================>.........] - ETA: 1s - loss: 0.0491 - accuracy: 0.9842
1239/1688 [=====================>........] - ETA: 1s - loss: 0.0488 - accuracy: 0.9843
1262/1688 [=====================>........] - ETA: 0s - loss: 0.0487 - accuracy: 0.9844
1285/1688 [=====================>........] - ETA: 0s - loss: 0.0486 - accuracy: 0.9844
1307/1688 [======================>.......] - ETA: 0s - loss: 0.0483 - accuracy: 0.9845
1329/1688 [======================>.......] - ETA: 0s - loss: 0.0483 - accuracy: 0.9844
1352/1688 [=======================>......] - ETA: 0s - loss: 0.0481 - accuracy: 0.9845
1374/1688 [=======================>......] - ETA: 0s - loss: 0.0486 - accuracy: 0.9844
1397/1688 [=======================>......] - ETA: 0s - loss: 0.0489 - accuracy: 0.9844
1420/1688 [========================>.....] - ETA: 0s - loss: 0.0494 - accuracy: 0.9843
1442/1688 [========================>.....] - ETA: 0s - loss: 0.0493 - accuracy: 0.9844
1464/1688 [=========================>....] - ETA: 0s - loss: 0.0499 - accuracy: 0.9843
1486/1688 [=========================>....] - ETA: 0s - loss: 0.0502 - accuracy: 0.9842
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0501 - accuracy: 0.9842
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0500 - accuracy: 0.9842
1554/1688 [==========================>...] - ETA: 0s - loss: 0.0497 - accuracy: 0.9843
1577/1688 [===========================>..] - ETA: 0s - loss: 0.0498 - accuracy: 0.9843
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0496 - accuracy: 0.9843
1623/1688 [===========================>..] - ETA: 0s - loss: 0.0495 - accuracy: 0.9843
1646/1688 [============================>.] - ETA: 0s - loss: 0.0493 - accuracy: 0.9844
1669/1688 [============================>.] - ETA: 0s - loss: 0.0492 - accuracy: 0.9844
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0491 - accuracy: 0.9845 - val_loss: 0.0545 - val_accuracy: 0.9843
Epoch 4/10
1/1688 [..............................] - ETA: 4s - loss: 0.0028 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0291 - accuracy: 0.9878
45/1688 [..............................] - ETA: 3s - loss: 0.0365 - accuracy: 0.9854
68/1688 [>.............................] - ETA: 3s - loss: 0.0357 - accuracy: 0.9867
91/1688 [>.............................] - ETA: 3s - loss: 0.0349 - accuracy: 0.9876
113/1688 [=>............................] - ETA: 3s - loss: 0.0349 - accuracy: 0.9876
137/1688 [=>............................] - ETA: 3s - loss: 0.0313 - accuracy: 0.9895
160/1688 [=>............................] - ETA: 3s - loss: 0.0327 - accuracy: 0.9885
182/1688 [==>...........................] - ETA: 3s - loss: 0.0337 - accuracy: 0.9880
204/1688 [==>...........................] - ETA: 3s - loss: 0.0326 - accuracy: 0.9881
227/1688 [===>..........................] - ETA: 3s - loss: 0.0330 - accuracy: 0.9884
249/1688 [===>..........................] - ETA: 3s - loss: 0.0349 - accuracy: 0.9882
271/1688 [===>..........................] - ETA: 3s - loss: 0.0365 - accuracy: 0.9884
294/1688 [====>.........................] - ETA: 3s - loss: 0.0349 - accuracy: 0.9891
316/1688 [====>.........................] - ETA: 3s - loss: 0.0341 - accuracy: 0.9892
339/1688 [=====>........................] - ETA: 3s - loss: 0.0335 - accuracy: 0.9894
362/1688 [=====>........................] - ETA: 2s - loss: 0.0335 - accuracy: 0.9896
384/1688 [=====>........................] - ETA: 2s - loss: 0.0336 - accuracy: 0.9896
407/1688 [======>.......................] - ETA: 2s - loss: 0.0339 - accuracy: 0.9897
430/1688 [======>.......................] - ETA: 2s - loss: 0.0342 - accuracy: 0.9896
451/1688 [=======>......................] - ETA: 2s - loss: 0.0338 - accuracy: 0.9895
473/1688 [=======>......................] - ETA: 2s - loss: 0.0342 - accuracy: 0.9895
496/1688 [=======>......................] - ETA: 2s - loss: 0.0344 - accuracy: 0.9892
520/1688 [========>.....................] - ETA: 2s - loss: 0.0341 - accuracy: 0.9892
543/1688 [========>.....................] - ETA: 2s - loss: 0.0345 - accuracy: 0.9892
567/1688 [=========>....................] - ETA: 2s - loss: 0.0348 - accuracy: 0.9893
590/1688 [=========>....................] - ETA: 2s - loss: 0.0349 - accuracy: 0.9892
613/1688 [=========>....................] - ETA: 2s - loss: 0.0344 - accuracy: 0.9893
635/1688 [==========>...................] - ETA: 2s - loss: 0.0341 - accuracy: 0.9894
658/1688 [==========>...................] - ETA: 2s - loss: 0.0344 - accuracy: 0.9891
680/1688 [===========>..................] - ETA: 2s - loss: 0.0342 - accuracy: 0.9892
702/1688 [===========>..................] - ETA: 2s - loss: 0.0344 - accuracy: 0.9892
724/1688 [===========>..................] - ETA: 2s - loss: 0.0352 - accuracy: 0.9890
746/1688 [============>.................] - ETA: 2s - loss: 0.0364 - accuracy: 0.9886
769/1688 [============>.................] - ETA: 2s - loss: 0.0369 - accuracy: 0.9885
791/1688 [=============>................] - ETA: 2s - loss: 0.0377 - accuracy: 0.9883
814/1688 [=============>................] - ETA: 1s - loss: 0.0384 - accuracy: 0.9879
836/1688 [=============>................] - ETA: 1s - loss: 0.0397 - accuracy: 0.9876
859/1688 [==============>...............] - ETA: 1s - loss: 0.0402 - accuracy: 0.9874
882/1688 [==============>...............] - ETA: 1s - loss: 0.0405 - accuracy: 0.9875
905/1688 [===============>..............] - ETA: 1s - loss: 0.0404 - accuracy: 0.9873
927/1688 [===============>..............] - ETA: 1s - loss: 0.0406 - accuracy: 0.9872
949/1688 [===============>..............] - ETA: 1s - loss: 0.0413 - accuracy: 0.9870
972/1688 [================>.............] - ETA: 1s - loss: 0.0416 - accuracy: 0.9869
994/1688 [================>.............] - ETA: 1s - loss: 0.0415 - accuracy: 0.9869
1017/1688 [=================>............] - ETA: 1s - loss: 0.0413 - accuracy: 0.9869
1040/1688 [=================>............] - ETA: 1s - loss: 0.0411 - accuracy: 0.9870
1062/1688 [=================>............] - ETA: 1s - loss: 0.0412 - accuracy: 0.9869
1085/1688 [==================>...........] - ETA: 1s - loss: 0.0410 - accuracy: 0.9870
1107/1688 [==================>...........] - ETA: 1s - loss: 0.0410 - accuracy: 0.9869
1130/1688 [===================>..........] - ETA: 1s - loss: 0.0411 - accuracy: 0.9869
1152/1688 [===================>..........] - ETA: 1s - loss: 0.0410 - accuracy: 0.9870
1175/1688 [===================>..........] - ETA: 1s - loss: 0.0408 - accuracy: 0.9871
1198/1688 [====================>.........] - ETA: 1s - loss: 0.0406 - accuracy: 0.9871
1219/1688 [====================>.........] - ETA: 1s - loss: 0.0407 - accuracy: 0.9870
1242/1688 [=====================>........] - ETA: 1s - loss: 0.0408 - accuracy: 0.9871
1264/1688 [=====================>........] - ETA: 0s - loss: 0.0407 - accuracy: 0.9870
1286/1688 [=====================>........] - ETA: 0s - loss: 0.0409 - accuracy: 0.9870
1309/1688 [======================>.......] - ETA: 0s - loss: 0.0411 - accuracy: 0.9870
1332/1688 [======================>.......] - ETA: 0s - loss: 0.0411 - accuracy: 0.9870
1355/1688 [=======================>......] - ETA: 0s - loss: 0.0414 - accuracy: 0.9868
1378/1688 [=======================>......] - ETA: 0s - loss: 0.0414 - accuracy: 0.9868
1401/1688 [=======================>......] - ETA: 0s - loss: 0.0416 - accuracy: 0.9867
1423/1688 [========================>.....] - ETA: 0s - loss: 0.0416 - accuracy: 0.9868
1447/1688 [========================>.....] - ETA: 0s - loss: 0.0414 - accuracy: 0.9869
1470/1688 [=========================>....] - ETA: 0s - loss: 0.0413 - accuracy: 0.9868
1493/1688 [=========================>....] - ETA: 0s - loss: 0.0413 - accuracy: 0.9868
1515/1688 [=========================>....] - ETA: 0s - loss: 0.0414 - accuracy: 0.9867
1537/1688 [==========================>...] - ETA: 0s - loss: 0.0415 - accuracy: 0.9867
1560/1688 [==========================>...] - ETA: 0s - loss: 0.0421 - accuracy: 0.9866
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0417 - accuracy: 0.9867
1605/1688 [===========================>..] - ETA: 0s - loss: 0.0415 - accuracy: 0.9867
1628/1688 [===========================>..] - ETA: 0s - loss: 0.0419 - accuracy: 0.9866
1651/1688 [============================>.] - ETA: 0s - loss: 0.0423 - accuracy: 0.9865
1674/1688 [============================>.] - ETA: 0s - loss: 0.0420 - accuracy: 0.9867
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0418 - accuracy: 0.9867 - val_loss: 0.0541 - val_accuracy: 0.9867
Epoch 5/10
1/1688 [..............................] - ETA: 4s - loss: 0.0714 - accuracy: 0.9688
23/1688 [..............................] - ETA: 3s - loss: 0.0172 - accuracy: 0.9918
45/1688 [..............................] - ETA: 3s - loss: 0.0187 - accuracy: 0.9924
67/1688 [>.............................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9930
89/1688 [>.............................] - ETA: 3s - loss: 0.0201 - accuracy: 0.9933
111/1688 [>.............................] - ETA: 3s - loss: 0.0223 - accuracy: 0.9930
134/1688 [=>............................] - ETA: 3s - loss: 0.0264 - accuracy: 0.9925
156/1688 [=>............................] - ETA: 3s - loss: 0.0247 - accuracy: 0.9930
178/1688 [==>...........................] - ETA: 3s - loss: 0.0232 - accuracy: 0.9935
201/1688 [==>...........................] - ETA: 3s - loss: 0.0231 - accuracy: 0.9932
224/1688 [==>...........................] - ETA: 3s - loss: 0.0240 - accuracy: 0.9927
246/1688 [===>..........................] - ETA: 3s - loss: 0.0235 - accuracy: 0.9930
268/1688 [===>..........................] - ETA: 3s - loss: 0.0231 - accuracy: 0.9931
291/1688 [====>.........................] - ETA: 3s - loss: 0.0223 - accuracy: 0.9932
315/1688 [====>.........................] - ETA: 3s - loss: 0.0224 - accuracy: 0.9931
338/1688 [=====>........................] - ETA: 3s - loss: 0.0219 - accuracy: 0.9932
361/1688 [=====>........................] - ETA: 3s - loss: 0.0215 - accuracy: 0.9932
383/1688 [=====>........................] - ETA: 2s - loss: 0.0222 - accuracy: 0.9931
405/1688 [======>.......................] - ETA: 2s - loss: 0.0223 - accuracy: 0.9927
428/1688 [======>.......................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9928
450/1688 [======>.......................] - ETA: 2s - loss: 0.0236 - accuracy: 0.9926
472/1688 [=======>......................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9919
494/1688 [=======>......................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9920
516/1688 [========>.....................] - ETA: 2s - loss: 0.0248 - accuracy: 0.9919
538/1688 [========>.....................] - ETA: 2s - loss: 0.0251 - accuracy: 0.9919
561/1688 [========>.....................] - ETA: 2s - loss: 0.0263 - accuracy: 0.9915
584/1688 [=========>....................] - ETA: 2s - loss: 0.0269 - accuracy: 0.9913
607/1688 [=========>....................] - ETA: 2s - loss: 0.0272 - accuracy: 0.9912
630/1688 [==========>...................] - ETA: 2s - loss: 0.0270 - accuracy: 0.9912
653/1688 [==========>...................] - ETA: 2s - loss: 0.0270 - accuracy: 0.9911
675/1688 [==========>...................] - ETA: 2s - loss: 0.0275 - accuracy: 0.9910
697/1688 [===========>..................] - ETA: 2s - loss: 0.0272 - accuracy: 0.9910
719/1688 [===========>..................] - ETA: 2s - loss: 0.0275 - accuracy: 0.9909
742/1688 [============>.................] - ETA: 2s - loss: 0.0282 - accuracy: 0.9908
764/1688 [============>.................] - ETA: 2s - loss: 0.0283 - accuracy: 0.9907
786/1688 [============>.................] - ETA: 2s - loss: 0.0285 - accuracy: 0.9906
809/1688 [=============>................] - ETA: 2s - loss: 0.0283 - accuracy: 0.9905
831/1688 [=============>................] - ETA: 1s - loss: 0.0281 - accuracy: 0.9906
853/1688 [==============>...............] - ETA: 1s - loss: 0.0287 - accuracy: 0.9903
875/1688 [==============>...............] - ETA: 1s - loss: 0.0295 - accuracy: 0.9900
897/1688 [==============>...............] - ETA: 1s - loss: 0.0298 - accuracy: 0.9899
921/1688 [===============>..............] - ETA: 1s - loss: 0.0300 - accuracy: 0.9898
943/1688 [===============>..............] - ETA: 1s - loss: 0.0297 - accuracy: 0.9899
966/1688 [================>.............] - ETA: 1s - loss: 0.0298 - accuracy: 0.9898
989/1688 [================>.............] - ETA: 1s - loss: 0.0299 - accuracy: 0.9898
1011/1688 [================>.............] - ETA: 1s - loss: 0.0296 - accuracy: 0.9899
1033/1688 [=================>............] - ETA: 1s - loss: 0.0295 - accuracy: 0.9900
1056/1688 [=================>............] - ETA: 1s - loss: 0.0293 - accuracy: 0.9901
1078/1688 [==================>...........] - ETA: 1s - loss: 0.0292 - accuracy: 0.9902
1100/1688 [==================>...........] - ETA: 1s - loss: 0.0292 - accuracy: 0.9902
1123/1688 [==================>...........] - ETA: 1s - loss: 0.0300 - accuracy: 0.9900
1146/1688 [===================>..........] - ETA: 1s - loss: 0.0300 - accuracy: 0.9901
1170/1688 [===================>..........] - ETA: 1s - loss: 0.0298 - accuracy: 0.9902
1193/1688 [====================>.........] - ETA: 1s - loss: 0.0300 - accuracy: 0.9901
1215/1688 [====================>.........] - ETA: 1s - loss: 0.0305 - accuracy: 0.9899
1237/1688 [====================>.........] - ETA: 1s - loss: 0.0313 - accuracy: 0.9897
1260/1688 [=====================>........] - ETA: 0s - loss: 0.0315 - accuracy: 0.9896
1282/1688 [=====================>........] - ETA: 0s - loss: 0.0313 - accuracy: 0.9897
1304/1688 [======================>.......] - ETA: 0s - loss: 0.0318 - accuracy: 0.9896
1326/1688 [======================>.......] - ETA: 0s - loss: 0.0316 - accuracy: 0.9897
1348/1688 [======================>.......] - ETA: 0s - loss: 0.0315 - accuracy: 0.9897
1370/1688 [=======================>......] - ETA: 0s - loss: 0.0315 - accuracy: 0.9897
1393/1688 [=======================>......] - ETA: 0s - loss: 0.0315 - accuracy: 0.9897
1415/1688 [========================>.....] - ETA: 0s - loss: 0.0313 - accuracy: 0.9897
1437/1688 [========================>.....] - ETA: 0s - loss: 0.0314 - accuracy: 0.9897
1459/1688 [========================>.....] - ETA: 0s - loss: 0.0313 - accuracy: 0.9897
1481/1688 [=========================>....] - ETA: 0s - loss: 0.0314 - accuracy: 0.9897
1504/1688 [=========================>....] - ETA: 0s - loss: 0.0318 - accuracy: 0.9896
1526/1688 [==========================>...] - ETA: 0s - loss: 0.0320 - accuracy: 0.9895
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0321 - accuracy: 0.9895
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0325 - accuracy: 0.9894
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0328 - accuracy: 0.9894
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0326 - accuracy: 0.9894
1640/1688 [============================>.] - ETA: 0s - loss: 0.0326 - accuracy: 0.9894
1662/1688 [============================>.] - ETA: 0s - loss: 0.0329 - accuracy: 0.9894
1684/1688 [============================>.] - ETA: 0s - loss: 0.0329 - accuracy: 0.9893
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0330 - accuracy: 0.9893 - val_loss: 0.0552 - val_accuracy: 0.9872
Epoch 6/10
1/1688 [..............................] - ETA: 4s - loss: 0.0331 - accuracy: 0.9688
23/1688 [..............................] - ETA: 3s - loss: 0.0219 - accuracy: 0.9959
45/1688 [..............................] - ETA: 3s - loss: 0.0194 - accuracy: 0.9937
67/1688 [>.............................] - ETA: 3s - loss: 0.0205 - accuracy: 0.9939
90/1688 [>.............................] - ETA: 3s - loss: 0.0209 - accuracy: 0.9944
112/1688 [>.............................] - ETA: 3s - loss: 0.0237 - accuracy: 0.9941
135/1688 [=>............................] - ETA: 3s - loss: 0.0244 - accuracy: 0.9937
158/1688 [=>............................] - ETA: 3s - loss: 0.0266 - accuracy: 0.9933
180/1688 [==>...........................] - ETA: 3s - loss: 0.0274 - accuracy: 0.9925
202/1688 [==>...........................] - ETA: 3s - loss: 0.0269 - accuracy: 0.9924
225/1688 [==>...........................] - ETA: 3s - loss: 0.0276 - accuracy: 0.9917
248/1688 [===>..........................] - ETA: 3s - loss: 0.0267 - accuracy: 0.9918
270/1688 [===>..........................] - ETA: 3s - loss: 0.0269 - accuracy: 0.9918
292/1688 [====>.........................] - ETA: 3s - loss: 0.0256 - accuracy: 0.9923
314/1688 [====>.........................] - ETA: 3s - loss: 0.0246 - accuracy: 0.9926
336/1688 [====>.........................] - ETA: 3s - loss: 0.0241 - accuracy: 0.9928
358/1688 [=====>........................] - ETA: 3s - loss: 0.0239 - accuracy: 0.9928
380/1688 [=====>........................] - ETA: 2s - loss: 0.0235 - accuracy: 0.9928
402/1688 [======>.......................] - ETA: 2s - loss: 0.0246 - accuracy: 0.9925
424/1688 [======>.......................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9926
447/1688 [======>.......................] - ETA: 2s - loss: 0.0252 - accuracy: 0.9927
470/1688 [=======>......................] - ETA: 2s - loss: 0.0254 - accuracy: 0.9926
491/1688 [=======>......................] - ETA: 2s - loss: 0.0251 - accuracy: 0.9927
513/1688 [========>.....................] - ETA: 2s - loss: 0.0251 - accuracy: 0.9926
535/1688 [========>.....................] - ETA: 2s - loss: 0.0255 - accuracy: 0.9923
558/1688 [========>.....................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9924
580/1688 [=========>....................] - ETA: 2s - loss: 0.0256 - accuracy: 0.9923
602/1688 [=========>....................] - ETA: 2s - loss: 0.0260 - accuracy: 0.9922
625/1688 [==========>...................] - ETA: 2s - loss: 0.0258 - accuracy: 0.9923
647/1688 [==========>...................] - ETA: 2s - loss: 0.0261 - accuracy: 0.9922
669/1688 [==========>...................] - ETA: 2s - loss: 0.0259 - accuracy: 0.9922
692/1688 [===========>..................] - ETA: 2s - loss: 0.0263 - accuracy: 0.9921
714/1688 [===========>..................] - ETA: 2s - loss: 0.0263 - accuracy: 0.9922
736/1688 [============>.................] - ETA: 2s - loss: 0.0262 - accuracy: 0.9921
758/1688 [============>.................] - ETA: 2s - loss: 0.0262 - accuracy: 0.9920
781/1688 [============>.................] - ETA: 2s - loss: 0.0262 - accuracy: 0.9919
803/1688 [=============>................] - ETA: 2s - loss: 0.0267 - accuracy: 0.9917
826/1688 [=============>................] - ETA: 1s - loss: 0.0263 - accuracy: 0.9918
849/1688 [==============>...............] - ETA: 1s - loss: 0.0262 - accuracy: 0.9919
871/1688 [==============>...............] - ETA: 1s - loss: 0.0260 - accuracy: 0.9919
895/1688 [==============>...............] - ETA: 1s - loss: 0.0257 - accuracy: 0.9920
918/1688 [===============>..............] - ETA: 1s - loss: 0.0258 - accuracy: 0.9920
941/1688 [===============>..............] - ETA: 1s - loss: 0.0256 - accuracy: 0.9921
963/1688 [================>.............] - ETA: 1s - loss: 0.0256 - accuracy: 0.9920
986/1688 [================>.............] - ETA: 1s - loss: 0.0259 - accuracy: 0.9919
1009/1688 [================>.............] - ETA: 1s - loss: 0.0258 - accuracy: 0.9919
1031/1688 [=================>............] - ETA: 1s - loss: 0.0259 - accuracy: 0.9918
1053/1688 [=================>............] - ETA: 1s - loss: 0.0256 - accuracy: 0.9919
1075/1688 [==================>...........] - ETA: 1s - loss: 0.0255 - accuracy: 0.9919
1097/1688 [==================>...........] - ETA: 1s - loss: 0.0259 - accuracy: 0.9917
1120/1688 [==================>...........] - ETA: 1s - loss: 0.0260 - accuracy: 0.9917
1143/1688 [===================>..........] - ETA: 1s - loss: 0.0264 - accuracy: 0.9916
1165/1688 [===================>..........] - ETA: 1s - loss: 0.0264 - accuracy: 0.9915
1188/1688 [====================>.........] - ETA: 1s - loss: 0.0263 - accuracy: 0.9916
1210/1688 [====================>.........] - ETA: 1s - loss: 0.0262 - accuracy: 0.9915
1232/1688 [====================>.........] - ETA: 1s - loss: 0.0262 - accuracy: 0.9915
1254/1688 [=====================>........] - ETA: 0s - loss: 0.0263 - accuracy: 0.9915
1276/1688 [=====================>........] - ETA: 0s - loss: 0.0263 - accuracy: 0.9915
1298/1688 [======================>.......] - ETA: 0s - loss: 0.0264 - accuracy: 0.9914
1321/1688 [======================>.......] - ETA: 0s - loss: 0.0264 - accuracy: 0.9914
1343/1688 [======================>.......] - ETA: 0s - loss: 0.0263 - accuracy: 0.9914
1365/1688 [=======================>......] - ETA: 0s - loss: 0.0263 - accuracy: 0.9914
1387/1688 [=======================>......] - ETA: 0s - loss: 0.0262 - accuracy: 0.9914
1410/1688 [========================>.....] - ETA: 0s - loss: 0.0262 - accuracy: 0.9914
1432/1688 [========================>.....] - ETA: 0s - loss: 0.0265 - accuracy: 0.9913
1454/1688 [========================>.....] - ETA: 0s - loss: 0.0267 - accuracy: 0.9912
1477/1688 [=========================>....] - ETA: 0s - loss: 0.0268 - accuracy: 0.9912
1499/1688 [=========================>....] - ETA: 0s - loss: 0.0268 - accuracy: 0.9912
1522/1688 [==========================>...] - ETA: 0s - loss: 0.0270 - accuracy: 0.9911
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0274 - accuracy: 0.9910
1568/1688 [==========================>...] - ETA: 0s - loss: 0.0274 - accuracy: 0.9910
1591/1688 [===========================>..] - ETA: 0s - loss: 0.0273 - accuracy: 0.9910
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0274 - accuracy: 0.9910
1635/1688 [============================>.] - ETA: 0s - loss: 0.0274 - accuracy: 0.9909
1658/1688 [============================>.] - ETA: 0s - loss: 0.0276 - accuracy: 0.9908
1680/1688 [============================>.] - ETA: 0s - loss: 0.0277 - accuracy: 0.9908
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0277 - accuracy: 0.9908 - val_loss: 0.0540 - val_accuracy: 0.9867
Epoch 7/10
1/1688 [..............................] - ETA: 4s - loss: 0.0058 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0174 - accuracy: 0.9948
47/1688 [..............................] - ETA: 3s - loss: 0.0185 - accuracy: 0.9927
69/1688 [>.............................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9941
91/1688 [>.............................] - ETA: 3s - loss: 0.0187 - accuracy: 0.9948
113/1688 [=>............................] - ETA: 3s - loss: 0.0188 - accuracy: 0.9950
135/1688 [=>............................] - ETA: 3s - loss: 0.0218 - accuracy: 0.9947
157/1688 [=>............................] - ETA: 3s - loss: 0.0210 - accuracy: 0.9948
179/1688 [==>...........................] - ETA: 3s - loss: 0.0213 - accuracy: 0.9944
201/1688 [==>...........................] - ETA: 3s - loss: 0.0216 - accuracy: 0.9946
223/1688 [==>...........................] - ETA: 3s - loss: 0.0204 - accuracy: 0.9948
246/1688 [===>..........................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9950
269/1688 [===>..........................] - ETA: 3s - loss: 0.0192 - accuracy: 0.9949
292/1688 [====>.........................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9950
315/1688 [====>.........................] - ETA: 3s - loss: 0.0187 - accuracy: 0.9952
337/1688 [====>.........................] - ETA: 3s - loss: 0.0185 - accuracy: 0.9949
359/1688 [=====>........................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9950
382/1688 [=====>........................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9951
404/1688 [======>.......................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9949
426/1688 [======>.......................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9949
449/1688 [======>.......................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9949
471/1688 [=======>......................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9948
493/1688 [=======>......................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9947
516/1688 [========>.....................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9947
538/1688 [========>.....................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9948
560/1688 [========>.....................] - ETA: 2s - loss: 0.0176 - accuracy: 0.9949
583/1688 [=========>....................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9949
606/1688 [=========>....................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9950
628/1688 [==========>...................] - ETA: 2s - loss: 0.0175 - accuracy: 0.9950
651/1688 [==========>...................] - ETA: 2s - loss: 0.0174 - accuracy: 0.9951
674/1688 [==========>...................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9948
696/1688 [===========>..................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9945
718/1688 [===========>..................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9944
739/1688 [============>.................] - ETA: 2s - loss: 0.0187 - accuracy: 0.9943
761/1688 [============>.................] - ETA: 2s - loss: 0.0185 - accuracy: 0.9943
784/1688 [============>.................] - ETA: 2s - loss: 0.0189 - accuracy: 0.9943
806/1688 [=============>................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9941
829/1688 [=============>................] - ETA: 1s - loss: 0.0192 - accuracy: 0.9941
853/1688 [==============>...............] - ETA: 1s - loss: 0.0201 - accuracy: 0.9939
875/1688 [==============>...............] - ETA: 1s - loss: 0.0201 - accuracy: 0.9939
897/1688 [==============>...............] - ETA: 1s - loss: 0.0203 - accuracy: 0.9938
919/1688 [===============>..............] - ETA: 1s - loss: 0.0201 - accuracy: 0.9938
942/1688 [===============>..............] - ETA: 1s - loss: 0.0205 - accuracy: 0.9937
965/1688 [================>.............] - ETA: 1s - loss: 0.0209 - accuracy: 0.9935
989/1688 [================>.............] - ETA: 1s - loss: 0.0211 - accuracy: 0.9934
1012/1688 [================>.............] - ETA: 1s - loss: 0.0211 - accuracy: 0.9934
1034/1688 [=================>............] - ETA: 1s - loss: 0.0210 - accuracy: 0.9935
1056/1688 [=================>............] - ETA: 1s - loss: 0.0210 - accuracy: 0.9934
1078/1688 [==================>...........] - ETA: 1s - loss: 0.0213 - accuracy: 0.9933
1101/1688 [==================>...........] - ETA: 1s - loss: 0.0213 - accuracy: 0.9933
1124/1688 [==================>...........] - ETA: 1s - loss: 0.0212 - accuracy: 0.9933
1147/1688 [===================>..........] - ETA: 1s - loss: 0.0215 - accuracy: 0.9932
1169/1688 [===================>..........] - ETA: 1s - loss: 0.0217 - accuracy: 0.9932
1191/1688 [====================>.........] - ETA: 1s - loss: 0.0219 - accuracy: 0.9931
1214/1688 [====================>.........] - ETA: 1s - loss: 0.0220 - accuracy: 0.9930
1236/1688 [====================>.........] - ETA: 1s - loss: 0.0223 - accuracy: 0.9929
1258/1688 [=====================>........] - ETA: 0s - loss: 0.0224 - accuracy: 0.9929
1281/1688 [=====================>........] - ETA: 0s - loss: 0.0224 - accuracy: 0.9930
1303/1688 [======================>.......] - ETA: 0s - loss: 0.0223 - accuracy: 0.9929
1326/1688 [======================>.......] - ETA: 0s - loss: 0.0224 - accuracy: 0.9929
1349/1688 [======================>.......] - ETA: 0s - loss: 0.0225 - accuracy: 0.9928
1371/1688 [=======================>......] - ETA: 0s - loss: 0.0226 - accuracy: 0.9928
1394/1688 [=======================>......] - ETA: 0s - loss: 0.0228 - accuracy: 0.9927
1417/1688 [========================>.....] - ETA: 0s - loss: 0.0227 - accuracy: 0.9927
1439/1688 [========================>.....] - ETA: 0s - loss: 0.0228 - accuracy: 0.9926
1460/1688 [========================>.....] - ETA: 0s - loss: 0.0229 - accuracy: 0.9927
1482/1688 [=========================>....] - ETA: 0s - loss: 0.0229 - accuracy: 0.9927
1504/1688 [=========================>....] - ETA: 0s - loss: 0.0231 - accuracy: 0.9926
1526/1688 [==========================>...] - ETA: 0s - loss: 0.0231 - accuracy: 0.9926
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0230 - accuracy: 0.9926
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0233 - accuracy: 0.9926
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0235 - accuracy: 0.9925
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0235 - accuracy: 0.9925
1636/1688 [============================>.] - ETA: 0s - loss: 0.0234 - accuracy: 0.9926
1659/1688 [============================>.] - ETA: 0s - loss: 0.0234 - accuracy: 0.9925
1681/1688 [============================>.] - ETA: 0s - loss: 0.0233 - accuracy: 0.9926
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0233 - accuracy: 0.9926 - val_loss: 0.0528 - val_accuracy: 0.9873
Epoch 8/10
1/1688 [..............................] - ETA: 4s - loss: 0.0018 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0128 - accuracy: 0.9946
45/1688 [..............................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9958
68/1688 [>.............................] - ETA: 3s - loss: 0.0140 - accuracy: 0.9954
90/1688 [>.............................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9955
113/1688 [=>............................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9950
136/1688 [=>............................] - ETA: 3s - loss: 0.0159 - accuracy: 0.9943
159/1688 [=>............................] - ETA: 3s - loss: 0.0155 - accuracy: 0.9945
181/1688 [==>...........................] - ETA: 3s - loss: 0.0170 - accuracy: 0.9941
204/1688 [==>...........................] - ETA: 3s - loss: 0.0161 - accuracy: 0.9946
227/1688 [===>..........................] - ETA: 3s - loss: 0.0156 - accuracy: 0.9949
249/1688 [===>..........................] - ETA: 3s - loss: 0.0160 - accuracy: 0.9946
271/1688 [===>..........................] - ETA: 3s - loss: 0.0154 - accuracy: 0.9948
293/1688 [====>.........................] - ETA: 3s - loss: 0.0150 - accuracy: 0.9950
315/1688 [====>.........................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9951
338/1688 [=====>........................] - ETA: 3s - loss: 0.0139 - accuracy: 0.9954
361/1688 [=====>........................] - ETA: 3s - loss: 0.0140 - accuracy: 0.9955
384/1688 [=====>........................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9957
407/1688 [======>.......................] - ETA: 2s - loss: 0.0133 - accuracy: 0.9958
431/1688 [======>.......................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9958
454/1688 [=======>......................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9960
475/1688 [=======>......................] - ETA: 2s - loss: 0.0128 - accuracy: 0.9960
498/1688 [=======>......................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9959
521/1688 [========>.....................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9957
542/1688 [========>.....................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9956
565/1688 [=========>....................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9957
588/1688 [=========>....................] - ETA: 2s - loss: 0.0137 - accuracy: 0.9955
611/1688 [=========>....................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9955
634/1688 [==========>...................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9955
656/1688 [==========>...................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9954
678/1688 [===========>..................] - ETA: 2s - loss: 0.0138 - accuracy: 0.9954
701/1688 [===========>..................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9952
724/1688 [===========>..................] - ETA: 2s - loss: 0.0139 - accuracy: 0.9953
747/1688 [============>.................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9952
769/1688 [============>.................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9951
792/1688 [=============>................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9950
814/1688 [=============>................] - ETA: 1s - loss: 0.0145 - accuracy: 0.9950
837/1688 [=============>................] - ETA: 1s - loss: 0.0142 - accuracy: 0.9951
859/1688 [==============>...............] - ETA: 1s - loss: 0.0144 - accuracy: 0.9950
882/1688 [==============>...............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9950
904/1688 [===============>..............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9950
926/1688 [===============>..............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9950
949/1688 [===============>..............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9950
972/1688 [================>.............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9949
994/1688 [================>.............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9947
1017/1688 [=================>............] - ETA: 1s - loss: 0.0154 - accuracy: 0.9947
1040/1688 [=================>............] - ETA: 1s - loss: 0.0155 - accuracy: 0.9947
1062/1688 [=================>............] - ETA: 1s - loss: 0.0156 - accuracy: 0.9946
1085/1688 [==================>...........] - ETA: 1s - loss: 0.0160 - accuracy: 0.9946
1108/1688 [==================>...........] - ETA: 1s - loss: 0.0161 - accuracy: 0.9946
1131/1688 [===================>..........] - ETA: 1s - loss: 0.0164 - accuracy: 0.9945
1154/1688 [===================>..........] - ETA: 1s - loss: 0.0167 - accuracy: 0.9943
1176/1688 [===================>..........] - ETA: 1s - loss: 0.0168 - accuracy: 0.9943
1198/1688 [====================>.........] - ETA: 1s - loss: 0.0171 - accuracy: 0.9942
1221/1688 [====================>.........] - ETA: 1s - loss: 0.0174 - accuracy: 0.9941
1244/1688 [=====================>........] - ETA: 1s - loss: 0.0174 - accuracy: 0.9941
1266/1688 [=====================>........] - ETA: 0s - loss: 0.0177 - accuracy: 0.9941
1288/1688 [=====================>........] - ETA: 0s - loss: 0.0180 - accuracy: 0.9940
1310/1688 [======================>.......] - ETA: 0s - loss: 0.0185 - accuracy: 0.9939
1332/1688 [======================>.......] - ETA: 0s - loss: 0.0187 - accuracy: 0.9938
1355/1688 [=======================>......] - ETA: 0s - loss: 0.0186 - accuracy: 0.9938
1377/1688 [=======================>......] - ETA: 0s - loss: 0.0186 - accuracy: 0.9938
1399/1688 [=======================>......] - ETA: 0s - loss: 0.0186 - accuracy: 0.9938
1421/1688 [========================>.....] - ETA: 0s - loss: 0.0188 - accuracy: 0.9938
1443/1688 [========================>.....] - ETA: 0s - loss: 0.0187 - accuracy: 0.9938
1466/1688 [=========================>....] - ETA: 0s - loss: 0.0185 - accuracy: 0.9939
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0185 - accuracy: 0.9939
1513/1688 [=========================>....] - ETA: 0s - loss: 0.0186 - accuracy: 0.9938
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0187 - accuracy: 0.9937
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0188 - accuracy: 0.9937
1581/1688 [===========================>..] - ETA: 0s - loss: 0.0190 - accuracy: 0.9937
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0191 - accuracy: 0.9936
1627/1688 [===========================>..] - ETA: 0s - loss: 0.0194 - accuracy: 0.9935
1649/1688 [============================>.] - ETA: 0s - loss: 0.0195 - accuracy: 0.9934
1672/1688 [============================>.] - ETA: 0s - loss: 0.0196 - accuracy: 0.9934
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0195 - accuracy: 0.9934 - val_loss: 0.0575 - val_accuracy: 0.9872
Epoch 9/10
1/1688 [..............................] - ETA: 4s - loss: 1.0254e-04 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9961
47/1688 [..............................] - ETA: 3s - loss: 0.0108 - accuracy: 0.9967
69/1688 [>.............................] - ETA: 3s - loss: 0.0133 - accuracy: 0.9955
91/1688 [>.............................] - ETA: 3s - loss: 0.0159 - accuracy: 0.9942
114/1688 [=>............................] - ETA: 3s - loss: 0.0138 - accuracy: 0.9951
136/1688 [=>............................] - ETA: 3s - loss: 0.0130 - accuracy: 0.9952
159/1688 [=>............................] - ETA: 3s - loss: 0.0143 - accuracy: 0.9949
182/1688 [==>...........................] - ETA: 3s - loss: 0.0140 - accuracy: 0.9952
204/1688 [==>...........................] - ETA: 3s - loss: 0.0130 - accuracy: 0.9957
227/1688 [===>..........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9959
250/1688 [===>..........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9956
271/1688 [===>..........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9957
293/1688 [====>.........................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9957
315/1688 [====>.........................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9957
337/1688 [====>.........................] - ETA: 3s - loss: 0.0128 - accuracy: 0.9958
360/1688 [=====>........................] - ETA: 3s - loss: 0.0127 - accuracy: 0.9959
383/1688 [=====>........................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9960
405/1688 [======>.......................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9961
427/1688 [======>.......................] - ETA: 2s - loss: 0.0125 - accuracy: 0.9959
449/1688 [======>.......................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9958
471/1688 [=======>......................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9956
494/1688 [=======>......................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9953
516/1688 [========>.....................] - ETA: 2s - loss: 0.0136 - accuracy: 0.9954
539/1688 [========>.....................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9952
561/1688 [========>.....................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9950
584/1688 [=========>....................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9950
607/1688 [=========>....................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9951
629/1688 [==========>...................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9952
651/1688 [==========>...................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9952
674/1688 [==========>...................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9951
697/1688 [===========>..................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9952
720/1688 [===========>..................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9954
743/1688 [============>.................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9953
765/1688 [============>.................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9953
788/1688 [=============>................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9953
811/1688 [=============>................] - ETA: 1s - loss: 0.0144 - accuracy: 0.9952
834/1688 [=============>................] - ETA: 1s - loss: 0.0145 - accuracy: 0.9953
856/1688 [==============>...............] - ETA: 1s - loss: 0.0151 - accuracy: 0.9952
879/1688 [==============>...............] - ETA: 1s - loss: 0.0153 - accuracy: 0.9953
902/1688 [===============>..............] - ETA: 1s - loss: 0.0152 - accuracy: 0.9953
925/1688 [===============>..............] - ETA: 1s - loss: 0.0151 - accuracy: 0.9952
947/1688 [===============>..............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9953
969/1688 [================>.............] - ETA: 1s - loss: 0.0149 - accuracy: 0.9953
991/1688 [================>.............] - ETA: 1s - loss: 0.0150 - accuracy: 0.9953
1014/1688 [=================>............] - ETA: 1s - loss: 0.0148 - accuracy: 0.9953
1036/1688 [=================>............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9954
1059/1688 [=================>............] - ETA: 1s - loss: 0.0146 - accuracy: 0.9954
1082/1688 [==================>...........] - ETA: 1s - loss: 0.0144 - accuracy: 0.9954
1104/1688 [==================>...........] - ETA: 1s - loss: 0.0145 - accuracy: 0.9953
1127/1688 [===================>..........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9952
1150/1688 [===================>..........] - ETA: 1s - loss: 0.0146 - accuracy: 0.9953
1173/1688 [===================>..........] - ETA: 1s - loss: 0.0146 - accuracy: 0.9953
1195/1688 [====================>.........] - ETA: 1s - loss: 0.0148 - accuracy: 0.9952
1218/1688 [====================>.........] - ETA: 1s - loss: 0.0148 - accuracy: 0.9951
1240/1688 [=====================>........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9951
1262/1688 [=====================>........] - ETA: 0s - loss: 0.0150 - accuracy: 0.9950
1284/1688 [=====================>........] - ETA: 0s - loss: 0.0151 - accuracy: 0.9950
1306/1688 [======================>.......] - ETA: 0s - loss: 0.0151 - accuracy: 0.9950
1328/1688 [======================>.......] - ETA: 0s - loss: 0.0154 - accuracy: 0.9948
1350/1688 [======================>.......] - ETA: 0s - loss: 0.0158 - accuracy: 0.9948
1372/1688 [=======================>......] - ETA: 0s - loss: 0.0156 - accuracy: 0.9948
1396/1688 [=======================>......] - ETA: 0s - loss: 0.0155 - accuracy: 0.9949
1419/1688 [========================>.....] - ETA: 0s - loss: 0.0156 - accuracy: 0.9948
1441/1688 [========================>.....] - ETA: 0s - loss: 0.0157 - accuracy: 0.9948
1463/1688 [=========================>....] - ETA: 0s - loss: 0.0158 - accuracy: 0.9947
1486/1688 [=========================>....] - ETA: 0s - loss: 0.0160 - accuracy: 0.9947
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0161 - accuracy: 0.9946
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0160 - accuracy: 0.9947
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0161 - accuracy: 0.9946
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0163 - accuracy: 0.9946
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0164 - accuracy: 0.9945
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0165 - accuracy: 0.9945
1639/1688 [============================>.] - ETA: 0s - loss: 0.0166 - accuracy: 0.9944
1661/1688 [============================>.] - ETA: 0s - loss: 0.0167 - accuracy: 0.9944
1684/1688 [============================>.] - ETA: 0s - loss: 0.0167 - accuracy: 0.9944
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0168 - accuracy: 0.9944 - val_loss: 0.0559 - val_accuracy: 0.9878
Epoch 10/10
1/1688 [..............................] - ETA: 4s - loss: 1.7250e-04 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0231 - accuracy: 0.9932
45/1688 [..............................] - ETA: 3s - loss: 0.0175 - accuracy: 0.9944
68/1688 [>.............................] - ETA: 3s - loss: 0.0151 - accuracy: 0.9954
91/1688 [>.............................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9955
114/1688 [=>............................] - ETA: 3s - loss: 0.0138 - accuracy: 0.9953
136/1688 [=>............................] - ETA: 3s - loss: 0.0130 - accuracy: 0.9959
159/1688 [=>............................] - ETA: 3s - loss: 0.0128 - accuracy: 0.9955
180/1688 [==>...........................] - ETA: 3s - loss: 0.0122 - accuracy: 0.9957
203/1688 [==>...........................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9957
226/1688 [===>..........................] - ETA: 3s - loss: 0.0122 - accuracy: 0.9959
249/1688 [===>..........................] - ETA: 3s - loss: 0.0115 - accuracy: 0.9962
271/1688 [===>..........................] - ETA: 3s - loss: 0.0115 - accuracy: 0.9961
294/1688 [====>.........................] - ETA: 3s - loss: 0.0113 - accuracy: 0.9961
317/1688 [====>.........................] - ETA: 3s - loss: 0.0110 - accuracy: 0.9963
340/1688 [=====>........................] - ETA: 3s - loss: 0.0117 - accuracy: 0.9959
363/1688 [=====>........................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9960
386/1688 [=====>........................] - ETA: 2s - loss: 0.0115 - accuracy: 0.9960
409/1688 [======>.......................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9959
431/1688 [======>.......................] - ETA: 2s - loss: 0.0114 - accuracy: 0.9959
454/1688 [=======>......................] - ETA: 2s - loss: 0.0114 - accuracy: 0.9959
476/1688 [=======>......................] - ETA: 2s - loss: 0.0115 - accuracy: 0.9959
499/1688 [=======>......................] - ETA: 2s - loss: 0.0115 - accuracy: 0.9959
521/1688 [========>.....................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9959
544/1688 [========>.....................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9960
566/1688 [=========>....................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9959
588/1688 [=========>....................] - ETA: 2s - loss: 0.0117 - accuracy: 0.9959
611/1688 [=========>....................] - ETA: 2s - loss: 0.0116 - accuracy: 0.9959
633/1688 [==========>...................] - ETA: 2s - loss: 0.0115 - accuracy: 0.9960
655/1688 [==========>...................] - ETA: 2s - loss: 0.0118 - accuracy: 0.9958
678/1688 [===========>..................] - ETA: 2s - loss: 0.0119 - accuracy: 0.9957
701/1688 [===========>..................] - ETA: 2s - loss: 0.0120 - accuracy: 0.9956
723/1688 [===========>..................] - ETA: 2s - loss: 0.0120 - accuracy: 0.9956
746/1688 [============>.................] - ETA: 2s - loss: 0.0123 - accuracy: 0.9956
769/1688 [============>.................] - ETA: 2s - loss: 0.0122 - accuracy: 0.9956
791/1688 [=============>................] - ETA: 2s - loss: 0.0124 - accuracy: 0.9956
813/1688 [=============>................] - ETA: 1s - loss: 0.0124 - accuracy: 0.9956
836/1688 [=============>................] - ETA: 1s - loss: 0.0122 - accuracy: 0.9957
859/1688 [==============>...............] - ETA: 1s - loss: 0.0122 - accuracy: 0.9956
881/1688 [==============>...............] - ETA: 1s - loss: 0.0123 - accuracy: 0.9956
903/1688 [===============>..............] - ETA: 1s - loss: 0.0127 - accuracy: 0.9955
926/1688 [===============>..............] - ETA: 1s - loss: 0.0128 - accuracy: 0.9956
949/1688 [===============>..............] - ETA: 1s - loss: 0.0133 - accuracy: 0.9954
972/1688 [================>.............] - ETA: 1s - loss: 0.0137 - accuracy: 0.9952
994/1688 [================>.............] - ETA: 1s - loss: 0.0136 - accuracy: 0.9953
1018/1688 [=================>............] - ETA: 1s - loss: 0.0136 - accuracy: 0.9952
1040/1688 [=================>............] - ETA: 1s - loss: 0.0139 - accuracy: 0.9952
1063/1688 [=================>............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9951
1085/1688 [==================>...........] - ETA: 1s - loss: 0.0146 - accuracy: 0.9952
1108/1688 [==================>...........] - ETA: 1s - loss: 0.0143 - accuracy: 0.9952
1131/1688 [===================>..........] - ETA: 1s - loss: 0.0145 - accuracy: 0.9951
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0144 - accuracy: 0.9951
1176/1688 [===================>..........] - ETA: 1s - loss: 0.0143 - accuracy: 0.9952
1198/1688 [====================>.........] - ETA: 1s - loss: 0.0142 - accuracy: 0.9952
1221/1688 [====================>.........] - ETA: 1s - loss: 0.0142 - accuracy: 0.9952
1243/1688 [=====================>........] - ETA: 1s - loss: 0.0143 - accuracy: 0.9951
1266/1688 [=====================>........] - ETA: 0s - loss: 0.0145 - accuracy: 0.9950
1289/1688 [=====================>........] - ETA: 0s - loss: 0.0149 - accuracy: 0.9949
1311/1688 [======================>.......] - ETA: 0s - loss: 0.0148 - accuracy: 0.9950
1333/1688 [======================>.......] - ETA: 0s - loss: 0.0147 - accuracy: 0.9950
1356/1688 [=======================>......] - ETA: 0s - loss: 0.0147 - accuracy: 0.9950
1379/1688 [=======================>......] - ETA: 0s - loss: 0.0147 - accuracy: 0.9950
1401/1688 [=======================>......] - ETA: 0s - loss: 0.0146 - accuracy: 0.9950
1423/1688 [========================>.....] - ETA: 0s - loss: 0.0146 - accuracy: 0.9950
1445/1688 [========================>.....] - ETA: 0s - loss: 0.0145 - accuracy: 0.9950
1467/1688 [=========================>....] - ETA: 0s - loss: 0.0145 - accuracy: 0.9950
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0146 - accuracy: 0.9950
1511/1688 [=========================>....] - ETA: 0s - loss: 0.0145 - accuracy: 0.9950
1535/1688 [==========================>...] - ETA: 0s - loss: 0.0146 - accuracy: 0.9950
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0148 - accuracy: 0.9950
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0147 - accuracy: 0.9950
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0149 - accuracy: 0.9950
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0148 - accuracy: 0.9950
1648/1688 [============================>.] - ETA: 0s - loss: 0.0147 - accuracy: 0.9950
1671/1688 [============================>.] - ETA: 0s - loss: 0.0148 - accuracy: 0.9950
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0148 - accuracy: 0.9950 - val_loss: 0.0562 - val_accuracy: 0.9882
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.9860000014305115
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bit integer activations and weights. Therefore, the floating-point TF-Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a TF-Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating-point TF-Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a TF-Keras model with custom QuantizeML quantized layers substituted for the original TF-Keras layers.
All TF-Keras API functions can be applied on this new model: summary(), compile(),
fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:577: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples. Continuing execution.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate. "
1/1024 [..............................] - ETA: 2:24
52/1024 [>.............................] - ETA: 0s
103/1024 [==>...........................] - ETA: 0s
154/1024 [===>..........................] - ETA: 0s
206/1024 [=====>........................] - ETA: 0s
258/1024 [======>.......................] - ETA: 0s
309/1024 [========>.....................] - ETA: 0s
361/1024 [=========>....................] - ETA: 0s
414/1024 [===========>..................] - ETA: 0s
466/1024 [============>.................] - ETA: 0s
517/1024 [==============>...............] - ETA: 0s
568/1024 [===============>..............] - ETA: 0s
620/1024 [=================>............] - ETA: 0s
671/1024 [==================>...........] - ETA: 0s
722/1024 [====================>.........] - ETA: 0s
774/1024 [=====================>........] - ETA: 0s
826/1024 [=======================>......] - ETA: 0s
878/1024 [========================>.....] - ETA: 0s
930/1024 [==========================>...] - ETA: 0s
983/1024 [===========================>..] - ETA: 0s
1024/1024 [==============================] - 1s 977us/step
model_quantized.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input (InputLayer) [(None, 28, 28, 1)] 0
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.9810000061988831
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.9832000136375427
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating-point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.9797999858856201
2.4. Model fine-tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine-tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine-tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 1:47:44 - loss: 0.0202 - accuracy: 1.0000
10/1688 [..............................] - ETA: 10s - loss: 0.0402 - accuracy: 0.9875
19/1688 [..............................] - ETA: 10s - loss: 0.0538 - accuracy: 0.9819
28/1688 [..............................] - ETA: 10s - loss: 0.0456 - accuracy: 0.9833
36/1688 [..............................] - ETA: 10s - loss: 0.0394 - accuracy: 0.9861
45/1688 [..............................] - ETA: 10s - loss: 0.0376 - accuracy: 0.9861
54/1688 [..............................] - ETA: 10s - loss: 0.0376 - accuracy: 0.9861
63/1688 [>.............................] - ETA: 10s - loss: 0.0352 - accuracy: 0.9871
72/1688 [>.............................] - ETA: 9s - loss: 0.0334 - accuracy: 0.9874
81/1688 [>.............................] - ETA: 9s - loss: 0.0317 - accuracy: 0.9884
90/1688 [>.............................] - ETA: 9s - loss: 0.0300 - accuracy: 0.9896
98/1688 [>.............................] - ETA: 9s - loss: 0.0299 - accuracy: 0.9898
107/1688 [>.............................] - ETA: 9s - loss: 0.0317 - accuracy: 0.9898
116/1688 [=>............................] - ETA: 9s - loss: 0.0314 - accuracy: 0.9900
124/1688 [=>............................] - ETA: 9s - loss: 0.0313 - accuracy: 0.9902
133/1688 [=>............................] - ETA: 9s - loss: 0.0314 - accuracy: 0.9904
141/1688 [=>............................] - ETA: 9s - loss: 0.0313 - accuracy: 0.9900
150/1688 [=>............................] - ETA: 9s - loss: 0.0300 - accuracy: 0.9904
159/1688 [=>............................] - ETA: 9s - loss: 0.0293 - accuracy: 0.9906
168/1688 [=>............................] - ETA: 9s - loss: 0.0293 - accuracy: 0.9905
176/1688 [==>...........................] - ETA: 9s - loss: 0.0287 - accuracy: 0.9909
184/1688 [==>...........................] - ETA: 9s - loss: 0.0289 - accuracy: 0.9912
193/1688 [==>...........................] - ETA: 9s - loss: 0.0283 - accuracy: 0.9914
201/1688 [==>...........................] - ETA: 9s - loss: 0.0278 - accuracy: 0.9916
210/1688 [==>...........................] - ETA: 9s - loss: 0.0273 - accuracy: 0.9918
218/1688 [==>...........................] - ETA: 9s - loss: 0.0270 - accuracy: 0.9918
226/1688 [===>..........................] - ETA: 8s - loss: 0.0267 - accuracy: 0.9918
235/1688 [===>..........................] - ETA: 8s - loss: 0.0265 - accuracy: 0.9920
244/1688 [===>..........................] - ETA: 8s - loss: 0.0265 - accuracy: 0.9921
253/1688 [===>..........................] - ETA: 8s - loss: 0.0259 - accuracy: 0.9923
262/1688 [===>..........................] - ETA: 8s - loss: 0.0260 - accuracy: 0.9921
271/1688 [===>..........................] - ETA: 8s - loss: 0.0264 - accuracy: 0.9919
279/1688 [===>..........................] - ETA: 8s - loss: 0.0260 - accuracy: 0.9922
287/1688 [====>.........................] - ETA: 8s - loss: 0.0257 - accuracy: 0.9923
295/1688 [====>.........................] - ETA: 8s - loss: 0.0254 - accuracy: 0.9924
304/1688 [====>.........................] - ETA: 8s - loss: 0.0255 - accuracy: 0.9923
313/1688 [====>.........................] - ETA: 8s - loss: 0.0260 - accuracy: 0.9920
322/1688 [====>.........................] - ETA: 8s - loss: 0.0257 - accuracy: 0.9921
331/1688 [====>.........................] - ETA: 8s - loss: 0.0256 - accuracy: 0.9922
339/1688 [=====>........................] - ETA: 8s - loss: 0.0256 - accuracy: 0.9923
348/1688 [=====>........................] - ETA: 8s - loss: 0.0254 - accuracy: 0.9924
357/1688 [=====>........................] - ETA: 8s - loss: 0.0253 - accuracy: 0.9924
365/1688 [=====>........................] - ETA: 8s - loss: 0.0251 - accuracy: 0.9926
374/1688 [=====>........................] - ETA: 8s - loss: 0.0253 - accuracy: 0.9924
383/1688 [=====>........................] - ETA: 8s - loss: 0.0251 - accuracy: 0.9923
392/1688 [=====>........................] - ETA: 7s - loss: 0.0254 - accuracy: 0.9923
401/1688 [======>.......................] - ETA: 7s - loss: 0.0254 - accuracy: 0.9924
410/1688 [======>.......................] - ETA: 7s - loss: 0.0252 - accuracy: 0.9925
419/1688 [======>.......................] - ETA: 7s - loss: 0.0250 - accuracy: 0.9925
428/1688 [======>.......................] - ETA: 7s - loss: 0.0247 - accuracy: 0.9927
437/1688 [======>.......................] - ETA: 7s - loss: 0.0251 - accuracy: 0.9926
446/1688 [======>.......................] - ETA: 7s - loss: 0.0249 - accuracy: 0.9927
455/1688 [=======>......................] - ETA: 7s - loss: 0.0245 - accuracy: 0.9929
464/1688 [=======>......................] - ETA: 7s - loss: 0.0244 - accuracy: 0.9929
473/1688 [=======>......................] - ETA: 7s - loss: 0.0242 - accuracy: 0.9929
482/1688 [=======>......................] - ETA: 7s - loss: 0.0241 - accuracy: 0.9930
491/1688 [=======>......................] - ETA: 7s - loss: 0.0238 - accuracy: 0.9931
500/1688 [=======>......................] - ETA: 7s - loss: 0.0236 - accuracy: 0.9932
508/1688 [========>.....................] - ETA: 7s - loss: 0.0235 - accuracy: 0.9933
516/1688 [========>.....................] - ETA: 7s - loss: 0.0234 - accuracy: 0.9933
525/1688 [========>.....................] - ETA: 7s - loss: 0.0233 - accuracy: 0.9932
534/1688 [========>.....................] - ETA: 7s - loss: 0.0235 - accuracy: 0.9930
543/1688 [========>.....................] - ETA: 7s - loss: 0.0232 - accuracy: 0.9932
552/1688 [========>.....................] - ETA: 6s - loss: 0.0231 - accuracy: 0.9933
560/1688 [========>.....................] - ETA: 6s - loss: 0.0228 - accuracy: 0.9934
569/1688 [=========>....................] - ETA: 6s - loss: 0.0227 - accuracy: 0.9935
578/1688 [=========>....................] - ETA: 6s - loss: 0.0226 - accuracy: 0.9935
586/1688 [=========>....................] - ETA: 6s - loss: 0.0225 - accuracy: 0.9935
595/1688 [=========>....................] - ETA: 6s - loss: 0.0225 - accuracy: 0.9935
604/1688 [=========>....................] - ETA: 6s - loss: 0.0223 - accuracy: 0.9936
613/1688 [=========>....................] - ETA: 6s - loss: 0.0224 - accuracy: 0.9935
622/1688 [==========>...................] - ETA: 6s - loss: 0.0225 - accuracy: 0.9935
630/1688 [==========>...................] - ETA: 6s - loss: 0.0223 - accuracy: 0.9935
639/1688 [==========>...................] - ETA: 6s - loss: 0.0222 - accuracy: 0.9935
648/1688 [==========>...................] - ETA: 6s - loss: 0.0221 - accuracy: 0.9935
657/1688 [==========>...................] - ETA: 6s - loss: 0.0221 - accuracy: 0.9935
666/1688 [==========>...................] - ETA: 6s - loss: 0.0221 - accuracy: 0.9935
675/1688 [==========>...................] - ETA: 6s - loss: 0.0222 - accuracy: 0.9934
684/1688 [===========>..................] - ETA: 6s - loss: 0.0221 - accuracy: 0.9934
693/1688 [===========>..................] - ETA: 6s - loss: 0.0222 - accuracy: 0.9934
702/1688 [===========>..................] - ETA: 6s - loss: 0.0223 - accuracy: 0.9933
711/1688 [===========>..................] - ETA: 5s - loss: 0.0222 - accuracy: 0.9933
720/1688 [===========>..................] - ETA: 5s - loss: 0.0225 - accuracy: 0.9932
729/1688 [===========>..................] - ETA: 5s - loss: 0.0224 - accuracy: 0.9933
738/1688 [============>.................] - ETA: 5s - loss: 0.0224 - accuracy: 0.9933
747/1688 [============>.................] - ETA: 5s - loss: 0.0225 - accuracy: 0.9932
756/1688 [============>.................] - ETA: 5s - loss: 0.0224 - accuracy: 0.9933
765/1688 [============>.................] - ETA: 5s - loss: 0.0223 - accuracy: 0.9933
774/1688 [============>.................] - ETA: 5s - loss: 0.0223 - accuracy: 0.9933
783/1688 [============>.................] - ETA: 5s - loss: 0.0223 - accuracy: 0.9933
791/1688 [=============>................] - ETA: 5s - loss: 0.0222 - accuracy: 0.9933
800/1688 [=============>................] - ETA: 5s - loss: 0.0222 - accuracy: 0.9933
809/1688 [=============>................] - ETA: 5s - loss: 0.0222 - accuracy: 0.9934
818/1688 [=============>................] - ETA: 5s - loss: 0.0221 - accuracy: 0.9934
827/1688 [=============>................] - ETA: 5s - loss: 0.0222 - accuracy: 0.9933
836/1688 [=============>................] - ETA: 5s - loss: 0.0221 - accuracy: 0.9933
844/1688 [==============>...............] - ETA: 5s - loss: 0.0221 - accuracy: 0.9934
853/1688 [==============>...............] - ETA: 5s - loss: 0.0219 - accuracy: 0.9934
862/1688 [==============>...............] - ETA: 5s - loss: 0.0219 - accuracy: 0.9935
871/1688 [==============>...............] - ETA: 4s - loss: 0.0218 - accuracy: 0.9935
880/1688 [==============>...............] - ETA: 4s - loss: 0.0217 - accuracy: 0.9935
889/1688 [==============>...............] - ETA: 4s - loss: 0.0216 - accuracy: 0.9935
898/1688 [==============>...............] - ETA: 4s - loss: 0.0216 - accuracy: 0.9935
907/1688 [===============>..............] - ETA: 4s - loss: 0.0216 - accuracy: 0.9935
916/1688 [===============>..............] - ETA: 4s - loss: 0.0215 - accuracy: 0.9936
925/1688 [===============>..............] - ETA: 4s - loss: 0.0214 - accuracy: 0.9936
934/1688 [===============>..............] - ETA: 4s - loss: 0.0214 - accuracy: 0.9937
943/1688 [===============>..............] - ETA: 4s - loss: 0.0212 - accuracy: 0.9937
952/1688 [===============>..............] - ETA: 4s - loss: 0.0211 - accuracy: 0.9938
961/1688 [================>.............] - ETA: 4s - loss: 0.0211 - accuracy: 0.9938
970/1688 [================>.............] - ETA: 4s - loss: 0.0211 - accuracy: 0.9937
979/1688 [================>.............] - ETA: 4s - loss: 0.0211 - accuracy: 0.9938
988/1688 [================>.............] - ETA: 4s - loss: 0.0211 - accuracy: 0.9938
997/1688 [================>.............] - ETA: 4s - loss: 0.0212 - accuracy: 0.9936
1006/1688 [================>.............] - ETA: 4s - loss: 0.0213 - accuracy: 0.9936
1014/1688 [=================>............] - ETA: 4s - loss: 0.0213 - accuracy: 0.9936
1023/1688 [=================>............] - ETA: 4s - loss: 0.0212 - accuracy: 0.9936
1032/1688 [=================>............] - ETA: 4s - loss: 0.0213 - accuracy: 0.9936
1041/1688 [=================>............] - ETA: 3s - loss: 0.0213 - accuracy: 0.9936
1050/1688 [=================>............] - ETA: 3s - loss: 0.0212 - accuracy: 0.9936
1059/1688 [=================>............] - ETA: 3s - loss: 0.0211 - accuracy: 0.9937
1068/1688 [=================>............] - ETA: 3s - loss: 0.0212 - accuracy: 0.9936
1077/1688 [==================>...........] - ETA: 3s - loss: 0.0211 - accuracy: 0.9937
1086/1688 [==================>...........] - ETA: 3s - loss: 0.0211 - accuracy: 0.9937
1094/1688 [==================>...........] - ETA: 3s - loss: 0.0210 - accuracy: 0.9937
1103/1688 [==================>...........] - ETA: 3s - loss: 0.0210 - accuracy: 0.9937
1112/1688 [==================>...........] - ETA: 3s - loss: 0.0211 - accuracy: 0.9937
1121/1688 [==================>...........] - ETA: 3s - loss: 0.0210 - accuracy: 0.9936
1130/1688 [===================>..........] - ETA: 3s - loss: 0.0210 - accuracy: 0.9937
1139/1688 [===================>..........] - ETA: 3s - loss: 0.0209 - accuracy: 0.9937
1148/1688 [===================>..........] - ETA: 3s - loss: 0.0208 - accuracy: 0.9937
1157/1688 [===================>..........] - ETA: 3s - loss: 0.0208 - accuracy: 0.9938
1165/1688 [===================>..........] - ETA: 3s - loss: 0.0208 - accuracy: 0.9938
1173/1688 [===================>..........] - ETA: 3s - loss: 0.0207 - accuracy: 0.9938
1182/1688 [====================>.........] - ETA: 3s - loss: 0.0207 - accuracy: 0.9938
1191/1688 [====================>.........] - ETA: 3s - loss: 0.0207 - accuracy: 0.9938
1200/1688 [====================>.........] - ETA: 2s - loss: 0.0207 - accuracy: 0.9938
1209/1688 [====================>.........] - ETA: 2s - loss: 0.0208 - accuracy: 0.9937
1218/1688 [====================>.........] - ETA: 2s - loss: 0.0208 - accuracy: 0.9937
1227/1688 [====================>.........] - ETA: 2s - loss: 0.0207 - accuracy: 0.9938
1235/1688 [====================>.........] - ETA: 2s - loss: 0.0206 - accuracy: 0.9938
1243/1688 [=====================>........] - ETA: 2s - loss: 0.0207 - accuracy: 0.9938
1252/1688 [=====================>........] - ETA: 2s - loss: 0.0207 - accuracy: 0.9937
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0206 - accuracy: 0.9938
1269/1688 [=====================>........] - ETA: 2s - loss: 0.0206 - accuracy: 0.9937
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0206 - accuracy: 0.9937
1287/1688 [=====================>........] - ETA: 2s - loss: 0.0207 - accuracy: 0.9937
1296/1688 [======================>.......] - ETA: 2s - loss: 0.0207 - accuracy: 0.9937
1305/1688 [======================>.......] - ETA: 2s - loss: 0.0206 - accuracy: 0.9937
1314/1688 [======================>.......] - ETA: 2s - loss: 0.0206 - accuracy: 0.9937
1323/1688 [======================>.......] - ETA: 2s - loss: 0.0206 - accuracy: 0.9938
1332/1688 [======================>.......] - ETA: 2s - loss: 0.0205 - accuracy: 0.9938
1341/1688 [======================>.......] - ETA: 2s - loss: 0.0205 - accuracy: 0.9938
1350/1688 [======================>.......] - ETA: 2s - loss: 0.0204 - accuracy: 0.9939
1358/1688 [=======================>......] - ETA: 2s - loss: 0.0204 - accuracy: 0.9939
1367/1688 [=======================>......] - ETA: 1s - loss: 0.0204 - accuracy: 0.9939
1376/1688 [=======================>......] - ETA: 1s - loss: 0.0203 - accuracy: 0.9939
1385/1688 [=======================>......] - ETA: 1s - loss: 0.0202 - accuracy: 0.9940
1394/1688 [=======================>......] - ETA: 1s - loss: 0.0202 - accuracy: 0.9940
1403/1688 [=======================>......] - ETA: 1s - loss: 0.0201 - accuracy: 0.9940
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0201 - accuracy: 0.9940
1421/1688 [========================>.....] - ETA: 1s - loss: 0.0200 - accuracy: 0.9941
1430/1688 [========================>.....] - ETA: 1s - loss: 0.0200 - accuracy: 0.9941
1439/1688 [========================>.....] - ETA: 1s - loss: 0.0199 - accuracy: 0.9941
1448/1688 [========================>.....] - ETA: 1s - loss: 0.0199 - accuracy: 0.9941
1457/1688 [========================>.....] - ETA: 1s - loss: 0.0199 - accuracy: 0.9941
1466/1688 [=========================>....] - ETA: 1s - loss: 0.0198 - accuracy: 0.9942
1475/1688 [=========================>....] - ETA: 1s - loss: 0.0198 - accuracy: 0.9942
1483/1688 [=========================>....] - ETA: 1s - loss: 0.0198 - accuracy: 0.9942
1492/1688 [=========================>....] - ETA: 1s - loss: 0.0198 - accuracy: 0.9942
1501/1688 [=========================>....] - ETA: 1s - loss: 0.0198 - accuracy: 0.9942
1509/1688 [=========================>....] - ETA: 1s - loss: 0.0198 - accuracy: 0.9942
1518/1688 [=========================>....] - ETA: 1s - loss: 0.0198 - accuracy: 0.9942
1527/1688 [==========================>...] - ETA: 0s - loss: 0.0198 - accuracy: 0.9942
1536/1688 [==========================>...] - ETA: 0s - loss: 0.0197 - accuracy: 0.9942
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0197 - accuracy: 0.9942
1554/1688 [==========================>...] - ETA: 0s - loss: 0.0197 - accuracy: 0.9942
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0197 - accuracy: 0.9942
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0196 - accuracy: 0.9942
1581/1688 [===========================>..] - ETA: 0s - loss: 0.0196 - accuracy: 0.9942
1590/1688 [===========================>..] - ETA: 0s - loss: 0.0196 - accuracy: 0.9942
1599/1688 [===========================>..] - ETA: 0s - loss: 0.0196 - accuracy: 0.9942
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0196 - accuracy: 0.9942
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0195 - accuracy: 0.9943
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0195 - accuracy: 0.9943
1635/1688 [============================>.] - ETA: 0s - loss: 0.0194 - accuracy: 0.9943
1643/1688 [============================>.] - ETA: 0s - loss: 0.0194 - accuracy: 0.9943
1652/1688 [============================>.] - ETA: 0s - loss: 0.0193 - accuracy: 0.9943
1661/1688 [============================>.] - ETA: 0s - loss: 0.0194 - accuracy: 0.9943
1670/1688 [============================>.] - ETA: 0s - loss: 0.0195 - accuracy: 0.9942
1679/1688 [============================>.] - ETA: 0s - loss: 0.0194 - accuracy: 0.9943
1688/1688 [==============================] - ETA: 0s - loss: 0.0194 - accuracy: 0.9943
1688/1688 [==============================] - 16s 7ms/step - loss: 0.0194 - accuracy: 0.9943 - val_loss: 0.0433 - val_accuracy: 0.9882
Epoch 2/5
1/1688 [..............................] - ETA: 11s - loss: 0.0094 - accuracy: 1.0000
10/1688 [..............................] - ETA: 10s - loss: 0.0110 - accuracy: 0.9969
19/1688 [..............................] - ETA: 10s - loss: 0.0089 - accuracy: 0.9984
28/1688 [..............................] - ETA: 10s - loss: 0.0093 - accuracy: 0.9978
37/1688 [..............................] - ETA: 10s - loss: 0.0110 - accuracy: 0.9966
46/1688 [..............................] - ETA: 10s - loss: 0.0107 - accuracy: 0.9973
55/1688 [..............................] - ETA: 9s - loss: 0.0112 - accuracy: 0.9966
64/1688 [>.............................] - ETA: 9s - loss: 0.0114 - accuracy: 0.9971
73/1688 [>.............................] - ETA: 9s - loss: 0.0106 - accuracy: 0.9974
82/1688 [>.............................] - ETA: 9s - loss: 0.0110 - accuracy: 0.9973
91/1688 [>.............................] - ETA: 9s - loss: 0.0118 - accuracy: 0.9966
99/1688 [>.............................] - ETA: 9s - loss: 0.0123 - accuracy: 0.9965
108/1688 [>.............................] - ETA: 9s - loss: 0.0143 - accuracy: 0.9959
117/1688 [=>............................] - ETA: 9s - loss: 0.0148 - accuracy: 0.9957
126/1688 [=>............................] - ETA: 9s - loss: 0.0142 - accuracy: 0.9960
135/1688 [=>............................] - ETA: 9s - loss: 0.0138 - accuracy: 0.9963
143/1688 [=>............................] - ETA: 9s - loss: 0.0139 - accuracy: 0.9963
152/1688 [=>............................] - ETA: 9s - loss: 0.0135 - accuracy: 0.9965
161/1688 [=>............................] - ETA: 9s - loss: 0.0134 - accuracy: 0.9967
170/1688 [==>...........................] - ETA: 9s - loss: 0.0131 - accuracy: 0.9969
178/1688 [==>...........................] - ETA: 9s - loss: 0.0130 - accuracy: 0.9970
186/1688 [==>...........................] - ETA: 9s - loss: 0.0131 - accuracy: 0.9968
194/1688 [==>...........................] - ETA: 9s - loss: 0.0134 - accuracy: 0.9966
203/1688 [==>...........................] - ETA: 9s - loss: 0.0135 - accuracy: 0.9965
212/1688 [==>...........................] - ETA: 9s - loss: 0.0133 - accuracy: 0.9966
221/1688 [==>...........................] - ETA: 8s - loss: 0.0134 - accuracy: 0.9966
229/1688 [===>..........................] - ETA: 8s - loss: 0.0132 - accuracy: 0.9967
237/1688 [===>..........................] - ETA: 8s - loss: 0.0128 - accuracy: 0.9968
246/1688 [===>..........................] - ETA: 8s - loss: 0.0127 - accuracy: 0.9968
255/1688 [===>..........................] - ETA: 8s - loss: 0.0125 - accuracy: 0.9969
263/1688 [===>..........................] - ETA: 8s - loss: 0.0127 - accuracy: 0.9968
271/1688 [===>..........................] - ETA: 8s - loss: 0.0127 - accuracy: 0.9968
280/1688 [===>..........................] - ETA: 8s - loss: 0.0128 - accuracy: 0.9968
288/1688 [====>.........................] - ETA: 8s - loss: 0.0129 - accuracy: 0.9966
297/1688 [====>.........................] - ETA: 8s - loss: 0.0130 - accuracy: 0.9965
305/1688 [====>.........................] - ETA: 8s - loss: 0.0133 - accuracy: 0.9964
314/1688 [====>.........................] - ETA: 8s - loss: 0.0135 - accuracy: 0.9964
323/1688 [====>.........................] - ETA: 8s - loss: 0.0135 - accuracy: 0.9964
332/1688 [====>.........................] - ETA: 8s - loss: 0.0139 - accuracy: 0.9962
341/1688 [=====>........................] - ETA: 8s - loss: 0.0138 - accuracy: 0.9962
350/1688 [=====>........................] - ETA: 8s - loss: 0.0138 - accuracy: 0.9962
359/1688 [=====>........................] - ETA: 8s - loss: 0.0138 - accuracy: 0.9962
367/1688 [=====>........................] - ETA: 8s - loss: 0.0137 - accuracy: 0.9963
376/1688 [=====>........................] - ETA: 8s - loss: 0.0137 - accuracy: 0.9963
384/1688 [=====>........................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9962
393/1688 [=====>........................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9962
402/1688 [======>.......................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9963
411/1688 [======>.......................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9963
420/1688 [======>.......................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9964
429/1688 [======>.......................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9963
437/1688 [======>.......................] - ETA: 7s - loss: 0.0139 - accuracy: 0.9962
446/1688 [======>.......................] - ETA: 7s - loss: 0.0139 - accuracy: 0.9963
455/1688 [=======>......................] - ETA: 7s - loss: 0.0141 - accuracy: 0.9962
464/1688 [=======>......................] - ETA: 7s - loss: 0.0139 - accuracy: 0.9963
473/1688 [=======>......................] - ETA: 7s - loss: 0.0139 - accuracy: 0.9963
481/1688 [=======>......................] - ETA: 7s - loss: 0.0138 - accuracy: 0.9963
489/1688 [=======>......................] - ETA: 7s - loss: 0.0137 - accuracy: 0.9964
498/1688 [=======>......................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9964
507/1688 [========>.....................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9964
516/1688 [========>.....................] - ETA: 7s - loss: 0.0136 - accuracy: 0.9964
525/1688 [========>.....................] - ETA: 7s - loss: 0.0134 - accuracy: 0.9965
534/1688 [========>.....................] - ETA: 7s - loss: 0.0133 - accuracy: 0.9965
542/1688 [========>.....................] - ETA: 7s - loss: 0.0134 - accuracy: 0.9965
551/1688 [========>.....................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9965
559/1688 [========>.....................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9966
568/1688 [=========>....................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9966
577/1688 [=========>....................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9965
586/1688 [=========>....................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9965
595/1688 [=========>....................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9965
604/1688 [=========>....................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9965
613/1688 [=========>....................] - ETA: 6s - loss: 0.0134 - accuracy: 0.9965
622/1688 [==========>...................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9965
631/1688 [==========>...................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9966
640/1688 [==========>...................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9966
649/1688 [==========>...................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9966
658/1688 [==========>...................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9966
667/1688 [==========>...................] - ETA: 6s - loss: 0.0131 - accuracy: 0.9966
676/1688 [===========>..................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9966
685/1688 [===========>..................] - ETA: 6s - loss: 0.0132 - accuracy: 0.9965
694/1688 [===========>..................] - ETA: 6s - loss: 0.0133 - accuracy: 0.9965
703/1688 [===========>..................] - ETA: 6s - loss: 0.0135 - accuracy: 0.9964
712/1688 [===========>..................] - ETA: 5s - loss: 0.0136 - accuracy: 0.9964
721/1688 [===========>..................] - ETA: 5s - loss: 0.0135 - accuracy: 0.9964
730/1688 [===========>..................] - ETA: 5s - loss: 0.0135 - accuracy: 0.9963
739/1688 [============>.................] - ETA: 5s - loss: 0.0136 - accuracy: 0.9963
748/1688 [============>.................] - ETA: 5s - loss: 0.0135 - accuracy: 0.9963
757/1688 [============>.................] - ETA: 5s - loss: 0.0135 - accuracy: 0.9963
766/1688 [============>.................] - ETA: 5s - loss: 0.0136 - accuracy: 0.9963
775/1688 [============>.................] - ETA: 5s - loss: 0.0136 - accuracy: 0.9962
784/1688 [============>.................] - ETA: 5s - loss: 0.0136 - accuracy: 0.9963
793/1688 [=============>................] - ETA: 5s - loss: 0.0135 - accuracy: 0.9963
802/1688 [=============>................] - ETA: 5s - loss: 0.0135 - accuracy: 0.9963
811/1688 [=============>................] - ETA: 5s - loss: 0.0135 - accuracy: 0.9963
820/1688 [=============>................] - ETA: 5s - loss: 0.0137 - accuracy: 0.9963
829/1688 [=============>................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9962
838/1688 [=============>................] - ETA: 5s - loss: 0.0138 - accuracy: 0.9962
847/1688 [==============>...............] - ETA: 5s - loss: 0.0137 - accuracy: 0.9963
856/1688 [==============>...............] - ETA: 5s - loss: 0.0138 - accuracy: 0.9963
865/1688 [==============>...............] - ETA: 5s - loss: 0.0138 - accuracy: 0.9962
873/1688 [==============>...............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9963
882/1688 [==============>...............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9963
891/1688 [==============>...............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9963
900/1688 [==============>...............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9964
908/1688 [===============>..............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9964
917/1688 [===============>..............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9964
926/1688 [===============>..............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9963
935/1688 [===============>..............] - ETA: 4s - loss: 0.0139 - accuracy: 0.9963
943/1688 [===============>..............] - ETA: 4s - loss: 0.0139 - accuracy: 0.9963
952/1688 [===============>..............] - ETA: 4s - loss: 0.0139 - accuracy: 0.9963
961/1688 [================>.............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9963
970/1688 [================>.............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9963
979/1688 [================>.............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9963
988/1688 [================>.............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9963
997/1688 [================>.............] - ETA: 4s - loss: 0.0137 - accuracy: 0.9963
1006/1688 [================>.............] - ETA: 4s - loss: 0.0139 - accuracy: 0.9962
1015/1688 [=================>............] - ETA: 4s - loss: 0.0138 - accuracy: 0.9962
1024/1688 [=================>............] - ETA: 4s - loss: 0.0140 - accuracy: 0.9962
1033/1688 [=================>............] - ETA: 4s - loss: 0.0139 - accuracy: 0.9962
1041/1688 [=================>............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9962
1050/1688 [=================>............] - ETA: 3s - loss: 0.0140 - accuracy: 0.9962
1059/1688 [=================>............] - ETA: 3s - loss: 0.0139 - accuracy: 0.9963
1068/1688 [=================>............] - ETA: 3s - loss: 0.0140 - accuracy: 0.9963
1077/1688 [==================>...........] - ETA: 3s - loss: 0.0139 - accuracy: 0.9963
1085/1688 [==================>...........] - ETA: 3s - loss: 0.0138 - accuracy: 0.9963
1094/1688 [==================>...........] - ETA: 3s - loss: 0.0138 - accuracy: 0.9963
1103/1688 [==================>...........] - ETA: 3s - loss: 0.0138 - accuracy: 0.9963
1112/1688 [==================>...........] - ETA: 3s - loss: 0.0137 - accuracy: 0.9964
1121/1688 [==================>...........] - ETA: 3s - loss: 0.0137 - accuracy: 0.9963
1130/1688 [===================>..........] - ETA: 3s - loss: 0.0138 - accuracy: 0.9963
1138/1688 [===================>..........] - ETA: 3s - loss: 0.0139 - accuracy: 0.9963
1147/1688 [===================>..........] - ETA: 3s - loss: 0.0139 - accuracy: 0.9963
1156/1688 [===================>..........] - ETA: 3s - loss: 0.0139 - accuracy: 0.9962
1165/1688 [===================>..........] - ETA: 3s - loss: 0.0139 - accuracy: 0.9962
1174/1688 [===================>..........] - ETA: 3s - loss: 0.0138 - accuracy: 0.9962
1183/1688 [====================>.........] - ETA: 3s - loss: 0.0138 - accuracy: 0.9962
1192/1688 [====================>.........] - ETA: 3s - loss: 0.0138 - accuracy: 0.9962
1201/1688 [====================>.........] - ETA: 2s - loss: 0.0138 - accuracy: 0.9962
1209/1688 [====================>.........] - ETA: 2s - loss: 0.0139 - accuracy: 0.9962
1218/1688 [====================>.........] - ETA: 2s - loss: 0.0139 - accuracy: 0.9962
1227/1688 [====================>.........] - ETA: 2s - loss: 0.0139 - accuracy: 0.9962
1236/1688 [====================>.........] - ETA: 2s - loss: 0.0139 - accuracy: 0.9962
1245/1688 [=====================>........] - ETA: 2s - loss: 0.0140 - accuracy: 0.9961
1254/1688 [=====================>........] - ETA: 2s - loss: 0.0140 - accuracy: 0.9961
1263/1688 [=====================>........] - ETA: 2s - loss: 0.0140 - accuracy: 0.9961
1272/1688 [=====================>........] - ETA: 2s - loss: 0.0141 - accuracy: 0.9961
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0141 - accuracy: 0.9961
1290/1688 [=====================>........] - ETA: 2s - loss: 0.0140 - accuracy: 0.9961
1299/1688 [======================>.......] - ETA: 2s - loss: 0.0140 - accuracy: 0.9961
1308/1688 [======================>.......] - ETA: 2s - loss: 0.0141 - accuracy: 0.9961
1317/1688 [======================>.......] - ETA: 2s - loss: 0.0141 - accuracy: 0.9961
1326/1688 [======================>.......] - ETA: 2s - loss: 0.0142 - accuracy: 0.9960
1335/1688 [======================>.......] - ETA: 2s - loss: 0.0142 - accuracy: 0.9960
1344/1688 [======================>.......] - ETA: 2s - loss: 0.0142 - accuracy: 0.9960
1353/1688 [=======================>......] - ETA: 2s - loss: 0.0143 - accuracy: 0.9960
1362/1688 [=======================>......] - ETA: 1s - loss: 0.0143 - accuracy: 0.9960
1371/1688 [=======================>......] - ETA: 1s - loss: 0.0142 - accuracy: 0.9960
1379/1688 [=======================>......] - ETA: 1s - loss: 0.0142 - accuracy: 0.9960
1388/1688 [=======================>......] - ETA: 1s - loss: 0.0141 - accuracy: 0.9960
1397/1688 [=======================>......] - ETA: 1s - loss: 0.0141 - accuracy: 0.9961
1406/1688 [=======================>......] - ETA: 1s - loss: 0.0141 - accuracy: 0.9961
1415/1688 [========================>.....] - ETA: 1s - loss: 0.0141 - accuracy: 0.9961
1424/1688 [========================>.....] - ETA: 1s - loss: 0.0141 - accuracy: 0.9960
1433/1688 [========================>.....] - ETA: 1s - loss: 0.0141 - accuracy: 0.9961
1441/1688 [========================>.....] - ETA: 1s - loss: 0.0141 - accuracy: 0.9961
1450/1688 [========================>.....] - ETA: 1s - loss: 0.0141 - accuracy: 0.9961
1459/1688 [========================>.....] - ETA: 1s - loss: 0.0141 - accuracy: 0.9961
1468/1688 [=========================>....] - ETA: 1s - loss: 0.0142 - accuracy: 0.9961
1476/1688 [=========================>....] - ETA: 1s - loss: 0.0142 - accuracy: 0.9960
1485/1688 [=========================>....] - ETA: 1s - loss: 0.0142 - accuracy: 0.9960
1494/1688 [=========================>....] - ETA: 1s - loss: 0.0141 - accuracy: 0.9960
1503/1688 [=========================>....] - ETA: 1s - loss: 0.0141 - accuracy: 0.9960
1512/1688 [=========================>....] - ETA: 1s - loss: 0.0141 - accuracy: 0.9961
1521/1688 [==========================>...] - ETA: 1s - loss: 0.0141 - accuracy: 0.9961
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0140 - accuracy: 0.9961
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0140 - accuracy: 0.9961
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0140 - accuracy: 0.9961
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0140 - accuracy: 0.9961
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0140 - accuracy: 0.9961
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0140 - accuracy: 0.9961
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1591/1688 [===========================>..] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1599/1688 [===========================>..] - ETA: 0s - loss: 0.0139 - accuracy: 0.9962
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0139 - accuracy: 0.9962
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0139 - accuracy: 0.9962
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1635/1688 [============================>.] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1644/1688 [============================>.] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1653/1688 [============================>.] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1662/1688 [============================>.] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1670/1688 [============================>.] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1678/1688 [============================>.] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1687/1688 [============================>.] - ETA: 0s - loss: 0.0140 - accuracy: 0.9962
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0140 - accuracy: 0.9961 - val_loss: 0.0414 - val_accuracy: 0.9882
Epoch 3/5
1/1688 [..............................] - ETA: 10s - loss: 0.0058 - accuracy: 1.0000
10/1688 [..............................] - ETA: 9s - loss: 0.0048 - accuracy: 1.0000
18/1688 [..............................] - ETA: 10s - loss: 0.0074 - accuracy: 0.9983
27/1688 [..............................] - ETA: 10s - loss: 0.0075 - accuracy: 0.9988
36/1688 [..............................] - ETA: 10s - loss: 0.0083 - accuracy: 0.9983
45/1688 [..............................] - ETA: 10s - loss: 0.0089 - accuracy: 0.9986
54/1688 [..............................] - ETA: 10s - loss: 0.0097 - accuracy: 0.9988
63/1688 [>.............................] - ETA: 9s - loss: 0.0093 - accuracy: 0.9990
72/1688 [>.............................] - ETA: 9s - loss: 0.0113 - accuracy: 0.9987
81/1688 [>.............................] - ETA: 9s - loss: 0.0107 - accuracy: 0.9988
90/1688 [>.............................] - ETA: 9s - loss: 0.0124 - accuracy: 0.9979
99/1688 [>.............................] - ETA: 9s - loss: 0.0126 - accuracy: 0.9981
108/1688 [>.............................] - ETA: 9s - loss: 0.0124 - accuracy: 0.9983
117/1688 [=>............................] - ETA: 9s - loss: 0.0120 - accuracy: 0.9984
126/1688 [=>............................] - ETA: 9s - loss: 0.0128 - accuracy: 0.9983
135/1688 [=>............................] - ETA: 9s - loss: 0.0128 - accuracy: 0.9981
144/1688 [=>............................] - ETA: 9s - loss: 0.0132 - accuracy: 0.9976
152/1688 [=>............................] - ETA: 9s - loss: 0.0127 - accuracy: 0.9977
160/1688 [=>............................] - ETA: 9s - loss: 0.0129 - accuracy: 0.9977
169/1688 [==>...........................] - ETA: 9s - loss: 0.0124 - accuracy: 0.9978
178/1688 [==>...........................] - ETA: 9s - loss: 0.0120 - accuracy: 0.9979
187/1688 [==>...........................] - ETA: 9s - loss: 0.0122 - accuracy: 0.9978
196/1688 [==>...........................] - ETA: 9s - loss: 0.0120 - accuracy: 0.9978
205/1688 [==>...........................] - ETA: 9s - loss: 0.0124 - accuracy: 0.9973
214/1688 [==>...........................] - ETA: 8s - loss: 0.0122 - accuracy: 0.9974
223/1688 [==>...........................] - ETA: 8s - loss: 0.0122 - accuracy: 0.9973
231/1688 [===>..........................] - ETA: 8s - loss: 0.0125 - accuracy: 0.9970
240/1688 [===>..........................] - ETA: 8s - loss: 0.0124 - accuracy: 0.9971
249/1688 [===>..........................] - ETA: 8s - loss: 0.0121 - accuracy: 0.9972
258/1688 [===>..........................] - ETA: 8s - loss: 0.0128 - accuracy: 0.9966
267/1688 [===>..........................] - ETA: 8s - loss: 0.0129 - accuracy: 0.9965
276/1688 [===>..........................] - ETA: 8s - loss: 0.0129 - accuracy: 0.9966
284/1688 [====>.........................] - ETA: 8s - loss: 0.0127 - accuracy: 0.9967
293/1688 [====>.........................] - ETA: 8s - loss: 0.0125 - accuracy: 0.9968
301/1688 [====>.........................] - ETA: 8s - loss: 0.0124 - accuracy: 0.9969
310/1688 [====>.........................] - ETA: 8s - loss: 0.0125 - accuracy: 0.9968
318/1688 [====>.........................] - ETA: 8s - loss: 0.0124 - accuracy: 0.9968
327/1688 [====>.........................] - ETA: 8s - loss: 0.0123 - accuracy: 0.9968
335/1688 [====>.........................] - ETA: 8s - loss: 0.0122 - accuracy: 0.9969
343/1688 [=====>........................] - ETA: 8s - loss: 0.0122 - accuracy: 0.9968
352/1688 [=====>........................] - ETA: 8s - loss: 0.0122 - accuracy: 0.9968
360/1688 [=====>........................] - ETA: 8s - loss: 0.0122 - accuracy: 0.9968
368/1688 [=====>........................] - ETA: 8s - loss: 0.0122 - accuracy: 0.9968
377/1688 [=====>........................] - ETA: 8s - loss: 0.0120 - accuracy: 0.9969
385/1688 [=====>........................] - ETA: 7s - loss: 0.0119 - accuracy: 0.9969
393/1688 [=====>........................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9970
402/1688 [======>.......................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9970
411/1688 [======>.......................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9971
420/1688 [======>.......................] - ETA: 7s - loss: 0.0115 - accuracy: 0.9972
429/1688 [======>.......................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9972
438/1688 [======>.......................] - ETA: 7s - loss: 0.0116 - accuracy: 0.9972
446/1688 [======>.......................] - ETA: 7s - loss: 0.0116 - accuracy: 0.9972
455/1688 [=======>......................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9971
464/1688 [=======>......................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9971
473/1688 [=======>......................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9972
481/1688 [=======>......................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9971
489/1688 [=======>......................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9971
498/1688 [=======>......................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9971
507/1688 [========>.....................] - ETA: 7s - loss: 0.0119 - accuracy: 0.9970
516/1688 [========>.....................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9969
524/1688 [========>.....................] - ETA: 7s - loss: 0.0123 - accuracy: 0.9968
532/1688 [========>.....................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9969
541/1688 [========>.....................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9968
550/1688 [========>.....................] - ETA: 6s - loss: 0.0123 - accuracy: 0.9968
559/1688 [========>.....................] - ETA: 6s - loss: 0.0123 - accuracy: 0.9968
568/1688 [=========>....................] - ETA: 6s - loss: 0.0124 - accuracy: 0.9966
577/1688 [=========>....................] - ETA: 6s - loss: 0.0125 - accuracy: 0.9966
586/1688 [=========>....................] - ETA: 6s - loss: 0.0125 - accuracy: 0.9966
595/1688 [=========>....................] - ETA: 6s - loss: 0.0124 - accuracy: 0.9966
604/1688 [=========>....................] - ETA: 6s - loss: 0.0123 - accuracy: 0.9966
612/1688 [=========>....................] - ETA: 6s - loss: 0.0123 - accuracy: 0.9967
621/1688 [==========>...................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9967
629/1688 [==========>...................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9967
638/1688 [==========>...................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9967
647/1688 [==========>...................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9967
655/1688 [==========>...................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9967
664/1688 [==========>...................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9967
672/1688 [==========>...................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9967
681/1688 [===========>..................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9967
689/1688 [===========>..................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9967
698/1688 [===========>..................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9967
707/1688 [===========>..................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9967
716/1688 [===========>..................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9966
725/1688 [===========>..................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9966
734/1688 [============>.................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9966
743/1688 [============>.................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9965
752/1688 [============>.................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9965
761/1688 [============>.................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9965
770/1688 [============>.................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9965
779/1688 [============>.................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9965
788/1688 [=============>................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9964
797/1688 [=============>................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9965
806/1688 [=============>................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9965
815/1688 [=============>................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9965
824/1688 [=============>................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9964
833/1688 [=============>................] - ETA: 5s - loss: 0.0124 - accuracy: 0.9965
842/1688 [=============>................] - ETA: 5s - loss: 0.0125 - accuracy: 0.9964
851/1688 [==============>...............] - ETA: 5s - loss: 0.0124 - accuracy: 0.9965
859/1688 [==============>...............] - ETA: 5s - loss: 0.0124 - accuracy: 0.9965
868/1688 [==============>...............] - ETA: 5s - loss: 0.0124 - accuracy: 0.9965
876/1688 [==============>...............] - ETA: 4s - loss: 0.0123 - accuracy: 0.9965
885/1688 [==============>...............] - ETA: 4s - loss: 0.0122 - accuracy: 0.9965
894/1688 [==============>...............] - ETA: 4s - loss: 0.0122 - accuracy: 0.9966
903/1688 [===============>..............] - ETA: 4s - loss: 0.0122 - accuracy: 0.9966
911/1688 [===============>..............] - ETA: 4s - loss: 0.0122 - accuracy: 0.9966
919/1688 [===============>..............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9966
928/1688 [===============>..............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9967
937/1688 [===============>..............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9966
946/1688 [===============>..............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9966
955/1688 [===============>..............] - ETA: 4s - loss: 0.0122 - accuracy: 0.9966
964/1688 [================>.............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9966
972/1688 [================>.............] - ETA: 4s - loss: 0.0122 - accuracy: 0.9965
981/1688 [================>.............] - ETA: 4s - loss: 0.0122 - accuracy: 0.9965
990/1688 [================>.............] - ETA: 4s - loss: 0.0122 - accuracy: 0.9965
998/1688 [================>.............] - ETA: 4s - loss: 0.0122 - accuracy: 0.9965
1007/1688 [================>.............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9965
1016/1688 [=================>............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9966
1025/1688 [=================>............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9966
1034/1688 [=================>............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9966
1042/1688 [=================>............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9966
1051/1688 [=================>............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9967
1060/1688 [=================>............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9967
1069/1688 [=================>............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9967
1077/1688 [==================>...........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9967
1086/1688 [==================>...........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9967
1095/1688 [==================>...........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9967
1103/1688 [==================>...........] - ETA: 3s - loss: 0.0120 - accuracy: 0.9967
1112/1688 [==================>...........] - ETA: 3s - loss: 0.0120 - accuracy: 0.9966
1121/1688 [==================>...........] - ETA: 3s - loss: 0.0120 - accuracy: 0.9967
1130/1688 [===================>..........] - ETA: 3s - loss: 0.0120 - accuracy: 0.9967
1139/1688 [===================>..........] - ETA: 3s - loss: 0.0120 - accuracy: 0.9967
1148/1688 [===================>..........] - ETA: 3s - loss: 0.0120 - accuracy: 0.9966
1157/1688 [===================>..........] - ETA: 3s - loss: 0.0121 - accuracy: 0.9966
1166/1688 [===================>..........] - ETA: 3s - loss: 0.0121 - accuracy: 0.9966
1175/1688 [===================>..........] - ETA: 3s - loss: 0.0120 - accuracy: 0.9966
1184/1688 [====================>.........] - ETA: 3s - loss: 0.0121 - accuracy: 0.9966
1193/1688 [====================>.........] - ETA: 3s - loss: 0.0120 - accuracy: 0.9967
1202/1688 [====================>.........] - ETA: 2s - loss: 0.0120 - accuracy: 0.9967
1211/1688 [====================>.........] - ETA: 2s - loss: 0.0120 - accuracy: 0.9967
1220/1688 [====================>.........] - ETA: 2s - loss: 0.0120 - accuracy: 0.9967
1229/1688 [====================>.........] - ETA: 2s - loss: 0.0121 - accuracy: 0.9966
1238/1688 [=====================>........] - ETA: 2s - loss: 0.0121 - accuracy: 0.9966
1247/1688 [=====================>........] - ETA: 2s - loss: 0.0121 - accuracy: 0.9965
1256/1688 [=====================>........] - ETA: 2s - loss: 0.0121 - accuracy: 0.9965
1265/1688 [=====================>........] - ETA: 2s - loss: 0.0121 - accuracy: 0.9966
1274/1688 [=====================>........] - ETA: 2s - loss: 0.0121 - accuracy: 0.9966
1283/1688 [=====================>........] - ETA: 2s - loss: 0.0121 - accuracy: 0.9965
1292/1688 [=====================>........] - ETA: 2s - loss: 0.0121 - accuracy: 0.9966
1301/1688 [======================>.......] - ETA: 2s - loss: 0.0121 - accuracy: 0.9966
1310/1688 [======================>.......] - ETA: 2s - loss: 0.0120 - accuracy: 0.9966
1318/1688 [======================>.......] - ETA: 2s - loss: 0.0121 - accuracy: 0.9966
1327/1688 [======================>.......] - ETA: 2s - loss: 0.0120 - accuracy: 0.9966
1335/1688 [======================>.......] - ETA: 2s - loss: 0.0120 - accuracy: 0.9966
1344/1688 [======================>.......] - ETA: 2s - loss: 0.0120 - accuracy: 0.9966
1353/1688 [=======================>......] - ETA: 2s - loss: 0.0120 - accuracy: 0.9966
1362/1688 [=======================>......] - ETA: 1s - loss: 0.0120 - accuracy: 0.9966
1370/1688 [=======================>......] - ETA: 1s - loss: 0.0120 - accuracy: 0.9966
1379/1688 [=======================>......] - ETA: 1s - loss: 0.0121 - accuracy: 0.9966
1388/1688 [=======================>......] - ETA: 1s - loss: 0.0121 - accuracy: 0.9965
1397/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9964
1406/1688 [=======================>......] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1415/1688 [========================>.....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9965
1423/1688 [========================>.....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9965
1432/1688 [========================>.....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9965
1441/1688 [========================>.....] - ETA: 1s - loss: 0.0121 - accuracy: 0.9965
1449/1688 [========================>.....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1457/1688 [========================>.....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1465/1688 [=========================>....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1474/1688 [=========================>....] - ETA: 1s - loss: 0.0123 - accuracy: 0.9965
1482/1688 [=========================>....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1491/1688 [=========================>....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1500/1688 [=========================>....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1509/1688 [=========================>....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1518/1688 [=========================>....] - ETA: 1s - loss: 0.0122 - accuracy: 0.9965
1527/1688 [==========================>...] - ETA: 0s - loss: 0.0123 - accuracy: 0.9965
1536/1688 [==========================>...] - ETA: 0s - loss: 0.0123 - accuracy: 0.9964
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0123 - accuracy: 0.9965
1554/1688 [==========================>...] - ETA: 0s - loss: 0.0124 - accuracy: 0.9964
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0124 - accuracy: 0.9964
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0124 - accuracy: 0.9964
1581/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9964
1590/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9964
1598/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9965
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9965
1616/1688 [===========================>..] - ETA: 0s - loss: 0.0124 - accuracy: 0.9965
1625/1688 [===========================>..] - ETA: 0s - loss: 0.0123 - accuracy: 0.9965
1634/1688 [============================>.] - ETA: 0s - loss: 0.0123 - accuracy: 0.9965
1643/1688 [============================>.] - ETA: 0s - loss: 0.0123 - accuracy: 0.9965
1652/1688 [============================>.] - ETA: 0s - loss: 0.0123 - accuracy: 0.9965
1660/1688 [============================>.] - ETA: 0s - loss: 0.0123 - accuracy: 0.9965
1669/1688 [============================>.] - ETA: 0s - loss: 0.0123 - accuracy: 0.9965
1678/1688 [============================>.] - ETA: 0s - loss: 0.0123 - accuracy: 0.9965
1687/1688 [============================>.] - ETA: 0s - loss: 0.0124 - accuracy: 0.9964
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0124 - accuracy: 0.9964 - val_loss: 0.0446 - val_accuracy: 0.9883
Epoch 4/5
1/1688 [..............................] - ETA: 11s - loss: 0.0043 - accuracy: 1.0000
10/1688 [..............................] - ETA: 10s - loss: 0.0215 - accuracy: 0.9937
19/1688 [..............................] - ETA: 10s - loss: 0.0164 - accuracy: 0.9951
28/1688 [..............................] - ETA: 9s - loss: 0.0159 - accuracy: 0.9955
37/1688 [..............................] - ETA: 9s - loss: 0.0143 - accuracy: 0.9958
45/1688 [..............................] - ETA: 9s - loss: 0.0131 - accuracy: 0.9965
53/1688 [..............................] - ETA: 9s - loss: 0.0135 - accuracy: 0.9965
62/1688 [>.............................] - ETA: 9s - loss: 0.0125 - accuracy: 0.9970
71/1688 [>.............................] - ETA: 9s - loss: 0.0119 - accuracy: 0.9974
80/1688 [>.............................] - ETA: 9s - loss: 0.0120 - accuracy: 0.9973
89/1688 [>.............................] - ETA: 9s - loss: 0.0124 - accuracy: 0.9972
97/1688 [>.............................] - ETA: 9s - loss: 0.0123 - accuracy: 0.9971
106/1688 [>.............................] - ETA: 9s - loss: 0.0136 - accuracy: 0.9968
115/1688 [=>............................] - ETA: 9s - loss: 0.0130 - accuracy: 0.9970
124/1688 [=>............................] - ETA: 9s - loss: 0.0129 - accuracy: 0.9970
133/1688 [=>............................] - ETA: 9s - loss: 0.0124 - accuracy: 0.9972
142/1688 [=>............................] - ETA: 9s - loss: 0.0126 - accuracy: 0.9971
151/1688 [=>............................] - ETA: 9s - loss: 0.0129 - accuracy: 0.9971
160/1688 [=>............................] - ETA: 9s - loss: 0.0134 - accuracy: 0.9969
169/1688 [==>...........................] - ETA: 9s - loss: 0.0133 - accuracy: 0.9969
177/1688 [==>...........................] - ETA: 9s - loss: 0.0132 - accuracy: 0.9970
186/1688 [==>...........................] - ETA: 9s - loss: 0.0132 - accuracy: 0.9970
195/1688 [==>...........................] - ETA: 9s - loss: 0.0135 - accuracy: 0.9968
203/1688 [==>...........................] - ETA: 9s - loss: 0.0134 - accuracy: 0.9968
212/1688 [==>...........................] - ETA: 8s - loss: 0.0136 - accuracy: 0.9968
221/1688 [==>...........................] - ETA: 8s - loss: 0.0134 - accuracy: 0.9969
230/1688 [===>..........................] - ETA: 8s - loss: 0.0133 - accuracy: 0.9969
238/1688 [===>..........................] - ETA: 8s - loss: 0.0131 - accuracy: 0.9970
247/1688 [===>..........................] - ETA: 8s - loss: 0.0133 - accuracy: 0.9968
255/1688 [===>..........................] - ETA: 8s - loss: 0.0131 - accuracy: 0.9969
264/1688 [===>..........................] - ETA: 8s - loss: 0.0129 - accuracy: 0.9969
273/1688 [===>..........................] - ETA: 8s - loss: 0.0131 - accuracy: 0.9969
282/1688 [====>.........................] - ETA: 8s - loss: 0.0130 - accuracy: 0.9969
291/1688 [====>.........................] - ETA: 8s - loss: 0.0134 - accuracy: 0.9967
300/1688 [====>.........................] - ETA: 8s - loss: 0.0132 - accuracy: 0.9968
308/1688 [====>.........................] - ETA: 8s - loss: 0.0131 - accuracy: 0.9968
317/1688 [====>.........................] - ETA: 8s - loss: 0.0133 - accuracy: 0.9967
325/1688 [====>.........................] - ETA: 8s - loss: 0.0132 - accuracy: 0.9968
334/1688 [====>.........................] - ETA: 8s - loss: 0.0136 - accuracy: 0.9967
343/1688 [=====>........................] - ETA: 8s - loss: 0.0134 - accuracy: 0.9968
352/1688 [=====>........................] - ETA: 8s - loss: 0.0132 - accuracy: 0.9969
361/1688 [=====>........................] - ETA: 8s - loss: 0.0130 - accuracy: 0.9969
370/1688 [=====>........................] - ETA: 8s - loss: 0.0129 - accuracy: 0.9970
379/1688 [=====>........................] - ETA: 7s - loss: 0.0128 - accuracy: 0.9970
388/1688 [=====>........................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9971
397/1688 [======>.......................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9971
406/1688 [======>.......................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9972
415/1688 [======>.......................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9970
424/1688 [======>.......................] - ETA: 7s - loss: 0.0129 - accuracy: 0.9969
433/1688 [======>.......................] - ETA: 7s - loss: 0.0129 - accuracy: 0.9970
441/1688 [======>.......................] - ETA: 7s - loss: 0.0128 - accuracy: 0.9970
449/1688 [======>.......................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9970
458/1688 [=======>......................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9970
467/1688 [=======>......................] - ETA: 7s - loss: 0.0125 - accuracy: 0.9971
476/1688 [=======>......................] - ETA: 7s - loss: 0.0124 - accuracy: 0.9971
485/1688 [=======>......................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9972
493/1688 [=======>......................] - ETA: 7s - loss: 0.0121 - accuracy: 0.9972
502/1688 [=======>......................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9973
511/1688 [========>.....................] - ETA: 7s - loss: 0.0123 - accuracy: 0.9971
520/1688 [========>.....................] - ETA: 7s - loss: 0.0121 - accuracy: 0.9972
529/1688 [========>.....................] - ETA: 7s - loss: 0.0121 - accuracy: 0.9972
538/1688 [========>.....................] - ETA: 7s - loss: 0.0121 - accuracy: 0.9973
547/1688 [========>.....................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9973
556/1688 [========>.....................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9973
565/1688 [=========>....................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9973
574/1688 [=========>....................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9973
582/1688 [=========>....................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9973
591/1688 [=========>....................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9973
600/1688 [=========>....................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9973
609/1688 [=========>....................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9973
618/1688 [=========>....................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9974
626/1688 [==========>...................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9974
635/1688 [==========>...................] - ETA: 6s - loss: 0.0116 - accuracy: 0.9974
644/1688 [==========>...................] - ETA: 6s - loss: 0.0115 - accuracy: 0.9975
652/1688 [==========>...................] - ETA: 6s - loss: 0.0114 - accuracy: 0.9975
661/1688 [==========>...................] - ETA: 6s - loss: 0.0113 - accuracy: 0.9975
670/1688 [==========>...................] - ETA: 6s - loss: 0.0113 - accuracy: 0.9975
679/1688 [===========>..................] - ETA: 6s - loss: 0.0115 - accuracy: 0.9974
688/1688 [===========>..................] - ETA: 6s - loss: 0.0115 - accuracy: 0.9974
697/1688 [===========>..................] - ETA: 6s - loss: 0.0116 - accuracy: 0.9973
705/1688 [===========>..................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9973
714/1688 [===========>..................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9973
723/1688 [===========>..................] - ETA: 5s - loss: 0.0116 - accuracy: 0.9973
732/1688 [============>.................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9974
741/1688 [============>.................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9974
749/1688 [============>.................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9974
758/1688 [============>.................] - ETA: 5s - loss: 0.0116 - accuracy: 0.9974
767/1688 [============>.................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9972
776/1688 [============>.................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9971
785/1688 [============>.................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9971
793/1688 [=============>................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9971
802/1688 [=============>................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9971
811/1688 [=============>................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9971
819/1688 [=============>................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9971
827/1688 [=============>................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9971
836/1688 [=============>................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9971
845/1688 [==============>...............] - ETA: 5s - loss: 0.0120 - accuracy: 0.9971
853/1688 [==============>...............] - ETA: 5s - loss: 0.0122 - accuracy: 0.9970
862/1688 [==============>...............] - ETA: 5s - loss: 0.0122 - accuracy: 0.9971
871/1688 [==============>...............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9971
880/1688 [==============>...............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9971
889/1688 [==============>...............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9971
898/1688 [==============>...............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
907/1688 [===============>..............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
916/1688 [===============>..............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9971
925/1688 [===============>..............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
934/1688 [===============>..............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
943/1688 [===============>..............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
952/1688 [===============>..............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
960/1688 [================>.............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
969/1688 [================>.............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9971
977/1688 [================>.............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9971
986/1688 [================>.............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9971
994/1688 [================>.............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9971
1003/1688 [================>.............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9971
1012/1688 [================>.............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
1021/1688 [=================>............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9971
1029/1688 [=================>............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
1038/1688 [=================>............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9970
1047/1688 [=================>............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9971
1056/1688 [=================>............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9970
1065/1688 [=================>............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9970
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9970
1083/1688 [==================>...........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9971
1092/1688 [==================>...........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9971
1101/1688 [==================>...........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9971
1109/1688 [==================>...........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9971
1118/1688 [==================>...........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9971
1127/1688 [===================>..........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9971
1136/1688 [===================>..........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9971
1145/1688 [===================>..........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9971
1153/1688 [===================>..........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9970
1162/1688 [===================>..........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9970
1171/1688 [===================>..........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9970
1180/1688 [===================>..........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9970
1188/1688 [====================>.........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9970
1197/1688 [====================>.........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9970
1206/1688 [====================>.........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9971
1215/1688 [====================>.........] - ETA: 2s - loss: 0.0118 - accuracy: 0.9970
1223/1688 [====================>.........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9970
1232/1688 [====================>.........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9970
1241/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9970
1250/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9970
1258/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9971
1266/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9971
1275/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9971
1284/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9971
1293/1688 [=====================>........] - ETA: 2s - loss: 0.0119 - accuracy: 0.9971
1302/1688 [======================>.......] - ETA: 2s - loss: 0.0119 - accuracy: 0.9971
1311/1688 [======================>.......] - ETA: 2s - loss: 0.0118 - accuracy: 0.9971
1319/1688 [======================>.......] - ETA: 2s - loss: 0.0118 - accuracy: 0.9971
1328/1688 [======================>.......] - ETA: 2s - loss: 0.0119 - accuracy: 0.9971
1336/1688 [======================>.......] - ETA: 2s - loss: 0.0119 - accuracy: 0.9971
1345/1688 [======================>.......] - ETA: 2s - loss: 0.0119 - accuracy: 0.9971
1354/1688 [=======================>......] - ETA: 2s - loss: 0.0119 - accuracy: 0.9971
1363/1688 [=======================>......] - ETA: 1s - loss: 0.0120 - accuracy: 0.9971
1372/1688 [=======================>......] - ETA: 1s - loss: 0.0120 - accuracy: 0.9971
1381/1688 [=======================>......] - ETA: 1s - loss: 0.0120 - accuracy: 0.9971
1390/1688 [=======================>......] - ETA: 1s - loss: 0.0120 - accuracy: 0.9970
1399/1688 [=======================>......] - ETA: 1s - loss: 0.0120 - accuracy: 0.9970
1408/1688 [========================>.....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9970
1417/1688 [========================>.....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9971
1426/1688 [========================>.....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9970
1435/1688 [========================>.....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9970
1443/1688 [========================>.....] - ETA: 1s - loss: 0.0120 - accuracy: 0.9970
1452/1688 [========================>.....] - ETA: 1s - loss: 0.0120 - accuracy: 0.9970
1461/1688 [========================>.....] - ETA: 1s - loss: 0.0120 - accuracy: 0.9970
1470/1688 [=========================>....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9970
1478/1688 [=========================>....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9970
1486/1688 [=========================>....] - ETA: 1s - loss: 0.0119 - accuracy: 0.9971
1495/1688 [=========================>....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9971
1504/1688 [=========================>....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9971
1513/1688 [=========================>....] - ETA: 1s - loss: 0.0118 - accuracy: 0.9971
1522/1688 [==========================>...] - ETA: 1s - loss: 0.0117 - accuracy: 0.9971
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0117 - accuracy: 0.9971
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0118 - accuracy: 0.9971
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0118 - accuracy: 0.9971
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0118 - accuracy: 0.9971
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0118 - accuracy: 0.9971
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0118 - accuracy: 0.9971
1585/1688 [===========================>..] - ETA: 0s - loss: 0.0118 - accuracy: 0.9971
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0118 - accuracy: 0.9971
1603/1688 [===========================>..] - ETA: 0s - loss: 0.0118 - accuracy: 0.9971
1612/1688 [===========================>..] - ETA: 0s - loss: 0.0119 - accuracy: 0.9971
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0119 - accuracy: 0.9970
1628/1688 [===========================>..] - ETA: 0s - loss: 0.0120 - accuracy: 0.9970
1637/1688 [============================>.] - ETA: 0s - loss: 0.0120 - accuracy: 0.9970
1645/1688 [============================>.] - ETA: 0s - loss: 0.0120 - accuracy: 0.9970
1654/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9970
1663/1688 [============================>.] - ETA: 0s - loss: 0.0120 - accuracy: 0.9970
1671/1688 [============================>.] - ETA: 0s - loss: 0.0120 - accuracy: 0.9970
1680/1688 [============================>.] - ETA: 0s - loss: 0.0121 - accuracy: 0.9970
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0121 - accuracy: 0.9970 - val_loss: 0.0490 - val_accuracy: 0.9877
Epoch 5/5
1/1688 [..............................] - ETA: 11s - loss: 0.0043 - accuracy: 1.0000
10/1688 [..............................] - ETA: 10s - loss: 0.0165 - accuracy: 0.9969
19/1688 [..............................] - ETA: 10s - loss: 0.0143 - accuracy: 0.9967
28/1688 [..............................] - ETA: 10s - loss: 0.0129 - accuracy: 0.9967
37/1688 [..............................] - ETA: 10s - loss: 0.0132 - accuracy: 0.9975
45/1688 [..............................] - ETA: 10s - loss: 0.0115 - accuracy: 0.9979
54/1688 [..............................] - ETA: 9s - loss: 0.0113 - accuracy: 0.9977
63/1688 [>.............................] - ETA: 9s - loss: 0.0110 - accuracy: 0.9975
72/1688 [>.............................] - ETA: 9s - loss: 0.0102 - accuracy: 0.9978
81/1688 [>.............................] - ETA: 9s - loss: 0.0095 - accuracy: 0.9981
90/1688 [>.............................] - ETA: 9s - loss: 0.0092 - accuracy: 0.9983
98/1688 [>.............................] - ETA: 9s - loss: 0.0101 - accuracy: 0.9981
107/1688 [>.............................] - ETA: 9s - loss: 0.0101 - accuracy: 0.9980
116/1688 [=>............................] - ETA: 9s - loss: 0.0107 - accuracy: 0.9978
125/1688 [=>............................] - ETA: 9s - loss: 0.0106 - accuracy: 0.9977
134/1688 [=>............................] - ETA: 9s - loss: 0.0106 - accuracy: 0.9977
143/1688 [=>............................] - ETA: 9s - loss: 0.0104 - accuracy: 0.9978
152/1688 [=>............................] - ETA: 9s - loss: 0.0108 - accuracy: 0.9973
161/1688 [=>............................] - ETA: 9s - loss: 0.0108 - accuracy: 0.9975
170/1688 [==>...........................] - ETA: 9s - loss: 0.0111 - accuracy: 0.9974
179/1688 [==>...........................] - ETA: 9s - loss: 0.0110 - accuracy: 0.9976
188/1688 [==>...........................] - ETA: 9s - loss: 0.0115 - accuracy: 0.9972
197/1688 [==>...........................] - ETA: 9s - loss: 0.0112 - accuracy: 0.9973
205/1688 [==>...........................] - ETA: 9s - loss: 0.0114 - accuracy: 0.9973
214/1688 [==>...........................] - ETA: 8s - loss: 0.0114 - accuracy: 0.9972
223/1688 [==>...........................] - ETA: 8s - loss: 0.0113 - accuracy: 0.9973
232/1688 [===>..........................] - ETA: 8s - loss: 0.0114 - accuracy: 0.9974
241/1688 [===>..........................] - ETA: 8s - loss: 0.0111 - accuracy: 0.9975
250/1688 [===>..........................] - ETA: 8s - loss: 0.0109 - accuracy: 0.9976
258/1688 [===>..........................] - ETA: 8s - loss: 0.0107 - accuracy: 0.9977
266/1688 [===>..........................] - ETA: 8s - loss: 0.0107 - accuracy: 0.9977
275/1688 [===>..........................] - ETA: 8s - loss: 0.0105 - accuracy: 0.9977
284/1688 [====>.........................] - ETA: 8s - loss: 0.0107 - accuracy: 0.9976
293/1688 [====>.........................] - ETA: 8s - loss: 0.0105 - accuracy: 0.9977
302/1688 [====>.........................] - ETA: 8s - loss: 0.0106 - accuracy: 0.9977
311/1688 [====>.........................] - ETA: 8s - loss: 0.0107 - accuracy: 0.9976
320/1688 [====>.........................] - ETA: 8s - loss: 0.0108 - accuracy: 0.9975
329/1688 [====>.........................] - ETA: 8s - loss: 0.0108 - accuracy: 0.9974
338/1688 [=====>........................] - ETA: 8s - loss: 0.0107 - accuracy: 0.9975
347/1688 [=====>........................] - ETA: 8s - loss: 0.0106 - accuracy: 0.9976
356/1688 [=====>........................] - ETA: 8s - loss: 0.0109 - accuracy: 0.9975
365/1688 [=====>........................] - ETA: 8s - loss: 0.0110 - accuracy: 0.9974
374/1688 [=====>........................] - ETA: 8s - loss: 0.0110 - accuracy: 0.9974
383/1688 [=====>........................] - ETA: 7s - loss: 0.0112 - accuracy: 0.9973
392/1688 [=====>........................] - ETA: 7s - loss: 0.0111 - accuracy: 0.9974
401/1688 [======>.......................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9974
410/1688 [======>.......................] - ETA: 7s - loss: 0.0110 - accuracy: 0.9975
419/1688 [======>.......................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9973
428/1688 [======>.......................] - ETA: 7s - loss: 0.0113 - accuracy: 0.9973
437/1688 [======>.......................] - ETA: 7s - loss: 0.0115 - accuracy: 0.9971
446/1688 [======>.......................] - ETA: 7s - loss: 0.0117 - accuracy: 0.9971
454/1688 [=======>......................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9971
463/1688 [=======>......................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9971
471/1688 [=======>......................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9971
480/1688 [=======>......................] - ETA: 7s - loss: 0.0118 - accuracy: 0.9971
488/1688 [=======>......................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9970
497/1688 [=======>......................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9969
506/1688 [=======>......................] - ETA: 7s - loss: 0.0121 - accuracy: 0.9969
515/1688 [========>.....................] - ETA: 7s - loss: 0.0121 - accuracy: 0.9969
524/1688 [========>.....................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9968
533/1688 [========>.....................] - ETA: 7s - loss: 0.0123 - accuracy: 0.9969
542/1688 [========>.....................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9969
551/1688 [========>.....................] - ETA: 6s - loss: 0.0123 - accuracy: 0.9968
560/1688 [========>.....................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9968
569/1688 [=========>....................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9969
578/1688 [=========>....................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9969
587/1688 [=========>....................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9969
595/1688 [=========>....................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9969
604/1688 [=========>....................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9969
613/1688 [=========>....................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9969
621/1688 [==========>...................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9969
630/1688 [==========>...................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9970
639/1688 [==========>...................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9970
648/1688 [==========>...................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9970
657/1688 [==========>...................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9970
665/1688 [==========>...................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9970
673/1688 [==========>...................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9970
682/1688 [===========>..................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9970
691/1688 [===========>..................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9970
700/1688 [===========>..................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9970
709/1688 [===========>..................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9970
718/1688 [===========>..................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9970
726/1688 [===========>..................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9970
734/1688 [============>.................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9971
743/1688 [============>.................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9971
752/1688 [============>.................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9971
761/1688 [============>.................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9971
770/1688 [============>.................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9971
779/1688 [============>.................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9972
788/1688 [=============>................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9971
796/1688 [=============>................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9971
805/1688 [=============>................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9972
814/1688 [=============>................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9972
823/1688 [=============>................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9972
832/1688 [=============>................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9971
841/1688 [=============>................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9972
850/1688 [==============>...............] - ETA: 5s - loss: 0.0118 - accuracy: 0.9971
859/1688 [==============>...............] - ETA: 5s - loss: 0.0119 - accuracy: 0.9971
867/1688 [==============>...............] - ETA: 5s - loss: 0.0118 - accuracy: 0.9971
876/1688 [==============>...............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9971
885/1688 [==============>...............] - ETA: 4s - loss: 0.0118 - accuracy: 0.9971
894/1688 [==============>...............] - ETA: 4s - loss: 0.0118 - accuracy: 0.9972
903/1688 [===============>..............] - ETA: 4s - loss: 0.0118 - accuracy: 0.9972
912/1688 [===============>..............] - ETA: 4s - loss: 0.0118 - accuracy: 0.9972
921/1688 [===============>..............] - ETA: 4s - loss: 0.0118 - accuracy: 0.9971
929/1688 [===============>..............] - ETA: 4s - loss: 0.0118 - accuracy: 0.9971
938/1688 [===============>..............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9971
947/1688 [===============>..............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
956/1688 [===============>..............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
965/1688 [================>.............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9971
974/1688 [================>.............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9971
983/1688 [================>.............] - ETA: 4s - loss: 0.0119 - accuracy: 0.9971
992/1688 [================>.............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
1001/1688 [================>.............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9970
1010/1688 [================>.............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
1018/1688 [=================>............] - ETA: 4s - loss: 0.0120 - accuracy: 0.9971
1027/1688 [=================>............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9970
1036/1688 [=================>............] - ETA: 3s - loss: 0.0121 - accuracy: 0.9970
1045/1688 [=================>............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9970
1054/1688 [=================>............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9970
1062/1688 [=================>............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9970
1071/1688 [==================>...........] - ETA: 3s - loss: 0.0120 - accuracy: 0.9970
1080/1688 [==================>...........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9970
1089/1688 [==================>...........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9971
1098/1688 [==================>...........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9971
1106/1688 [==================>...........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9971
1114/1688 [==================>...........] - ETA: 3s - loss: 0.0120 - accuracy: 0.9971
1123/1688 [==================>...........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9971
1132/1688 [===================>..........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9970
1140/1688 [===================>..........] - ETA: 3s - loss: 0.0119 - accuracy: 0.9970
1149/1688 [===================>..........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9971
1157/1688 [===================>..........] - ETA: 3s - loss: 0.0118 - accuracy: 0.9971
1166/1688 [===================>..........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9971
1175/1688 [===================>..........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9971
1184/1688 [====================>.........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9971
1193/1688 [====================>.........] - ETA: 3s - loss: 0.0117 - accuracy: 0.9971
1202/1688 [====================>.........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1210/1688 [====================>.........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1219/1688 [====================>.........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1237/1688 [====================>.........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1245/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1253/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1262/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1287/1688 [=====================>........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1296/1688 [======================>.......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1304/1688 [======================>.......] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1313/1688 [======================>.......] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1322/1688 [======================>.......] - ETA: 2s - loss: 0.0116 - accuracy: 0.9971
1330/1688 [======================>.......] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1339/1688 [======================>.......] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1347/1688 [======================>.......] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1355/1688 [=======================>......] - ETA: 2s - loss: 0.0117 - accuracy: 0.9971
1364/1688 [=======================>......] - ETA: 1s - loss: 0.0116 - accuracy: 0.9971
1373/1688 [=======================>......] - ETA: 1s - loss: 0.0116 - accuracy: 0.9971
1382/1688 [=======================>......] - ETA: 1s - loss: 0.0116 - accuracy: 0.9971
1391/1688 [=======================>......] - ETA: 1s - loss: 0.0116 - accuracy: 0.9971
1399/1688 [=======================>......] - ETA: 1s - loss: 0.0116 - accuracy: 0.9971
1408/1688 [========================>.....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9971
1417/1688 [========================>.....] - ETA: 1s - loss: 0.0117 - accuracy: 0.9971
1426/1688 [========================>.....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9971
1434/1688 [========================>.....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9971
1443/1688 [========================>.....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9971
1452/1688 [========================>.....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9971
1461/1688 [========================>.....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9971
1470/1688 [=========================>....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9971
1479/1688 [=========================>....] - ETA: 1s - loss: 0.0117 - accuracy: 0.9970
1488/1688 [=========================>....] - ETA: 1s - loss: 0.0116 - accuracy: 0.9970
1497/1688 [=========================>....] - ETA: 1s - loss: 0.0117 - accuracy: 0.9970
1506/1688 [=========================>....] - ETA: 1s - loss: 0.0117 - accuracy: 0.9970
1515/1688 [=========================>....] - ETA: 1s - loss: 0.0117 - accuracy: 0.9970
1524/1688 [==========================>...] - ETA: 1s - loss: 0.0117 - accuracy: 0.9970
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0116 - accuracy: 0.9971
1542/1688 [==========================>...] - ETA: 0s - loss: 0.0116 - accuracy: 0.9970
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0116 - accuracy: 0.9970
1560/1688 [==========================>...] - ETA: 0s - loss: 0.0116 - accuracy: 0.9971
1569/1688 [==========================>...] - ETA: 0s - loss: 0.0116 - accuracy: 0.9971
1578/1688 [===========================>..] - ETA: 0s - loss: 0.0116 - accuracy: 0.9970
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0116 - accuracy: 0.9971
1596/1688 [===========================>..] - ETA: 0s - loss: 0.0117 - accuracy: 0.9971
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0117 - accuracy: 0.9971
1612/1688 [===========================>..] - ETA: 0s - loss: 0.0116 - accuracy: 0.9971
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0116 - accuracy: 0.9971
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0116 - accuracy: 0.9971
1638/1688 [============================>.] - ETA: 0s - loss: 0.0116 - accuracy: 0.9971
1647/1688 [============================>.] - ETA: 0s - loss: 0.0117 - accuracy: 0.9970
1656/1688 [============================>.] - ETA: 0s - loss: 0.0117 - accuracy: 0.9970
1665/1688 [============================>.] - ETA: 0s - loss: 0.0117 - accuracy: 0.9970
1674/1688 [============================>.] - ETA: 0s - loss: 0.0118 - accuracy: 0.9970
1683/1688 [============================>.] - ETA: 0s - loss: 0.0118 - accuracy: 0.9970
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0118 - accuracy: 0.9970 - val_loss: 0.0449 - val_accuracy: 0.9890
<tf_keras.src.callbacks.History object at 0x7ef54cb6a190>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine-tuning:', score)
Test accuracy after fine-tuning: 0.986299991607666
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with TF-Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test.astype(np.int32))
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.9871000051498413
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[ -7.5071154 -7.0124483 -2.898366 -1.1856002 -11.311951 -5.371859
-18.247828 7.8625703 -3.8049154 -1.9394721]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 13.070 seconds)