Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a TF-Keras floating point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original TF-Keras floating point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf_keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from tf_keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, width, height, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
16384/11490434 [..............................] - ETA: 43s
49152/11490434 [..............................] - ETA: 35s
81920/11490434 [..............................] - ETA: 33s
147456/11490434 [..............................] - ETA: 23s
212992/11490434 [..............................] - ETA: 19s
327680/11490434 [..............................] - ETA: 14s
475136/11490434 [>.............................] - ETA: 10s
753664/11490434 [>.............................] - ETA: 7s
1130496/11490434 [=>............................] - ETA: 5s
1720320/11490434 [===>..........................] - ETA: 3s
2678784/11490434 [=====>........................] - ETA: 2s
2973696/11490434 [======>.......................] - ETA: 2s
3858432/11490434 [=========>....................] - ETA: 1s
4612096/11490434 [===========>..................] - ETA: 1s
4702208/11490434 [===========>..................] - ETA: 1s
6160384/11490434 [===============>..............] - ETA: 0s
6750208/11490434 [================>.............] - ETA: 0s
7421952/11490434 [==================>...........] - ETA: 0s
8060928/11490434 [====================>.........] - ETA: 0s
8667136/11490434 [=====================>........] - ETA: 0s
9273344/11490434 [=======================>......] - ETA: 0s
9887744/11490434 [========================>.....] - ETA: 0s
10469376/11490434 [==========================>...] - ETA: 0s
11075584/11490434 [===========================>..] - ETA: 0s
11490434/11490434 [==============================] - 1s 0us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard TF-Keras CNN model.
import tf_keras as keras
model_keras = keras.models.Sequential([
keras.layers.Rescaling(1. / 255, input_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from tf_keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 1:14:10 - loss: 2.9487 - accuracy: 0.0000e+00
21/1688 [..............................] - ETA: 4s - loss: 1.5952 - accuracy: 0.4673
43/1688 [..............................] - ETA: 3s - loss: 1.1236 - accuracy: 0.6432
65/1688 [>.............................] - ETA: 3s - loss: 0.9032 - accuracy: 0.7159
88/1688 [>.............................] - ETA: 3s - loss: 0.7725 - accuracy: 0.7592
111/1688 [>.............................] - ETA: 3s - loss: 0.6787 - accuracy: 0.7891
133/1688 [=>............................] - ETA: 3s - loss: 0.6124 - accuracy: 0.8127
156/1688 [=>............................] - ETA: 3s - loss: 0.5699 - accuracy: 0.8269
176/1688 [==>...........................] - ETA: 3s - loss: 0.5439 - accuracy: 0.8354
197/1688 [==>...........................] - ETA: 3s - loss: 0.5140 - accuracy: 0.8441
219/1688 [==>...........................] - ETA: 3s - loss: 0.4879 - accuracy: 0.8529
241/1688 [===>..........................] - ETA: 3s - loss: 0.4720 - accuracy: 0.8575
263/1688 [===>..........................] - ETA: 3s - loss: 0.4517 - accuracy: 0.8634
285/1688 [====>.........................] - ETA: 3s - loss: 0.4333 - accuracy: 0.8685
307/1688 [====>.........................] - ETA: 3s - loss: 0.4148 - accuracy: 0.8744
331/1688 [====>.........................] - ETA: 3s - loss: 0.3969 - accuracy: 0.8798
354/1688 [=====>........................] - ETA: 3s - loss: 0.3842 - accuracy: 0.8840
376/1688 [=====>........................] - ETA: 3s - loss: 0.3765 - accuracy: 0.8866
399/1688 [======>.......................] - ETA: 2s - loss: 0.3662 - accuracy: 0.8893
421/1688 [======>.......................] - ETA: 2s - loss: 0.3566 - accuracy: 0.8921
444/1688 [======>.......................] - ETA: 2s - loss: 0.3515 - accuracy: 0.8934
467/1688 [=======>......................] - ETA: 2s - loss: 0.3440 - accuracy: 0.8954
490/1688 [=======>......................] - ETA: 2s - loss: 0.3361 - accuracy: 0.8977
513/1688 [========>.....................] - ETA: 2s - loss: 0.3272 - accuracy: 0.9003
536/1688 [========>.....................] - ETA: 2s - loss: 0.3182 - accuracy: 0.9030
560/1688 [========>.....................] - ETA: 2s - loss: 0.3110 - accuracy: 0.9054
583/1688 [=========>....................] - ETA: 2s - loss: 0.3047 - accuracy: 0.9071
605/1688 [=========>....................] - ETA: 2s - loss: 0.2987 - accuracy: 0.9089
627/1688 [==========>...................] - ETA: 2s - loss: 0.2922 - accuracy: 0.9110
650/1688 [==========>...................] - ETA: 2s - loss: 0.2876 - accuracy: 0.9125
672/1688 [==========>...................] - ETA: 2s - loss: 0.2823 - accuracy: 0.9141
694/1688 [===========>..................] - ETA: 2s - loss: 0.2775 - accuracy: 0.9155
716/1688 [===========>..................] - ETA: 2s - loss: 0.2729 - accuracy: 0.9167
739/1688 [============>.................] - ETA: 2s - loss: 0.2676 - accuracy: 0.9184
762/1688 [============>.................] - ETA: 2s - loss: 0.2644 - accuracy: 0.9196
785/1688 [============>.................] - ETA: 2s - loss: 0.2607 - accuracy: 0.9206
807/1688 [=============>................] - ETA: 2s - loss: 0.2570 - accuracy: 0.9216
829/1688 [=============>................] - ETA: 1s - loss: 0.2540 - accuracy: 0.9226
851/1688 [==============>...............] - ETA: 1s - loss: 0.2502 - accuracy: 0.9235
873/1688 [==============>...............] - ETA: 1s - loss: 0.2470 - accuracy: 0.9246
897/1688 [==============>...............] - ETA: 1s - loss: 0.2435 - accuracy: 0.9257
919/1688 [===============>..............] - ETA: 1s - loss: 0.2403 - accuracy: 0.9267
941/1688 [===============>..............] - ETA: 1s - loss: 0.2369 - accuracy: 0.9277
963/1688 [================>.............] - ETA: 1s - loss: 0.2344 - accuracy: 0.9285
985/1688 [================>.............] - ETA: 1s - loss: 0.2320 - accuracy: 0.9292
1008/1688 [================>.............] - ETA: 1s - loss: 0.2292 - accuracy: 0.9300
1030/1688 [=================>............] - ETA: 1s - loss: 0.2270 - accuracy: 0.9307
1052/1688 [=================>............] - ETA: 1s - loss: 0.2244 - accuracy: 0.9316
1075/1688 [==================>...........] - ETA: 1s - loss: 0.2219 - accuracy: 0.9324
1097/1688 [==================>...........] - ETA: 1s - loss: 0.2201 - accuracy: 0.9331
1119/1688 [==================>...........] - ETA: 1s - loss: 0.2179 - accuracy: 0.9338
1142/1688 [===================>..........] - ETA: 1s - loss: 0.2160 - accuracy: 0.9343
1164/1688 [===================>..........] - ETA: 1s - loss: 0.2144 - accuracy: 0.9347
1186/1688 [====================>.........] - ETA: 1s - loss: 0.2121 - accuracy: 0.9355
1208/1688 [====================>.........] - ETA: 1s - loss: 0.2098 - accuracy: 0.9361
1230/1688 [====================>.........] - ETA: 1s - loss: 0.2076 - accuracy: 0.9368
1253/1688 [=====================>........] - ETA: 0s - loss: 0.2056 - accuracy: 0.9375
1276/1688 [=====================>........] - ETA: 0s - loss: 0.2040 - accuracy: 0.9383
1299/1688 [======================>.......] - ETA: 0s - loss: 0.2021 - accuracy: 0.9389
1321/1688 [======================>.......] - ETA: 0s - loss: 0.2006 - accuracy: 0.9394
1343/1688 [======================>.......] - ETA: 0s - loss: 0.1992 - accuracy: 0.9397
1365/1688 [=======================>......] - ETA: 0s - loss: 0.1975 - accuracy: 0.9403
1387/1688 [=======================>......] - ETA: 0s - loss: 0.1955 - accuracy: 0.9409
1410/1688 [========================>.....] - ETA: 0s - loss: 0.1939 - accuracy: 0.9414
1433/1688 [========================>.....] - ETA: 0s - loss: 0.1925 - accuracy: 0.9417
1455/1688 [========================>.....] - ETA: 0s - loss: 0.1913 - accuracy: 0.9421
1478/1688 [=========================>....] - ETA: 0s - loss: 0.1895 - accuracy: 0.9426
1501/1688 [=========================>....] - ETA: 0s - loss: 0.1880 - accuracy: 0.9430
1523/1688 [==========================>...] - ETA: 0s - loss: 0.1860 - accuracy: 0.9436
1545/1688 [==========================>...] - ETA: 0s - loss: 0.1842 - accuracy: 0.9442
1567/1688 [==========================>...] - ETA: 0s - loss: 0.1828 - accuracy: 0.9446
1590/1688 [===========================>..] - ETA: 0s - loss: 0.1813 - accuracy: 0.9449
1613/1688 [===========================>..] - ETA: 0s - loss: 0.1805 - accuracy: 0.9453
1635/1688 [============================>.] - ETA: 0s - loss: 0.1795 - accuracy: 0.9456
1657/1688 [============================>.] - ETA: 0s - loss: 0.1781 - accuracy: 0.9460
1679/1688 [============================>.] - ETA: 0s - loss: 0.1772 - accuracy: 0.9463
1688/1688 [==============================] - ETA: 0s - loss: 0.1766 - accuracy: 0.9465
1688/1688 [==============================] - 7s 3ms/step - loss: 0.1766 - accuracy: 0.9465 - val_loss: 0.0821 - val_accuracy: 0.9742
Epoch 2/10
1/1688 [..............................] - ETA: 6s - loss: 0.0573 - accuracy: 1.0000
18/1688 [..............................] - ETA: 4s - loss: 0.0653 - accuracy: 0.9809
41/1688 [..............................] - ETA: 4s - loss: 0.0818 - accuracy: 0.9748
63/1688 [>.............................] - ETA: 4s - loss: 0.0708 - accuracy: 0.9777
86/1688 [>.............................] - ETA: 3s - loss: 0.0677 - accuracy: 0.9793
108/1688 [>.............................] - ETA: 3s - loss: 0.0722 - accuracy: 0.9783
131/1688 [=>............................] - ETA: 3s - loss: 0.0681 - accuracy: 0.9800
153/1688 [=>............................] - ETA: 3s - loss: 0.0670 - accuracy: 0.9804
176/1688 [==>...........................] - ETA: 3s - loss: 0.0699 - accuracy: 0.9799
199/1688 [==>...........................] - ETA: 3s - loss: 0.0694 - accuracy: 0.9796
222/1688 [==>...........................] - ETA: 3s - loss: 0.0710 - accuracy: 0.9778
245/1688 [===>..........................] - ETA: 3s - loss: 0.0684 - accuracy: 0.9786
267/1688 [===>..........................] - ETA: 3s - loss: 0.0677 - accuracy: 0.9789
290/1688 [====>.........................] - ETA: 3s - loss: 0.0679 - accuracy: 0.9787
312/1688 [====>.........................] - ETA: 3s - loss: 0.0665 - accuracy: 0.9790
334/1688 [====>.........................] - ETA: 3s - loss: 0.0689 - accuracy: 0.9785
357/1688 [=====>........................] - ETA: 3s - loss: 0.0683 - accuracy: 0.9786
379/1688 [=====>........................] - ETA: 3s - loss: 0.0676 - accuracy: 0.9788
401/1688 [======>.......................] - ETA: 2s - loss: 0.0673 - accuracy: 0.9788
422/1688 [======>.......................] - ETA: 2s - loss: 0.0666 - accuracy: 0.9790
445/1688 [======>.......................] - ETA: 2s - loss: 0.0666 - accuracy: 0.9790
468/1688 [=======>......................] - ETA: 2s - loss: 0.0659 - accuracy: 0.9792
491/1688 [=======>......................] - ETA: 2s - loss: 0.0667 - accuracy: 0.9788
514/1688 [========>.....................] - ETA: 2s - loss: 0.0671 - accuracy: 0.9787
536/1688 [========>.....................] - ETA: 2s - loss: 0.0669 - accuracy: 0.9790
558/1688 [========>.....................] - ETA: 2s - loss: 0.0665 - accuracy: 0.9792
581/1688 [=========>....................] - ETA: 2s - loss: 0.0670 - accuracy: 0.9789
604/1688 [=========>....................] - ETA: 2s - loss: 0.0678 - accuracy: 0.9788
626/1688 [==========>...................] - ETA: 2s - loss: 0.0677 - accuracy: 0.9786
649/1688 [==========>...................] - ETA: 2s - loss: 0.0672 - accuracy: 0.9787
671/1688 [==========>...................] - ETA: 2s - loss: 0.0666 - accuracy: 0.9789
694/1688 [===========>..................] - ETA: 2s - loss: 0.0667 - accuracy: 0.9790
716/1688 [===========>..................] - ETA: 2s - loss: 0.0666 - accuracy: 0.9791
739/1688 [============>.................] - ETA: 2s - loss: 0.0665 - accuracy: 0.9791
761/1688 [============>.................] - ETA: 2s - loss: 0.0676 - accuracy: 0.9788
783/1688 [============>.................] - ETA: 2s - loss: 0.0680 - accuracy: 0.9786
805/1688 [=============>................] - ETA: 2s - loss: 0.0677 - accuracy: 0.9785
827/1688 [=============>................] - ETA: 1s - loss: 0.0682 - accuracy: 0.9783
849/1688 [==============>...............] - ETA: 1s - loss: 0.0682 - accuracy: 0.9784
871/1688 [==============>...............] - ETA: 1s - loss: 0.0686 - accuracy: 0.9785
893/1688 [==============>...............] - ETA: 1s - loss: 0.0684 - accuracy: 0.9787
915/1688 [===============>..............] - ETA: 1s - loss: 0.0688 - accuracy: 0.9785
938/1688 [===============>..............] - ETA: 1s - loss: 0.0680 - accuracy: 0.9786
961/1688 [================>.............] - ETA: 1s - loss: 0.0679 - accuracy: 0.9786
983/1688 [================>.............] - ETA: 1s - loss: 0.0677 - accuracy: 0.9787
1005/1688 [================>.............] - ETA: 1s - loss: 0.0680 - accuracy: 0.9786
1027/1688 [=================>............] - ETA: 1s - loss: 0.0678 - accuracy: 0.9787
1050/1688 [=================>............] - ETA: 1s - loss: 0.0679 - accuracy: 0.9786
1072/1688 [==================>...........] - ETA: 1s - loss: 0.0678 - accuracy: 0.9784
1094/1688 [==================>...........] - ETA: 1s - loss: 0.0674 - accuracy: 0.9786
1116/1688 [==================>...........] - ETA: 1s - loss: 0.0677 - accuracy: 0.9784
1139/1688 [===================>..........] - ETA: 1s - loss: 0.0678 - accuracy: 0.9785
1161/1688 [===================>..........] - ETA: 1s - loss: 0.0679 - accuracy: 0.9785
1184/1688 [====================>.........] - ETA: 1s - loss: 0.0674 - accuracy: 0.9786
1206/1688 [====================>.........] - ETA: 1s - loss: 0.0674 - accuracy: 0.9786
1228/1688 [====================>.........] - ETA: 1s - loss: 0.0674 - accuracy: 0.9788
1250/1688 [=====================>........] - ETA: 1s - loss: 0.0671 - accuracy: 0.9789
1272/1688 [=====================>........] - ETA: 0s - loss: 0.0669 - accuracy: 0.9789
1295/1688 [======================>.......] - ETA: 0s - loss: 0.0670 - accuracy: 0.9790
1318/1688 [======================>.......] - ETA: 0s - loss: 0.0677 - accuracy: 0.9789
1341/1688 [======================>.......] - ETA: 0s - loss: 0.0678 - accuracy: 0.9789
1363/1688 [=======================>......] - ETA: 0s - loss: 0.0676 - accuracy: 0.9789
1385/1688 [=======================>......] - ETA: 0s - loss: 0.0678 - accuracy: 0.9788
1406/1688 [=======================>......] - ETA: 0s - loss: 0.0676 - accuracy: 0.9788
1428/1688 [========================>.....] - ETA: 0s - loss: 0.0678 - accuracy: 0.9787
1451/1688 [========================>.....] - ETA: 0s - loss: 0.0680 - accuracy: 0.9786
1473/1688 [=========================>....] - ETA: 0s - loss: 0.0676 - accuracy: 0.9787
1496/1688 [=========================>....] - ETA: 0s - loss: 0.0676 - accuracy: 0.9787
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0679 - accuracy: 0.9786
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0680 - accuracy: 0.9786
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0680 - accuracy: 0.9786
1588/1688 [===========================>..] - ETA: 0s - loss: 0.0682 - accuracy: 0.9785
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0682 - accuracy: 0.9785
1633/1688 [============================>.] - ETA: 0s - loss: 0.0683 - accuracy: 0.9785
1656/1688 [============================>.] - ETA: 0s - loss: 0.0686 - accuracy: 0.9785
1679/1688 [============================>.] - ETA: 0s - loss: 0.0686 - accuracy: 0.9784
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0687 - accuracy: 0.9785 - val_loss: 0.0683 - val_accuracy: 0.9797
Epoch 3/10
1/1688 [..............................] - ETA: 4s - loss: 0.0048 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0362 - accuracy: 0.9891
46/1688 [..............................] - ETA: 3s - loss: 0.0444 - accuracy: 0.9837
69/1688 [>.............................] - ETA: 3s - loss: 0.0402 - accuracy: 0.9864
92/1688 [>.............................] - ETA: 3s - loss: 0.0385 - accuracy: 0.9861
114/1688 [=>............................] - ETA: 3s - loss: 0.0427 - accuracy: 0.9855
136/1688 [=>............................] - ETA: 3s - loss: 0.0439 - accuracy: 0.9855
158/1688 [=>............................] - ETA: 3s - loss: 0.0416 - accuracy: 0.9864
181/1688 [==>...........................] - ETA: 3s - loss: 0.0403 - accuracy: 0.9869
204/1688 [==>...........................] - ETA: 3s - loss: 0.0383 - accuracy: 0.9877
227/1688 [===>..........................] - ETA: 3s - loss: 0.0390 - accuracy: 0.9876
249/1688 [===>..........................] - ETA: 3s - loss: 0.0395 - accuracy: 0.9869
271/1688 [===>..........................] - ETA: 3s - loss: 0.0392 - accuracy: 0.9871
293/1688 [====>.........................] - ETA: 3s - loss: 0.0399 - accuracy: 0.9869
316/1688 [====>.........................] - ETA: 3s - loss: 0.0399 - accuracy: 0.9871
339/1688 [=====>........................] - ETA: 3s - loss: 0.0400 - accuracy: 0.9873
361/1688 [=====>........................] - ETA: 3s - loss: 0.0404 - accuracy: 0.9874
383/1688 [=====>........................] - ETA: 2s - loss: 0.0397 - accuracy: 0.9877
405/1688 [======>.......................] - ETA: 2s - loss: 0.0417 - accuracy: 0.9872
428/1688 [======>.......................] - ETA: 2s - loss: 0.0436 - accuracy: 0.9866
451/1688 [=======>......................] - ETA: 2s - loss: 0.0450 - accuracy: 0.9861
473/1688 [=======>......................] - ETA: 2s - loss: 0.0442 - accuracy: 0.9864
495/1688 [=======>......................] - ETA: 2s - loss: 0.0443 - accuracy: 0.9861
518/1688 [========>.....................] - ETA: 2s - loss: 0.0438 - accuracy: 0.9863
540/1688 [========>.....................] - ETA: 2s - loss: 0.0442 - accuracy: 0.9863
562/1688 [========>.....................] - ETA: 2s - loss: 0.0448 - accuracy: 0.9859
584/1688 [=========>....................] - ETA: 2s - loss: 0.0455 - accuracy: 0.9855
606/1688 [=========>....................] - ETA: 2s - loss: 0.0452 - accuracy: 0.9855
628/1688 [==========>...................] - ETA: 2s - loss: 0.0448 - accuracy: 0.9856
651/1688 [==========>...................] - ETA: 2s - loss: 0.0447 - accuracy: 0.9857
673/1688 [==========>...................] - ETA: 2s - loss: 0.0439 - accuracy: 0.9859
696/1688 [===========>..................] - ETA: 2s - loss: 0.0448 - accuracy: 0.9860
719/1688 [===========>..................] - ETA: 2s - loss: 0.0452 - accuracy: 0.9858
741/1688 [============>.................] - ETA: 2s - loss: 0.0453 - accuracy: 0.9857
763/1688 [============>.................] - ETA: 2s - loss: 0.0454 - accuracy: 0.9858
785/1688 [============>.................] - ETA: 2s - loss: 0.0461 - accuracy: 0.9856
808/1688 [=============>................] - ETA: 2s - loss: 0.0466 - accuracy: 0.9855
831/1688 [=============>................] - ETA: 1s - loss: 0.0464 - accuracy: 0.9854
854/1688 [==============>...............] - ETA: 1s - loss: 0.0473 - accuracy: 0.9854
876/1688 [==============>...............] - ETA: 1s - loss: 0.0474 - accuracy: 0.9854
898/1688 [==============>...............] - ETA: 1s - loss: 0.0478 - accuracy: 0.9853
920/1688 [===============>..............] - ETA: 1s - loss: 0.0482 - accuracy: 0.9851
942/1688 [===============>..............] - ETA: 1s - loss: 0.0495 - accuracy: 0.9848
965/1688 [================>.............] - ETA: 1s - loss: 0.0504 - accuracy: 0.9845
987/1688 [================>.............] - ETA: 1s - loss: 0.0506 - accuracy: 0.9844
1010/1688 [================>.............] - ETA: 1s - loss: 0.0502 - accuracy: 0.9846
1033/1688 [=================>............] - ETA: 1s - loss: 0.0502 - accuracy: 0.9847
1055/1688 [=================>............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9847
1077/1688 [==================>...........] - ETA: 1s - loss: 0.0499 - accuracy: 0.9849
1099/1688 [==================>...........] - ETA: 1s - loss: 0.0501 - accuracy: 0.9847
1121/1688 [==================>...........] - ETA: 1s - loss: 0.0503 - accuracy: 0.9847
1145/1688 [===================>..........] - ETA: 1s - loss: 0.0501 - accuracy: 0.9847
1168/1688 [===================>..........] - ETA: 1s - loss: 0.0506 - accuracy: 0.9846
1190/1688 [====================>.........] - ETA: 1s - loss: 0.0503 - accuracy: 0.9846
1211/1688 [====================>.........] - ETA: 1s - loss: 0.0502 - accuracy: 0.9845
1234/1688 [====================>.........] - ETA: 1s - loss: 0.0500 - accuracy: 0.9846
1256/1688 [=====================>........] - ETA: 0s - loss: 0.0499 - accuracy: 0.9845
1278/1688 [=====================>........] - ETA: 0s - loss: 0.0502 - accuracy: 0.9844
1301/1688 [======================>.......] - ETA: 0s - loss: 0.0505 - accuracy: 0.9844
1323/1688 [======================>.......] - ETA: 0s - loss: 0.0508 - accuracy: 0.9843
1345/1688 [======================>.......] - ETA: 0s - loss: 0.0510 - accuracy: 0.9842
1367/1688 [=======================>......] - ETA: 0s - loss: 0.0515 - accuracy: 0.9841
1388/1688 [=======================>......] - ETA: 0s - loss: 0.0513 - accuracy: 0.9842
1409/1688 [========================>.....] - ETA: 0s - loss: 0.0514 - accuracy: 0.9843
1431/1688 [========================>.....] - ETA: 0s - loss: 0.0513 - accuracy: 0.9843
1452/1688 [========================>.....] - ETA: 0s - loss: 0.0509 - accuracy: 0.9844
1474/1688 [=========================>....] - ETA: 0s - loss: 0.0508 - accuracy: 0.9844
1497/1688 [=========================>....] - ETA: 0s - loss: 0.0509 - accuracy: 0.9844
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0508 - accuracy: 0.9844
1541/1688 [==========================>...] - ETA: 0s - loss: 0.0508 - accuracy: 0.9844
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0509 - accuracy: 0.9843
1586/1688 [===========================>..] - ETA: 0s - loss: 0.0515 - accuracy: 0.9842
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0515 - accuracy: 0.9843
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0518 - accuracy: 0.9842
1652/1688 [============================>.] - ETA: 0s - loss: 0.0520 - accuracy: 0.9841
1674/1688 [============================>.] - ETA: 0s - loss: 0.0518 - accuracy: 0.9841
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0520 - accuracy: 0.9840 - val_loss: 0.0559 - val_accuracy: 0.9835
Epoch 4/10
1/1688 [..............................] - ETA: 4s - loss: 0.0032 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0187 - accuracy: 0.9946
45/1688 [..............................] - ETA: 3s - loss: 0.0193 - accuracy: 0.9951
67/1688 [>.............................] - ETA: 3s - loss: 0.0213 - accuracy: 0.9949
89/1688 [>.............................] - ETA: 3s - loss: 0.0203 - accuracy: 0.9944
111/1688 [>.............................] - ETA: 3s - loss: 0.0217 - accuracy: 0.9932
133/1688 [=>............................] - ETA: 3s - loss: 0.0243 - accuracy: 0.9930
156/1688 [=>............................] - ETA: 3s - loss: 0.0262 - accuracy: 0.9920
179/1688 [==>...........................] - ETA: 3s - loss: 0.0277 - accuracy: 0.9914
201/1688 [==>...........................] - ETA: 3s - loss: 0.0304 - accuracy: 0.9910
224/1688 [==>...........................] - ETA: 3s - loss: 0.0327 - accuracy: 0.9901
246/1688 [===>..........................] - ETA: 3s - loss: 0.0329 - accuracy: 0.9897
268/1688 [===>..........................] - ETA: 3s - loss: 0.0334 - accuracy: 0.9899
289/1688 [====>.........................] - ETA: 3s - loss: 0.0348 - accuracy: 0.9895
311/1688 [====>.........................] - ETA: 3s - loss: 0.0349 - accuracy: 0.9893
333/1688 [====>.........................] - ETA: 3s - loss: 0.0348 - accuracy: 0.9894
355/1688 [=====>........................] - ETA: 3s - loss: 0.0340 - accuracy: 0.9896
378/1688 [=====>........................] - ETA: 3s - loss: 0.0338 - accuracy: 0.9897
400/1688 [======>.......................] - ETA: 2s - loss: 0.0334 - accuracy: 0.9899
422/1688 [======>.......................] - ETA: 2s - loss: 0.0342 - accuracy: 0.9896
445/1688 [======>.......................] - ETA: 2s - loss: 0.0355 - accuracy: 0.9893
467/1688 [=======>......................] - ETA: 2s - loss: 0.0362 - accuracy: 0.9891
490/1688 [=======>......................] - ETA: 2s - loss: 0.0358 - accuracy: 0.9892
513/1688 [========>.....................] - ETA: 2s - loss: 0.0362 - accuracy: 0.9889
536/1688 [========>.....................] - ETA: 2s - loss: 0.0360 - accuracy: 0.9891
559/1688 [========>.....................] - ETA: 2s - loss: 0.0365 - accuracy: 0.9890
581/1688 [=========>....................] - ETA: 2s - loss: 0.0370 - accuracy: 0.9887
604/1688 [=========>....................] - ETA: 2s - loss: 0.0375 - accuracy: 0.9887
626/1688 [==========>...................] - ETA: 2s - loss: 0.0378 - accuracy: 0.9887
649/1688 [==========>...................] - ETA: 2s - loss: 0.0383 - accuracy: 0.9883
671/1688 [==========>...................] - ETA: 2s - loss: 0.0392 - accuracy: 0.9883
693/1688 [===========>..................] - ETA: 2s - loss: 0.0399 - accuracy: 0.9882
716/1688 [===========>..................] - ETA: 2s - loss: 0.0404 - accuracy: 0.9881
738/1688 [============>.................] - ETA: 2s - loss: 0.0407 - accuracy: 0.9880
761/1688 [============>.................] - ETA: 2s - loss: 0.0413 - accuracy: 0.9876
784/1688 [============>.................] - ETA: 2s - loss: 0.0411 - accuracy: 0.9877
806/1688 [=============>................] - ETA: 2s - loss: 0.0410 - accuracy: 0.9878
829/1688 [=============>................] - ETA: 1s - loss: 0.0410 - accuracy: 0.9878
851/1688 [==============>...............] - ETA: 1s - loss: 0.0407 - accuracy: 0.9878
874/1688 [==============>...............] - ETA: 1s - loss: 0.0406 - accuracy: 0.9879
897/1688 [==============>...............] - ETA: 1s - loss: 0.0403 - accuracy: 0.9880
920/1688 [===============>..............] - ETA: 1s - loss: 0.0406 - accuracy: 0.9878
943/1688 [===============>..............] - ETA: 1s - loss: 0.0408 - accuracy: 0.9879
966/1688 [================>.............] - ETA: 1s - loss: 0.0405 - accuracy: 0.9880
988/1688 [================>.............] - ETA: 1s - loss: 0.0404 - accuracy: 0.9880
1010/1688 [================>.............] - ETA: 1s - loss: 0.0404 - accuracy: 0.9881
1033/1688 [=================>............] - ETA: 1s - loss: 0.0406 - accuracy: 0.9880
1056/1688 [=================>............] - ETA: 1s - loss: 0.0405 - accuracy: 0.9880
1078/1688 [==================>...........] - ETA: 1s - loss: 0.0403 - accuracy: 0.9880
1100/1688 [==================>...........] - ETA: 1s - loss: 0.0405 - accuracy: 0.9880
1123/1688 [==================>...........] - ETA: 1s - loss: 0.0405 - accuracy: 0.9880
1145/1688 [===================>..........] - ETA: 1s - loss: 0.0405 - accuracy: 0.9880
1168/1688 [===================>..........] - ETA: 1s - loss: 0.0403 - accuracy: 0.9881
1191/1688 [====================>.........] - ETA: 1s - loss: 0.0400 - accuracy: 0.9882
1212/1688 [====================>.........] - ETA: 1s - loss: 0.0398 - accuracy: 0.9882
1236/1688 [====================>.........] - ETA: 1s - loss: 0.0394 - accuracy: 0.9883
1259/1688 [=====================>........] - ETA: 0s - loss: 0.0394 - accuracy: 0.9883
1281/1688 [=====================>........] - ETA: 0s - loss: 0.0398 - accuracy: 0.9882
1303/1688 [======================>.......] - ETA: 0s - loss: 0.0396 - accuracy: 0.9882
1325/1688 [======================>.......] - ETA: 0s - loss: 0.0396 - accuracy: 0.9883
1348/1688 [======================>.......] - ETA: 0s - loss: 0.0401 - accuracy: 0.9882
1370/1688 [=======================>......] - ETA: 0s - loss: 0.0404 - accuracy: 0.9881
1393/1688 [=======================>......] - ETA: 0s - loss: 0.0403 - accuracy: 0.9881
1415/1688 [========================>.....] - ETA: 0s - loss: 0.0402 - accuracy: 0.9881
1438/1688 [========================>.....] - ETA: 0s - loss: 0.0401 - accuracy: 0.9881
1460/1688 [========================>.....] - ETA: 0s - loss: 0.0399 - accuracy: 0.9882
1482/1688 [=========================>....] - ETA: 0s - loss: 0.0396 - accuracy: 0.9882
1504/1688 [=========================>....] - ETA: 0s - loss: 0.0399 - accuracy: 0.9881
1526/1688 [==========================>...] - ETA: 0s - loss: 0.0400 - accuracy: 0.9881
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0400 - accuracy: 0.9880
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0402 - accuracy: 0.9880
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0405 - accuracy: 0.9878
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0406 - accuracy: 0.9877
1636/1688 [============================>.] - ETA: 0s - loss: 0.0406 - accuracy: 0.9877
1659/1688 [============================>.] - ETA: 0s - loss: 0.0406 - accuracy: 0.9877
1681/1688 [============================>.] - ETA: 0s - loss: 0.0411 - accuracy: 0.9875
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0410 - accuracy: 0.9875 - val_loss: 0.0633 - val_accuracy: 0.9825
Epoch 5/10
1/1688 [..............................] - ETA: 3s - loss: 0.0820 - accuracy: 0.9688
23/1688 [..............................] - ETA: 3s - loss: 0.0339 - accuracy: 0.9878
45/1688 [..............................] - ETA: 3s - loss: 0.0325 - accuracy: 0.9868
67/1688 [>.............................] - ETA: 3s - loss: 0.0273 - accuracy: 0.9893
89/1688 [>.............................] - ETA: 3s - loss: 0.0254 - accuracy: 0.9909
111/1688 [>.............................] - ETA: 3s - loss: 0.0262 - accuracy: 0.9910
133/1688 [=>............................] - ETA: 3s - loss: 0.0262 - accuracy: 0.9908
156/1688 [=>............................] - ETA: 3s - loss: 0.0275 - accuracy: 0.9900
178/1688 [==>...........................] - ETA: 3s - loss: 0.0275 - accuracy: 0.9898
201/1688 [==>...........................] - ETA: 3s - loss: 0.0276 - accuracy: 0.9899
224/1688 [==>...........................] - ETA: 3s - loss: 0.0260 - accuracy: 0.9904
247/1688 [===>..........................] - ETA: 3s - loss: 0.0251 - accuracy: 0.9908
269/1688 [===>..........................] - ETA: 3s - loss: 0.0251 - accuracy: 0.9908
291/1688 [====>.........................] - ETA: 3s - loss: 0.0255 - accuracy: 0.9908
314/1688 [====>.........................] - ETA: 3s - loss: 0.0261 - accuracy: 0.9907
336/1688 [====>.........................] - ETA: 3s - loss: 0.0263 - accuracy: 0.9908
358/1688 [=====>........................] - ETA: 3s - loss: 0.0274 - accuracy: 0.9904
380/1688 [=====>........................] - ETA: 3s - loss: 0.0285 - accuracy: 0.9902
402/1688 [======>.......................] - ETA: 2s - loss: 0.0277 - accuracy: 0.9904
425/1688 [======>.......................] - ETA: 2s - loss: 0.0285 - accuracy: 0.9903
447/1688 [======>.......................] - ETA: 2s - loss: 0.0282 - accuracy: 0.9904
469/1688 [=======>......................] - ETA: 2s - loss: 0.0275 - accuracy: 0.9907
492/1688 [=======>......................] - ETA: 2s - loss: 0.0286 - accuracy: 0.9903
515/1688 [========>.....................] - ETA: 2s - loss: 0.0285 - accuracy: 0.9905
538/1688 [========>.....................] - ETA: 2s - loss: 0.0291 - accuracy: 0.9904
560/1688 [========>.....................] - ETA: 2s - loss: 0.0291 - accuracy: 0.9905
583/1688 [=========>....................] - ETA: 2s - loss: 0.0292 - accuracy: 0.9904
605/1688 [=========>....................] - ETA: 2s - loss: 0.0293 - accuracy: 0.9903
627/1688 [==========>...................] - ETA: 2s - loss: 0.0301 - accuracy: 0.9902
650/1688 [==========>...................] - ETA: 2s - loss: 0.0299 - accuracy: 0.9901
672/1688 [==========>...................] - ETA: 2s - loss: 0.0308 - accuracy: 0.9899
694/1688 [===========>..................] - ETA: 2s - loss: 0.0309 - accuracy: 0.9899
716/1688 [===========>..................] - ETA: 2s - loss: 0.0310 - accuracy: 0.9898
739/1688 [============>.................] - ETA: 2s - loss: 0.0308 - accuracy: 0.9899
761/1688 [============>.................] - ETA: 2s - loss: 0.0310 - accuracy: 0.9898
783/1688 [============>.................] - ETA: 2s - loss: 0.0311 - accuracy: 0.9899
806/1688 [=============>................] - ETA: 2s - loss: 0.0315 - accuracy: 0.9895
828/1688 [=============>................] - ETA: 1s - loss: 0.0317 - accuracy: 0.9894
850/1688 [==============>...............] - ETA: 1s - loss: 0.0319 - accuracy: 0.9894
873/1688 [==============>...............] - ETA: 1s - loss: 0.0322 - accuracy: 0.9893
895/1688 [==============>...............] - ETA: 1s - loss: 0.0321 - accuracy: 0.9894
917/1688 [===============>..............] - ETA: 1s - loss: 0.0327 - accuracy: 0.9892
939/1688 [===============>..............] - ETA: 1s - loss: 0.0326 - accuracy: 0.9892
962/1688 [================>.............] - ETA: 1s - loss: 0.0328 - accuracy: 0.9892
984/1688 [================>.............] - ETA: 1s - loss: 0.0328 - accuracy: 0.9891
1006/1688 [================>.............] - ETA: 1s - loss: 0.0330 - accuracy: 0.9891
1028/1688 [=================>............] - ETA: 1s - loss: 0.0329 - accuracy: 0.9891
1050/1688 [=================>............] - ETA: 1s - loss: 0.0327 - accuracy: 0.9891
1072/1688 [==================>...........] - ETA: 1s - loss: 0.0325 - accuracy: 0.9892
1095/1688 [==================>...........] - ETA: 1s - loss: 0.0322 - accuracy: 0.9893
1117/1688 [==================>...........] - ETA: 1s - loss: 0.0325 - accuracy: 0.9893
1140/1688 [===================>..........] - ETA: 1s - loss: 0.0325 - accuracy: 0.9893
1162/1688 [===================>..........] - ETA: 1s - loss: 0.0327 - accuracy: 0.9894
1185/1688 [====================>.........] - ETA: 1s - loss: 0.0326 - accuracy: 0.9894
1206/1688 [====================>.........] - ETA: 1s - loss: 0.0326 - accuracy: 0.9893
1228/1688 [====================>.........] - ETA: 1s - loss: 0.0326 - accuracy: 0.9894
1250/1688 [=====================>........] - ETA: 1s - loss: 0.0329 - accuracy: 0.9893
1272/1688 [=====================>........] - ETA: 0s - loss: 0.0326 - accuracy: 0.9894
1294/1688 [=====================>........] - ETA: 0s - loss: 0.0327 - accuracy: 0.9894
1315/1688 [======================>.......] - ETA: 0s - loss: 0.0329 - accuracy: 0.9894
1338/1688 [======================>.......] - ETA: 0s - loss: 0.0332 - accuracy: 0.9893
1360/1688 [=======================>......] - ETA: 0s - loss: 0.0332 - accuracy: 0.9892
1382/1688 [=======================>......] - ETA: 0s - loss: 0.0331 - accuracy: 0.9893
1405/1688 [=======================>......] - ETA: 0s - loss: 0.0331 - accuracy: 0.9892
1426/1688 [========================>.....] - ETA: 0s - loss: 0.0332 - accuracy: 0.9892
1448/1688 [========================>.....] - ETA: 0s - loss: 0.0332 - accuracy: 0.9892
1470/1688 [=========================>....] - ETA: 0s - loss: 0.0332 - accuracy: 0.9892
1493/1688 [=========================>....] - ETA: 0s - loss: 0.0331 - accuracy: 0.9893
1515/1688 [=========================>....] - ETA: 0s - loss: 0.0333 - accuracy: 0.9893
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0331 - accuracy: 0.9893
1561/1688 [==========================>...] - ETA: 0s - loss: 0.0331 - accuracy: 0.9893
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0332 - accuracy: 0.9892
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0333 - accuracy: 0.9893
1628/1688 [===========================>..] - ETA: 0s - loss: 0.0334 - accuracy: 0.9892
1650/1688 [============================>.] - ETA: 0s - loss: 0.0335 - accuracy: 0.9892
1672/1688 [============================>.] - ETA: 0s - loss: 0.0333 - accuracy: 0.9892
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0336 - accuracy: 0.9891 - val_loss: 0.0535 - val_accuracy: 0.9858
Epoch 6/10
1/1688 [..............................] - ETA: 4s - loss: 0.0053 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0261 - accuracy: 0.9891
45/1688 [..............................] - ETA: 3s - loss: 0.0266 - accuracy: 0.9896
67/1688 [>.............................] - ETA: 3s - loss: 0.0270 - accuracy: 0.9916
90/1688 [>.............................] - ETA: 3s - loss: 0.0249 - accuracy: 0.9924
112/1688 [>.............................] - ETA: 3s - loss: 0.0245 - accuracy: 0.9927
135/1688 [=>............................] - ETA: 3s - loss: 0.0222 - accuracy: 0.9935
157/1688 [=>............................] - ETA: 3s - loss: 0.0204 - accuracy: 0.9940
179/1688 [==>...........................] - ETA: 3s - loss: 0.0189 - accuracy: 0.9946
201/1688 [==>...........................] - ETA: 3s - loss: 0.0192 - accuracy: 0.9942
223/1688 [==>...........................] - ETA: 3s - loss: 0.0201 - accuracy: 0.9938
245/1688 [===>..........................] - ETA: 3s - loss: 0.0204 - accuracy: 0.9936
266/1688 [===>..........................] - ETA: 3s - loss: 0.0209 - accuracy: 0.9934
289/1688 [====>.........................] - ETA: 3s - loss: 0.0208 - accuracy: 0.9936
312/1688 [====>.........................] - ETA: 3s - loss: 0.0206 - accuracy: 0.9934
334/1688 [====>.........................] - ETA: 3s - loss: 0.0206 - accuracy: 0.9931
356/1688 [=====>........................] - ETA: 3s - loss: 0.0209 - accuracy: 0.9928
379/1688 [=====>........................] - ETA: 3s - loss: 0.0210 - accuracy: 0.9927
401/1688 [======>.......................] - ETA: 2s - loss: 0.0215 - accuracy: 0.9927
423/1688 [======>.......................] - ETA: 2s - loss: 0.0224 - accuracy: 0.9926
446/1688 [======>.......................] - ETA: 2s - loss: 0.0220 - accuracy: 0.9927
469/1688 [=======>......................] - ETA: 2s - loss: 0.0223 - accuracy: 0.9927
490/1688 [=======>......................] - ETA: 2s - loss: 0.0225 - accuracy: 0.9927
512/1688 [========>.....................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9924
534/1688 [========>.....................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9925
557/1688 [========>.....................] - ETA: 2s - loss: 0.0223 - accuracy: 0.9927
579/1688 [=========>....................] - ETA: 2s - loss: 0.0221 - accuracy: 0.9927
601/1688 [=========>....................] - ETA: 2s - loss: 0.0227 - accuracy: 0.9926
623/1688 [==========>...................] - ETA: 2s - loss: 0.0228 - accuracy: 0.9926
645/1688 [==========>...................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9926
668/1688 [==========>...................] - ETA: 2s - loss: 0.0235 - accuracy: 0.9924
690/1688 [===========>..................] - ETA: 2s - loss: 0.0238 - accuracy: 0.9923
713/1688 [===========>..................] - ETA: 2s - loss: 0.0244 - accuracy: 0.9920
735/1688 [============>.................] - ETA: 2s - loss: 0.0249 - accuracy: 0.9918
757/1688 [============>.................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9918
779/1688 [============>.................] - ETA: 2s - loss: 0.0244 - accuracy: 0.9919
801/1688 [=============>................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9917
824/1688 [=============>................] - ETA: 1s - loss: 0.0247 - accuracy: 0.9917
847/1688 [==============>...............] - ETA: 1s - loss: 0.0244 - accuracy: 0.9919
871/1688 [==============>...............] - ETA: 1s - loss: 0.0243 - accuracy: 0.9919
894/1688 [==============>...............] - ETA: 1s - loss: 0.0247 - accuracy: 0.9918
916/1688 [===============>..............] - ETA: 1s - loss: 0.0247 - accuracy: 0.9918
939/1688 [===============>..............] - ETA: 1s - loss: 0.0255 - accuracy: 0.9916
961/1688 [================>.............] - ETA: 1s - loss: 0.0259 - accuracy: 0.9914
983/1688 [================>.............] - ETA: 1s - loss: 0.0259 - accuracy: 0.9914
1005/1688 [================>.............] - ETA: 1s - loss: 0.0259 - accuracy: 0.9913
1027/1688 [=================>............] - ETA: 1s - loss: 0.0261 - accuracy: 0.9912
1049/1688 [=================>............] - ETA: 1s - loss: 0.0260 - accuracy: 0.9913
1071/1688 [==================>...........] - ETA: 1s - loss: 0.0261 - accuracy: 0.9913
1093/1688 [==================>...........] - ETA: 1s - loss: 0.0263 - accuracy: 0.9913
1116/1688 [==================>...........] - ETA: 1s - loss: 0.0262 - accuracy: 0.9913
1139/1688 [===================>..........] - ETA: 1s - loss: 0.0263 - accuracy: 0.9911
1161/1688 [===================>..........] - ETA: 1s - loss: 0.0264 - accuracy: 0.9911
1183/1688 [====================>.........] - ETA: 1s - loss: 0.0270 - accuracy: 0.9910
1205/1688 [====================>.........] - ETA: 1s - loss: 0.0276 - accuracy: 0.9907
1228/1688 [====================>.........] - ETA: 1s - loss: 0.0274 - accuracy: 0.9908
1250/1688 [=====================>........] - ETA: 1s - loss: 0.0274 - accuracy: 0.9908
1272/1688 [=====================>........] - ETA: 0s - loss: 0.0273 - accuracy: 0.9909
1295/1688 [======================>.......] - ETA: 0s - loss: 0.0271 - accuracy: 0.9910
1318/1688 [======================>.......] - ETA: 0s - loss: 0.0271 - accuracy: 0.9910
1341/1688 [======================>.......] - ETA: 0s - loss: 0.0275 - accuracy: 0.9910
1364/1688 [=======================>......] - ETA: 0s - loss: 0.0275 - accuracy: 0.9910
1387/1688 [=======================>......] - ETA: 0s - loss: 0.0276 - accuracy: 0.9909
1409/1688 [========================>.....] - ETA: 0s - loss: 0.0278 - accuracy: 0.9909
1431/1688 [========================>.....] - ETA: 0s - loss: 0.0281 - accuracy: 0.9908
1453/1688 [========================>.....] - ETA: 0s - loss: 0.0282 - accuracy: 0.9908
1476/1688 [=========================>....] - ETA: 0s - loss: 0.0281 - accuracy: 0.9907
1498/1688 [=========================>....] - ETA: 0s - loss: 0.0280 - accuracy: 0.9908
1521/1688 [==========================>...] - ETA: 0s - loss: 0.0282 - accuracy: 0.9907
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0282 - accuracy: 0.9907
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0284 - accuracy: 0.9906
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0283 - accuracy: 0.9906
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0284 - accuracy: 0.9906
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0284 - accuracy: 0.9905
1654/1688 [============================>.] - ETA: 0s - loss: 0.0285 - accuracy: 0.9905
1676/1688 [============================>.] - ETA: 0s - loss: 0.0286 - accuracy: 0.9905
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0285 - accuracy: 0.9905 - val_loss: 0.0573 - val_accuracy: 0.9858
Epoch 7/10
1/1688 [..............................] - ETA: 3s - loss: 0.0593 - accuracy: 0.9688
23/1688 [..............................] - ETA: 3s - loss: 0.0346 - accuracy: 0.9878
45/1688 [..............................] - ETA: 3s - loss: 0.0317 - accuracy: 0.9889
68/1688 [>.............................] - ETA: 3s - loss: 0.0313 - accuracy: 0.9894
90/1688 [>.............................] - ETA: 3s - loss: 0.0275 - accuracy: 0.9903
112/1688 [>.............................] - ETA: 3s - loss: 0.0271 - accuracy: 0.9902
134/1688 [=>............................] - ETA: 3s - loss: 0.0268 - accuracy: 0.9904
157/1688 [=>............................] - ETA: 3s - loss: 0.0254 - accuracy: 0.9906
179/1688 [==>...........................] - ETA: 3s - loss: 0.0241 - accuracy: 0.9913
201/1688 [==>...........................] - ETA: 3s - loss: 0.0232 - accuracy: 0.9916
222/1688 [==>...........................] - ETA: 3s - loss: 0.0228 - accuracy: 0.9917
245/1688 [===>..........................] - ETA: 3s - loss: 0.0218 - accuracy: 0.9921
268/1688 [===>..........................] - ETA: 3s - loss: 0.0211 - accuracy: 0.9924
290/1688 [====>.........................] - ETA: 3s - loss: 0.0209 - accuracy: 0.9925
312/1688 [====>.........................] - ETA: 3s - loss: 0.0207 - accuracy: 0.9924
335/1688 [====>.........................] - ETA: 3s - loss: 0.0211 - accuracy: 0.9926
357/1688 [=====>........................] - ETA: 3s - loss: 0.0210 - accuracy: 0.9927
380/1688 [=====>........................] - ETA: 2s - loss: 0.0206 - accuracy: 0.9929
402/1688 [======>.......................] - ETA: 2s - loss: 0.0201 - accuracy: 0.9931
424/1688 [======>.......................] - ETA: 2s - loss: 0.0202 - accuracy: 0.9931
446/1688 [======>.......................] - ETA: 2s - loss: 0.0208 - accuracy: 0.9931
469/1688 [=======>......................] - ETA: 2s - loss: 0.0211 - accuracy: 0.9932
492/1688 [=======>......................] - ETA: 2s - loss: 0.0218 - accuracy: 0.9930
515/1688 [========>.....................] - ETA: 2s - loss: 0.0213 - accuracy: 0.9931
537/1688 [========>.....................] - ETA: 2s - loss: 0.0213 - accuracy: 0.9932
560/1688 [========>.....................] - ETA: 2s - loss: 0.0219 - accuracy: 0.9931
582/1688 [=========>....................] - ETA: 2s - loss: 0.0226 - accuracy: 0.9928
603/1688 [=========>....................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9927
625/1688 [==========>...................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9927
648/1688 [==========>...................] - ETA: 2s - loss: 0.0229 - accuracy: 0.9928
670/1688 [==========>...................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9926
693/1688 [===========>..................] - ETA: 2s - loss: 0.0231 - accuracy: 0.9927
715/1688 [===========>..................] - ETA: 2s - loss: 0.0236 - accuracy: 0.9927
738/1688 [============>.................] - ETA: 2s - loss: 0.0232 - accuracy: 0.9928
760/1688 [============>.................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9929
782/1688 [============>.................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9929
804/1688 [=============>................] - ETA: 2s - loss: 0.0230 - accuracy: 0.9928
827/1688 [=============>................] - ETA: 1s - loss: 0.0230 - accuracy: 0.9928
849/1688 [==============>...............] - ETA: 1s - loss: 0.0229 - accuracy: 0.9929
872/1688 [==============>...............] - ETA: 1s - loss: 0.0232 - accuracy: 0.9928
894/1688 [==============>...............] - ETA: 1s - loss: 0.0234 - accuracy: 0.9927
916/1688 [===============>..............] - ETA: 1s - loss: 0.0234 - accuracy: 0.9927
938/1688 [===============>..............] - ETA: 1s - loss: 0.0236 - accuracy: 0.9926
960/1688 [================>.............] - ETA: 1s - loss: 0.0238 - accuracy: 0.9926
983/1688 [================>.............] - ETA: 1s - loss: 0.0239 - accuracy: 0.9926
1005/1688 [================>.............] - ETA: 1s - loss: 0.0236 - accuracy: 0.9927
1027/1688 [=================>............] - ETA: 1s - loss: 0.0233 - accuracy: 0.9928
1049/1688 [=================>............] - ETA: 1s - loss: 0.0235 - accuracy: 0.9926
1072/1688 [==================>...........] - ETA: 1s - loss: 0.0236 - accuracy: 0.9926
1095/1688 [==================>...........] - ETA: 1s - loss: 0.0236 - accuracy: 0.9926
1117/1688 [==================>...........] - ETA: 1s - loss: 0.0235 - accuracy: 0.9926
1140/1688 [===================>..........] - ETA: 1s - loss: 0.0235 - accuracy: 0.9926
1163/1688 [===================>..........] - ETA: 1s - loss: 0.0235 - accuracy: 0.9927
1186/1688 [====================>.........] - ETA: 1s - loss: 0.0237 - accuracy: 0.9926
1208/1688 [====================>.........] - ETA: 1s - loss: 0.0237 - accuracy: 0.9925
1231/1688 [====================>.........] - ETA: 1s - loss: 0.0240 - accuracy: 0.9925
1253/1688 [=====================>........] - ETA: 0s - loss: 0.0239 - accuracy: 0.9925
1275/1688 [=====================>........] - ETA: 0s - loss: 0.0238 - accuracy: 0.9925
1298/1688 [======================>.......] - ETA: 0s - loss: 0.0238 - accuracy: 0.9925
1320/1688 [======================>.......] - ETA: 0s - loss: 0.0241 - accuracy: 0.9924
1342/1688 [======================>.......] - ETA: 0s - loss: 0.0242 - accuracy: 0.9924
1364/1688 [=======================>......] - ETA: 0s - loss: 0.0243 - accuracy: 0.9923
1386/1688 [=======================>......] - ETA: 0s - loss: 0.0242 - accuracy: 0.9924
1408/1688 [========================>.....] - ETA: 0s - loss: 0.0241 - accuracy: 0.9924
1430/1688 [========================>.....] - ETA: 0s - loss: 0.0240 - accuracy: 0.9924
1451/1688 [========================>.....] - ETA: 0s - loss: 0.0240 - accuracy: 0.9924
1473/1688 [=========================>....] - ETA: 0s - loss: 0.0239 - accuracy: 0.9924
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0240 - accuracy: 0.9924
1517/1688 [=========================>....] - ETA: 0s - loss: 0.0238 - accuracy: 0.9925
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0236 - accuracy: 0.9925
1562/1688 [==========================>...] - ETA: 0s - loss: 0.0236 - accuracy: 0.9926
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0236 - accuracy: 0.9926
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0236 - accuracy: 0.9926
1628/1688 [===========================>..] - ETA: 0s - loss: 0.0237 - accuracy: 0.9925
1649/1688 [============================>.] - ETA: 0s - loss: 0.0237 - accuracy: 0.9925
1671/1688 [============================>.] - ETA: 0s - loss: 0.0236 - accuracy: 0.9925
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0237 - accuracy: 0.9924 - val_loss: 0.0571 - val_accuracy: 0.9855
Epoch 8/10
1/1688 [..............................] - ETA: 4s - loss: 0.0023 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0242 - accuracy: 0.9932
45/1688 [..............................] - ETA: 3s - loss: 0.0197 - accuracy: 0.9931
67/1688 [>.............................] - ETA: 3s - loss: 0.0162 - accuracy: 0.9939
89/1688 [>.............................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9944
111/1688 [>.............................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9949
133/1688 [=>............................] - ETA: 3s - loss: 0.0161 - accuracy: 0.9948
156/1688 [=>............................] - ETA: 3s - loss: 0.0156 - accuracy: 0.9950
178/1688 [==>...........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9953
200/1688 [==>...........................] - ETA: 3s - loss: 0.0148 - accuracy: 0.9953
222/1688 [==>...........................] - ETA: 3s - loss: 0.0151 - accuracy: 0.9952
244/1688 [===>..........................] - ETA: 3s - loss: 0.0154 - accuracy: 0.9950
267/1688 [===>..........................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9953
289/1688 [====>.........................] - ETA: 3s - loss: 0.0145 - accuracy: 0.9954
312/1688 [====>.........................] - ETA: 3s - loss: 0.0144 - accuracy: 0.9952
334/1688 [====>.........................] - ETA: 3s - loss: 0.0139 - accuracy: 0.9954
357/1688 [=====>........................] - ETA: 3s - loss: 0.0139 - accuracy: 0.9954
380/1688 [=====>........................] - ETA: 3s - loss: 0.0142 - accuracy: 0.9953
402/1688 [======>.......................] - ETA: 2s - loss: 0.0143 - accuracy: 0.9953
425/1688 [======>.......................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9954
447/1688 [======>.......................] - ETA: 2s - loss: 0.0141 - accuracy: 0.9955
469/1688 [=======>......................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9954
491/1688 [=======>......................] - ETA: 2s - loss: 0.0142 - accuracy: 0.9955
513/1688 [========>.....................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9954
535/1688 [========>.....................] - ETA: 2s - loss: 0.0148 - accuracy: 0.9954
556/1688 [========>.....................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9953
578/1688 [=========>....................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9953
600/1688 [=========>....................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9953
623/1688 [==========>...................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9951
646/1688 [==========>...................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9951
668/1688 [==========>...................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9952
691/1688 [===========>..................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9952
714/1688 [===========>..................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9952
736/1688 [============>.................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9953
760/1688 [============>.................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9952
782/1688 [============>.................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9951
805/1688 [=============>................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9951
827/1688 [=============>................] - ETA: 1s - loss: 0.0158 - accuracy: 0.9950
849/1688 [==============>...............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9951
871/1688 [==============>...............] - ETA: 1s - loss: 0.0160 - accuracy: 0.9950
893/1688 [==============>...............] - ETA: 1s - loss: 0.0158 - accuracy: 0.9951
915/1688 [===============>..............] - ETA: 1s - loss: 0.0159 - accuracy: 0.9950
937/1688 [===============>..............] - ETA: 1s - loss: 0.0157 - accuracy: 0.9951
960/1688 [================>.............] - ETA: 1s - loss: 0.0158 - accuracy: 0.9950
983/1688 [================>.............] - ETA: 1s - loss: 0.0162 - accuracy: 0.9949
1005/1688 [================>.............] - ETA: 1s - loss: 0.0165 - accuracy: 0.9947
1027/1688 [=================>............] - ETA: 1s - loss: 0.0172 - accuracy: 0.9946
1049/1688 [=================>............] - ETA: 1s - loss: 0.0174 - accuracy: 0.9945
1071/1688 [==================>...........] - ETA: 1s - loss: 0.0175 - accuracy: 0.9944
1094/1688 [==================>...........] - ETA: 1s - loss: 0.0175 - accuracy: 0.9943
1116/1688 [==================>...........] - ETA: 1s - loss: 0.0176 - accuracy: 0.9942
1138/1688 [===================>..........] - ETA: 1s - loss: 0.0177 - accuracy: 0.9942
1161/1688 [===================>..........] - ETA: 1s - loss: 0.0181 - accuracy: 0.9941
1183/1688 [====================>.........] - ETA: 1s - loss: 0.0184 - accuracy: 0.9941
1205/1688 [====================>.........] - ETA: 1s - loss: 0.0184 - accuracy: 0.9941
1227/1688 [====================>.........] - ETA: 1s - loss: 0.0182 - accuracy: 0.9942
1249/1688 [=====================>........] - ETA: 1s - loss: 0.0182 - accuracy: 0.9942
1271/1688 [=====================>........] - ETA: 0s - loss: 0.0181 - accuracy: 0.9942
1294/1688 [=====================>........] - ETA: 0s - loss: 0.0179 - accuracy: 0.9943
1317/1688 [======================>.......] - ETA: 0s - loss: 0.0181 - accuracy: 0.9943
1339/1688 [======================>.......] - ETA: 0s - loss: 0.0181 - accuracy: 0.9943
1361/1688 [=======================>......] - ETA: 0s - loss: 0.0181 - accuracy: 0.9942
1383/1688 [=======================>......] - ETA: 0s - loss: 0.0183 - accuracy: 0.9941
1406/1688 [=======================>......] - ETA: 0s - loss: 0.0184 - accuracy: 0.9941
1428/1688 [========================>.....] - ETA: 0s - loss: 0.0186 - accuracy: 0.9940
1450/1688 [========================>.....] - ETA: 0s - loss: 0.0186 - accuracy: 0.9939
1472/1688 [=========================>....] - ETA: 0s - loss: 0.0186 - accuracy: 0.9939
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0186 - accuracy: 0.9939
1517/1688 [=========================>....] - ETA: 0s - loss: 0.0187 - accuracy: 0.9939
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0186 - accuracy: 0.9938
1562/1688 [==========================>...] - ETA: 0s - loss: 0.0187 - accuracy: 0.9938
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0188 - accuracy: 0.9938
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0189 - accuracy: 0.9938
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0190 - accuracy: 0.9936
1652/1688 [============================>.] - ETA: 0s - loss: 0.0189 - accuracy: 0.9936
1674/1688 [============================>.] - ETA: 0s - loss: 0.0192 - accuracy: 0.9936
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0191 - accuracy: 0.9936 - val_loss: 0.0642 - val_accuracy: 0.9833
Epoch 9/10
1/1688 [..............................] - ETA: 3s - loss: 0.0669 - accuracy: 0.9688
24/1688 [..............................] - ETA: 3s - loss: 0.0283 - accuracy: 0.9909
47/1688 [..............................] - ETA: 3s - loss: 0.0196 - accuracy: 0.9934
69/1688 [>.............................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9946
91/1688 [>.............................] - ETA: 3s - loss: 0.0167 - accuracy: 0.9948
113/1688 [=>............................] - ETA: 3s - loss: 0.0160 - accuracy: 0.9950
136/1688 [=>............................] - ETA: 3s - loss: 0.0152 - accuracy: 0.9949
158/1688 [=>............................] - ETA: 3s - loss: 0.0147 - accuracy: 0.9951
180/1688 [==>...........................] - ETA: 3s - loss: 0.0154 - accuracy: 0.9948
201/1688 [==>...........................] - ETA: 3s - loss: 0.0164 - accuracy: 0.9941
222/1688 [==>...........................] - ETA: 3s - loss: 0.0158 - accuracy: 0.9942
244/1688 [===>..........................] - ETA: 3s - loss: 0.0167 - accuracy: 0.9940
266/1688 [===>..........................] - ETA: 3s - loss: 0.0174 - accuracy: 0.9938
288/1688 [====>.........................] - ETA: 3s - loss: 0.0177 - accuracy: 0.9939
310/1688 [====>.........................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9942
332/1688 [====>.........................] - ETA: 3s - loss: 0.0173 - accuracy: 0.9938
354/1688 [=====>........................] - ETA: 3s - loss: 0.0179 - accuracy: 0.9937
376/1688 [=====>........................] - ETA: 3s - loss: 0.0178 - accuracy: 0.9938
398/1688 [======>.......................] - ETA: 2s - loss: 0.0186 - accuracy: 0.9935
420/1688 [======>.......................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9937
442/1688 [======>.......................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9939
464/1688 [=======>......................] - ETA: 2s - loss: 0.0173 - accuracy: 0.9940
486/1688 [=======>......................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9941
508/1688 [========>.....................] - ETA: 2s - loss: 0.0176 - accuracy: 0.9941
531/1688 [========>.....................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9939
553/1688 [========>.....................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9941
575/1688 [=========>....................] - ETA: 2s - loss: 0.0173 - accuracy: 0.9942
598/1688 [=========>....................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9944
621/1688 [==========>...................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9945
643/1688 [==========>...................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9945
665/1688 [==========>...................] - ETA: 2s - loss: 0.0164 - accuracy: 0.9945
687/1688 [===========>..................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9946
709/1688 [===========>..................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9946
731/1688 [===========>..................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9947
753/1688 [============>.................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9947
775/1688 [============>.................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9947
798/1688 [=============>................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9948
820/1688 [=============>................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9947
843/1688 [=============>................] - ETA: 1s - loss: 0.0166 - accuracy: 0.9947
866/1688 [==============>...............] - ETA: 1s - loss: 0.0164 - accuracy: 0.9947
888/1688 [==============>...............] - ETA: 1s - loss: 0.0164 - accuracy: 0.9948
910/1688 [===============>..............] - ETA: 1s - loss: 0.0166 - accuracy: 0.9947
933/1688 [===============>..............] - ETA: 1s - loss: 0.0165 - accuracy: 0.9947
955/1688 [===============>..............] - ETA: 1s - loss: 0.0165 - accuracy: 0.9947
978/1688 [================>.............] - ETA: 1s - loss: 0.0166 - accuracy: 0.9947
1001/1688 [================>.............] - ETA: 1s - loss: 0.0164 - accuracy: 0.9947
1023/1688 [=================>............] - ETA: 1s - loss: 0.0166 - accuracy: 0.9946
1045/1688 [=================>............] - ETA: 1s - loss: 0.0165 - accuracy: 0.9946
1067/1688 [=================>............] - ETA: 1s - loss: 0.0172 - accuracy: 0.9944
1090/1688 [==================>...........] - ETA: 1s - loss: 0.0172 - accuracy: 0.9944
1112/1688 [==================>...........] - ETA: 1s - loss: 0.0175 - accuracy: 0.9943
1135/1688 [===================>..........] - ETA: 1s - loss: 0.0174 - accuracy: 0.9943
1157/1688 [===================>..........] - ETA: 1s - loss: 0.0173 - accuracy: 0.9943
1179/1688 [===================>..........] - ETA: 1s - loss: 0.0171 - accuracy: 0.9944
1201/1688 [====================>.........] - ETA: 1s - loss: 0.0170 - accuracy: 0.9945
1224/1688 [====================>.........] - ETA: 1s - loss: 0.0172 - accuracy: 0.9943
1246/1688 [=====================>........] - ETA: 1s - loss: 0.0172 - accuracy: 0.9943
1268/1688 [=====================>........] - ETA: 0s - loss: 0.0171 - accuracy: 0.9943
1291/1688 [=====================>........] - ETA: 0s - loss: 0.0170 - accuracy: 0.9944
1313/1688 [======================>.......] - ETA: 0s - loss: 0.0170 - accuracy: 0.9944
1336/1688 [======================>.......] - ETA: 0s - loss: 0.0173 - accuracy: 0.9943
1357/1688 [=======================>......] - ETA: 0s - loss: 0.0173 - accuracy: 0.9943
1379/1688 [=======================>......] - ETA: 0s - loss: 0.0175 - accuracy: 0.9943
1401/1688 [=======================>......] - ETA: 0s - loss: 0.0175 - accuracy: 0.9943
1423/1688 [========================>.....] - ETA: 0s - loss: 0.0175 - accuracy: 0.9943
1445/1688 [========================>.....] - ETA: 0s - loss: 0.0174 - accuracy: 0.9943
1467/1688 [=========================>....] - ETA: 0s - loss: 0.0176 - accuracy: 0.9942
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0177 - accuracy: 0.9941
1511/1688 [=========================>....] - ETA: 0s - loss: 0.0177 - accuracy: 0.9941
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0178 - accuracy: 0.9941
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0176 - accuracy: 0.9942
1578/1688 [===========================>..] - ETA: 0s - loss: 0.0178 - accuracy: 0.9942
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0177 - accuracy: 0.9942
1623/1688 [===========================>..] - ETA: 0s - loss: 0.0176 - accuracy: 0.9942
1646/1688 [============================>.] - ETA: 0s - loss: 0.0175 - accuracy: 0.9942
1668/1688 [============================>.] - ETA: 0s - loss: 0.0175 - accuracy: 0.9942
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0173 - accuracy: 0.9943 - val_loss: 0.0540 - val_accuracy: 0.9873
Epoch 10/10
1/1688 [..............................] - ETA: 3s - loss: 0.0053 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0209 - accuracy: 0.9932
45/1688 [..............................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9958
67/1688 [>.............................] - ETA: 3s - loss: 0.0121 - accuracy: 0.9967
90/1688 [>.............................] - ETA: 3s - loss: 0.0110 - accuracy: 0.9972
112/1688 [>.............................] - ETA: 3s - loss: 0.0134 - accuracy: 0.9972
134/1688 [=>............................] - ETA: 3s - loss: 0.0149 - accuracy: 0.9963
156/1688 [=>............................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9966
178/1688 [==>...........................] - ETA: 3s - loss: 0.0166 - accuracy: 0.9956
200/1688 [==>...........................] - ETA: 3s - loss: 0.0166 - accuracy: 0.9956
223/1688 [==>...........................] - ETA: 3s - loss: 0.0164 - accuracy: 0.9957
245/1688 [===>..........................] - ETA: 3s - loss: 0.0162 - accuracy: 0.9955
267/1688 [===>..........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9958
289/1688 [====>.........................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9960
311/1688 [====>.........................] - ETA: 3s - loss: 0.0140 - accuracy: 0.9963
333/1688 [====>.........................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9962
355/1688 [=====>........................] - ETA: 3s - loss: 0.0139 - accuracy: 0.9961
377/1688 [=====>........................] - ETA: 3s - loss: 0.0143 - accuracy: 0.9959
400/1688 [======>.......................] - ETA: 2s - loss: 0.0150 - accuracy: 0.9954
423/1688 [======>.......................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9954
445/1688 [======>.......................] - ETA: 2s - loss: 0.0145 - accuracy: 0.9956
466/1688 [=======>......................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9954
487/1688 [=======>......................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9956
510/1688 [========>.....................] - ETA: 2s - loss: 0.0155 - accuracy: 0.9954
533/1688 [========>.....................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9955
555/1688 [========>.....................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9955
577/1688 [=========>....................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9955
599/1688 [=========>....................] - ETA: 2s - loss: 0.0151 - accuracy: 0.9955
621/1688 [==========>...................] - ETA: 2s - loss: 0.0154 - accuracy: 0.9954
644/1688 [==========>...................] - ETA: 2s - loss: 0.0153 - accuracy: 0.9954
666/1688 [==========>...................] - ETA: 2s - loss: 0.0152 - accuracy: 0.9954
688/1688 [===========>..................] - ETA: 2s - loss: 0.0149 - accuracy: 0.9955
710/1688 [===========>..................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9956
733/1688 [============>.................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9956
755/1688 [============>.................] - ETA: 2s - loss: 0.0144 - accuracy: 0.9956
778/1688 [============>.................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9956
801/1688 [=============>................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9954
823/1688 [=============>................] - ETA: 1s - loss: 0.0145 - accuracy: 0.9955
846/1688 [==============>...............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9956
868/1688 [==============>...............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9955
891/1688 [==============>...............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9956
913/1688 [===============>..............] - ETA: 1s - loss: 0.0144 - accuracy: 0.9954
935/1688 [===============>..............] - ETA: 1s - loss: 0.0142 - accuracy: 0.9955
958/1688 [================>.............] - ETA: 1s - loss: 0.0143 - accuracy: 0.9955
980/1688 [================>.............] - ETA: 1s - loss: 0.0144 - accuracy: 0.9953
1002/1688 [================>.............] - ETA: 1s - loss: 0.0144 - accuracy: 0.9953
1024/1688 [=================>............] - ETA: 1s - loss: 0.0145 - accuracy: 0.9952
1046/1688 [=================>............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9951
1069/1688 [=================>............] - ETA: 1s - loss: 0.0147 - accuracy: 0.9951
1091/1688 [==================>...........] - ETA: 1s - loss: 0.0151 - accuracy: 0.9950
1113/1688 [==================>...........] - ETA: 1s - loss: 0.0150 - accuracy: 0.9949
1134/1688 [===================>..........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9949
1156/1688 [===================>..........] - ETA: 1s - loss: 0.0149 - accuracy: 0.9949
1178/1688 [===================>..........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9950
1200/1688 [====================>.........] - ETA: 1s - loss: 0.0145 - accuracy: 0.9951
1223/1688 [====================>.........] - ETA: 1s - loss: 0.0145 - accuracy: 0.9951
1246/1688 [=====================>........] - ETA: 1s - loss: 0.0145 - accuracy: 0.9950
1268/1688 [=====================>........] - ETA: 0s - loss: 0.0145 - accuracy: 0.9950
1291/1688 [=====================>........] - ETA: 0s - loss: 0.0146 - accuracy: 0.9950
1313/1688 [======================>.......] - ETA: 0s - loss: 0.0146 - accuracy: 0.9950
1335/1688 [======================>.......] - ETA: 0s - loss: 0.0146 - accuracy: 0.9950
1357/1688 [=======================>......] - ETA: 0s - loss: 0.0145 - accuracy: 0.9951
1380/1688 [=======================>......] - ETA: 0s - loss: 0.0143 - accuracy: 0.9952
1402/1688 [=======================>......] - ETA: 0s - loss: 0.0144 - accuracy: 0.9952
1424/1688 [========================>.....] - ETA: 0s - loss: 0.0143 - accuracy: 0.9952
1447/1688 [========================>.....] - ETA: 0s - loss: 0.0142 - accuracy: 0.9952
1469/1688 [=========================>....] - ETA: 0s - loss: 0.0141 - accuracy: 0.9952
1492/1688 [=========================>....] - ETA: 0s - loss: 0.0141 - accuracy: 0.9952
1514/1688 [=========================>....] - ETA: 0s - loss: 0.0140 - accuracy: 0.9953
1536/1688 [==========================>...] - ETA: 0s - loss: 0.0139 - accuracy: 0.9953
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0141 - accuracy: 0.9952
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0141 - accuracy: 0.9952
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0141 - accuracy: 0.9951
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0144 - accuracy: 0.9951
1646/1688 [============================>.] - ETA: 0s - loss: 0.0145 - accuracy: 0.9951
1669/1688 [============================>.] - ETA: 0s - loss: 0.0145 - accuracy: 0.9951
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0147 - accuracy: 0.9950 - val_loss: 0.0549 - val_accuracy: 0.9878
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.98580002784729
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bits integer activations and weights. Therefore, the floating point TF-Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a TF-Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating point TF-Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a TF-Keras model with custom QuantizeML quantized layers substituted for the original TF-Keras layers.
All TF-Keras API functions can be applied on this new model: summary(), compile(),
fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:489: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate. "
1/1024 [..............................] - ETA: 2:28
51/1024 [>.............................] - ETA: 0s
103/1024 [==>...........................] - ETA: 0s
154/1024 [===>..........................] - ETA: 0s
206/1024 [=====>........................] - ETA: 0s
258/1024 [======>.......................] - ETA: 0s
310/1024 [========>.....................] - ETA: 0s
361/1024 [=========>....................] - ETA: 0s
412/1024 [===========>..................] - ETA: 0s
464/1024 [============>.................] - ETA: 0s
516/1024 [==============>...............] - ETA: 0s
568/1024 [===============>..............] - ETA: 0s
620/1024 [=================>............] - ETA: 0s
672/1024 [==================>...........] - ETA: 0s
724/1024 [====================>.........] - ETA: 0s
776/1024 [=====================>........] - ETA: 0s
828/1024 [=======================>......] - ETA: 0s
879/1024 [========================>.....] - ETA: 0s
931/1024 [==========================>...] - ETA: 0s
982/1024 [===========================>..] - ETA: 0s
1024/1024 [==============================] - 1s 977us/step
model_quantized.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling_input (InputLaye [(None, 28, 28, 1)] 0
r)
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.9790999889373779
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.9842000007629395
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.9814000129699707
2.4. Model fine tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 1:48:25 - loss: 9.6093e-04 - accuracy: 1.0000
10/1688 [..............................] - ETA: 10s - loss: 0.0144 - accuracy: 0.9969
19/1688 [..............................] - ETA: 10s - loss: 0.0151 - accuracy: 0.9951
28/1688 [..............................] - ETA: 10s - loss: 0.0157 - accuracy: 0.9944
37/1688 [..............................] - ETA: 10s - loss: 0.0173 - accuracy: 0.9941
45/1688 [..............................] - ETA: 10s - loss: 0.0173 - accuracy: 0.9944
54/1688 [..............................] - ETA: 10s - loss: 0.0158 - accuracy: 0.9954
62/1688 [>.............................] - ETA: 10s - loss: 0.0162 - accuracy: 0.9950
71/1688 [>.............................] - ETA: 10s - loss: 0.0177 - accuracy: 0.9947
80/1688 [>.............................] - ETA: 10s - loss: 0.0176 - accuracy: 0.9941
89/1688 [>.............................] - ETA: 9s - loss: 0.0164 - accuracy: 0.9947
98/1688 [>.............................] - ETA: 9s - loss: 0.0156 - accuracy: 0.9949
106/1688 [>.............................] - ETA: 9s - loss: 0.0157 - accuracy: 0.9944
115/1688 [=>............................] - ETA: 9s - loss: 0.0154 - accuracy: 0.9946
124/1688 [=>............................] - ETA: 9s - loss: 0.0148 - accuracy: 0.9947
133/1688 [=>............................] - ETA: 9s - loss: 0.0156 - accuracy: 0.9944
142/1688 [=>............................] - ETA: 9s - loss: 0.0152 - accuracy: 0.9945
151/1688 [=>............................] - ETA: 9s - loss: 0.0152 - accuracy: 0.9942
160/1688 [=>............................] - ETA: 9s - loss: 0.0154 - accuracy: 0.9941
169/1688 [==>...........................] - ETA: 9s - loss: 0.0158 - accuracy: 0.9939
177/1688 [==>...........................] - ETA: 9s - loss: 0.0152 - accuracy: 0.9942
186/1688 [==>...........................] - ETA: 9s - loss: 0.0161 - accuracy: 0.9938
195/1688 [==>...........................] - ETA: 9s - loss: 0.0157 - accuracy: 0.9939
204/1688 [==>...........................] - ETA: 9s - loss: 0.0154 - accuracy: 0.9940
212/1688 [==>...........................] - ETA: 9s - loss: 0.0158 - accuracy: 0.9938
221/1688 [==>...........................] - ETA: 9s - loss: 0.0153 - accuracy: 0.9941
230/1688 [===>..........................] - ETA: 8s - loss: 0.0150 - accuracy: 0.9943
239/1688 [===>..........................] - ETA: 8s - loss: 0.0146 - accuracy: 0.9944
248/1688 [===>..........................] - ETA: 8s - loss: 0.0146 - accuracy: 0.9942
257/1688 [===>..........................] - ETA: 8s - loss: 0.0144 - accuracy: 0.9944
266/1688 [===>..........................] - ETA: 8s - loss: 0.0142 - accuracy: 0.9945
275/1688 [===>..........................] - ETA: 8s - loss: 0.0146 - accuracy: 0.9944
284/1688 [====>.........................] - ETA: 8s - loss: 0.0144 - accuracy: 0.9945
293/1688 [====>.........................] - ETA: 8s - loss: 0.0141 - accuracy: 0.9947
302/1688 [====>.........................] - ETA: 8s - loss: 0.0137 - accuracy: 0.9948
310/1688 [====>.........................] - ETA: 8s - loss: 0.0134 - accuracy: 0.9950
319/1688 [====>.........................] - ETA: 8s - loss: 0.0132 - accuracy: 0.9950
328/1688 [====>.........................] - ETA: 8s - loss: 0.0131 - accuracy: 0.9950
337/1688 [====>.........................] - ETA: 8s - loss: 0.0128 - accuracy: 0.9951
346/1688 [=====>........................] - ETA: 8s - loss: 0.0126 - accuracy: 0.9952
355/1688 [=====>........................] - ETA: 8s - loss: 0.0130 - accuracy: 0.9952
363/1688 [=====>........................] - ETA: 8s - loss: 0.0131 - accuracy: 0.9951
372/1688 [=====>........................] - ETA: 8s - loss: 0.0129 - accuracy: 0.9950
381/1688 [=====>........................] - ETA: 8s - loss: 0.0128 - accuracy: 0.9951
390/1688 [=====>........................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9952
399/1688 [======>.......................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9952
408/1688 [======>.......................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9953
417/1688 [======>.......................] - ETA: 7s - loss: 0.0125 - accuracy: 0.9953
425/1688 [======>.......................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9952
434/1688 [======>.......................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9952
443/1688 [======>.......................] - ETA: 7s - loss: 0.0129 - accuracy: 0.9951
452/1688 [=======>......................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9952
461/1688 [=======>......................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9953
470/1688 [=======>......................] - ETA: 7s - loss: 0.0127 - accuracy: 0.9953
479/1688 [=======>......................] - ETA: 7s - loss: 0.0125 - accuracy: 0.9954
487/1688 [=======>......................] - ETA: 7s - loss: 0.0123 - accuracy: 0.9954
496/1688 [=======>......................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9955
505/1688 [=======>......................] - ETA: 7s - loss: 0.0121 - accuracy: 0.9955
513/1688 [========>.....................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9956
522/1688 [========>.....................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9954
531/1688 [========>.....................] - ETA: 7s - loss: 0.0122 - accuracy: 0.9954
539/1688 [========>.....................] - ETA: 7s - loss: 0.0120 - accuracy: 0.9955
547/1688 [========>.....................] - ETA: 7s - loss: 0.0119 - accuracy: 0.9955
556/1688 [========>.....................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9956
565/1688 [=========>....................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9956
573/1688 [=========>....................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9955
582/1688 [=========>....................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9955
590/1688 [=========>....................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9954
599/1688 [=========>....................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9955
608/1688 [=========>....................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9954
617/1688 [=========>....................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9954
626/1688 [==========>...................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9955
635/1688 [==========>...................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9956
644/1688 [==========>...................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9956
653/1688 [==========>...................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9956
662/1688 [==========>...................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9956
671/1688 [==========>...................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9956
679/1688 [===========>..................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9957
688/1688 [===========>..................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9957
697/1688 [===========>..................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9957
705/1688 [===========>..................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9957
714/1688 [===========>..................] - ETA: 5s - loss: 0.0116 - accuracy: 0.9958
723/1688 [===========>..................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9958
732/1688 [============>.................] - ETA: 5s - loss: 0.0114 - accuracy: 0.9959
741/1688 [============>.................] - ETA: 5s - loss: 0.0115 - accuracy: 0.9959
749/1688 [============>.................] - ETA: 5s - loss: 0.0114 - accuracy: 0.9959
758/1688 [============>.................] - ETA: 5s - loss: 0.0113 - accuracy: 0.9959
767/1688 [============>.................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9960
776/1688 [============>.................] - ETA: 5s - loss: 0.0111 - accuracy: 0.9960
785/1688 [============>.................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9960
794/1688 [=============>................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9960
803/1688 [=============>................] - ETA: 5s - loss: 0.0112 - accuracy: 0.9960
812/1688 [=============>................] - ETA: 5s - loss: 0.0111 - accuracy: 0.9960
821/1688 [=============>................] - ETA: 5s - loss: 0.0111 - accuracy: 0.9960
829/1688 [=============>................] - ETA: 5s - loss: 0.0111 - accuracy: 0.9960
838/1688 [=============>................] - ETA: 5s - loss: 0.0111 - accuracy: 0.9960
846/1688 [==============>...............] - ETA: 5s - loss: 0.0112 - accuracy: 0.9960
855/1688 [==============>...............] - ETA: 5s - loss: 0.0112 - accuracy: 0.9960
864/1688 [==============>...............] - ETA: 5s - loss: 0.0112 - accuracy: 0.9960
873/1688 [==============>...............] - ETA: 5s - loss: 0.0111 - accuracy: 0.9960
882/1688 [==============>...............] - ETA: 4s - loss: 0.0110 - accuracy: 0.9961
891/1688 [==============>...............] - ETA: 4s - loss: 0.0110 - accuracy: 0.9960
899/1688 [==============>...............] - ETA: 4s - loss: 0.0110 - accuracy: 0.9960
907/1688 [===============>..............] - ETA: 4s - loss: 0.0110 - accuracy: 0.9961
916/1688 [===============>..............] - ETA: 4s - loss: 0.0109 - accuracy: 0.9961
925/1688 [===============>..............] - ETA: 4s - loss: 0.0109 - accuracy: 0.9961
934/1688 [===============>..............] - ETA: 4s - loss: 0.0108 - accuracy: 0.9962
942/1688 [===============>..............] - ETA: 4s - loss: 0.0108 - accuracy: 0.9962
950/1688 [===============>..............] - ETA: 4s - loss: 0.0107 - accuracy: 0.9962
959/1688 [================>.............] - ETA: 4s - loss: 0.0108 - accuracy: 0.9962
968/1688 [================>.............] - ETA: 4s - loss: 0.0108 - accuracy: 0.9962
976/1688 [================>.............] - ETA: 4s - loss: 0.0107 - accuracy: 0.9962
985/1688 [================>.............] - ETA: 4s - loss: 0.0107 - accuracy: 0.9963
994/1688 [================>.............] - ETA: 4s - loss: 0.0106 - accuracy: 0.9963
1003/1688 [================>.............] - ETA: 4s - loss: 0.0106 - accuracy: 0.9963
1012/1688 [================>.............] - ETA: 4s - loss: 0.0105 - accuracy: 0.9963
1021/1688 [=================>............] - ETA: 4s - loss: 0.0105 - accuracy: 0.9963
1030/1688 [=================>............] - ETA: 4s - loss: 0.0105 - accuracy: 0.9963
1039/1688 [=================>............] - ETA: 3s - loss: 0.0105 - accuracy: 0.9963
1047/1688 [=================>............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9963
1056/1688 [=================>............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9963
1065/1688 [=================>............] - ETA: 3s - loss: 0.0104 - accuracy: 0.9963
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0104 - accuracy: 0.9964
1082/1688 [==================>...........] - ETA: 3s - loss: 0.0103 - accuracy: 0.9964
1091/1688 [==================>...........] - ETA: 3s - loss: 0.0104 - accuracy: 0.9964
1099/1688 [==================>...........] - ETA: 3s - loss: 0.0104 - accuracy: 0.9964
1108/1688 [==================>...........] - ETA: 3s - loss: 0.0103 - accuracy: 0.9964
1116/1688 [==================>...........] - ETA: 3s - loss: 0.0103 - accuracy: 0.9964
1125/1688 [==================>...........] - ETA: 3s - loss: 0.0103 - accuracy: 0.9964
1134/1688 [===================>..........] - ETA: 3s - loss: 0.0103 - accuracy: 0.9964
1142/1688 [===================>..........] - ETA: 3s - loss: 0.0102 - accuracy: 0.9964
1151/1688 [===================>..........] - ETA: 3s - loss: 0.0102 - accuracy: 0.9964
1160/1688 [===================>..........] - ETA: 3s - loss: 0.0102 - accuracy: 0.9964
1168/1688 [===================>..........] - ETA: 3s - loss: 0.0102 - accuracy: 0.9964
1177/1688 [===================>..........] - ETA: 3s - loss: 0.0104 - accuracy: 0.9964
1186/1688 [====================>.........] - ETA: 3s - loss: 0.0104 - accuracy: 0.9964
1195/1688 [====================>.........] - ETA: 3s - loss: 0.0104 - accuracy: 0.9963
1204/1688 [====================>.........] - ETA: 2s - loss: 0.0104 - accuracy: 0.9964
1213/1688 [====================>.........] - ETA: 2s - loss: 0.0103 - accuracy: 0.9964
1221/1688 [====================>.........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9964
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9964
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9964
1247/1688 [=====================>........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9964
1255/1688 [=====================>........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9964
1264/1688 [=====================>........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9964
1273/1688 [=====================>........] - ETA: 2s - loss: 0.0103 - accuracy: 0.9964
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0103 - accuracy: 0.9964
1289/1688 [=====================>........] - ETA: 2s - loss: 0.0102 - accuracy: 0.9964
1297/1688 [======================>.......] - ETA: 2s - loss: 0.0101 - accuracy: 0.9964
1305/1688 [======================>.......] - ETA: 2s - loss: 0.0102 - accuracy: 0.9964
1313/1688 [======================>.......] - ETA: 2s - loss: 0.0102 - accuracy: 0.9964
1322/1688 [======================>.......] - ETA: 2s - loss: 0.0101 - accuracy: 0.9964
1331/1688 [======================>.......] - ETA: 2s - loss: 0.0101 - accuracy: 0.9965
1340/1688 [======================>.......] - ETA: 2s - loss: 0.0101 - accuracy: 0.9964
1349/1688 [======================>.......] - ETA: 2s - loss: 0.0101 - accuracy: 0.9964
1358/1688 [=======================>......] - ETA: 2s - loss: 0.0101 - accuracy: 0.9964
1367/1688 [=======================>......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9964
1375/1688 [=======================>......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9965
1384/1688 [=======================>......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9965
1392/1688 [=======================>......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9964
1401/1688 [=======================>......] - ETA: 1s - loss: 0.0100 - accuracy: 0.9965
1410/1688 [========================>.....] - ETA: 1s - loss: 0.0100 - accuracy: 0.9965
1419/1688 [========================>.....] - ETA: 1s - loss: 0.0101 - accuracy: 0.9964
1428/1688 [========================>.....] - ETA: 1s - loss: 0.0100 - accuracy: 0.9964
1437/1688 [========================>.....] - ETA: 1s - loss: 0.0101 - accuracy: 0.9964
1446/1688 [========================>.....] - ETA: 1s - loss: 0.0101 - accuracy: 0.9964
1455/1688 [========================>.....] - ETA: 1s - loss: 0.0100 - accuracy: 0.9964
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0100 - accuracy: 0.9964
1473/1688 [=========================>....] - ETA: 1s - loss: 0.0100 - accuracy: 0.9964
1482/1688 [=========================>....] - ETA: 1s - loss: 0.0099 - accuracy: 0.9964
1491/1688 [=========================>....] - ETA: 1s - loss: 0.0099 - accuracy: 0.9964
1500/1688 [=========================>....] - ETA: 1s - loss: 0.0099 - accuracy: 0.9964
1509/1688 [=========================>....] - ETA: 1s - loss: 0.0099 - accuracy: 0.9965
1518/1688 [=========================>....] - ETA: 1s - loss: 0.0099 - accuracy: 0.9964
1527/1688 [==========================>...] - ETA: 0s - loss: 0.0098 - accuracy: 0.9965
1536/1688 [==========================>...] - ETA: 0s - loss: 0.0099 - accuracy: 0.9965
1545/1688 [==========================>...] - ETA: 0s - loss: 0.0098 - accuracy: 0.9965
1553/1688 [==========================>...] - ETA: 0s - loss: 0.0098 - accuracy: 0.9965
1561/1688 [==========================>...] - ETA: 0s - loss: 0.0098 - accuracy: 0.9965
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0098 - accuracy: 0.9965
1579/1688 [===========================>..] - ETA: 0s - loss: 0.0097 - accuracy: 0.9965
1588/1688 [===========================>..] - ETA: 0s - loss: 0.0097 - accuracy: 0.9965
1597/1688 [===========================>..] - ETA: 0s - loss: 0.0097 - accuracy: 0.9965
1606/1688 [===========================>..] - ETA: 0s - loss: 0.0097 - accuracy: 0.9965
1615/1688 [===========================>..] - ETA: 0s - loss: 0.0096 - accuracy: 0.9966
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0096 - accuracy: 0.9966
1633/1688 [============================>.] - ETA: 0s - loss: 0.0097 - accuracy: 0.9965
1642/1688 [============================>.] - ETA: 0s - loss: 0.0097 - accuracy: 0.9965
1650/1688 [============================>.] - ETA: 0s - loss: 0.0097 - accuracy: 0.9966
1659/1688 [============================>.] - ETA: 0s - loss: 0.0098 - accuracy: 0.9965
1668/1688 [============================>.] - ETA: 0s - loss: 0.0097 - accuracy: 0.9965
1676/1688 [============================>.] - ETA: 0s - loss: 0.0097 - accuracy: 0.9965
1684/1688 [============================>.] - ETA: 0s - loss: 0.0098 - accuracy: 0.9965
1688/1688 [==============================] - 16s 7ms/step - loss: 0.0098 - accuracy: 0.9965 - val_loss: 0.0473 - val_accuracy: 0.9887
Epoch 2/5
1/1688 [..............................] - ETA: 10s - loss: 0.0015 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0020 - accuracy: 1.0000
17/1688 [..............................] - ETA: 10s - loss: 0.0055 - accuracy: 0.9982
26/1688 [..............................] - ETA: 10s - loss: 0.0044 - accuracy: 0.9988
35/1688 [..............................] - ETA: 10s - loss: 0.0040 - accuracy: 0.9991
44/1688 [..............................] - ETA: 10s - loss: 0.0037 - accuracy: 0.9993
53/1688 [..............................] - ETA: 10s - loss: 0.0035 - accuracy: 0.9994
61/1688 [>.............................] - ETA: 10s - loss: 0.0051 - accuracy: 0.9990
70/1688 [>.............................] - ETA: 10s - loss: 0.0051 - accuracy: 0.9987
79/1688 [>.............................] - ETA: 10s - loss: 0.0052 - accuracy: 0.9988
88/1688 [>.............................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9989
97/1688 [>.............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9984
106/1688 [>.............................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9985
115/1688 [=>............................] - ETA: 9s - loss: 0.0049 - accuracy: 0.9986
124/1688 [=>............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9985
133/1688 [=>............................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9986
141/1688 [=>............................] - ETA: 9s - loss: 0.0048 - accuracy: 0.9987
149/1688 [=>............................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9985
158/1688 [=>............................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9984
167/1688 [=>............................] - ETA: 9s - loss: 0.0051 - accuracy: 0.9985
176/1688 [==>...........................] - ETA: 9s - loss: 0.0050 - accuracy: 0.9986
185/1688 [==>...........................] - ETA: 9s - loss: 0.0056 - accuracy: 0.9985
194/1688 [==>...........................] - ETA: 9s - loss: 0.0055 - accuracy: 0.9986
203/1688 [==>...........................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9986
212/1688 [==>...........................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9985
220/1688 [==>...........................] - ETA: 9s - loss: 0.0052 - accuracy: 0.9986
228/1688 [===>..........................] - ETA: 9s - loss: 0.0053 - accuracy: 0.9985
237/1688 [===>..........................] - ETA: 8s - loss: 0.0055 - accuracy: 0.9983
246/1688 [===>..........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9982
255/1688 [===>..........................] - ETA: 8s - loss: 0.0055 - accuracy: 0.9983
264/1688 [===>..........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9983
273/1688 [===>..........................] - ETA: 8s - loss: 0.0055 - accuracy: 0.9983
282/1688 [====>.........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9983
291/1688 [====>.........................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9984
300/1688 [====>.........................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9983
308/1688 [====>.........................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9983
317/1688 [====>.........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9982
326/1688 [====>.........................] - ETA: 8s - loss: 0.0052 - accuracy: 0.9983
335/1688 [====>.........................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9982
344/1688 [=====>........................] - ETA: 8s - loss: 0.0055 - accuracy: 0.9982
353/1688 [=====>........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9982
362/1688 [=====>........................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9983
371/1688 [=====>........................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9982
380/1688 [=====>........................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9983
389/1688 [=====>........................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9982
398/1688 [======>.......................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9982
407/1688 [======>.......................] - ETA: 7s - loss: 0.0057 - accuracy: 0.9982
415/1688 [======>.......................] - ETA: 7s - loss: 0.0058 - accuracy: 0.9981
424/1688 [======>.......................] - ETA: 7s - loss: 0.0058 - accuracy: 0.9981
433/1688 [======>.......................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9981
441/1688 [======>.......................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9981
449/1688 [======>.......................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9981
457/1688 [=======>......................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9980
466/1688 [=======>......................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9981
475/1688 [=======>......................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9981
484/1688 [=======>......................] - ETA: 7s - loss: 0.0059 - accuracy: 0.9981
493/1688 [=======>......................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9981
501/1688 [=======>......................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9981
509/1688 [========>.....................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9981
518/1688 [========>.....................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9981
527/1688 [========>.....................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9980
535/1688 [========>.....................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9981
544/1688 [========>.....................] - ETA: 7s - loss: 0.0060 - accuracy: 0.9981
553/1688 [========>.....................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9981
562/1688 [========>.....................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9982
571/1688 [=========>....................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9981
580/1688 [=========>....................] - ETA: 6s - loss: 0.0059 - accuracy: 0.9982
589/1688 [=========>....................] - ETA: 6s - loss: 0.0061 - accuracy: 0.9981
598/1688 [=========>....................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9982
606/1688 [=========>....................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9981
614/1688 [=========>....................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9982
623/1688 [==========>...................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9982
632/1688 [==========>...................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9982
641/1688 [==========>...................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9982
650/1688 [==========>...................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9982
659/1688 [==========>...................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9982
668/1688 [==========>...................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9982
677/1688 [===========>..................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9982
685/1688 [===========>..................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9982
694/1688 [===========>..................] - ETA: 6s - loss: 0.0060 - accuracy: 0.9982
703/1688 [===========>..................] - ETA: 6s - loss: 0.0061 - accuracy: 0.9981
711/1688 [===========>..................] - ETA: 6s - loss: 0.0061 - accuracy: 0.9981
720/1688 [===========>..................] - ETA: 5s - loss: 0.0061 - accuracy: 0.9981
729/1688 [===========>..................] - ETA: 5s - loss: 0.0061 - accuracy: 0.9981
738/1688 [============>.................] - ETA: 5s - loss: 0.0061 - accuracy: 0.9981
747/1688 [============>.................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9980
756/1688 [============>.................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9981
765/1688 [============>.................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9981
773/1688 [============>.................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9980
782/1688 [============>.................] - ETA: 5s - loss: 0.0064 - accuracy: 0.9980
790/1688 [=============>................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9980
799/1688 [=============>................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9980
808/1688 [=============>................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9980
817/1688 [=============>................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9980
826/1688 [=============>................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9980
834/1688 [=============>................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9980
843/1688 [=============>................] - ETA: 5s - loss: 0.0063 - accuracy: 0.9980
852/1688 [==============>...............] - ETA: 5s - loss: 0.0062 - accuracy: 0.9980
861/1688 [==============>...............] - ETA: 5s - loss: 0.0063 - accuracy: 0.9980
870/1688 [==============>...............] - ETA: 5s - loss: 0.0062 - accuracy: 0.9980
879/1688 [==============>...............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9980
888/1688 [==============>...............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9980
897/1688 [==============>...............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9980
906/1688 [===============>..............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9980
915/1688 [===============>..............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9980
924/1688 [===============>..............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9980
933/1688 [===============>..............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9980
942/1688 [===============>..............] - ETA: 4s - loss: 0.0063 - accuracy: 0.9980
950/1688 [===============>..............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9980
959/1688 [================>.............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9980
968/1688 [================>.............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9981
976/1688 [================>.............] - ETA: 4s - loss: 0.0061 - accuracy: 0.9981
985/1688 [================>.............] - ETA: 4s - loss: 0.0061 - accuracy: 0.9981
994/1688 [================>.............] - ETA: 4s - loss: 0.0061 - accuracy: 0.9981
1003/1688 [================>.............] - ETA: 4s - loss: 0.0061 - accuracy: 0.9981
1012/1688 [================>.............] - ETA: 4s - loss: 0.0062 - accuracy: 0.9981
1020/1688 [=================>............] - ETA: 4s - loss: 0.0061 - accuracy: 0.9981
1029/1688 [=================>............] - ETA: 4s - loss: 0.0061 - accuracy: 0.9981
1038/1688 [=================>............] - ETA: 3s - loss: 0.0061 - accuracy: 0.9982
1047/1688 [=================>............] - ETA: 3s - loss: 0.0061 - accuracy: 0.9982
1056/1688 [=================>............] - ETA: 3s - loss: 0.0061 - accuracy: 0.9982
1065/1688 [=================>............] - ETA: 3s - loss: 0.0061 - accuracy: 0.9981
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9981
1083/1688 [==================>...........] - ETA: 3s - loss: 0.0061 - accuracy: 0.9981
1091/1688 [==================>...........] - ETA: 3s - loss: 0.0061 - accuracy: 0.9981
1100/1688 [==================>...........] - ETA: 3s - loss: 0.0061 - accuracy: 0.9981
1109/1688 [==================>...........] - ETA: 3s - loss: 0.0061 - accuracy: 0.9981
1117/1688 [==================>...........] - ETA: 3s - loss: 0.0061 - accuracy: 0.9981
1125/1688 [==================>...........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9980
1134/1688 [===================>..........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9980
1143/1688 [===================>..........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9980
1151/1688 [===================>..........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9980
1160/1688 [===================>..........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9980
1169/1688 [===================>..........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9980
1178/1688 [===================>..........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9980
1186/1688 [====================>.........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9980
1195/1688 [====================>.........] - ETA: 3s - loss: 0.0062 - accuracy: 0.9980
1204/1688 [====================>.........] - ETA: 2s - loss: 0.0062 - accuracy: 0.9980
1213/1688 [====================>.........] - ETA: 2s - loss: 0.0062 - accuracy: 0.9980
1221/1688 [====================>.........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1230/1688 [====================>.........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1239/1688 [=====================>........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1247/1688 [=====================>........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1256/1688 [=====================>........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1265/1688 [=====================>........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1274/1688 [=====================>........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1282/1688 [=====================>........] - ETA: 2s - loss: 0.0060 - accuracy: 0.9981
1291/1688 [=====================>........] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1300/1688 [======================>.......] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1309/1688 [======================>.......] - ETA: 2s - loss: 0.0061 - accuracy: 0.9980
1318/1688 [======================>.......] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1327/1688 [======================>.......] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1336/1688 [======================>.......] - ETA: 2s - loss: 0.0062 - accuracy: 0.9981
1345/1688 [======================>.......] - ETA: 2s - loss: 0.0062 - accuracy: 0.9981
1354/1688 [=======================>......] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1362/1688 [=======================>......] - ETA: 2s - loss: 0.0061 - accuracy: 0.9981
1371/1688 [=======================>......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1380/1688 [=======================>......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1389/1688 [=======================>......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1398/1688 [=======================>......] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1407/1688 [========================>.....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1416/1688 [========================>.....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1425/1688 [========================>.....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1434/1688 [========================>.....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1442/1688 [========================>.....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1451/1688 [========================>.....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1460/1688 [========================>.....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1469/1688 [=========================>....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1477/1688 [=========================>....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1486/1688 [=========================>....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1495/1688 [=========================>....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1504/1688 [=========================>....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9982
1513/1688 [=========================>....] - ETA: 1s - loss: 0.0061 - accuracy: 0.9981
1522/1688 [==========================>...] - ETA: 1s - loss: 0.0061 - accuracy: 0.9982
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0061 - accuracy: 0.9981
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0061 - accuracy: 0.9982
1548/1688 [==========================>...] - ETA: 0s - loss: 0.0062 - accuracy: 0.9981
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0061 - accuracy: 0.9981
1565/1688 [==========================>...] - ETA: 0s - loss: 0.0061 - accuracy: 0.9981
1574/1688 [==========================>...] - ETA: 0s - loss: 0.0062 - accuracy: 0.9981
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0061 - accuracy: 0.9981
1591/1688 [===========================>..] - ETA: 0s - loss: 0.0061 - accuracy: 0.9981
1599/1688 [===========================>..] - ETA: 0s - loss: 0.0062 - accuracy: 0.9981
1608/1688 [===========================>..] - ETA: 0s - loss: 0.0062 - accuracy: 0.9981
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0062 - accuracy: 0.9981
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0062 - accuracy: 0.9981
1635/1688 [============================>.] - ETA: 0s - loss: 0.0061 - accuracy: 0.9981
1643/1688 [============================>.] - ETA: 0s - loss: 0.0062 - accuracy: 0.9981
1652/1688 [============================>.] - ETA: 0s - loss: 0.0062 - accuracy: 0.9981
1661/1688 [============================>.] - ETA: 0s - loss: 0.0062 - accuracy: 0.9981
1670/1688 [============================>.] - ETA: 0s - loss: 0.0061 - accuracy: 0.9981
1679/1688 [============================>.] - ETA: 0s - loss: 0.0061 - accuracy: 0.9981
1688/1688 [==============================] - ETA: 0s - loss: 0.0062 - accuracy: 0.9981
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0062 - accuracy: 0.9981 - val_loss: 0.0450 - val_accuracy: 0.9888
Epoch 3/5
1/1688 [..............................] - ETA: 11s - loss: 0.0065 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0134 - accuracy: 0.9931
17/1688 [..............................] - ETA: 10s - loss: 0.0103 - accuracy: 0.9945
26/1688 [..............................] - ETA: 10s - loss: 0.0086 - accuracy: 0.9964
35/1688 [..............................] - ETA: 10s - loss: 0.0068 - accuracy: 0.9973
44/1688 [..............................] - ETA: 10s - loss: 0.0074 - accuracy: 0.9972
53/1688 [..............................] - ETA: 9s - loss: 0.0076 - accuracy: 0.9971
62/1688 [>.............................] - ETA: 9s - loss: 0.0067 - accuracy: 0.9975
71/1688 [>.............................] - ETA: 9s - loss: 0.0068 - accuracy: 0.9974
80/1688 [>.............................] - ETA: 9s - loss: 0.0061 - accuracy: 0.9977
89/1688 [>.............................] - ETA: 9s - loss: 0.0061 - accuracy: 0.9975
97/1688 [>.............................] - ETA: 9s - loss: 0.0060 - accuracy: 0.9977
106/1688 [>.............................] - ETA: 9s - loss: 0.0061 - accuracy: 0.9976
114/1688 [=>............................] - ETA: 9s - loss: 0.0058 - accuracy: 0.9978
123/1688 [=>............................] - ETA: 9s - loss: 0.0057 - accuracy: 0.9980
132/1688 [=>............................] - ETA: 9s - loss: 0.0063 - accuracy: 0.9979
141/1688 [=>............................] - ETA: 9s - loss: 0.0061 - accuracy: 0.9980
150/1688 [=>............................] - ETA: 9s - loss: 0.0059 - accuracy: 0.9981
158/1688 [=>............................] - ETA: 9s - loss: 0.0059 - accuracy: 0.9982
167/1688 [=>............................] - ETA: 9s - loss: 0.0057 - accuracy: 0.9983
175/1688 [==>...........................] - ETA: 9s - loss: 0.0055 - accuracy: 0.9984
183/1688 [==>...........................] - ETA: 9s - loss: 0.0055 - accuracy: 0.9985
192/1688 [==>...........................] - ETA: 9s - loss: 0.0059 - accuracy: 0.9984
201/1688 [==>...........................] - ETA: 9s - loss: 0.0059 - accuracy: 0.9984
209/1688 [==>...........................] - ETA: 9s - loss: 0.0058 - accuracy: 0.9985
218/1688 [==>...........................] - ETA: 9s - loss: 0.0059 - accuracy: 0.9984
227/1688 [===>..........................] - ETA: 8s - loss: 0.0059 - accuracy: 0.9983
236/1688 [===>..........................] - ETA: 8s - loss: 0.0058 - accuracy: 0.9984
245/1688 [===>..........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9985
253/1688 [===>..........................] - ETA: 8s - loss: 0.0058 - accuracy: 0.9984
262/1688 [===>..........................] - ETA: 8s - loss: 0.0057 - accuracy: 0.9984
271/1688 [===>..........................] - ETA: 8s - loss: 0.0056 - accuracy: 0.9985
280/1688 [===>..........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9985
289/1688 [====>.........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9986
298/1688 [====>.........................] - ETA: 8s - loss: 0.0055 - accuracy: 0.9985
307/1688 [====>.........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9986
316/1688 [====>.........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9986
325/1688 [====>.........................] - ETA: 8s - loss: 0.0055 - accuracy: 0.9986
334/1688 [====>.........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9986
343/1688 [=====>........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9986
352/1688 [=====>........................] - ETA: 8s - loss: 0.0054 - accuracy: 0.9987
361/1688 [=====>........................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9987
370/1688 [=====>........................] - ETA: 8s - loss: 0.0053 - accuracy: 0.9987
378/1688 [=====>........................] - ETA: 8s - loss: 0.0052 - accuracy: 0.9988
387/1688 [=====>........................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9988
395/1688 [======>.......................] - ETA: 7s - loss: 0.0051 - accuracy: 0.9988
403/1688 [======>.......................] - ETA: 7s - loss: 0.0050 - accuracy: 0.9988
411/1688 [======>.......................] - ETA: 7s - loss: 0.0056 - accuracy: 0.9987
420/1688 [======>.......................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9987
429/1688 [======>.......................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9988
438/1688 [======>.......................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9987
446/1688 [======>.......................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9987
454/1688 [=======>......................] - ETA: 7s - loss: 0.0055 - accuracy: 0.9988
463/1688 [=======>......................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9988
471/1688 [=======>......................] - ETA: 7s - loss: 0.0054 - accuracy: 0.9988
480/1688 [=======>......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9988
488/1688 [=======>......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9988
497/1688 [=======>......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9988
506/1688 [=======>......................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9988
515/1688 [========>.....................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9988
523/1688 [========>.....................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9988
532/1688 [========>.....................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9988
541/1688 [========>.....................] - ETA: 7s - loss: 0.0052 - accuracy: 0.9988
550/1688 [========>.....................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9989
558/1688 [========>.....................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9989
567/1688 [=========>....................] - ETA: 6s - loss: 0.0051 - accuracy: 0.9989
576/1688 [=========>....................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
585/1688 [=========>....................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
594/1688 [=========>....................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
603/1688 [=========>....................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9990
611/1688 [=========>....................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
620/1688 [==========>...................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
628/1688 [==========>...................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9990
636/1688 [==========>...................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9989
644/1688 [==========>...................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9989
653/1688 [==========>...................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9989
661/1688 [==========>...................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9990
669/1688 [==========>...................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9989
678/1688 [===========>..................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9989
687/1688 [===========>..................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9990
696/1688 [===========>..................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9990
705/1688 [===========>..................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9989
713/1688 [===========>..................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
722/1688 [===========>..................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9990
730/1688 [===========>..................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9990
738/1688 [============>.................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
747/1688 [============>.................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
756/1688 [============>.................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
765/1688 [============>.................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
773/1688 [============>.................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
782/1688 [============>.................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
790/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
799/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
808/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
816/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
825/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
834/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9988
843/1688 [=============>................] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
852/1688 [==============>...............] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
861/1688 [==============>...............] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
870/1688 [==============>...............] - ETA: 5s - loss: 0.0049 - accuracy: 0.9989
878/1688 [==============>...............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9989
887/1688 [==============>...............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9989
896/1688 [==============>...............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9989
905/1688 [===============>..............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
913/1688 [===============>..............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
921/1688 [===============>..............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
930/1688 [===============>..............] - ETA: 4s - loss: 0.0051 - accuracy: 0.9987
939/1688 [===============>..............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9987
948/1688 [===============>..............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9987
956/1688 [===============>..............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
964/1688 [================>.............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
972/1688 [================>.............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
981/1688 [================>.............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
990/1688 [================>.............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
999/1688 [================>.............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9988
1008/1688 [================>.............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9988
1017/1688 [=================>............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9988
1025/1688 [=================>............] - ETA: 4s - loss: 0.0049 - accuracy: 0.9988
1034/1688 [=================>............] - ETA: 4s - loss: 0.0050 - accuracy: 0.9988
1043/1688 [=================>............] - ETA: 3s - loss: 0.0050 - accuracy: 0.9988
1052/1688 [=================>............] - ETA: 3s - loss: 0.0050 - accuracy: 0.9988
1060/1688 [=================>............] - ETA: 3s - loss: 0.0050 - accuracy: 0.9988
1068/1688 [=================>............] - ETA: 3s - loss: 0.0050 - accuracy: 0.9988
1077/1688 [==================>...........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1086/1688 [==================>...........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1095/1688 [==================>...........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1104/1688 [==================>...........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1113/1688 [==================>...........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1122/1688 [==================>...........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1131/1688 [===================>..........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1140/1688 [===================>..........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1148/1688 [===================>..........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1157/1688 [===================>..........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1165/1688 [===================>..........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1174/1688 [===================>..........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1183/1688 [====================>.........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1192/1688 [====================>.........] - ETA: 3s - loss: 0.0049 - accuracy: 0.9988
1201/1688 [====================>.........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1210/1688 [====================>.........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1219/1688 [====================>.........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1237/1688 [====================>.........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1245/1688 [=====================>........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1254/1688 [=====================>........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1263/1688 [=====================>........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1272/1688 [=====================>........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1280/1688 [=====================>........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1289/1688 [=====================>........] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1298/1688 [======================>.......] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1307/1688 [======================>.......] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1315/1688 [======================>.......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9987
1324/1688 [======================>.......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9987
1333/1688 [======================>.......] - ETA: 2s - loss: 0.0049 - accuracy: 0.9988
1341/1688 [======================>.......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9988
1349/1688 [======================>.......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9988
1357/1688 [=======================>......] - ETA: 2s - loss: 0.0050 - accuracy: 0.9988
1365/1688 [=======================>......] - ETA: 1s - loss: 0.0049 - accuracy: 0.9988
1374/1688 [=======================>......] - ETA: 1s - loss: 0.0049 - accuracy: 0.9988
1383/1688 [=======================>......] - ETA: 1s - loss: 0.0049 - accuracy: 0.9988
1392/1688 [=======================>......] - ETA: 1s - loss: 0.0049 - accuracy: 0.9988
1401/1688 [=======================>......] - ETA: 1s - loss: 0.0049 - accuracy: 0.9988
1410/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9988
1419/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1428/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1436/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1445/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1453/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1462/1688 [========================>.....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1470/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1478/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1487/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1496/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1505/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1513/1688 [=========================>....] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1522/1688 [==========================>...] - ETA: 1s - loss: 0.0050 - accuracy: 0.9987
1531/1688 [==========================>...] - ETA: 0s - loss: 0.0050 - accuracy: 0.9987
1540/1688 [==========================>...] - ETA: 0s - loss: 0.0050 - accuracy: 0.9987
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0050 - accuracy: 0.9987
1558/1688 [==========================>...] - ETA: 0s - loss: 0.0050 - accuracy: 0.9987
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0050 - accuracy: 0.9987
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0049 - accuracy: 0.9987
1584/1688 [===========================>..] - ETA: 0s - loss: 0.0049 - accuracy: 0.9987
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0049 - accuracy: 0.9987
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0049 - accuracy: 0.9987
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0049 - accuracy: 0.9987
1619/1688 [===========================>..] - ETA: 0s - loss: 0.0049 - accuracy: 0.9987
1628/1688 [===========================>..] - ETA: 0s - loss: 0.0050 - accuracy: 0.9987
1636/1688 [============================>.] - ETA: 0s - loss: 0.0050 - accuracy: 0.9986
1645/1688 [============================>.] - ETA: 0s - loss: 0.0051 - accuracy: 0.9986
1653/1688 [============================>.] - ETA: 0s - loss: 0.0051 - accuracy: 0.9986
1661/1688 [============================>.] - ETA: 0s - loss: 0.0051 - accuracy: 0.9986
1669/1688 [============================>.] - ETA: 0s - loss: 0.0051 - accuracy: 0.9986
1677/1688 [============================>.] - ETA: 0s - loss: 0.0051 - accuracy: 0.9986
1686/1688 [============================>.] - ETA: 0s - loss: 0.0051 - accuracy: 0.9986
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0051 - accuracy: 0.9986 - val_loss: 0.0454 - val_accuracy: 0.9880
Epoch 4/5
1/1688 [..............................] - ETA: 11s - loss: 0.0016 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0035 - accuracy: 1.0000
18/1688 [..............................] - ETA: 10s - loss: 0.0043 - accuracy: 1.0000
27/1688 [..............................] - ETA: 10s - loss: 0.0050 - accuracy: 0.9988
36/1688 [..............................] - ETA: 10s - loss: 0.0041 - accuracy: 0.9991
45/1688 [..............................] - ETA: 10s - loss: 0.0042 - accuracy: 0.9986
53/1688 [..............................] - ETA: 10s - loss: 0.0044 - accuracy: 0.9988
62/1688 [>.............................] - ETA: 9s - loss: 0.0042 - accuracy: 0.9990
71/1688 [>.............................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9991
79/1688 [>.............................] - ETA: 9s - loss: 0.0036 - accuracy: 0.9992
88/1688 [>.............................] - ETA: 9s - loss: 0.0035 - accuracy: 0.9993
97/1688 [>.............................] - ETA: 9s - loss: 0.0035 - accuracy: 0.9994
106/1688 [>.............................] - ETA: 9s - loss: 0.0033 - accuracy: 0.9994
115/1688 [=>............................] - ETA: 9s - loss: 0.0035 - accuracy: 0.9995
124/1688 [=>............................] - ETA: 9s - loss: 0.0041 - accuracy: 0.9992
133/1688 [=>............................] - ETA: 9s - loss: 0.0040 - accuracy: 0.9993
142/1688 [=>............................] - ETA: 9s - loss: 0.0039 - accuracy: 0.9993
151/1688 [=>............................] - ETA: 9s - loss: 0.0037 - accuracy: 0.9994
160/1688 [=>............................] - ETA: 9s - loss: 0.0037 - accuracy: 0.9994
168/1688 [=>............................] - ETA: 9s - loss: 0.0037 - accuracy: 0.9994
177/1688 [==>...........................] - ETA: 9s - loss: 0.0036 - accuracy: 0.9995
186/1688 [==>...........................] - ETA: 9s - loss: 0.0035 - accuracy: 0.9995
194/1688 [==>...........................] - ETA: 9s - loss: 0.0034 - accuracy: 0.9995
203/1688 [==>...........................] - ETA: 9s - loss: 0.0035 - accuracy: 0.9995
212/1688 [==>...........................] - ETA: 9s - loss: 0.0034 - accuracy: 0.9996
221/1688 [==>...........................] - ETA: 9s - loss: 0.0034 - accuracy: 0.9996
230/1688 [===>..........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9996
239/1688 [===>..........................] - ETA: 8s - loss: 0.0033 - accuracy: 0.9996
248/1688 [===>..........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9996
257/1688 [===>..........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9996
265/1688 [===>..........................] - ETA: 8s - loss: 0.0035 - accuracy: 0.9995
274/1688 [===>..........................] - ETA: 8s - loss: 0.0035 - accuracy: 0.9995
283/1688 [====>.........................] - ETA: 8s - loss: 0.0035 - accuracy: 0.9996
291/1688 [====>.........................] - ETA: 8s - loss: 0.0035 - accuracy: 0.9996
300/1688 [====>.........................] - ETA: 8s - loss: 0.0035 - accuracy: 0.9996
308/1688 [====>.........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9996
317/1688 [====>.........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9996
326/1688 [====>.........................] - ETA: 8s - loss: 0.0033 - accuracy: 0.9996
335/1688 [====>.........................] - ETA: 8s - loss: 0.0033 - accuracy: 0.9996
344/1688 [=====>........................] - ETA: 8s - loss: 0.0033 - accuracy: 0.9996
353/1688 [=====>........................] - ETA: 8s - loss: 0.0033 - accuracy: 0.9996
361/1688 [=====>........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9995
369/1688 [=====>........................] - ETA: 8s - loss: 0.0033 - accuracy: 0.9995
378/1688 [=====>........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9994
387/1688 [=====>........................] - ETA: 8s - loss: 0.0035 - accuracy: 0.9994
396/1688 [======>.......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9993
405/1688 [======>.......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9993
413/1688 [======>.......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9992
422/1688 [======>.......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9993
431/1688 [======>.......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9993
439/1688 [======>.......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9993
448/1688 [======>.......................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9993
457/1688 [=======>......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9992
466/1688 [=======>......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9993
474/1688 [=======>......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9993
483/1688 [=======>......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9993
492/1688 [=======>......................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9993
500/1688 [=======>......................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9992
509/1688 [========>.....................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9993
517/1688 [========>.....................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9993
526/1688 [========>.....................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9993
535/1688 [========>.....................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9993
544/1688 [========>.....................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9993
553/1688 [========>.....................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9993
562/1688 [========>.....................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
571/1688 [=========>....................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
580/1688 [=========>....................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
589/1688 [=========>....................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9993
597/1688 [=========>....................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9993
606/1688 [=========>....................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9992
615/1688 [=========>....................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9992
623/1688 [==========>...................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
632/1688 [==========>...................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9993
641/1688 [==========>...................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9993
649/1688 [==========>...................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9992
657/1688 [==========>...................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9992
666/1688 [==========>...................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9992
675/1688 [==========>...................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
684/1688 [===========>..................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
693/1688 [===========>..................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
701/1688 [===========>..................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
710/1688 [===========>..................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9993
719/1688 [===========>..................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
728/1688 [===========>..................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
737/1688 [============>.................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
746/1688 [============>.................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
755/1688 [============>.................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
763/1688 [============>.................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
772/1688 [============>.................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
780/1688 [============>.................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
789/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
798/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
807/1688 [=============>................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9993
816/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
825/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
834/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
843/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
852/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
861/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
870/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9993
879/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9993
887/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9993
896/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9993
905/1688 [===============>..............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
914/1688 [===============>..............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
923/1688 [===============>..............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9993
932/1688 [===============>..............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
941/1688 [===============>..............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
950/1688 [===============>..............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
959/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
968/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
977/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
985/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
994/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9992
1003/1688 [================>.............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9992
1011/1688 [================>.............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9991
1020/1688 [=================>............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9991
1028/1688 [=================>............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9991
1037/1688 [=================>............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9991
1045/1688 [=================>............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9991
1053/1688 [=================>............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1062/1688 [=================>............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1071/1688 [==================>...........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1080/1688 [==================>...........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1089/1688 [==================>...........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1097/1688 [==================>...........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1105/1688 [==================>...........] - ETA: 3s - loss: 0.0041 - accuracy: 0.9991
1114/1688 [==================>...........] - ETA: 3s - loss: 0.0041 - accuracy: 0.9991
1123/1688 [==================>...........] - ETA: 3s - loss: 0.0041 - accuracy: 0.9991
1131/1688 [===================>..........] - ETA: 3s - loss: 0.0041 - accuracy: 0.9991
1139/1688 [===================>..........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1148/1688 [===================>..........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1157/1688 [===================>..........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1166/1688 [===================>..........] - ETA: 3s - loss: 0.0041 - accuracy: 0.9991
1175/1688 [===================>..........] - ETA: 3s - loss: 0.0041 - accuracy: 0.9991
1183/1688 [====================>.........] - ETA: 3s - loss: 0.0041 - accuracy: 0.9992
1192/1688 [====================>.........] - ETA: 3s - loss: 0.0041 - accuracy: 0.9992
1201/1688 [====================>.........] - ETA: 3s - loss: 0.0041 - accuracy: 0.9992
1210/1688 [====================>.........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1219/1688 [====================>.........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9992
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1237/1688 [====================>.........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1246/1688 [=====================>........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1254/1688 [=====================>........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1262/1688 [=====================>........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1271/1688 [=====================>........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1279/1688 [=====================>........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1288/1688 [=====================>........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1297/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1306/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1315/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1324/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1332/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1341/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9991
1349/1688 [======================>.......] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1358/1688 [=======================>......] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1366/1688 [=======================>......] - ETA: 1s - loss: 0.0040 - accuracy: 0.9991
1374/1688 [=======================>......] - ETA: 1s - loss: 0.0040 - accuracy: 0.9991
1383/1688 [=======================>......] - ETA: 1s - loss: 0.0041 - accuracy: 0.9991
1392/1688 [=======================>......] - ETA: 1s - loss: 0.0040 - accuracy: 0.9991
1400/1688 [=======================>......] - ETA: 1s - loss: 0.0040 - accuracy: 0.9991
1408/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9991
1417/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9991
1425/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9991
1434/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9991
1443/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9991
1452/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9991
1461/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9991
1470/1688 [=========================>....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9991
1479/1688 [=========================>....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9991
1488/1688 [=========================>....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9991
1497/1688 [=========================>....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9991
1506/1688 [=========================>....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9991
1514/1688 [=========================>....] - ETA: 1s - loss: 0.0040 - accuracy: 0.9991
1522/1688 [==========================>...] - ETA: 1s - loss: 0.0040 - accuracy: 0.9991
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0040 - accuracy: 0.9991
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0040 - accuracy: 0.9991
1547/1688 [==========================>...] - ETA: 0s - loss: 0.0040 - accuracy: 0.9991
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0040 - accuracy: 0.9991
1564/1688 [==========================>...] - ETA: 0s - loss: 0.0040 - accuracy: 0.9991
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0040 - accuracy: 0.9991
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0040 - accuracy: 0.9991
1591/1688 [===========================>..] - ETA: 0s - loss: 0.0040 - accuracy: 0.9991
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0040 - accuracy: 0.9991
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0042 - accuracy: 0.9991
1618/1688 [===========================>..] - ETA: 0s - loss: 0.0042 - accuracy: 0.9991
1627/1688 [===========================>..] - ETA: 0s - loss: 0.0041 - accuracy: 0.9991
1635/1688 [============================>.] - ETA: 0s - loss: 0.0041 - accuracy: 0.9991
1644/1688 [============================>.] - ETA: 0s - loss: 0.0041 - accuracy: 0.9991
1652/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9991
1661/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9991
1669/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9991
1678/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9991
1687/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9991
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0042 - accuracy: 0.9991 - val_loss: 0.0478 - val_accuracy: 0.9885
Epoch 5/5
1/1688 [..............................] - ETA: 10s - loss: 4.6126e-04 - accuracy: 1.0000
9/1688 [..............................] - ETA: 10s - loss: 0.0043 - accuracy: 1.0000
17/1688 [..............................] - ETA: 10s - loss: 0.0053 - accuracy: 0.9982
26/1688 [..............................] - ETA: 10s - loss: 0.0043 - accuracy: 0.9988
35/1688 [..............................] - ETA: 10s - loss: 0.0039 - accuracy: 0.9991
44/1688 [..............................] - ETA: 10s - loss: 0.0041 - accuracy: 0.9993
52/1688 [..............................] - ETA: 10s - loss: 0.0037 - accuracy: 0.9994
61/1688 [>.............................] - ETA: 10s - loss: 0.0034 - accuracy: 0.9995
69/1688 [>.............................] - ETA: 10s - loss: 0.0032 - accuracy: 0.9995
77/1688 [>.............................] - ETA: 10s - loss: 0.0030 - accuracy: 0.9996
85/1688 [>.............................] - ETA: 10s - loss: 0.0032 - accuracy: 0.9996
94/1688 [>.............................] - ETA: 9s - loss: 0.0032 - accuracy: 0.9997
103/1688 [>.............................] - ETA: 9s - loss: 0.0031 - accuracy: 0.9997
112/1688 [>.............................] - ETA: 9s - loss: 0.0031 - accuracy: 0.9994
121/1688 [=>............................] - ETA: 9s - loss: 0.0030 - accuracy: 0.9995
130/1688 [=>............................] - ETA: 9s - loss: 0.0030 - accuracy: 0.9995
139/1688 [=>............................] - ETA: 9s - loss: 0.0030 - accuracy: 0.9996
148/1688 [=>............................] - ETA: 9s - loss: 0.0030 - accuracy: 0.9996
157/1688 [=>............................] - ETA: 9s - loss: 0.0030 - accuracy: 0.9996
166/1688 [=>............................] - ETA: 9s - loss: 0.0030 - accuracy: 0.9996
175/1688 [==>...........................] - ETA: 9s - loss: 0.0030 - accuracy: 0.9996
183/1688 [==>...........................] - ETA: 9s - loss: 0.0029 - accuracy: 0.9997
192/1688 [==>...........................] - ETA: 9s - loss: 0.0028 - accuracy: 0.9997
201/1688 [==>...........................] - ETA: 9s - loss: 0.0030 - accuracy: 0.9995
209/1688 [==>...........................] - ETA: 9s - loss: 0.0031 - accuracy: 0.9994
217/1688 [==>...........................] - ETA: 9s - loss: 0.0031 - accuracy: 0.9994
226/1688 [===>..........................] - ETA: 9s - loss: 0.0032 - accuracy: 0.9993
235/1688 [===>..........................] - ETA: 8s - loss: 0.0031 - accuracy: 0.9993
244/1688 [===>..........................] - ETA: 8s - loss: 0.0032 - accuracy: 0.9994
253/1688 [===>..........................] - ETA: 8s - loss: 0.0032 - accuracy: 0.9994
261/1688 [===>..........................] - ETA: 8s - loss: 0.0032 - accuracy: 0.9994
269/1688 [===>..........................] - ETA: 8s - loss: 0.0031 - accuracy: 0.9994
278/1688 [===>..........................] - ETA: 8s - loss: 0.0032 - accuracy: 0.9993
286/1688 [====>.........................] - ETA: 8s - loss: 0.0032 - accuracy: 0.9993
295/1688 [====>.........................] - ETA: 8s - loss: 0.0031 - accuracy: 0.9994
304/1688 [====>.........................] - ETA: 8s - loss: 0.0031 - accuracy: 0.9994
313/1688 [====>.........................] - ETA: 8s - loss: 0.0030 - accuracy: 0.9994
322/1688 [====>.........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9992
330/1688 [====>.........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9992
339/1688 [=====>........................] - ETA: 8s - loss: 0.0033 - accuracy: 0.9993
347/1688 [=====>........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9992
356/1688 [=====>........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9992
365/1688 [=====>........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9992
374/1688 [=====>........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9992
383/1688 [=====>........................] - ETA: 8s - loss: 0.0034 - accuracy: 0.9993
391/1688 [=====>........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9993
400/1688 [======>.......................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9993
409/1688 [======>.......................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9993
418/1688 [======>.......................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9993
426/1688 [======>.......................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9993
435/1688 [======>.......................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9993
443/1688 [======>.......................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9993
452/1688 [=======>......................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9992
461/1688 [=======>......................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9992
470/1688 [=======>......................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9992
479/1688 [=======>......................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9992
488/1688 [=======>......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9991
497/1688 [=======>......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9991
505/1688 [=======>......................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9991
514/1688 [========>.....................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9991
523/1688 [========>.....................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9992
532/1688 [========>.....................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9991
540/1688 [========>.....................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9991
549/1688 [========>.....................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9991
558/1688 [========>.....................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9991
566/1688 [=========>....................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9991
574/1688 [=========>....................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9991
583/1688 [=========>....................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9990
592/1688 [=========>....................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9990
601/1688 [=========>....................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9991
610/1688 [=========>....................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9991
619/1688 [==========>...................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9991
627/1688 [==========>...................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9991
636/1688 [==========>...................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9991
644/1688 [==========>...................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9990
653/1688 [==========>...................] - ETA: 6s - loss: 0.0039 - accuracy: 0.9990
662/1688 [==========>...................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9991
670/1688 [==========>...................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9991
679/1688 [===========>..................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9991
687/1688 [===========>..................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9991
696/1688 [===========>..................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9990
705/1688 [===========>..................] - ETA: 6s - loss: 0.0040 - accuracy: 0.9990
714/1688 [===========>..................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9990
722/1688 [===========>..................] - ETA: 5s - loss: 0.0041 - accuracy: 0.9990
730/1688 [===========>..................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9990
739/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9990
747/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9990
756/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9990
765/1688 [============>.................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
774/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9990
783/1688 [============>.................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9990
792/1688 [=============>................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
801/1688 [=============>................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
810/1688 [=============>................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
819/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
828/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
836/1688 [=============>................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
845/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
854/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
862/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
870/1688 [==============>...............] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
878/1688 [==============>...............] - ETA: 4s - loss: 0.0039 - accuracy: 0.9991
887/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
895/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
903/1688 [===============>..............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
912/1688 [===============>..............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
921/1688 [===============>..............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
930/1688 [===============>..............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
938/1688 [===============>..............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
947/1688 [===============>..............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
956/1688 [===============>..............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
964/1688 [================>.............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
973/1688 [================>.............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9990
982/1688 [================>.............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
990/1688 [================>.............] - ETA: 4s - loss: 0.0042 - accuracy: 0.9989
999/1688 [================>.............] - ETA: 4s - loss: 0.0044 - accuracy: 0.9989
1008/1688 [================>.............] - ETA: 4s - loss: 0.0044 - accuracy: 0.9989
1017/1688 [=================>............] - ETA: 4s - loss: 0.0044 - accuracy: 0.9989
1026/1688 [=================>............] - ETA: 4s - loss: 0.0043 - accuracy: 0.9989
1035/1688 [=================>............] - ETA: 4s - loss: 0.0043 - accuracy: 0.9989
1043/1688 [=================>............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9989
1052/1688 [=================>............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9989
1061/1688 [=================>............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9989
1070/1688 [==================>...........] - ETA: 3s - loss: 0.0043 - accuracy: 0.9989
1079/1688 [==================>...........] - ETA: 3s - loss: 0.0043 - accuracy: 0.9989
1087/1688 [==================>...........] - ETA: 3s - loss: 0.0043 - accuracy: 0.9989
1095/1688 [==================>...........] - ETA: 3s - loss: 0.0043 - accuracy: 0.9989
1104/1688 [==================>...........] - ETA: 3s - loss: 0.0043 - accuracy: 0.9990
1112/1688 [==================>...........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9990
1121/1688 [==================>...........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9990
1130/1688 [===================>..........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9990
1139/1688 [===================>..........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9990
1148/1688 [===================>..........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9990
1157/1688 [===================>..........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9990
1166/1688 [===================>..........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9990
1175/1688 [===================>..........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9990
1184/1688 [====================>.........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9990
1193/1688 [====================>.........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9990
1202/1688 [====================>.........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9990
1211/1688 [====================>.........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9990
1220/1688 [====================>.........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9990
1228/1688 [====================>.........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9990
1237/1688 [====================>.........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9990
1246/1688 [=====================>........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9990
1254/1688 [=====================>........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9990
1262/1688 [=====================>........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9990
1270/1688 [=====================>........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9990
1279/1688 [=====================>........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9990
1287/1688 [=====================>........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9990
1296/1688 [======================>.......] - ETA: 2s - loss: 0.0042 - accuracy: 0.9990
1305/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9990
1314/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9990
1323/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9990
1332/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9990
1341/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9990
1349/1688 [======================>.......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9990
1358/1688 [=======================>......] - ETA: 2s - loss: 0.0041 - accuracy: 0.9990
1367/1688 [=======================>......] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1376/1688 [=======================>......] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1385/1688 [=======================>......] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1394/1688 [=======================>......] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1402/1688 [=======================>......] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1411/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1420/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1429/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1438/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1446/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1455/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1464/1688 [=========================>....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1472/1688 [=========================>....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1481/1688 [=========================>....] - ETA: 1s - loss: 0.0042 - accuracy: 0.9990
1490/1688 [=========================>....] - ETA: 1s - loss: 0.0042 - accuracy: 0.9990
1499/1688 [=========================>....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1508/1688 [=========================>....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1516/1688 [=========================>....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1525/1688 [==========================>...] - ETA: 1s - loss: 0.0042 - accuracy: 0.9990
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0042 - accuracy: 0.9990
1560/1688 [==========================>...] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1569/1688 [==========================>...] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1578/1688 [===========================>..] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1595/1688 [===========================>..] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1613/1688 [===========================>..] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1631/1688 [===========================>..] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1639/1688 [============================>.] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1648/1688 [============================>.] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1657/1688 [============================>.] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1666/1688 [============================>.] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1675/1688 [============================>.] - ETA: 0s - loss: 0.0041 - accuracy: 0.9990
1684/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9990
1688/1688 [==============================] - 11s 7ms/step - loss: 0.0041 - accuracy: 0.9990 - val_loss: 0.0469 - val_accuracy: 0.9883
<tf_keras.src.callbacks.History object at 0x7145955a6390>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine tuning:', score)
Test accuracy after fine tuning: 0.9883000254631042
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with TF-Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test.astype(np.int32))
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.9865000247955322
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[-17.85389 -10.923257 -7.0271673 -0.06355193 -14.291705
-11.592661 -27.31079 12.217054 -8.211378 -1.4405913 ]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 15.062 seconds)