Note
Go to the end to download the full example code.
Global Akida workflow
Using the MNIST dataset, this example shows the definition and training of a keras floating point model, its quantization to 8-bit with the help of calibration, its quantization to 4-bit using QAT and its conversion to Akida. Notice that the performance of the original keras floating point model is maintained throughout the Akida flow. Please refer to the Akida user guide for further information.
Note
Please refer to the TensorFlow tf.keras.models module for model creation/import details and the TensorFlow Guide for TensorFlow usage.
The MNIST example below is light enough so that a GPU is not needed for training.
Global Akida workflow
1. Create and train
1.1. Load and reshape MNIST dataset
import numpy as np
import matplotlib.cm as cm
import matplotlib.pyplot as plt
from keras.datasets import mnist
# Load MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Add a channels dimension to the image sets as Akida expects 4-D inputs (corresponding to
# (num_samples, width, height, channels). Note: MNIST is a grayscale dataset and is unusual
# in this respect - most image data already includes a channel dimension, and this step will
# not be necessary.
x_train = np.expand_dims(x_train, -1)
x_test = np.expand_dims(x_test, -1)
# Display a few images from the test set
f, axarr = plt.subplots(1, 4)
for i in range(0, 4):
axarr[i].imshow(x_test[i].reshape((28, 28)), cmap=cm.Greys_r)
axarr[i].set_title('Class %d' % y_test[i])
plt.show()
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
8192/11490434 [..............................] - ETA: 0s
16384/11490434 [..............................] - ETA: 1:37
32768/11490434 [..............................] - ETA: 1:09
49152/11490434 [..............................] - ETA: 1:00
65536/11490434 [..............................] - ETA: 56s
90112/11490434 [..............................] - ETA: 49s
114688/11490434 [..............................] - ETA: 44s
155648/11490434 [..............................] - ETA: 36s
196608/11490434 [..............................] - ETA: 32s
245760/11490434 [..............................] - ETA: 28s
311296/11490434 [..............................] - ETA: 23s
393216/11490434 [>.............................] - ETA: 20s
499712/11490434 [>.............................] - ETA: 17s
647168/11490434 [>.............................] - ETA: 13s
811008/11490434 [=>............................] - ETA: 11s
1015808/11490434 [=>............................] - ETA: 9s
1269760/11490434 [==>...........................] - ETA: 7s
1589248/11490434 [===>..........................] - ETA: 6s
1998848/11490434 [====>.........................] - ETA: 5s
2506752/11490434 [=====>........................] - ETA: 4s
3178496/11490434 [=======>......................] - ETA: 3s
3997696/11490434 [=========>....................] - ETA: 2s
4980736/11490434 [============>.................] - ETA: 1s
6111232/11490434 [==============>...............] - ETA: 1s
7127040/11490434 [=================>............] - ETA: 0s
8077312/11490434 [====================>.........] - ETA: 0s
9052160/11490434 [======================>.......] - ETA: 0s
10231808/11490434 [=========================>....] - ETA: 0s
11190272/11490434 [============================>.] - ETA: 0s
11490434/11490434 [==============================] - 2s 0us/step
1.2. Model definition
Note that at this stage, there is nothing specific to the Akida IP. The model constructed below, as inspired by this example, is a completely standard Keras CNN model.
import keras
model_keras = keras.models.Sequential([
keras.layers.Rescaling(1. / 255, input_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=32, kernel_size=3, strides=2),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
# Separable layer
keras.layers.DepthwiseConv2D(kernel_size=3, padding='same', strides=2),
keras.layers.Conv2D(filters=64, kernel_size=1, padding='same'),
keras.layers.BatchNormalization(),
keras.layers.ReLU(),
keras.layers.Flatten(),
keras.layers.Dense(10)
], 'mnistnet')
model_keras.summary()
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling (Rescaling) (None, 28, 28, 1) 0
conv2d (Conv2D) (None, 13, 13, 32) 320
batch_normalization (Batch (None, 13, 13, 32) 128
Normalization)
re_lu (ReLU) (None, 13, 13, 32) 0
depthwise_conv2d (Depthwis (None, 7, 7, 32) 320
eConv2D)
conv2d_1 (Conv2D) (None, 7, 7, 64) 2112
batch_normalization_1 (Bat (None, 7, 7, 64) 256
chNormalization)
re_lu_1 (ReLU) (None, 7, 7, 64) 0
flatten (Flatten) (None, 3136) 0
dense (Dense) (None, 10) 31370
=================================================================
Total params: 34506 (134.79 KB)
Trainable params: 34314 (134.04 KB)
Non-trainable params: 192 (768.00 Byte)
_________________________________________________________________
1.3. Model training
Given the model created above, train the model and check its accuracy. The model should achieve a test accuracy over 98% after 10 epochs.
from keras.optimizers import Adam
model_keras.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-3),
metrics=['accuracy'])
_ = model_keras.fit(x_train, y_train, epochs=10, validation_split=0.1)
Epoch 1/10
1/1688 [..............................] - ETA: 1:00:20 - loss: 2.9797 - accuracy: 0.0312
23/1688 [..............................] - ETA: 3s - loss: 1.4747 - accuracy: 0.5122
46/1688 [..............................] - ETA: 3s - loss: 1.0305 - accuracy: 0.6671
70/1688 [>.............................] - ETA: 3s - loss: 0.8442 - accuracy: 0.7299
93/1688 [>.............................] - ETA: 3s - loss: 0.7400 - accuracy: 0.7655
115/1688 [=>............................] - ETA: 3s - loss: 0.6662 - accuracy: 0.7908
137/1688 [=>............................] - ETA: 3s - loss: 0.6106 - accuracy: 0.8109
160/1688 [=>............................] - ETA: 3s - loss: 0.5748 - accuracy: 0.8219
183/1688 [==>...........................] - ETA: 3s - loss: 0.5417 - accuracy: 0.8323
206/1688 [==>...........................] - ETA: 3s - loss: 0.5079 - accuracy: 0.8448
229/1688 [===>..........................] - ETA: 3s - loss: 0.4813 - accuracy: 0.8533
252/1688 [===>..........................] - ETA: 3s - loss: 0.4571 - accuracy: 0.8604
275/1688 [===>..........................] - ETA: 3s - loss: 0.4367 - accuracy: 0.8660
298/1688 [====>.........................] - ETA: 3s - loss: 0.4207 - accuracy: 0.8715
321/1688 [====>.........................] - ETA: 3s - loss: 0.4051 - accuracy: 0.8766
344/1688 [=====>........................] - ETA: 3s - loss: 0.3920 - accuracy: 0.8808
367/1688 [=====>........................] - ETA: 2s - loss: 0.3805 - accuracy: 0.8845
390/1688 [=====>........................] - ETA: 2s - loss: 0.3713 - accuracy: 0.8873
413/1688 [======>.......................] - ETA: 2s - loss: 0.3619 - accuracy: 0.8898
435/1688 [======>.......................] - ETA: 2s - loss: 0.3521 - accuracy: 0.8930
458/1688 [=======>......................] - ETA: 2s - loss: 0.3450 - accuracy: 0.8953
481/1688 [=======>......................] - ETA: 2s - loss: 0.3373 - accuracy: 0.8980
503/1688 [=======>......................] - ETA: 2s - loss: 0.3295 - accuracy: 0.9003
526/1688 [========>.....................] - ETA: 2s - loss: 0.3216 - accuracy: 0.9026
549/1688 [========>.....................] - ETA: 2s - loss: 0.3151 - accuracy: 0.9049
572/1688 [=========>....................] - ETA: 2s - loss: 0.3106 - accuracy: 0.9064
595/1688 [=========>....................] - ETA: 2s - loss: 0.3035 - accuracy: 0.9084
618/1688 [=========>....................] - ETA: 2s - loss: 0.2973 - accuracy: 0.9104
640/1688 [==========>...................] - ETA: 2s - loss: 0.2916 - accuracy: 0.9123
663/1688 [==========>...................] - ETA: 2s - loss: 0.2857 - accuracy: 0.9140
686/1688 [===========>..................] - ETA: 2s - loss: 0.2806 - accuracy: 0.9156
709/1688 [===========>..................] - ETA: 2s - loss: 0.2774 - accuracy: 0.9165
732/1688 [============>.................] - ETA: 2s - loss: 0.2727 - accuracy: 0.9180
754/1688 [============>.................] - ETA: 2s - loss: 0.2684 - accuracy: 0.9193
776/1688 [============>.................] - ETA: 2s - loss: 0.2638 - accuracy: 0.9206
799/1688 [=============>................] - ETA: 1s - loss: 0.2608 - accuracy: 0.9217
822/1688 [=============>................] - ETA: 1s - loss: 0.2569 - accuracy: 0.9227
845/1688 [==============>...............] - ETA: 1s - loss: 0.2532 - accuracy: 0.9236
867/1688 [==============>...............] - ETA: 1s - loss: 0.2497 - accuracy: 0.9245
890/1688 [==============>...............] - ETA: 1s - loss: 0.2461 - accuracy: 0.9256
913/1688 [===============>..............] - ETA: 1s - loss: 0.2425 - accuracy: 0.9266
935/1688 [===============>..............] - ETA: 1s - loss: 0.2385 - accuracy: 0.9279
958/1688 [================>.............] - ETA: 1s - loss: 0.2352 - accuracy: 0.9290
981/1688 [================>.............] - ETA: 1s - loss: 0.2320 - accuracy: 0.9296
1004/1688 [================>.............] - ETA: 1s - loss: 0.2296 - accuracy: 0.9305
1026/1688 [=================>............] - ETA: 1s - loss: 0.2270 - accuracy: 0.9313
1050/1688 [=================>............] - ETA: 1s - loss: 0.2245 - accuracy: 0.9321
1072/1688 [==================>...........] - ETA: 1s - loss: 0.2222 - accuracy: 0.9326
1095/1688 [==================>...........] - ETA: 1s - loss: 0.2203 - accuracy: 0.9333
1118/1688 [==================>...........] - ETA: 1s - loss: 0.2186 - accuracy: 0.9339
1141/1688 [===================>..........] - ETA: 1s - loss: 0.2162 - accuracy: 0.9345
1164/1688 [===================>..........] - ETA: 1s - loss: 0.2139 - accuracy: 0.9352
1187/1688 [====================>.........] - ETA: 1s - loss: 0.2118 - accuracy: 0.9359
1210/1688 [====================>.........] - ETA: 1s - loss: 0.2103 - accuracy: 0.9363
1233/1688 [====================>.........] - ETA: 1s - loss: 0.2083 - accuracy: 0.9369
1256/1688 [=====================>........] - ETA: 0s - loss: 0.2064 - accuracy: 0.9374
1279/1688 [=====================>........] - ETA: 0s - loss: 0.2043 - accuracy: 0.9379
1302/1688 [======================>.......] - ETA: 0s - loss: 0.2023 - accuracy: 0.9386
1325/1688 [======================>.......] - ETA: 0s - loss: 0.2002 - accuracy: 0.9392
1348/1688 [======================>.......] - ETA: 0s - loss: 0.1981 - accuracy: 0.9398
1371/1688 [=======================>......] - ETA: 0s - loss: 0.1962 - accuracy: 0.9404
1394/1688 [=======================>......] - ETA: 0s - loss: 0.1942 - accuracy: 0.9410
1417/1688 [========================>.....] - ETA: 0s - loss: 0.1924 - accuracy: 0.9415
1438/1688 [========================>.....] - ETA: 0s - loss: 0.1910 - accuracy: 0.9419
1459/1688 [========================>.....] - ETA: 0s - loss: 0.1891 - accuracy: 0.9425
1480/1688 [=========================>....] - ETA: 0s - loss: 0.1884 - accuracy: 0.9427
1501/1688 [=========================>....] - ETA: 0s - loss: 0.1875 - accuracy: 0.9430
1523/1688 [==========================>...] - ETA: 0s - loss: 0.1868 - accuracy: 0.9432
1544/1688 [==========================>...] - ETA: 0s - loss: 0.1855 - accuracy: 0.9436
1565/1688 [==========================>...] - ETA: 0s - loss: 0.1842 - accuracy: 0.9440
1586/1688 [===========================>..] - ETA: 0s - loss: 0.1830 - accuracy: 0.9443
1608/1688 [===========================>..] - ETA: 0s - loss: 0.1822 - accuracy: 0.9446
1629/1688 [===========================>..] - ETA: 0s - loss: 0.1806 - accuracy: 0.9449
1650/1688 [============================>.] - ETA: 0s - loss: 0.1795 - accuracy: 0.9452
1671/1688 [============================>.] - ETA: 0s - loss: 0.1783 - accuracy: 0.9456
1688/1688 [==============================] - ETA: 0s - loss: 0.1778 - accuracy: 0.9458
1688/1688 [==============================] - 6s 3ms/step - loss: 0.1778 - accuracy: 0.9458 - val_loss: 0.0735 - val_accuracy: 0.9807
Epoch 2/10
1/1688 [..............................] - ETA: 3s - loss: 0.0393 - accuracy: 0.9688
24/1688 [..............................] - ETA: 3s - loss: 0.0557 - accuracy: 0.9818
47/1688 [..............................] - ETA: 3s - loss: 0.0833 - accuracy: 0.9747
69/1688 [>.............................] - ETA: 3s - loss: 0.0859 - accuracy: 0.9737
92/1688 [>.............................] - ETA: 3s - loss: 0.0808 - accuracy: 0.9742
115/1688 [=>............................] - ETA: 3s - loss: 0.0821 - accuracy: 0.9742
138/1688 [=>............................] - ETA: 3s - loss: 0.0830 - accuracy: 0.9742
160/1688 [=>............................] - ETA: 3s - loss: 0.0844 - accuracy: 0.9727
183/1688 [==>...........................] - ETA: 3s - loss: 0.0827 - accuracy: 0.9737
206/1688 [==>...........................] - ETA: 3s - loss: 0.0818 - accuracy: 0.9736
229/1688 [===>..........................] - ETA: 3s - loss: 0.0788 - accuracy: 0.9742
252/1688 [===>..........................] - ETA: 3s - loss: 0.0788 - accuracy: 0.9741
275/1688 [===>..........................] - ETA: 3s - loss: 0.0803 - accuracy: 0.9737
297/1688 [====>.........................] - ETA: 3s - loss: 0.0790 - accuracy: 0.9745
320/1688 [====>.........................] - ETA: 3s - loss: 0.0798 - accuracy: 0.9743
342/1688 [=====>........................] - ETA: 3s - loss: 0.0802 - accuracy: 0.9744
364/1688 [=====>........................] - ETA: 2s - loss: 0.0787 - accuracy: 0.9751
387/1688 [=====>........................] - ETA: 2s - loss: 0.0769 - accuracy: 0.9757
409/1688 [======>.......................] - ETA: 2s - loss: 0.0749 - accuracy: 0.9760
432/1688 [======>.......................] - ETA: 2s - loss: 0.0738 - accuracy: 0.9764
455/1688 [=======>......................] - ETA: 2s - loss: 0.0726 - accuracy: 0.9770
478/1688 [=======>......................] - ETA: 2s - loss: 0.0728 - accuracy: 0.9770
501/1688 [=======>......................] - ETA: 2s - loss: 0.0724 - accuracy: 0.9772
524/1688 [========>.....................] - ETA: 2s - loss: 0.0717 - accuracy: 0.9775
547/1688 [========>.....................] - ETA: 2s - loss: 0.0717 - accuracy: 0.9775
569/1688 [=========>....................] - ETA: 2s - loss: 0.0718 - accuracy: 0.9777
592/1688 [=========>....................] - ETA: 2s - loss: 0.0718 - accuracy: 0.9776
614/1688 [=========>....................] - ETA: 2s - loss: 0.0711 - accuracy: 0.9778
637/1688 [==========>...................] - ETA: 2s - loss: 0.0712 - accuracy: 0.9776
659/1688 [==========>...................] - ETA: 2s - loss: 0.0707 - accuracy: 0.9779
682/1688 [===========>..................] - ETA: 2s - loss: 0.0712 - accuracy: 0.9776
705/1688 [===========>..................] - ETA: 2s - loss: 0.0705 - accuracy: 0.9779
728/1688 [===========>..................] - ETA: 2s - loss: 0.0700 - accuracy: 0.9781
750/1688 [============>.................] - ETA: 2s - loss: 0.0705 - accuracy: 0.9778
772/1688 [============>.................] - ETA: 2s - loss: 0.0704 - accuracy: 0.9779
795/1688 [=============>................] - ETA: 2s - loss: 0.0706 - accuracy: 0.9779
818/1688 [=============>................] - ETA: 1s - loss: 0.0703 - accuracy: 0.9779
841/1688 [=============>................] - ETA: 1s - loss: 0.0698 - accuracy: 0.9781
864/1688 [==============>...............] - ETA: 1s - loss: 0.0701 - accuracy: 0.9783
887/1688 [==============>...............] - ETA: 1s - loss: 0.0697 - accuracy: 0.9783
910/1688 [===============>..............] - ETA: 1s - loss: 0.0709 - accuracy: 0.9780
933/1688 [===============>..............] - ETA: 1s - loss: 0.0706 - accuracy: 0.9781
955/1688 [===============>..............] - ETA: 1s - loss: 0.0701 - accuracy: 0.9783
978/1688 [================>.............] - ETA: 1s - loss: 0.0702 - accuracy: 0.9783
1000/1688 [================>.............] - ETA: 1s - loss: 0.0708 - accuracy: 0.9781
1023/1688 [=================>............] - ETA: 1s - loss: 0.0705 - accuracy: 0.9781
1046/1688 [=================>............] - ETA: 1s - loss: 0.0704 - accuracy: 0.9782
1068/1688 [=================>............] - ETA: 1s - loss: 0.0704 - accuracy: 0.9783
1091/1688 [==================>...........] - ETA: 1s - loss: 0.0707 - accuracy: 0.9782
1113/1688 [==================>...........] - ETA: 1s - loss: 0.0706 - accuracy: 0.9782
1136/1688 [===================>..........] - ETA: 1s - loss: 0.0704 - accuracy: 0.9783
1159/1688 [===================>..........] - ETA: 1s - loss: 0.0703 - accuracy: 0.9783
1182/1688 [====================>.........] - ETA: 1s - loss: 0.0708 - accuracy: 0.9782
1205/1688 [====================>.........] - ETA: 1s - loss: 0.0703 - accuracy: 0.9783
1228/1688 [====================>.........] - ETA: 1s - loss: 0.0699 - accuracy: 0.9784
1251/1688 [=====================>........] - ETA: 0s - loss: 0.0700 - accuracy: 0.9784
1274/1688 [=====================>........] - ETA: 0s - loss: 0.0705 - accuracy: 0.9782
1296/1688 [======================>.......] - ETA: 0s - loss: 0.0704 - accuracy: 0.9783
1319/1688 [======================>.......] - ETA: 0s - loss: 0.0702 - accuracy: 0.9783
1342/1688 [======================>.......] - ETA: 0s - loss: 0.0701 - accuracy: 0.9783
1365/1688 [=======================>......] - ETA: 0s - loss: 0.0702 - accuracy: 0.9783
1386/1688 [=======================>......] - ETA: 0s - loss: 0.0703 - accuracy: 0.9783
1408/1688 [========================>.....] - ETA: 0s - loss: 0.0705 - accuracy: 0.9782
1430/1688 [========================>.....] - ETA: 0s - loss: 0.0702 - accuracy: 0.9783
1451/1688 [========================>.....] - ETA: 0s - loss: 0.0706 - accuracy: 0.9782
1473/1688 [=========================>....] - ETA: 0s - loss: 0.0706 - accuracy: 0.9782
1494/1688 [=========================>....] - ETA: 0s - loss: 0.0705 - accuracy: 0.9783
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0703 - accuracy: 0.9783
1537/1688 [==========================>...] - ETA: 0s - loss: 0.0702 - accuracy: 0.9784
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0707 - accuracy: 0.9783
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0707 - accuracy: 0.9784
1601/1688 [===========================>..] - ETA: 0s - loss: 0.0706 - accuracy: 0.9784
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0706 - accuracy: 0.9784
1644/1688 [============================>.] - ETA: 0s - loss: 0.0706 - accuracy: 0.9785
1665/1688 [============================>.] - ETA: 0s - loss: 0.0703 - accuracy: 0.9786
1686/1688 [============================>.] - ETA: 0s - loss: 0.0703 - accuracy: 0.9786
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0703 - accuracy: 0.9786 - val_loss: 0.0641 - val_accuracy: 0.9830
Epoch 3/10
1/1688 [..............................] - ETA: 3s - loss: 0.0011 - accuracy: 1.0000
25/1688 [..............................] - ETA: 3s - loss: 0.0548 - accuracy: 0.9850
47/1688 [..............................] - ETA: 3s - loss: 0.0506 - accuracy: 0.9847
69/1688 [>.............................] - ETA: 3s - loss: 0.0560 - accuracy: 0.9841
91/1688 [>.............................] - ETA: 3s - loss: 0.0525 - accuracy: 0.9852
113/1688 [=>............................] - ETA: 3s - loss: 0.0475 - accuracy: 0.9862
136/1688 [=>............................] - ETA: 3s - loss: 0.0447 - accuracy: 0.9869
158/1688 [=>............................] - ETA: 3s - loss: 0.0470 - accuracy: 0.9862
179/1688 [==>...........................] - ETA: 3s - loss: 0.0469 - accuracy: 0.9860
199/1688 [==>...........................] - ETA: 3s - loss: 0.0484 - accuracy: 0.9860
220/1688 [==>...........................] - ETA: 3s - loss: 0.0502 - accuracy: 0.9857
241/1688 [===>..........................] - ETA: 3s - loss: 0.0500 - accuracy: 0.9856
262/1688 [===>..........................] - ETA: 3s - loss: 0.0498 - accuracy: 0.9854
283/1688 [====>.........................] - ETA: 3s - loss: 0.0494 - accuracy: 0.9855
305/1688 [====>.........................] - ETA: 3s - loss: 0.0502 - accuracy: 0.9851
326/1688 [====>.........................] - ETA: 3s - loss: 0.0502 - accuracy: 0.9851
347/1688 [=====>........................] - ETA: 3s - loss: 0.0520 - accuracy: 0.9851
368/1688 [=====>........................] - ETA: 3s - loss: 0.0514 - accuracy: 0.9851
389/1688 [=====>........................] - ETA: 3s - loss: 0.0509 - accuracy: 0.9851
410/1688 [======>.......................] - ETA: 3s - loss: 0.0508 - accuracy: 0.9851
431/1688 [======>.......................] - ETA: 2s - loss: 0.0505 - accuracy: 0.9853
452/1688 [=======>......................] - ETA: 2s - loss: 0.0506 - accuracy: 0.9851
473/1688 [=======>......................] - ETA: 2s - loss: 0.0501 - accuracy: 0.9851
495/1688 [=======>......................] - ETA: 2s - loss: 0.0513 - accuracy: 0.9848
516/1688 [========>.....................] - ETA: 2s - loss: 0.0505 - accuracy: 0.9850
537/1688 [========>.....................] - ETA: 2s - loss: 0.0497 - accuracy: 0.9853
558/1688 [========>.....................] - ETA: 2s - loss: 0.0494 - accuracy: 0.9853
579/1688 [=========>....................] - ETA: 2s - loss: 0.0497 - accuracy: 0.9850
600/1688 [=========>....................] - ETA: 2s - loss: 0.0494 - accuracy: 0.9851
621/1688 [==========>...................] - ETA: 2s - loss: 0.0498 - accuracy: 0.9848
642/1688 [==========>...................] - ETA: 2s - loss: 0.0505 - accuracy: 0.9847
663/1688 [==========>...................] - ETA: 2s - loss: 0.0498 - accuracy: 0.9848
684/1688 [===========>..................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9850
705/1688 [===========>..................] - ETA: 2s - loss: 0.0495 - accuracy: 0.9848
726/1688 [===========>..................] - ETA: 2s - loss: 0.0494 - accuracy: 0.9850
747/1688 [============>.................] - ETA: 2s - loss: 0.0503 - accuracy: 0.9848
768/1688 [============>.................] - ETA: 2s - loss: 0.0502 - accuracy: 0.9848
787/1688 [============>.................] - ETA: 2s - loss: 0.0501 - accuracy: 0.9849
807/1688 [=============>................] - ETA: 2s - loss: 0.0507 - accuracy: 0.9848
827/1688 [=============>................] - ETA: 2s - loss: 0.0505 - accuracy: 0.9848
847/1688 [==============>...............] - ETA: 2s - loss: 0.0505 - accuracy: 0.9847
867/1688 [==============>...............] - ETA: 1s - loss: 0.0502 - accuracy: 0.9847
887/1688 [==============>...............] - ETA: 1s - loss: 0.0511 - accuracy: 0.9846
907/1688 [===============>..............] - ETA: 1s - loss: 0.0511 - accuracy: 0.9845
927/1688 [===============>..............] - ETA: 1s - loss: 0.0511 - accuracy: 0.9844
947/1688 [===============>..............] - ETA: 1s - loss: 0.0510 - accuracy: 0.9845
967/1688 [================>.............] - ETA: 1s - loss: 0.0511 - accuracy: 0.9845
987/1688 [================>.............] - ETA: 1s - loss: 0.0508 - accuracy: 0.9846
1006/1688 [================>.............] - ETA: 1s - loss: 0.0508 - accuracy: 0.9846
1024/1688 [=================>............] - ETA: 1s - loss: 0.0508 - accuracy: 0.9845
1042/1688 [=================>............] - ETA: 1s - loss: 0.0508 - accuracy: 0.9846
1060/1688 [=================>............] - ETA: 1s - loss: 0.0508 - accuracy: 0.9845
1078/1688 [==================>...........] - ETA: 1s - loss: 0.0508 - accuracy: 0.9844
1096/1688 [==================>...........] - ETA: 1s - loss: 0.0508 - accuracy: 0.9844
1114/1688 [==================>...........] - ETA: 1s - loss: 0.0503 - accuracy: 0.9845
1133/1688 [===================>..........] - ETA: 1s - loss: 0.0498 - accuracy: 0.9847
1151/1688 [===================>..........] - ETA: 1s - loss: 0.0497 - accuracy: 0.9847
1170/1688 [===================>..........] - ETA: 1s - loss: 0.0500 - accuracy: 0.9846
1189/1688 [====================>.........] - ETA: 1s - loss: 0.0499 - accuracy: 0.9847
1207/1688 [====================>.........] - ETA: 1s - loss: 0.0497 - accuracy: 0.9847
1225/1688 [====================>.........] - ETA: 1s - loss: 0.0501 - accuracy: 0.9846
1243/1688 [=====================>........] - ETA: 1s - loss: 0.0502 - accuracy: 0.9846
1261/1688 [=====================>........] - ETA: 1s - loss: 0.0502 - accuracy: 0.9846
1279/1688 [=====================>........] - ETA: 1s - loss: 0.0501 - accuracy: 0.9847
1297/1688 [======================>.......] - ETA: 0s - loss: 0.0502 - accuracy: 0.9846
1315/1688 [======================>.......] - ETA: 0s - loss: 0.0507 - accuracy: 0.9844
1333/1688 [======================>.......] - ETA: 0s - loss: 0.0507 - accuracy: 0.9844
1351/1688 [=======================>......] - ETA: 0s - loss: 0.0505 - accuracy: 0.9844
1369/1688 [=======================>......] - ETA: 0s - loss: 0.0505 - accuracy: 0.9844
1387/1688 [=======================>......] - ETA: 0s - loss: 0.0507 - accuracy: 0.9843
1405/1688 [=======================>......] - ETA: 0s - loss: 0.0507 - accuracy: 0.9843
1424/1688 [========================>.....] - ETA: 0s - loss: 0.0507 - accuracy: 0.9843
1442/1688 [========================>.....] - ETA: 0s - loss: 0.0508 - accuracy: 0.9842
1460/1688 [========================>.....] - ETA: 0s - loss: 0.0506 - accuracy: 0.9842
1478/1688 [=========================>....] - ETA: 0s - loss: 0.0504 - accuracy: 0.9843
1497/1688 [=========================>....] - ETA: 0s - loss: 0.0502 - accuracy: 0.9843
1515/1688 [=========================>....] - ETA: 0s - loss: 0.0499 - accuracy: 0.9844
1533/1688 [==========================>...] - ETA: 0s - loss: 0.0499 - accuracy: 0.9844
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0500 - accuracy: 0.9844
1570/1688 [==========================>...] - ETA: 0s - loss: 0.0502 - accuracy: 0.9844
1588/1688 [===========================>..] - ETA: 0s - loss: 0.0506 - accuracy: 0.9842
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0507 - accuracy: 0.9841
1625/1688 [===========================>..] - ETA: 0s - loss: 0.0509 - accuracy: 0.9841
1644/1688 [============================>.] - ETA: 0s - loss: 0.0507 - accuracy: 0.9841
1663/1688 [============================>.] - ETA: 0s - loss: 0.0509 - accuracy: 0.9841
1681/1688 [============================>.] - ETA: 0s - loss: 0.0508 - accuracy: 0.9840
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0510 - accuracy: 0.9840 - val_loss: 0.0663 - val_accuracy: 0.9828
Epoch 4/10
1/1688 [..............................] - ETA: 4s - loss: 0.0075 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0616 - accuracy: 0.9857
47/1688 [..............................] - ETA: 3s - loss: 0.0517 - accuracy: 0.9847
70/1688 [>.............................] - ETA: 3s - loss: 0.0410 - accuracy: 0.9871
93/1688 [>.............................] - ETA: 3s - loss: 0.0423 - accuracy: 0.9866
116/1688 [=>............................] - ETA: 3s - loss: 0.0410 - accuracy: 0.9865
139/1688 [=>............................] - ETA: 3s - loss: 0.0399 - accuracy: 0.9872
162/1688 [=>............................] - ETA: 3s - loss: 0.0383 - accuracy: 0.9873
184/1688 [==>...........................] - ETA: 3s - loss: 0.0381 - accuracy: 0.9876
207/1688 [==>...........................] - ETA: 3s - loss: 0.0376 - accuracy: 0.9870
230/1688 [===>..........................] - ETA: 3s - loss: 0.0367 - accuracy: 0.9872
253/1688 [===>..........................] - ETA: 3s - loss: 0.0363 - accuracy: 0.9874
275/1688 [===>..........................] - ETA: 3s - loss: 0.0356 - accuracy: 0.9874
298/1688 [====>.........................] - ETA: 3s - loss: 0.0371 - accuracy: 0.9869
320/1688 [====>.........................] - ETA: 3s - loss: 0.0372 - accuracy: 0.9871
342/1688 [=====>........................] - ETA: 3s - loss: 0.0371 - accuracy: 0.9875
364/1688 [=====>........................] - ETA: 2s - loss: 0.0384 - accuracy: 0.9871
387/1688 [=====>........................] - ETA: 2s - loss: 0.0387 - accuracy: 0.9868
409/1688 [======>.......................] - ETA: 2s - loss: 0.0381 - accuracy: 0.9869
432/1688 [======>.......................] - ETA: 2s - loss: 0.0374 - accuracy: 0.9871
455/1688 [=======>......................] - ETA: 2s - loss: 0.0375 - accuracy: 0.9873
478/1688 [=======>......................] - ETA: 2s - loss: 0.0377 - accuracy: 0.9874
501/1688 [=======>......................] - ETA: 2s - loss: 0.0369 - accuracy: 0.9878
524/1688 [========>.....................] - ETA: 2s - loss: 0.0374 - accuracy: 0.9877
546/1688 [========>.....................] - ETA: 2s - loss: 0.0368 - accuracy: 0.9879
569/1688 [=========>....................] - ETA: 2s - loss: 0.0376 - accuracy: 0.9876
592/1688 [=========>....................] - ETA: 2s - loss: 0.0371 - accuracy: 0.9876
614/1688 [=========>....................] - ETA: 2s - loss: 0.0368 - accuracy: 0.9878
637/1688 [==========>...................] - ETA: 2s - loss: 0.0363 - accuracy: 0.9880
660/1688 [==========>...................] - ETA: 2s - loss: 0.0363 - accuracy: 0.9880
684/1688 [===========>..................] - ETA: 2s - loss: 0.0368 - accuracy: 0.9879
707/1688 [===========>..................] - ETA: 2s - loss: 0.0364 - accuracy: 0.9881
729/1688 [===========>..................] - ETA: 2s - loss: 0.0368 - accuracy: 0.9880
750/1688 [============>.................] - ETA: 2s - loss: 0.0376 - accuracy: 0.9880
770/1688 [============>.................] - ETA: 2s - loss: 0.0375 - accuracy: 0.9880
791/1688 [=============>................] - ETA: 2s - loss: 0.0375 - accuracy: 0.9880
812/1688 [=============>................] - ETA: 1s - loss: 0.0379 - accuracy: 0.9878
834/1688 [=============>................] - ETA: 1s - loss: 0.0380 - accuracy: 0.9877
855/1688 [==============>...............] - ETA: 1s - loss: 0.0377 - accuracy: 0.9879
876/1688 [==============>...............] - ETA: 1s - loss: 0.0379 - accuracy: 0.9877
897/1688 [==============>...............] - ETA: 1s - loss: 0.0387 - accuracy: 0.9875
918/1688 [===============>..............] - ETA: 1s - loss: 0.0389 - accuracy: 0.9873
939/1688 [===============>..............] - ETA: 1s - loss: 0.0391 - accuracy: 0.9874
960/1688 [================>.............] - ETA: 1s - loss: 0.0388 - accuracy: 0.9874
981/1688 [================>.............] - ETA: 1s - loss: 0.0384 - accuracy: 0.9875
1002/1688 [================>.............] - ETA: 1s - loss: 0.0385 - accuracy: 0.9875
1023/1688 [=================>............] - ETA: 1s - loss: 0.0385 - accuracy: 0.9876
1044/1688 [=================>............] - ETA: 1s - loss: 0.0391 - accuracy: 0.9874
1066/1688 [=================>............] - ETA: 1s - loss: 0.0390 - accuracy: 0.9874
1087/1688 [==================>...........] - ETA: 1s - loss: 0.0390 - accuracy: 0.9874
1108/1688 [==================>...........] - ETA: 1s - loss: 0.0391 - accuracy: 0.9873
1129/1688 [===================>..........] - ETA: 1s - loss: 0.0389 - accuracy: 0.9874
1149/1688 [===================>..........] - ETA: 1s - loss: 0.0389 - accuracy: 0.9874
1170/1688 [===================>..........] - ETA: 1s - loss: 0.0390 - accuracy: 0.9874
1191/1688 [====================>.........] - ETA: 1s - loss: 0.0392 - accuracy: 0.9873
1212/1688 [====================>.........] - ETA: 1s - loss: 0.0390 - accuracy: 0.9874
1233/1688 [====================>.........] - ETA: 1s - loss: 0.0390 - accuracy: 0.9874
1254/1688 [=====================>........] - ETA: 1s - loss: 0.0386 - accuracy: 0.9875
1275/1688 [=====================>........] - ETA: 0s - loss: 0.0386 - accuracy: 0.9875
1296/1688 [======================>.......] - ETA: 0s - loss: 0.0390 - accuracy: 0.9874
1317/1688 [======================>.......] - ETA: 0s - loss: 0.0390 - accuracy: 0.9875
1338/1688 [======================>.......] - ETA: 0s - loss: 0.0392 - accuracy: 0.9874
1360/1688 [=======================>......] - ETA: 0s - loss: 0.0392 - accuracy: 0.9874
1381/1688 [=======================>......] - ETA: 0s - loss: 0.0392 - accuracy: 0.9874
1402/1688 [=======================>......] - ETA: 0s - loss: 0.0389 - accuracy: 0.9875
1423/1688 [========================>.....] - ETA: 0s - loss: 0.0391 - accuracy: 0.9874
1445/1688 [========================>.....] - ETA: 0s - loss: 0.0387 - accuracy: 0.9876
1466/1688 [=========================>....] - ETA: 0s - loss: 0.0390 - accuracy: 0.9875
1487/1688 [=========================>....] - ETA: 0s - loss: 0.0391 - accuracy: 0.9875
1507/1688 [=========================>....] - ETA: 0s - loss: 0.0393 - accuracy: 0.9874
1527/1688 [==========================>...] - ETA: 0s - loss: 0.0391 - accuracy: 0.9875
1547/1688 [==========================>...] - ETA: 0s - loss: 0.0392 - accuracy: 0.9874
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0390 - accuracy: 0.9875
1587/1688 [===========================>..] - ETA: 0s - loss: 0.0390 - accuracy: 0.9875
1607/1688 [===========================>..] - ETA: 0s - loss: 0.0391 - accuracy: 0.9874
1627/1688 [===========================>..] - ETA: 0s - loss: 0.0390 - accuracy: 0.9874
1647/1688 [============================>.] - ETA: 0s - loss: 0.0389 - accuracy: 0.9874
1667/1688 [============================>.] - ETA: 0s - loss: 0.0389 - accuracy: 0.9875
1687/1688 [============================>.] - ETA: 0s - loss: 0.0392 - accuracy: 0.9874
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0392 - accuracy: 0.9874 - val_loss: 0.0828 - val_accuracy: 0.9790
Epoch 5/10
1/1688 [..............................] - ETA: 4s - loss: 3.7124e-04 - accuracy: 1.0000
21/1688 [..............................] - ETA: 4s - loss: 0.0169 - accuracy: 0.9911
41/1688 [..............................] - ETA: 4s - loss: 0.0261 - accuracy: 0.9878
60/1688 [>.............................] - ETA: 4s - loss: 0.0236 - accuracy: 0.9906
79/1688 [>.............................] - ETA: 4s - loss: 0.0253 - accuracy: 0.9901
99/1688 [>.............................] - ETA: 4s - loss: 0.0255 - accuracy: 0.9899
119/1688 [=>............................] - ETA: 4s - loss: 0.0259 - accuracy: 0.9905
139/1688 [=>............................] - ETA: 4s - loss: 0.0239 - accuracy: 0.9917
159/1688 [=>............................] - ETA: 3s - loss: 0.0250 - accuracy: 0.9914
179/1688 [==>...........................] - ETA: 3s - loss: 0.0260 - accuracy: 0.9909
199/1688 [==>...........................] - ETA: 3s - loss: 0.0266 - accuracy: 0.9910
219/1688 [==>...........................] - ETA: 3s - loss: 0.0271 - accuracy: 0.9906
239/1688 [===>..........................] - ETA: 3s - loss: 0.0261 - accuracy: 0.9910
259/1688 [===>..........................] - ETA: 3s - loss: 0.0283 - accuracy: 0.9900
279/1688 [===>..........................] - ETA: 3s - loss: 0.0279 - accuracy: 0.9900
298/1688 [====>.........................] - ETA: 3s - loss: 0.0278 - accuracy: 0.9902
318/1688 [====>.........................] - ETA: 3s - loss: 0.0281 - accuracy: 0.9904
338/1688 [=====>........................] - ETA: 3s - loss: 0.0283 - accuracy: 0.9904
357/1688 [=====>........................] - ETA: 3s - loss: 0.0283 - accuracy: 0.9904
377/1688 [=====>........................] - ETA: 3s - loss: 0.0278 - accuracy: 0.9904
397/1688 [======>.......................] - ETA: 3s - loss: 0.0271 - accuracy: 0.9907
417/1688 [======>.......................] - ETA: 3s - loss: 0.0275 - accuracy: 0.9906
437/1688 [======>.......................] - ETA: 3s - loss: 0.0296 - accuracy: 0.9901
457/1688 [=======>......................] - ETA: 3s - loss: 0.0299 - accuracy: 0.9899
477/1688 [=======>......................] - ETA: 3s - loss: 0.0293 - accuracy: 0.9901
496/1688 [=======>......................] - ETA: 3s - loss: 0.0292 - accuracy: 0.9903
516/1688 [========>.....................] - ETA: 3s - loss: 0.0293 - accuracy: 0.9904
536/1688 [========>.....................] - ETA: 2s - loss: 0.0292 - accuracy: 0.9905
556/1688 [========>.....................] - ETA: 2s - loss: 0.0292 - accuracy: 0.9906
576/1688 [=========>....................] - ETA: 2s - loss: 0.0299 - accuracy: 0.9903
596/1688 [=========>....................] - ETA: 2s - loss: 0.0295 - accuracy: 0.9903
616/1688 [=========>....................] - ETA: 2s - loss: 0.0299 - accuracy: 0.9900
636/1688 [==========>...................] - ETA: 2s - loss: 0.0299 - accuracy: 0.9901
655/1688 [==========>...................] - ETA: 2s - loss: 0.0294 - accuracy: 0.9903
676/1688 [===========>..................] - ETA: 2s - loss: 0.0294 - accuracy: 0.9902
695/1688 [===========>..................] - ETA: 2s - loss: 0.0299 - accuracy: 0.9900
715/1688 [===========>..................] - ETA: 2s - loss: 0.0301 - accuracy: 0.9900
735/1688 [============>.................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9899
755/1688 [============>.................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9899
775/1688 [============>.................] - ETA: 2s - loss: 0.0301 - accuracy: 0.9900
795/1688 [=============>................] - ETA: 2s - loss: 0.0309 - accuracy: 0.9897
814/1688 [=============>................] - ETA: 2s - loss: 0.0311 - accuracy: 0.9896
833/1688 [=============>................] - ETA: 2s - loss: 0.0313 - accuracy: 0.9896
853/1688 [==============>...............] - ETA: 2s - loss: 0.0308 - accuracy: 0.9897
873/1688 [==============>...............] - ETA: 2s - loss: 0.0312 - accuracy: 0.9895
893/1688 [==============>...............] - ETA: 2s - loss: 0.0317 - accuracy: 0.9894
912/1688 [===============>..............] - ETA: 2s - loss: 0.0319 - accuracy: 0.9893
932/1688 [===============>..............] - ETA: 1s - loss: 0.0321 - accuracy: 0.9893
952/1688 [===============>..............] - ETA: 1s - loss: 0.0322 - accuracy: 0.9892
972/1688 [================>.............] - ETA: 1s - loss: 0.0325 - accuracy: 0.9892
992/1688 [================>.............] - ETA: 1s - loss: 0.0325 - accuracy: 0.9891
1012/1688 [================>.............] - ETA: 1s - loss: 0.0324 - accuracy: 0.9891
1032/1688 [=================>............] - ETA: 1s - loss: 0.0322 - accuracy: 0.9892
1052/1688 [=================>............] - ETA: 1s - loss: 0.0324 - accuracy: 0.9891
1072/1688 [==================>...........] - ETA: 1s - loss: 0.0323 - accuracy: 0.9892
1092/1688 [==================>...........] - ETA: 1s - loss: 0.0325 - accuracy: 0.9892
1112/1688 [==================>...........] - ETA: 1s - loss: 0.0323 - accuracy: 0.9892
1132/1688 [===================>..........] - ETA: 1s - loss: 0.0322 - accuracy: 0.9893
1151/1688 [===================>..........] - ETA: 1s - loss: 0.0326 - accuracy: 0.9891
1171/1688 [===================>..........] - ETA: 1s - loss: 0.0326 - accuracy: 0.9890
1191/1688 [====================>.........] - ETA: 1s - loss: 0.0332 - accuracy: 0.9888
1211/1688 [====================>.........] - ETA: 1s - loss: 0.0338 - accuracy: 0.9887
1230/1688 [====================>.........] - ETA: 1s - loss: 0.0338 - accuracy: 0.9886
1249/1688 [=====================>........] - ETA: 1s - loss: 0.0335 - accuracy: 0.9887
1268/1688 [=====================>........] - ETA: 1s - loss: 0.0332 - accuracy: 0.9888
1286/1688 [=====================>........] - ETA: 1s - loss: 0.0332 - accuracy: 0.9888
1304/1688 [======================>.......] - ETA: 0s - loss: 0.0334 - accuracy: 0.9888
1322/1688 [======================>.......] - ETA: 0s - loss: 0.0334 - accuracy: 0.9888
1340/1688 [======================>.......] - ETA: 0s - loss: 0.0333 - accuracy: 0.9889
1358/1688 [=======================>......] - ETA: 0s - loss: 0.0333 - accuracy: 0.9889
1376/1688 [=======================>......] - ETA: 0s - loss: 0.0331 - accuracy: 0.9890
1394/1688 [=======================>......] - ETA: 0s - loss: 0.0332 - accuracy: 0.9889
1412/1688 [========================>.....] - ETA: 0s - loss: 0.0330 - accuracy: 0.9890
1430/1688 [========================>.....] - ETA: 0s - loss: 0.0332 - accuracy: 0.9889
1448/1688 [========================>.....] - ETA: 0s - loss: 0.0331 - accuracy: 0.9890
1466/1688 [=========================>....] - ETA: 0s - loss: 0.0329 - accuracy: 0.9891
1484/1688 [=========================>....] - ETA: 0s - loss: 0.0331 - accuracy: 0.9890
1502/1688 [=========================>....] - ETA: 0s - loss: 0.0333 - accuracy: 0.9889
1520/1688 [==========================>...] - ETA: 0s - loss: 0.0332 - accuracy: 0.9890
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0332 - accuracy: 0.9889
1556/1688 [==========================>...] - ETA: 0s - loss: 0.0332 - accuracy: 0.9890
1575/1688 [==========================>...] - ETA: 0s - loss: 0.0337 - accuracy: 0.9889
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0340 - accuracy: 0.9888
1611/1688 [===========================>..] - ETA: 0s - loss: 0.0342 - accuracy: 0.9887
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0343 - accuracy: 0.9887
1649/1688 [============================>.] - ETA: 0s - loss: 0.0344 - accuracy: 0.9887
1667/1688 [============================>.] - ETA: 0s - loss: 0.0344 - accuracy: 0.9887
1686/1688 [============================>.] - ETA: 0s - loss: 0.0342 - accuracy: 0.9888
1688/1688 [==============================] - 5s 3ms/step - loss: 0.0342 - accuracy: 0.9888 - val_loss: 0.0553 - val_accuracy: 0.9857
Epoch 6/10
1/1688 [..............................] - ETA: 3s - loss: 5.9146e-04 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0219 - accuracy: 0.9896
46/1688 [..............................] - ETA: 3s - loss: 0.0178 - accuracy: 0.9925
69/1688 [>.............................] - ETA: 3s - loss: 0.0192 - accuracy: 0.9928
92/1688 [>.............................] - ETA: 3s - loss: 0.0192 - accuracy: 0.9929
115/1688 [=>............................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9927
138/1688 [=>............................] - ETA: 3s - loss: 0.0197 - accuracy: 0.9930
160/1688 [=>............................] - ETA: 3s - loss: 0.0199 - accuracy: 0.9930
182/1688 [==>...........................] - ETA: 3s - loss: 0.0209 - accuracy: 0.9928
205/1688 [==>...........................] - ETA: 3s - loss: 0.0214 - accuracy: 0.9924
228/1688 [===>..........................] - ETA: 3s - loss: 0.0244 - accuracy: 0.9914
251/1688 [===>..........................] - ETA: 3s - loss: 0.0240 - accuracy: 0.9915
273/1688 [===>..........................] - ETA: 3s - loss: 0.0241 - accuracy: 0.9915
295/1688 [====>.........................] - ETA: 3s - loss: 0.0236 - accuracy: 0.9915
318/1688 [====>.........................] - ETA: 3s - loss: 0.0255 - accuracy: 0.9914
341/1688 [=====>........................] - ETA: 3s - loss: 0.0267 - accuracy: 0.9911
364/1688 [=====>........................] - ETA: 2s - loss: 0.0262 - accuracy: 0.9914
387/1688 [=====>........................] - ETA: 2s - loss: 0.0260 - accuracy: 0.9914
410/1688 [======>.......................] - ETA: 2s - loss: 0.0252 - accuracy: 0.9917
433/1688 [======>.......................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9918
456/1688 [=======>......................] - ETA: 2s - loss: 0.0259 - accuracy: 0.9915
479/1688 [=======>......................] - ETA: 2s - loss: 0.0259 - accuracy: 0.9916
502/1688 [=======>......................] - ETA: 2s - loss: 0.0257 - accuracy: 0.9916
524/1688 [========>.....................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9917
547/1688 [========>.....................] - ETA: 2s - loss: 0.0251 - accuracy: 0.9916
570/1688 [=========>....................] - ETA: 2s - loss: 0.0248 - accuracy: 0.9918
593/1688 [=========>....................] - ETA: 2s - loss: 0.0246 - accuracy: 0.9918
615/1688 [=========>....................] - ETA: 2s - loss: 0.0247 - accuracy: 0.9917
636/1688 [==========>...................] - ETA: 2s - loss: 0.0251 - accuracy: 0.9916
658/1688 [==========>...................] - ETA: 2s - loss: 0.0255 - accuracy: 0.9915
680/1688 [===========>..................] - ETA: 2s - loss: 0.0253 - accuracy: 0.9916
701/1688 [===========>..................] - ETA: 2s - loss: 0.0256 - accuracy: 0.9914
723/1688 [===========>..................] - ETA: 2s - loss: 0.0260 - accuracy: 0.9913
744/1688 [============>.................] - ETA: 2s - loss: 0.0258 - accuracy: 0.9913
765/1688 [============>.................] - ETA: 2s - loss: 0.0261 - accuracy: 0.9914
786/1688 [============>.................] - ETA: 2s - loss: 0.0266 - accuracy: 0.9912
807/1688 [=============>................] - ETA: 2s - loss: 0.0264 - accuracy: 0.9912
828/1688 [=============>................] - ETA: 1s - loss: 0.0267 - accuracy: 0.9913
849/1688 [==============>...............] - ETA: 1s - loss: 0.0268 - accuracy: 0.9912
870/1688 [==============>...............] - ETA: 1s - loss: 0.0271 - accuracy: 0.9911
891/1688 [==============>...............] - ETA: 1s - loss: 0.0269 - accuracy: 0.9912
912/1688 [===============>..............] - ETA: 1s - loss: 0.0268 - accuracy: 0.9912
934/1688 [===============>..............] - ETA: 1s - loss: 0.0268 - accuracy: 0.9912
955/1688 [===============>..............] - ETA: 1s - loss: 0.0268 - accuracy: 0.9913
977/1688 [================>.............] - ETA: 1s - loss: 0.0273 - accuracy: 0.9912
998/1688 [================>.............] - ETA: 1s - loss: 0.0273 - accuracy: 0.9912
1019/1688 [=================>............] - ETA: 1s - loss: 0.0272 - accuracy: 0.9913
1040/1688 [=================>............] - ETA: 1s - loss: 0.0272 - accuracy: 0.9912
1061/1688 [=================>............] - ETA: 1s - loss: 0.0273 - accuracy: 0.9912
1083/1688 [==================>...........] - ETA: 1s - loss: 0.0269 - accuracy: 0.9913
1104/1688 [==================>...........] - ETA: 1s - loss: 0.0272 - accuracy: 0.9913
1125/1688 [==================>...........] - ETA: 1s - loss: 0.0273 - accuracy: 0.9912
1146/1688 [===================>..........] - ETA: 1s - loss: 0.0272 - accuracy: 0.9913
1168/1688 [===================>..........] - ETA: 1s - loss: 0.0270 - accuracy: 0.9913
1189/1688 [====================>.........] - ETA: 1s - loss: 0.0269 - accuracy: 0.9914
1210/1688 [====================>.........] - ETA: 1s - loss: 0.0267 - accuracy: 0.9914
1231/1688 [====================>.........] - ETA: 1s - loss: 0.0267 - accuracy: 0.9913
1253/1688 [=====================>........] - ETA: 1s - loss: 0.0269 - accuracy: 0.9913
1274/1688 [=====================>........] - ETA: 0s - loss: 0.0269 - accuracy: 0.9913
1295/1688 [======================>.......] - ETA: 0s - loss: 0.0268 - accuracy: 0.9914
1317/1688 [======================>.......] - ETA: 0s - loss: 0.0266 - accuracy: 0.9914
1338/1688 [======================>.......] - ETA: 0s - loss: 0.0265 - accuracy: 0.9915
1359/1688 [=======================>......] - ETA: 0s - loss: 0.0264 - accuracy: 0.9915
1380/1688 [=======================>......] - ETA: 0s - loss: 0.0272 - accuracy: 0.9914
1401/1688 [=======================>......] - ETA: 0s - loss: 0.0274 - accuracy: 0.9913
1422/1688 [========================>.....] - ETA: 0s - loss: 0.0274 - accuracy: 0.9913
1444/1688 [========================>.....] - ETA: 0s - loss: 0.0274 - accuracy: 0.9912
1466/1688 [=========================>....] - ETA: 0s - loss: 0.0278 - accuracy: 0.9911
1487/1688 [=========================>....] - ETA: 0s - loss: 0.0277 - accuracy: 0.9911
1508/1688 [=========================>....] - ETA: 0s - loss: 0.0277 - accuracy: 0.9911
1530/1688 [==========================>...] - ETA: 0s - loss: 0.0275 - accuracy: 0.9911
1551/1688 [==========================>...] - ETA: 0s - loss: 0.0275 - accuracy: 0.9911
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0274 - accuracy: 0.9911
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0275 - accuracy: 0.9910
1615/1688 [===========================>..] - ETA: 0s - loss: 0.0277 - accuracy: 0.9910
1636/1688 [============================>.] - ETA: 0s - loss: 0.0276 - accuracy: 0.9910
1657/1688 [============================>.] - ETA: 0s - loss: 0.0279 - accuracy: 0.9909
1678/1688 [============================>.] - ETA: 0s - loss: 0.0280 - accuracy: 0.9908
1688/1688 [==============================] - 4s 2ms/step - loss: 0.0279 - accuracy: 0.9909 - val_loss: 0.0548 - val_accuracy: 0.9863
Epoch 7/10
1/1688 [..............................] - ETA: 4s - loss: 0.0093 - accuracy: 1.0000
23/1688 [..............................] - ETA: 3s - loss: 0.0161 - accuracy: 0.9959
44/1688 [..............................] - ETA: 3s - loss: 0.0123 - accuracy: 0.9964
66/1688 [>.............................] - ETA: 3s - loss: 0.0221 - accuracy: 0.9938
87/1688 [>.............................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9943
108/1688 [>.............................] - ETA: 3s - loss: 0.0204 - accuracy: 0.9945
130/1688 [=>............................] - ETA: 3s - loss: 0.0202 - accuracy: 0.9947
151/1688 [=>............................] - ETA: 3s - loss: 0.0195 - accuracy: 0.9948
172/1688 [==>...........................] - ETA: 3s - loss: 0.0189 - accuracy: 0.9947
193/1688 [==>...........................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9947
214/1688 [==>...........................] - ETA: 3s - loss: 0.0181 - accuracy: 0.9949
235/1688 [===>..........................] - ETA: 3s - loss: 0.0178 - accuracy: 0.9948
256/1688 [===>..........................] - ETA: 3s - loss: 0.0181 - accuracy: 0.9946
277/1688 [===>..........................] - ETA: 3s - loss: 0.0178 - accuracy: 0.9947
298/1688 [====>.........................] - ETA: 3s - loss: 0.0178 - accuracy: 0.9945
319/1688 [====>.........................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9944
340/1688 [=====>........................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9941
361/1688 [=====>........................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9941
382/1688 [=====>........................] - ETA: 3s - loss: 0.0184 - accuracy: 0.9943
403/1688 [======>.......................] - ETA: 3s - loss: 0.0178 - accuracy: 0.9945
424/1688 [======>.......................] - ETA: 3s - loss: 0.0180 - accuracy: 0.9945
445/1688 [======>.......................] - ETA: 3s - loss: 0.0180 - accuracy: 0.9945
466/1688 [=======>......................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9943
487/1688 [=======>......................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9944
508/1688 [========>.....................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9944
529/1688 [========>.....................] - ETA: 2s - loss: 0.0178 - accuracy: 0.9943
550/1688 [========>.....................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9942
571/1688 [=========>....................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9942
592/1688 [=========>....................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9942
614/1688 [=========>....................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9942
636/1688 [==========>...................] - ETA: 2s - loss: 0.0175 - accuracy: 0.9943
657/1688 [==========>...................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9943
678/1688 [===========>..................] - ETA: 2s - loss: 0.0180 - accuracy: 0.9943
699/1688 [===========>..................] - ETA: 2s - loss: 0.0183 - accuracy: 0.9943
720/1688 [===========>..................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9942
742/1688 [============>.................] - ETA: 2s - loss: 0.0184 - accuracy: 0.9942
763/1688 [============>.................] - ETA: 2s - loss: 0.0182 - accuracy: 0.9943
785/1688 [============>.................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9943
806/1688 [=============>................] - ETA: 2s - loss: 0.0179 - accuracy: 0.9943
828/1688 [=============>................] - ETA: 2s - loss: 0.0181 - accuracy: 0.9942
849/1688 [==============>...............] - ETA: 2s - loss: 0.0182 - accuracy: 0.9942
870/1688 [==============>...............] - ETA: 1s - loss: 0.0181 - accuracy: 0.9941
891/1688 [==============>...............] - ETA: 1s - loss: 0.0179 - accuracy: 0.9942
912/1688 [===============>..............] - ETA: 1s - loss: 0.0179 - accuracy: 0.9941
933/1688 [===============>..............] - ETA: 1s - loss: 0.0180 - accuracy: 0.9941
954/1688 [===============>..............] - ETA: 1s - loss: 0.0187 - accuracy: 0.9939
975/1688 [================>.............] - ETA: 1s - loss: 0.0193 - accuracy: 0.9937
996/1688 [================>.............] - ETA: 1s - loss: 0.0196 - accuracy: 0.9935
1017/1688 [=================>............] - ETA: 1s - loss: 0.0199 - accuracy: 0.9934
1039/1688 [=================>............] - ETA: 1s - loss: 0.0203 - accuracy: 0.9933
1060/1688 [=================>............] - ETA: 1s - loss: 0.0203 - accuracy: 0.9932
1081/1688 [==================>...........] - ETA: 1s - loss: 0.0203 - accuracy: 0.9932
1102/1688 [==================>...........] - ETA: 1s - loss: 0.0204 - accuracy: 0.9932
1123/1688 [==================>...........] - ETA: 1s - loss: 0.0205 - accuracy: 0.9931
1144/1688 [===================>..........] - ETA: 1s - loss: 0.0205 - accuracy: 0.9931
1165/1688 [===================>..........] - ETA: 1s - loss: 0.0205 - accuracy: 0.9931
1186/1688 [====================>.........] - ETA: 1s - loss: 0.0207 - accuracy: 0.9930
1207/1688 [====================>.........] - ETA: 1s - loss: 0.0206 - accuracy: 0.9930
1229/1688 [====================>.........] - ETA: 1s - loss: 0.0206 - accuracy: 0.9930
1250/1688 [=====================>........] - ETA: 1s - loss: 0.0206 - accuracy: 0.9930
1271/1688 [=====================>........] - ETA: 1s - loss: 0.0207 - accuracy: 0.9929
1292/1688 [=====================>........] - ETA: 0s - loss: 0.0208 - accuracy: 0.9929
1313/1688 [======================>.......] - ETA: 0s - loss: 0.0207 - accuracy: 0.9930
1335/1688 [======================>.......] - ETA: 0s - loss: 0.0210 - accuracy: 0.9929
1357/1688 [=======================>......] - ETA: 0s - loss: 0.0211 - accuracy: 0.9928
1379/1688 [=======================>......] - ETA: 0s - loss: 0.0210 - accuracy: 0.9928
1401/1688 [=======================>......] - ETA: 0s - loss: 0.0212 - accuracy: 0.9928
1422/1688 [========================>.....] - ETA: 0s - loss: 0.0213 - accuracy: 0.9928
1444/1688 [========================>.....] - ETA: 0s - loss: 0.0216 - accuracy: 0.9926
1465/1688 [=========================>....] - ETA: 0s - loss: 0.0217 - accuracy: 0.9926
1486/1688 [=========================>....] - ETA: 0s - loss: 0.0221 - accuracy: 0.9925
1507/1688 [=========================>....] - ETA: 0s - loss: 0.0223 - accuracy: 0.9924
1528/1688 [==========================>...] - ETA: 0s - loss: 0.0224 - accuracy: 0.9924
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0225 - accuracy: 0.9924
1569/1688 [==========================>...] - ETA: 0s - loss: 0.0224 - accuracy: 0.9925
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0227 - accuracy: 0.9924
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0229 - accuracy: 0.9924
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0229 - accuracy: 0.9923
1649/1688 [============================>.] - ETA: 0s - loss: 0.0231 - accuracy: 0.9922
1669/1688 [============================>.] - ETA: 0s - loss: 0.0231 - accuracy: 0.9923
1688/1688 [==============================] - ETA: 0s - loss: 0.0233 - accuracy: 0.9922
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0233 - accuracy: 0.9922 - val_loss: 0.0593 - val_accuracy: 0.9842
Epoch 8/10
1/1688 [..............................] - ETA: 3s - loss: 0.0013 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0088 - accuracy: 0.9974
46/1688 [..............................] - ETA: 3s - loss: 0.0094 - accuracy: 0.9973
69/1688 [>.............................] - ETA: 3s - loss: 0.0101 - accuracy: 0.9968
91/1688 [>.............................] - ETA: 3s - loss: 0.0092 - accuracy: 0.9973
113/1688 [=>............................] - ETA: 3s - loss: 0.0093 - accuracy: 0.9972
136/1688 [=>............................] - ETA: 3s - loss: 0.0102 - accuracy: 0.9968
158/1688 [=>............................] - ETA: 3s - loss: 0.0111 - accuracy: 0.9964
181/1688 [==>...........................] - ETA: 3s - loss: 0.0122 - accuracy: 0.9964
204/1688 [==>...........................] - ETA: 3s - loss: 0.0124 - accuracy: 0.9966
227/1688 [===>..........................] - ETA: 3s - loss: 0.0125 - accuracy: 0.9964
250/1688 [===>..........................] - ETA: 3s - loss: 0.0139 - accuracy: 0.9958
273/1688 [===>..........................] - ETA: 3s - loss: 0.0142 - accuracy: 0.9958
296/1688 [====>.........................] - ETA: 3s - loss: 0.0141 - accuracy: 0.9959
318/1688 [====>.........................] - ETA: 3s - loss: 0.0157 - accuracy: 0.9958
341/1688 [=====>........................] - ETA: 3s - loss: 0.0172 - accuracy: 0.9951
364/1688 [=====>........................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9954
386/1688 [=====>........................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9954
408/1688 [======>.......................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9950
431/1688 [======>.......................] - ETA: 2s - loss: 0.0170 - accuracy: 0.9949
454/1688 [=======>......................] - ETA: 2s - loss: 0.0173 - accuracy: 0.9948
477/1688 [=======>......................] - ETA: 2s - loss: 0.0169 - accuracy: 0.9950
499/1688 [=======>......................] - ETA: 2s - loss: 0.0168 - accuracy: 0.9951
522/1688 [========>.....................] - ETA: 2s - loss: 0.0174 - accuracy: 0.9949
544/1688 [========>.....................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9948
567/1688 [=========>....................] - ETA: 2s - loss: 0.0177 - accuracy: 0.9948
589/1688 [=========>....................] - ETA: 2s - loss: 0.0190 - accuracy: 0.9945
612/1688 [=========>....................] - ETA: 2s - loss: 0.0194 - accuracy: 0.9943
635/1688 [==========>...................] - ETA: 2s - loss: 0.0196 - accuracy: 0.9942
658/1688 [==========>...................] - ETA: 2s - loss: 0.0195 - accuracy: 0.9943
680/1688 [===========>..................] - ETA: 2s - loss: 0.0193 - accuracy: 0.9943
702/1688 [===========>..................] - ETA: 2s - loss: 0.0194 - accuracy: 0.9942
723/1688 [===========>..................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9943
744/1688 [============>.................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9942
765/1688 [============>.................] - ETA: 2s - loss: 0.0193 - accuracy: 0.9941
786/1688 [============>.................] - ETA: 2s - loss: 0.0192 - accuracy: 0.9941
808/1688 [=============>................] - ETA: 2s - loss: 0.0191 - accuracy: 0.9941
829/1688 [=============>................] - ETA: 1s - loss: 0.0194 - accuracy: 0.9941
851/1688 [==============>...............] - ETA: 1s - loss: 0.0196 - accuracy: 0.9941
872/1688 [==============>...............] - ETA: 1s - loss: 0.0197 - accuracy: 0.9940
893/1688 [==============>...............] - ETA: 1s - loss: 0.0198 - accuracy: 0.9941
914/1688 [===============>..............] - ETA: 1s - loss: 0.0200 - accuracy: 0.9939
936/1688 [===============>..............] - ETA: 1s - loss: 0.0198 - accuracy: 0.9940
958/1688 [================>.............] - ETA: 1s - loss: 0.0198 - accuracy: 0.9939
980/1688 [================>.............] - ETA: 1s - loss: 0.0198 - accuracy: 0.9939
1001/1688 [================>.............] - ETA: 1s - loss: 0.0199 - accuracy: 0.9938
1022/1688 [=================>............] - ETA: 1s - loss: 0.0199 - accuracy: 0.9938
1043/1688 [=================>............] - ETA: 1s - loss: 0.0200 - accuracy: 0.9937
1063/1688 [=================>............] - ETA: 1s - loss: 0.0200 - accuracy: 0.9937
1084/1688 [==================>...........] - ETA: 1s - loss: 0.0199 - accuracy: 0.9938
1105/1688 [==================>...........] - ETA: 1s - loss: 0.0200 - accuracy: 0.9938
1126/1688 [===================>..........] - ETA: 1s - loss: 0.0198 - accuracy: 0.9938
1147/1688 [===================>..........] - ETA: 1s - loss: 0.0197 - accuracy: 0.9938
1168/1688 [===================>..........] - ETA: 1s - loss: 0.0198 - accuracy: 0.9938
1189/1688 [====================>.........] - ETA: 1s - loss: 0.0199 - accuracy: 0.9938
1209/1688 [====================>.........] - ETA: 1s - loss: 0.0198 - accuracy: 0.9938
1229/1688 [====================>.........] - ETA: 1s - loss: 0.0196 - accuracy: 0.9939
1249/1688 [=====================>........] - ETA: 1s - loss: 0.0196 - accuracy: 0.9939
1269/1688 [=====================>........] - ETA: 0s - loss: 0.0198 - accuracy: 0.9939
1289/1688 [=====================>........] - ETA: 0s - loss: 0.0199 - accuracy: 0.9938
1309/1688 [======================>.......] - ETA: 0s - loss: 0.0199 - accuracy: 0.9938
1329/1688 [======================>.......] - ETA: 0s - loss: 0.0200 - accuracy: 0.9937
1349/1688 [======================>.......] - ETA: 0s - loss: 0.0200 - accuracy: 0.9937
1369/1688 [=======================>......] - ETA: 0s - loss: 0.0202 - accuracy: 0.9937
1389/1688 [=======================>......] - ETA: 0s - loss: 0.0205 - accuracy: 0.9935
1409/1688 [========================>.....] - ETA: 0s - loss: 0.0203 - accuracy: 0.9936
1429/1688 [========================>.....] - ETA: 0s - loss: 0.0201 - accuracy: 0.9937
1449/1688 [========================>.....] - ETA: 0s - loss: 0.0201 - accuracy: 0.9937
1469/1688 [=========================>....] - ETA: 0s - loss: 0.0206 - accuracy: 0.9935
1489/1688 [=========================>....] - ETA: 0s - loss: 0.0206 - accuracy: 0.9935
1509/1688 [=========================>....] - ETA: 0s - loss: 0.0206 - accuracy: 0.9935
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0206 - accuracy: 0.9934
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0207 - accuracy: 0.9934
1569/1688 [==========================>...] - ETA: 0s - loss: 0.0206 - accuracy: 0.9934
1589/1688 [===========================>..] - ETA: 0s - loss: 0.0206 - accuracy: 0.9934
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0205 - accuracy: 0.9935
1629/1688 [===========================>..] - ETA: 0s - loss: 0.0204 - accuracy: 0.9934
1649/1688 [============================>.] - ETA: 0s - loss: 0.0204 - accuracy: 0.9934
1669/1688 [============================>.] - ETA: 0s - loss: 0.0204 - accuracy: 0.9934
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0204 - accuracy: 0.9934 - val_loss: 0.0539 - val_accuracy: 0.9872
Epoch 9/10
1/1688 [..............................] - ETA: 4s - loss: 0.0291 - accuracy: 0.9688
21/1688 [..............................] - ETA: 4s - loss: 0.0222 - accuracy: 0.9911
40/1688 [..............................] - ETA: 4s - loss: 0.0171 - accuracy: 0.9930
60/1688 [>.............................] - ETA: 4s - loss: 0.0175 - accuracy: 0.9948
80/1688 [>.............................] - ETA: 4s - loss: 0.0152 - accuracy: 0.9949
100/1688 [>.............................] - ETA: 4s - loss: 0.0160 - accuracy: 0.9947
120/1688 [=>............................] - ETA: 4s - loss: 0.0165 - accuracy: 0.9948
140/1688 [=>............................] - ETA: 3s - loss: 0.0160 - accuracy: 0.9951
160/1688 [=>............................] - ETA: 3s - loss: 0.0180 - accuracy: 0.9947
180/1688 [==>...........................] - ETA: 3s - loss: 0.0187 - accuracy: 0.9941
200/1688 [==>...........................] - ETA: 3s - loss: 0.0179 - accuracy: 0.9945
220/1688 [==>...........................] - ETA: 3s - loss: 0.0175 - accuracy: 0.9947
240/1688 [===>..........................] - ETA: 3s - loss: 0.0182 - accuracy: 0.9947
260/1688 [===>..........................] - ETA: 3s - loss: 0.0176 - accuracy: 0.9946
280/1688 [===>..........................] - ETA: 3s - loss: 0.0175 - accuracy: 0.9948
300/1688 [====>.........................] - ETA: 3s - loss: 0.0166 - accuracy: 0.9951
320/1688 [====>.........................] - ETA: 3s - loss: 0.0164 - accuracy: 0.9952
340/1688 [=====>........................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9951
360/1688 [=====>........................] - ETA: 3s - loss: 0.0177 - accuracy: 0.9946
380/1688 [=====>........................] - ETA: 3s - loss: 0.0171 - accuracy: 0.9949
400/1688 [======>.......................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9950
419/1688 [======>.......................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9950
439/1688 [======>.......................] - ETA: 3s - loss: 0.0163 - accuracy: 0.9952
459/1688 [=======>......................] - ETA: 3s - loss: 0.0161 - accuracy: 0.9951
479/1688 [=======>......................] - ETA: 3s - loss: 0.0160 - accuracy: 0.9951
499/1688 [=======>......................] - ETA: 3s - loss: 0.0161 - accuracy: 0.9951
519/1688 [========>.....................] - ETA: 3s - loss: 0.0161 - accuracy: 0.9951
539/1688 [========>.....................] - ETA: 2s - loss: 0.0158 - accuracy: 0.9952
559/1688 [========>.....................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9951
579/1688 [=========>....................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9951
599/1688 [=========>....................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9951
619/1688 [==========>...................] - ETA: 2s - loss: 0.0163 - accuracy: 0.9951
639/1688 [==========>...................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9951
659/1688 [==========>...................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9951
679/1688 [===========>..................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9952
699/1688 [===========>..................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9953
719/1688 [===========>..................] - ETA: 2s - loss: 0.0159 - accuracy: 0.9953
739/1688 [============>.................] - ETA: 2s - loss: 0.0157 - accuracy: 0.9953
759/1688 [============>.................] - ETA: 2s - loss: 0.0161 - accuracy: 0.9951
779/1688 [============>.................] - ETA: 2s - loss: 0.0160 - accuracy: 0.9950
799/1688 [=============>................] - ETA: 2s - loss: 0.0162 - accuracy: 0.9949
819/1688 [=============>................] - ETA: 2s - loss: 0.0166 - accuracy: 0.9947
839/1688 [=============>................] - ETA: 2s - loss: 0.0165 - accuracy: 0.9947
859/1688 [==============>...............] - ETA: 2s - loss: 0.0164 - accuracy: 0.9947
879/1688 [==============>...............] - ETA: 2s - loss: 0.0165 - accuracy: 0.9947
899/1688 [==============>...............] - ETA: 2s - loss: 0.0168 - accuracy: 0.9946
919/1688 [===============>..............] - ETA: 1s - loss: 0.0170 - accuracy: 0.9945
939/1688 [===============>..............] - ETA: 1s - loss: 0.0169 - accuracy: 0.9946
959/1688 [================>.............] - ETA: 1s - loss: 0.0173 - accuracy: 0.9944
979/1688 [================>.............] - ETA: 1s - loss: 0.0173 - accuracy: 0.9943
999/1688 [================>.............] - ETA: 1s - loss: 0.0172 - accuracy: 0.9943
1019/1688 [=================>............] - ETA: 1s - loss: 0.0170 - accuracy: 0.9944
1039/1688 [=================>............] - ETA: 1s - loss: 0.0170 - accuracy: 0.9944
1059/1688 [=================>............] - ETA: 1s - loss: 0.0171 - accuracy: 0.9943
1079/1688 [==================>...........] - ETA: 1s - loss: 0.0171 - accuracy: 0.9943
1099/1688 [==================>...........] - ETA: 1s - loss: 0.0173 - accuracy: 0.9942
1119/1688 [==================>...........] - ETA: 1s - loss: 0.0172 - accuracy: 0.9942
1138/1688 [===================>..........] - ETA: 1s - loss: 0.0173 - accuracy: 0.9942
1158/1688 [===================>..........] - ETA: 1s - loss: 0.0172 - accuracy: 0.9943
1178/1688 [===================>..........] - ETA: 1s - loss: 0.0171 - accuracy: 0.9943
1198/1688 [====================>.........] - ETA: 1s - loss: 0.0169 - accuracy: 0.9943
1218/1688 [====================>.........] - ETA: 1s - loss: 0.0171 - accuracy: 0.9943
1238/1688 [=====================>........] - ETA: 1s - loss: 0.0172 - accuracy: 0.9943
1258/1688 [=====================>........] - ETA: 1s - loss: 0.0173 - accuracy: 0.9943
1279/1688 [=====================>........] - ETA: 1s - loss: 0.0172 - accuracy: 0.9943
1300/1688 [======================>.......] - ETA: 1s - loss: 0.0171 - accuracy: 0.9943
1321/1688 [======================>.......] - ETA: 0s - loss: 0.0171 - accuracy: 0.9942
1343/1688 [======================>.......] - ETA: 0s - loss: 0.0171 - accuracy: 0.9943
1365/1688 [=======================>......] - ETA: 0s - loss: 0.0171 - accuracy: 0.9943
1387/1688 [=======================>......] - ETA: 0s - loss: 0.0175 - accuracy: 0.9942
1409/1688 [========================>.....] - ETA: 0s - loss: 0.0177 - accuracy: 0.9941
1431/1688 [========================>.....] - ETA: 0s - loss: 0.0177 - accuracy: 0.9941
1452/1688 [========================>.....] - ETA: 0s - loss: 0.0178 - accuracy: 0.9941
1474/1688 [=========================>....] - ETA: 0s - loss: 0.0178 - accuracy: 0.9941
1495/1688 [=========================>....] - ETA: 0s - loss: 0.0180 - accuracy: 0.9941
1517/1688 [=========================>....] - ETA: 0s - loss: 0.0179 - accuracy: 0.9941
1538/1688 [==========================>...] - ETA: 0s - loss: 0.0178 - accuracy: 0.9942
1560/1688 [==========================>...] - ETA: 0s - loss: 0.0178 - accuracy: 0.9942
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0180 - accuracy: 0.9942
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0181 - accuracy: 0.9942
1626/1688 [===========================>..] - ETA: 0s - loss: 0.0182 - accuracy: 0.9941
1648/1688 [============================>.] - ETA: 0s - loss: 0.0182 - accuracy: 0.9941
1670/1688 [============================>.] - ETA: 0s - loss: 0.0182 - accuracy: 0.9941
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0182 - accuracy: 0.9941 - val_loss: 0.0583 - val_accuracy: 0.9882
Epoch 10/10
1/1688 [..............................] - ETA: 3s - loss: 0.0068 - accuracy: 1.0000
24/1688 [..............................] - ETA: 3s - loss: 0.0046 - accuracy: 0.9987
47/1688 [..............................] - ETA: 3s - loss: 0.0083 - accuracy: 0.9973
70/1688 [>.............................] - ETA: 3s - loss: 0.0097 - accuracy: 0.9969
93/1688 [>.............................] - ETA: 3s - loss: 0.0136 - accuracy: 0.9960
116/1688 [=>............................] - ETA: 3s - loss: 0.0126 - accuracy: 0.9965
139/1688 [=>............................] - ETA: 3s - loss: 0.0146 - accuracy: 0.9962
162/1688 [=>............................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9952
184/1688 [==>...........................] - ETA: 3s - loss: 0.0162 - accuracy: 0.9954
207/1688 [==>...........................] - ETA: 3s - loss: 0.0162 - accuracy: 0.9952
230/1688 [===>..........................] - ETA: 3s - loss: 0.0166 - accuracy: 0.9951
253/1688 [===>..........................] - ETA: 3s - loss: 0.0175 - accuracy: 0.9948
275/1688 [===>..........................] - ETA: 3s - loss: 0.0168 - accuracy: 0.9950
298/1688 [====>.........................] - ETA: 3s - loss: 0.0165 - accuracy: 0.9951
320/1688 [====>.........................] - ETA: 3s - loss: 0.0158 - accuracy: 0.9953
343/1688 [=====>........................] - ETA: 3s - loss: 0.0153 - accuracy: 0.9954
366/1688 [=====>........................] - ETA: 2s - loss: 0.0147 - accuracy: 0.9956
389/1688 [=====>........................] - ETA: 2s - loss: 0.0146 - accuracy: 0.9957
412/1688 [======>.......................] - ETA: 2s - loss: 0.0140 - accuracy: 0.9958
435/1688 [======>.......................] - ETA: 2s - loss: 0.0135 - accuracy: 0.9960
456/1688 [=======>......................] - ETA: 2s - loss: 0.0134 - accuracy: 0.9960
477/1688 [=======>......................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9961
498/1688 [=======>......................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9960
519/1688 [========>.....................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9961
540/1688 [========>.....................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9961
562/1688 [========>.....................] - ETA: 2s - loss: 0.0131 - accuracy: 0.9961
583/1688 [=========>....................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9961
604/1688 [=========>....................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9962
625/1688 [==========>...................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9962
646/1688 [==========>...................] - ETA: 2s - loss: 0.0128 - accuracy: 0.9963
667/1688 [==========>...................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9963
688/1688 [===========>..................] - ETA: 2s - loss: 0.0127 - accuracy: 0.9962
709/1688 [===========>..................] - ETA: 2s - loss: 0.0131 - accuracy: 0.9961
731/1688 [===========>..................] - ETA: 2s - loss: 0.0130 - accuracy: 0.9961
752/1688 [============>.................] - ETA: 2s - loss: 0.0129 - accuracy: 0.9962
773/1688 [============>.................] - ETA: 2s - loss: 0.0131 - accuracy: 0.9961
794/1688 [=============>................] - ETA: 2s - loss: 0.0131 - accuracy: 0.9960
815/1688 [=============>................] - ETA: 2s - loss: 0.0132 - accuracy: 0.9959
836/1688 [=============>................] - ETA: 1s - loss: 0.0135 - accuracy: 0.9957
858/1688 [==============>...............] - ETA: 1s - loss: 0.0134 - accuracy: 0.9957
879/1688 [==============>...............] - ETA: 1s - loss: 0.0134 - accuracy: 0.9957
900/1688 [==============>...............] - ETA: 1s - loss: 0.0137 - accuracy: 0.9956
921/1688 [===============>..............] - ETA: 1s - loss: 0.0136 - accuracy: 0.9956
942/1688 [===============>..............] - ETA: 1s - loss: 0.0138 - accuracy: 0.9956
963/1688 [================>.............] - ETA: 1s - loss: 0.0136 - accuracy: 0.9957
984/1688 [================>.............] - ETA: 1s - loss: 0.0136 - accuracy: 0.9956
1005/1688 [================>.............] - ETA: 1s - loss: 0.0137 - accuracy: 0.9955
1026/1688 [=================>............] - ETA: 1s - loss: 0.0137 - accuracy: 0.9955
1047/1688 [=================>............] - ETA: 1s - loss: 0.0139 - accuracy: 0.9953
1069/1688 [=================>............] - ETA: 1s - loss: 0.0141 - accuracy: 0.9952
1090/1688 [==================>...........] - ETA: 1s - loss: 0.0143 - accuracy: 0.9952
1111/1688 [==================>...........] - ETA: 1s - loss: 0.0144 - accuracy: 0.9952
1132/1688 [===================>..........] - ETA: 1s - loss: 0.0144 - accuracy: 0.9953
1153/1688 [===================>..........] - ETA: 1s - loss: 0.0146 - accuracy: 0.9952
1174/1688 [===================>..........] - ETA: 1s - loss: 0.0146 - accuracy: 0.9952
1195/1688 [====================>.........] - ETA: 1s - loss: 0.0146 - accuracy: 0.9952
1216/1688 [====================>.........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9951
1237/1688 [====================>.........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9951
1258/1688 [=====================>........] - ETA: 1s - loss: 0.0147 - accuracy: 0.9951
1280/1688 [=====================>........] - ETA: 0s - loss: 0.0145 - accuracy: 0.9951
1301/1688 [======================>.......] - ETA: 0s - loss: 0.0144 - accuracy: 0.9952
1322/1688 [======================>.......] - ETA: 0s - loss: 0.0146 - accuracy: 0.9951
1343/1688 [======================>.......] - ETA: 0s - loss: 0.0147 - accuracy: 0.9951
1364/1688 [=======================>......] - ETA: 0s - loss: 0.0149 - accuracy: 0.9951
1386/1688 [=======================>......] - ETA: 0s - loss: 0.0148 - accuracy: 0.9951
1407/1688 [========================>.....] - ETA: 0s - loss: 0.0148 - accuracy: 0.9950
1428/1688 [========================>.....] - ETA: 0s - loss: 0.0151 - accuracy: 0.9950
1449/1688 [========================>.....] - ETA: 0s - loss: 0.0155 - accuracy: 0.9948
1471/1688 [=========================>....] - ETA: 0s - loss: 0.0156 - accuracy: 0.9948
1492/1688 [=========================>....] - ETA: 0s - loss: 0.0155 - accuracy: 0.9948
1513/1688 [=========================>....] - ETA: 0s - loss: 0.0156 - accuracy: 0.9947
1534/1688 [==========================>...] - ETA: 0s - loss: 0.0156 - accuracy: 0.9948
1555/1688 [==========================>...] - ETA: 0s - loss: 0.0157 - accuracy: 0.9948
1576/1688 [===========================>..] - ETA: 0s - loss: 0.0159 - accuracy: 0.9947
1597/1688 [===========================>..] - ETA: 0s - loss: 0.0159 - accuracy: 0.9947
1617/1688 [===========================>..] - ETA: 0s - loss: 0.0160 - accuracy: 0.9947
1637/1688 [============================>.] - ETA: 0s - loss: 0.0159 - accuracy: 0.9947
1657/1688 [============================>.] - ETA: 0s - loss: 0.0160 - accuracy: 0.9947
1676/1688 [============================>.] - ETA: 0s - loss: 0.0159 - accuracy: 0.9947
1688/1688 [==============================] - 4s 3ms/step - loss: 0.0161 - accuracy: 0.9946 - val_loss: 0.0684 - val_accuracy: 0.9842
score = model_keras.evaluate(x_test, y_test, verbose=0)
print('Test accuracy:', score[1])
Test accuracy: 0.9842000007629395
2. Quantize
2.1. 8-bit quantization
An Akida accelerator processes 8 or 4-bits integer activations and weights. Therefore, the floating point Keras model must be quantized in preparation to run on an Akida accelerator.
The QuantizeML quantize function can be used to quantize a Keras model for Akida. For this step in this example, an “8/8/8” quantization scheme will be applied to the floating point Keras model to produce 8-bit weights in the first layer, 8-bit weights in all other layers, and 8-bit activations.
The quantization process results in a Keras model with custom QuantizeML quantized layers substituted for the original Keras layers.
All Keras API functions can be applied on this new model: summary(), compile(), fit(). etc.
Note
The quantize function applies several transformations to
the original model. For example, it folds the batch normalization layers into the
corresponding neural layers. The new weights are computed according to this folding
operation.
from quantizeml.models import quantize, QuantizationParams
qparams = QuantizationParams(input_weight_bits=8, weight_bits=8, activation_bits=8)
model_quantized = quantize(model_keras, qparams=qparams)
model_quantized.summary()
/usr/local/lib/python3.11/dist-packages/quantizeml/models/quantize.py:454: UserWarning: Quantizing per-axis with random calibration samples is not accurate. Set QuantizationParams.per_tensor_activations=True when calibrating with random samples.
warnings.warn("Quantizing per-axis with random calibration samples is not accurate.\
1/1024 [..............................] - ETA: 3:22
57/1024 [>.............................] - ETA: 0s
114/1024 [==>...........................] - ETA: 0s
171/1024 [====>.........................] - ETA: 0s
228/1024 [=====>........................] - ETA: 0s
286/1024 [=======>......................] - ETA: 0s
343/1024 [=========>....................] - ETA: 0s
399/1024 [==========>...................] - ETA: 0s
456/1024 [============>.................] - ETA: 0s
513/1024 [==============>...............] - ETA: 0s
570/1024 [===============>..............] - ETA: 0s
627/1024 [=================>............] - ETA: 0s
685/1024 [===================>..........] - ETA: 0s
743/1024 [====================>.........] - ETA: 0s
800/1024 [======================>.......] - ETA: 0s
858/1024 [========================>.....] - ETA: 0s
914/1024 [=========================>....] - ETA: 0s
971/1024 [===========================>..] - ETA: 0s
1024/1024 [==============================] - 1s 883us/step
Model: "mnistnet"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
rescaling_input (InputLaye [(None, 28, 28, 1)] 0
r)
rescaling (QuantizedRescal (None, 28, 28, 1) 0
ing)
conv2d (QuantizedConv2D) (None, 13, 13, 32) 320
re_lu (QuantizedReLU) (None, 13, 13, 32) 64
depthwise_conv2d (Quantize (None, 7, 7, 32) 384
dDepthwiseConv2D)
conv2d_1 (QuantizedConv2D) (None, 7, 7, 64) 2112
re_lu_1 (QuantizedReLU) (None, 7, 7, 64) 128
flatten (QuantizedFlatten) (None, 3136) 0
dense (QuantizedDense) (None, 10) 31370
dequantizer (Dequantizer) (None, 10) 0
=================================================================
Total params: 34378 (134.29 KB)
Trainable params: 34122 (133.29 KB)
Non-trainable params: 256 (1.00 KB)
_________________________________________________________________
Note
Note that the number of parameters for the floating and quantized models differs, a consequence of the BatchNormalization folding and the additional parameters added for quantization. For further details, please refer to their respective summary.
Check the quantized model accuracy.
def compile_evaluate(model):
""" Compiles and evaluates the model, then return accuracy score. """
model.compile(metrics=['accuracy'])
return model.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after 8-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 8-bit quantization: 0.9782000184059143
2.2. Effect of calibration
The previous call to quantize was made with random samples for calibration
(default parameters). While the observed drop in accuracy is minimal, that is
around 1%, it can be worse on more complex models. Therefore, it is advised to
use a set of real samples from the training set for calibration during a call
to quantize.
Note that this remains a calibration step rather than a training step in that
no output labels are required. Furthermore, any relevant data could be used for
calibration. The recommended settings for calibration that are widely used to
obtain the zoo performance are:
1024 samples
a batch size of 100
2 epochs
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 987us/step
Check the accuracy for the quantized and calibrated model.
print('Test accuracy after calibration:', compile_evaluate(model_quantized))
Test accuracy after calibration: 0.9829000234603882
Calibrating with real samples on this model recovers the initial float accuracy.
2.3. 4-bit quantization
The accuracy of the 8/8/8 quantized model is equal to that of the Keras floating point model. In some cases, a smaller memory size for the model is required. This can be accomplished through quantization of the model to smaller bitwidths.
The model will now be quantized to 8/4/4, that is 8-bit weights in the first layer with 4-bit weights and activations in all other layers. Such a quantization scheme will usually introduce a performance drop.
qparams = QuantizationParams(input_weight_bits=8, weight_bits=4, activation_bits=4)
model_quantized = quantize(model_keras, qparams=qparams,
samples=x_train, num_samples=1024, batch_size=100, epochs=2)
1/11 [=>............................] - ETA: 1s
11/11 [==============================] - 0s 1ms/step
1/11 [=>............................] - ETA: 0s
11/11 [==============================] - 0s 1ms/step
Check the 4-bit quantized accuracy.
print('Test accuracy after 4-bit quantization:', compile_evaluate(model_quantized))
Test accuracy after 4-bit quantization: 0.9835000038146973
2.4. Model fine tuning (Quantization Aware Training)
When a model suffers from an accuracy drop after quantization, fine tuning or Quantization Aware Training (QAT) may recover some or all of the original performance.
Note that since this is a fine tuning step, both the number of epochs and learning rate are expected to be lower than during the initial float training.
model_quantized.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=Adam(learning_rate=1e-4),
metrics=['accuracy'])
model_quantized.fit(x_train, y_train, epochs=5, validation_split=0.1)
Epoch 1/5
1/1688 [..............................] - ETA: 2:02:05 - loss: 0.0125 - accuracy: 1.0000
11/1688 [..............................] - ETA: 8s - loss: 0.0202 - accuracy: 0.9943
21/1688 [..............................] - ETA: 8s - loss: 0.0163 - accuracy: 0.9955
32/1688 [..............................] - ETA: 8s - loss: 0.0156 - accuracy: 0.9961
42/1688 [..............................] - ETA: 8s - loss: 0.0131 - accuracy: 0.9970
52/1688 [..............................] - ETA: 8s - loss: 0.0117 - accuracy: 0.9976
62/1688 [>.............................] - ETA: 8s - loss: 0.0135 - accuracy: 0.9965
71/1688 [>.............................] - ETA: 8s - loss: 0.0129 - accuracy: 0.9965
81/1688 [>.............................] - ETA: 8s - loss: 0.0153 - accuracy: 0.9958
91/1688 [>.............................] - ETA: 8s - loss: 0.0149 - accuracy: 0.9955
101/1688 [>.............................] - ETA: 8s - loss: 0.0142 - accuracy: 0.9960
111/1688 [>.............................] - ETA: 8s - loss: 0.0139 - accuracy: 0.9961
121/1688 [=>............................] - ETA: 7s - loss: 0.0133 - accuracy: 0.9964
131/1688 [=>............................] - ETA: 7s - loss: 0.0130 - accuracy: 0.9962
142/1688 [=>............................] - ETA: 7s - loss: 0.0130 - accuracy: 0.9960
152/1688 [=>............................] - ETA: 7s - loss: 0.0128 - accuracy: 0.9961
162/1688 [=>............................] - ETA: 7s - loss: 0.0124 - accuracy: 0.9963
172/1688 [==>...........................] - ETA: 7s - loss: 0.0123 - accuracy: 0.9964
183/1688 [==>...........................] - ETA: 7s - loss: 0.0133 - accuracy: 0.9959
193/1688 [==>...........................] - ETA: 7s - loss: 0.0129 - accuracy: 0.9960
203/1688 [==>...........................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9962
214/1688 [==>...........................] - ETA: 7s - loss: 0.0131 - accuracy: 0.9956
225/1688 [==>...........................] - ETA: 7s - loss: 0.0126 - accuracy: 0.9958
235/1688 [===>..........................] - ETA: 7s - loss: 0.0124 - accuracy: 0.9960
245/1688 [===>..........................] - ETA: 7s - loss: 0.0128 - accuracy: 0.9959
255/1688 [===>..........................] - ETA: 7s - loss: 0.0128 - accuracy: 0.9958
265/1688 [===>..........................] - ETA: 7s - loss: 0.0125 - accuracy: 0.9959
275/1688 [===>..........................] - ETA: 7s - loss: 0.0124 - accuracy: 0.9960
285/1688 [====>.........................] - ETA: 7s - loss: 0.0123 - accuracy: 0.9961
295/1688 [====>.........................] - ETA: 7s - loss: 0.0123 - accuracy: 0.9960
305/1688 [====>.........................] - ETA: 6s - loss: 0.0124 - accuracy: 0.9959
316/1688 [====>.........................] - ETA: 6s - loss: 0.0123 - accuracy: 0.9959
326/1688 [====>.........................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9960
336/1688 [====>.........................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9960
347/1688 [=====>........................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9959
357/1688 [=====>........................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9960
367/1688 [=====>........................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9960
377/1688 [=====>........................] - ETA: 6s - loss: 0.0122 - accuracy: 0.9959
387/1688 [=====>........................] - ETA: 6s - loss: 0.0119 - accuracy: 0.9960
397/1688 [======>.......................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9960
407/1688 [======>.......................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9959
417/1688 [======>.......................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9960
427/1688 [======>.......................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9959
438/1688 [======>.......................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9959
449/1688 [======>.......................] - ETA: 6s - loss: 0.0121 - accuracy: 0.9958
459/1688 [=======>......................] - ETA: 6s - loss: 0.0120 - accuracy: 0.9958
470/1688 [=======>......................] - ETA: 6s - loss: 0.0118 - accuracy: 0.9959
480/1688 [=======>......................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9959
490/1688 [=======>......................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9959
500/1688 [=======>......................] - ETA: 6s - loss: 0.0117 - accuracy: 0.9959
511/1688 [========>.....................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9958
522/1688 [========>.....................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9957
533/1688 [========>.....................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9957
543/1688 [========>.....................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9957
554/1688 [========>.....................] - ETA: 5s - loss: 0.0121 - accuracy: 0.9957
564/1688 [=========>....................] - ETA: 5s - loss: 0.0123 - accuracy: 0.9956
574/1688 [=========>....................] - ETA: 5s - loss: 0.0121 - accuracy: 0.9957
584/1688 [=========>....................] - ETA: 5s - loss: 0.0120 - accuracy: 0.9958
594/1688 [=========>....................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9958
604/1688 [=========>....................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9958
614/1688 [=========>....................] - ETA: 5s - loss: 0.0119 - accuracy: 0.9958
624/1688 [==========>...................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9958
634/1688 [==========>...................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9959
644/1688 [==========>...................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9960
655/1688 [==========>...................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9960
665/1688 [==========>...................] - ETA: 5s - loss: 0.0118 - accuracy: 0.9961
676/1688 [===========>..................] - ETA: 5s - loss: 0.0117 - accuracy: 0.9961
686/1688 [===========>..................] - ETA: 5s - loss: 0.0116 - accuracy: 0.9961
697/1688 [===========>..................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9961
708/1688 [===========>..................] - ETA: 4s - loss: 0.0116 - accuracy: 0.9961
718/1688 [===========>..................] - ETA: 4s - loss: 0.0114 - accuracy: 0.9961
728/1688 [===========>..................] - ETA: 4s - loss: 0.0118 - accuracy: 0.9961
738/1688 [============>.................] - ETA: 4s - loss: 0.0119 - accuracy: 0.9960
748/1688 [============>.................] - ETA: 4s - loss: 0.0119 - accuracy: 0.9960
758/1688 [============>.................] - ETA: 4s - loss: 0.0118 - accuracy: 0.9960
769/1688 [============>.................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9961
779/1688 [============>.................] - ETA: 4s - loss: 0.0117 - accuracy: 0.9961
790/1688 [=============>................] - ETA: 4s - loss: 0.0119 - accuracy: 0.9960
801/1688 [=============>................] - ETA: 4s - loss: 0.0118 - accuracy: 0.9960
811/1688 [=============>................] - ETA: 4s - loss: 0.0119 - accuracy: 0.9959
822/1688 [=============>................] - ETA: 4s - loss: 0.0119 - accuracy: 0.9959
833/1688 [=============>................] - ETA: 4s - loss: 0.0119 - accuracy: 0.9959
843/1688 [=============>................] - ETA: 4s - loss: 0.0121 - accuracy: 0.9958
853/1688 [==============>...............] - ETA: 4s - loss: 0.0122 - accuracy: 0.9957
863/1688 [==============>...............] - ETA: 4s - loss: 0.0122 - accuracy: 0.9957
874/1688 [==============>...............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9957
884/1688 [==============>...............] - ETA: 4s - loss: 0.0121 - accuracy: 0.9958
894/1688 [==============>...............] - ETA: 3s - loss: 0.0121 - accuracy: 0.9957
905/1688 [===============>..............] - ETA: 3s - loss: 0.0121 - accuracy: 0.9958
915/1688 [===============>..............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9958
925/1688 [===============>..............] - ETA: 3s - loss: 0.0120 - accuracy: 0.9958
936/1688 [===============>..............] - ETA: 3s - loss: 0.0119 - accuracy: 0.9958
946/1688 [===============>..............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9958
956/1688 [===============>..............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9958
966/1688 [================>.............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9959
976/1688 [================>.............] - ETA: 3s - loss: 0.0118 - accuracy: 0.9958
986/1688 [================>.............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9959
997/1688 [================>.............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9959
1007/1688 [================>.............] - ETA: 3s - loss: 0.0117 - accuracy: 0.9959
1017/1688 [=================>............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9959
1028/1688 [=================>............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9959
1039/1688 [=================>............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9959
1049/1688 [=================>............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9959
1060/1688 [=================>............] - ETA: 3s - loss: 0.0116 - accuracy: 0.9959
1070/1688 [==================>...........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9959
1080/1688 [==================>...........] - ETA: 3s - loss: 0.0115 - accuracy: 0.9959
1090/1688 [==================>...........] - ETA: 3s - loss: 0.0116 - accuracy: 0.9959
1100/1688 [==================>...........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9959
1111/1688 [==================>...........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9959
1122/1688 [==================>...........] - ETA: 2s - loss: 0.0115 - accuracy: 0.9959
1132/1688 [===================>..........] - ETA: 2s - loss: 0.0115 - accuracy: 0.9960
1142/1688 [===================>..........] - ETA: 2s - loss: 0.0114 - accuracy: 0.9960
1152/1688 [===================>..........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9959
1162/1688 [===================>..........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9959
1172/1688 [===================>..........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9959
1182/1688 [====================>.........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9959
1192/1688 [====================>.........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9959
1202/1688 [====================>.........] - ETA: 2s - loss: 0.0116 - accuracy: 0.9959
1212/1688 [====================>.........] - ETA: 2s - loss: 0.0115 - accuracy: 0.9959
1222/1688 [====================>.........] - ETA: 2s - loss: 0.0115 - accuracy: 0.9959
1233/1688 [====================>.........] - ETA: 2s - loss: 0.0114 - accuracy: 0.9960
1243/1688 [=====================>........] - ETA: 2s - loss: 0.0114 - accuracy: 0.9960
1253/1688 [=====================>........] - ETA: 2s - loss: 0.0113 - accuracy: 0.9960
1264/1688 [=====================>........] - ETA: 2s - loss: 0.0113 - accuracy: 0.9960
1274/1688 [=====================>........] - ETA: 2s - loss: 0.0113 - accuracy: 0.9960
1284/1688 [=====================>........] - ETA: 2s - loss: 0.0114 - accuracy: 0.9960
1294/1688 [=====================>........] - ETA: 1s - loss: 0.0113 - accuracy: 0.9960
1304/1688 [======================>.......] - ETA: 1s - loss: 0.0113 - accuracy: 0.9960
1315/1688 [======================>.......] - ETA: 1s - loss: 0.0112 - accuracy: 0.9961
1325/1688 [======================>.......] - ETA: 1s - loss: 0.0113 - accuracy: 0.9960
1335/1688 [======================>.......] - ETA: 1s - loss: 0.0113 - accuracy: 0.9961
1345/1688 [======================>.......] - ETA: 1s - loss: 0.0112 - accuracy: 0.9961
1355/1688 [=======================>......] - ETA: 1s - loss: 0.0112 - accuracy: 0.9961
1365/1688 [=======================>......] - ETA: 1s - loss: 0.0111 - accuracy: 0.9962
1375/1688 [=======================>......] - ETA: 1s - loss: 0.0111 - accuracy: 0.9962
1385/1688 [=======================>......] - ETA: 1s - loss: 0.0110 - accuracy: 0.9962
1395/1688 [=======================>......] - ETA: 1s - loss: 0.0110 - accuracy: 0.9962
1405/1688 [=======================>......] - ETA: 1s - loss: 0.0110 - accuracy: 0.9962
1415/1688 [========================>.....] - ETA: 1s - loss: 0.0110 - accuracy: 0.9962
1425/1688 [========================>.....] - ETA: 1s - loss: 0.0110 - accuracy: 0.9962
1435/1688 [========================>.....] - ETA: 1s - loss: 0.0110 - accuracy: 0.9962
1445/1688 [========================>.....] - ETA: 1s - loss: 0.0110 - accuracy: 0.9962
1455/1688 [========================>.....] - ETA: 1s - loss: 0.0110 - accuracy: 0.9962
1465/1688 [=========================>....] - ETA: 1s - loss: 0.0109 - accuracy: 0.9962
1476/1688 [=========================>....] - ETA: 1s - loss: 0.0109 - accuracy: 0.9962
1486/1688 [=========================>....] - ETA: 1s - loss: 0.0108 - accuracy: 0.9962
1496/1688 [=========================>....] - ETA: 0s - loss: 0.0108 - accuracy: 0.9962
1506/1688 [=========================>....] - ETA: 0s - loss: 0.0108 - accuracy: 0.9962
1516/1688 [=========================>....] - ETA: 0s - loss: 0.0108 - accuracy: 0.9962
1526/1688 [==========================>...] - ETA: 0s - loss: 0.0108 - accuracy: 0.9963
1536/1688 [==========================>...] - ETA: 0s - loss: 0.0107 - accuracy: 0.9963
1547/1688 [==========================>...] - ETA: 0s - loss: 0.0107 - accuracy: 0.9963
1557/1688 [==========================>...] - ETA: 0s - loss: 0.0107 - accuracy: 0.9963
1567/1688 [==========================>...] - ETA: 0s - loss: 0.0107 - accuracy: 0.9963
1577/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9963
1588/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9963
1599/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9963
1609/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9963
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0107 - accuracy: 0.9964
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0106 - accuracy: 0.9964
1641/1688 [============================>.] - ETA: 0s - loss: 0.0107 - accuracy: 0.9964
1651/1688 [============================>.] - ETA: 0s - loss: 0.0106 - accuracy: 0.9964
1661/1688 [============================>.] - ETA: 0s - loss: 0.0107 - accuracy: 0.9964
1672/1688 [============================>.] - ETA: 0s - loss: 0.0108 - accuracy: 0.9964
1682/1688 [============================>.] - ETA: 0s - loss: 0.0108 - accuracy: 0.9964
1688/1688 [==============================] - 15s 6ms/step - loss: 0.0108 - accuracy: 0.9964 - val_loss: 0.0585 - val_accuracy: 0.9862
Epoch 2/5
1/1688 [..............................] - ETA: 8s - loss: 0.0023 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0025 - accuracy: 1.0000
23/1688 [..............................] - ETA: 8s - loss: 0.0027 - accuracy: 1.0000
33/1688 [..............................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9991
43/1688 [..............................] - ETA: 8s - loss: 0.0058 - accuracy: 0.9978
53/1688 [..............................] - ETA: 8s - loss: 0.0066 - accuracy: 0.9971
63/1688 [>.............................] - ETA: 8s - loss: 0.0065 - accuracy: 0.9970
73/1688 [>.............................] - ETA: 8s - loss: 0.0066 - accuracy: 0.9974
83/1688 [>.............................] - ETA: 8s - loss: 0.0079 - accuracy: 0.9974
94/1688 [>.............................] - ETA: 8s - loss: 0.0086 - accuracy: 0.9970
104/1688 [>.............................] - ETA: 7s - loss: 0.0084 - accuracy: 0.9970
114/1688 [=>............................] - ETA: 7s - loss: 0.0080 - accuracy: 0.9973
124/1688 [=>............................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9975
134/1688 [=>............................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9977
144/1688 [=>............................] - ETA: 7s - loss: 0.0071 - accuracy: 0.9978
154/1688 [=>............................] - ETA: 7s - loss: 0.0071 - accuracy: 0.9980
164/1688 [=>............................] - ETA: 7s - loss: 0.0071 - accuracy: 0.9981
174/1688 [==>...........................] - ETA: 7s - loss: 0.0071 - accuracy: 0.9980
185/1688 [==>...........................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9978
195/1688 [==>...........................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9979
205/1688 [==>...........................] - ETA: 7s - loss: 0.0074 - accuracy: 0.9980
215/1688 [==>...........................] - ETA: 7s - loss: 0.0073 - accuracy: 0.9981
225/1688 [==>...........................] - ETA: 7s - loss: 0.0071 - accuracy: 0.9982
235/1688 [===>..........................] - ETA: 7s - loss: 0.0076 - accuracy: 0.9980
245/1688 [===>..........................] - ETA: 7s - loss: 0.0077 - accuracy: 0.9978
255/1688 [===>..........................] - ETA: 7s - loss: 0.0078 - accuracy: 0.9978
265/1688 [===>..........................] - ETA: 7s - loss: 0.0080 - accuracy: 0.9976
276/1688 [===>..........................] - ETA: 7s - loss: 0.0079 - accuracy: 0.9977
286/1688 [====>.........................] - ETA: 7s - loss: 0.0080 - accuracy: 0.9977
296/1688 [====>.........................] - ETA: 7s - loss: 0.0080 - accuracy: 0.9977
306/1688 [====>.........................] - ETA: 6s - loss: 0.0082 - accuracy: 0.9977
316/1688 [====>.........................] - ETA: 6s - loss: 0.0081 - accuracy: 0.9976
326/1688 [====>.........................] - ETA: 6s - loss: 0.0079 - accuracy: 0.9977
337/1688 [====>.........................] - ETA: 6s - loss: 0.0081 - accuracy: 0.9977
347/1688 [=====>........................] - ETA: 6s - loss: 0.0083 - accuracy: 0.9976
357/1688 [=====>........................] - ETA: 6s - loss: 0.0083 - accuracy: 0.9976
367/1688 [=====>........................] - ETA: 6s - loss: 0.0082 - accuracy: 0.9976
377/1688 [=====>........................] - ETA: 6s - loss: 0.0081 - accuracy: 0.9977
387/1688 [=====>........................] - ETA: 6s - loss: 0.0079 - accuracy: 0.9977
397/1688 [======>.......................] - ETA: 6s - loss: 0.0078 - accuracy: 0.9978
407/1688 [======>.......................] - ETA: 6s - loss: 0.0079 - accuracy: 0.9978
417/1688 [======>.......................] - ETA: 6s - loss: 0.0079 - accuracy: 0.9978
427/1688 [======>.......................] - ETA: 6s - loss: 0.0078 - accuracy: 0.9978
437/1688 [======>.......................] - ETA: 6s - loss: 0.0077 - accuracy: 0.9979
447/1688 [======>.......................] - ETA: 6s - loss: 0.0079 - accuracy: 0.9978
457/1688 [=======>......................] - ETA: 6s - loss: 0.0079 - accuracy: 0.9977
467/1688 [=======>......................] - ETA: 6s - loss: 0.0080 - accuracy: 0.9977
477/1688 [=======>......................] - ETA: 6s - loss: 0.0078 - accuracy: 0.9978
487/1688 [=======>......................] - ETA: 6s - loss: 0.0078 - accuracy: 0.9978
497/1688 [=======>......................] - ETA: 6s - loss: 0.0077 - accuracy: 0.9979
507/1688 [========>.....................] - ETA: 5s - loss: 0.0076 - accuracy: 0.9979
517/1688 [========>.....................] - ETA: 5s - loss: 0.0076 - accuracy: 0.9979
527/1688 [========>.....................] - ETA: 5s - loss: 0.0075 - accuracy: 0.9979
537/1688 [========>.....................] - ETA: 5s - loss: 0.0075 - accuracy: 0.9979
547/1688 [========>.....................] - ETA: 5s - loss: 0.0077 - accuracy: 0.9978
557/1688 [========>.....................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9978
567/1688 [=========>....................] - ETA: 5s - loss: 0.0081 - accuracy: 0.9977
577/1688 [=========>....................] - ETA: 5s - loss: 0.0080 - accuracy: 0.9978
587/1688 [=========>....................] - ETA: 5s - loss: 0.0079 - accuracy: 0.9978
597/1688 [=========>....................] - ETA: 5s - loss: 0.0079 - accuracy: 0.9978
607/1688 [=========>....................] - ETA: 5s - loss: 0.0080 - accuracy: 0.9978
617/1688 [=========>....................] - ETA: 5s - loss: 0.0079 - accuracy: 0.9978
627/1688 [==========>...................] - ETA: 5s - loss: 0.0079 - accuracy: 0.9979
637/1688 [==========>...................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9979
647/1688 [==========>...................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9979
657/1688 [==========>...................] - ETA: 5s - loss: 0.0078 - accuracy: 0.9979
667/1688 [==========>...................] - ETA: 5s - loss: 0.0077 - accuracy: 0.9979
677/1688 [===========>..................] - ETA: 5s - loss: 0.0076 - accuracy: 0.9980
687/1688 [===========>..................] - ETA: 5s - loss: 0.0076 - accuracy: 0.9979
697/1688 [===========>..................] - ETA: 5s - loss: 0.0076 - accuracy: 0.9979
707/1688 [===========>..................] - ETA: 4s - loss: 0.0075 - accuracy: 0.9980
717/1688 [===========>..................] - ETA: 4s - loss: 0.0075 - accuracy: 0.9980
727/1688 [===========>..................] - ETA: 4s - loss: 0.0075 - accuracy: 0.9980
737/1688 [============>.................] - ETA: 4s - loss: 0.0076 - accuracy: 0.9979
747/1688 [============>.................] - ETA: 4s - loss: 0.0076 - accuracy: 0.9980
757/1688 [============>.................] - ETA: 4s - loss: 0.0075 - accuracy: 0.9980
767/1688 [============>.................] - ETA: 4s - loss: 0.0075 - accuracy: 0.9980
778/1688 [============>.................] - ETA: 4s - loss: 0.0076 - accuracy: 0.9979
788/1688 [=============>................] - ETA: 4s - loss: 0.0076 - accuracy: 0.9979
798/1688 [=============>................] - ETA: 4s - loss: 0.0075 - accuracy: 0.9979
808/1688 [=============>................] - ETA: 4s - loss: 0.0076 - accuracy: 0.9979
818/1688 [=============>................] - ETA: 4s - loss: 0.0075 - accuracy: 0.9979
828/1688 [=============>................] - ETA: 4s - loss: 0.0078 - accuracy: 0.9978
838/1688 [=============>................] - ETA: 4s - loss: 0.0077 - accuracy: 0.9978
848/1688 [==============>...............] - ETA: 4s - loss: 0.0076 - accuracy: 0.9979
858/1688 [==============>...............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9979
868/1688 [==============>...............] - ETA: 4s - loss: 0.0077 - accuracy: 0.9979
878/1688 [==============>...............] - ETA: 4s - loss: 0.0076 - accuracy: 0.9979
889/1688 [==============>...............] - ETA: 4s - loss: 0.0076 - accuracy: 0.9979
899/1688 [==============>...............] - ETA: 3s - loss: 0.0075 - accuracy: 0.9979
909/1688 [===============>..............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9980
919/1688 [===============>..............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9980
930/1688 [===============>..............] - ETA: 3s - loss: 0.0075 - accuracy: 0.9980
940/1688 [===============>..............] - ETA: 3s - loss: 0.0075 - accuracy: 0.9980
950/1688 [===============>..............] - ETA: 3s - loss: 0.0075 - accuracy: 0.9980
960/1688 [================>.............] - ETA: 3s - loss: 0.0075 - accuracy: 0.9979
970/1688 [================>.............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9980
980/1688 [================>.............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9980
990/1688 [================>.............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9980
1000/1688 [================>.............] - ETA: 3s - loss: 0.0073 - accuracy: 0.9980
1010/1688 [================>.............] - ETA: 3s - loss: 0.0073 - accuracy: 0.9980
1020/1688 [=================>............] - ETA: 3s - loss: 0.0073 - accuracy: 0.9980
1030/1688 [=================>............] - ETA: 3s - loss: 0.0073 - accuracy: 0.9980
1041/1688 [=================>............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9979
1051/1688 [=================>............] - ETA: 3s - loss: 0.0074 - accuracy: 0.9979
1061/1688 [=================>............] - ETA: 3s - loss: 0.0073 - accuracy: 0.9980
1072/1688 [==================>...........] - ETA: 3s - loss: 0.0073 - accuracy: 0.9980
1083/1688 [==================>...........] - ETA: 3s - loss: 0.0073 - accuracy: 0.9980
1093/1688 [==================>...........] - ETA: 3s - loss: 0.0072 - accuracy: 0.9980
1103/1688 [==================>...........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1113/1688 [==================>...........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1123/1688 [==================>...........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1133/1688 [===================>..........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1143/1688 [===================>..........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1153/1688 [===================>..........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1163/1688 [===================>..........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1174/1688 [===================>..........] - ETA: 2s - loss: 0.0073 - accuracy: 0.9980
1185/1688 [====================>.........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1195/1688 [====================>.........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1206/1688 [====================>.........] - ETA: 2s - loss: 0.0073 - accuracy: 0.9980
1216/1688 [====================>.........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1226/1688 [====================>.........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1236/1688 [====================>.........] - ETA: 2s - loss: 0.0072 - accuracy: 0.9980
1246/1688 [=====================>........] - ETA: 2s - loss: 0.0071 - accuracy: 0.9980
1256/1688 [=====================>........] - ETA: 2s - loss: 0.0071 - accuracy: 0.9980
1266/1688 [=====================>........] - ETA: 2s - loss: 0.0071 - accuracy: 0.9980
1276/1688 [=====================>........] - ETA: 2s - loss: 0.0070 - accuracy: 0.9980
1287/1688 [=====================>........] - ETA: 2s - loss: 0.0070 - accuracy: 0.9981
1297/1688 [======================>.......] - ETA: 1s - loss: 0.0071 - accuracy: 0.9980
1308/1688 [======================>.......] - ETA: 1s - loss: 0.0071 - accuracy: 0.9980
1318/1688 [======================>.......] - ETA: 1s - loss: 0.0070 - accuracy: 0.9981
1328/1688 [======================>.......] - ETA: 1s - loss: 0.0070 - accuracy: 0.9980
1338/1688 [======================>.......] - ETA: 1s - loss: 0.0070 - accuracy: 0.9980
1348/1688 [======================>.......] - ETA: 1s - loss: 0.0070 - accuracy: 0.9980
1358/1688 [=======================>......] - ETA: 1s - loss: 0.0070 - accuracy: 0.9980
1369/1688 [=======================>......] - ETA: 1s - loss: 0.0070 - accuracy: 0.9981
1379/1688 [=======================>......] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1389/1688 [=======================>......] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1400/1688 [=======================>......] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1410/1688 [========================>.....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9980
1421/1688 [========================>.....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1431/1688 [========================>.....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1441/1688 [========================>.....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9980
1451/1688 [========================>.....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9980
1462/1688 [========================>.....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1472/1688 [=========================>....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1482/1688 [=========================>....] - ETA: 1s - loss: 0.0069 - accuracy: 0.9981
1492/1688 [=========================>....] - ETA: 0s - loss: 0.0068 - accuracy: 0.9981
1502/1688 [=========================>....] - ETA: 0s - loss: 0.0069 - accuracy: 0.9981
1512/1688 [=========================>....] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1522/1688 [==========================>...] - ETA: 0s - loss: 0.0070 - accuracy: 0.9980
1532/1688 [==========================>...] - ETA: 0s - loss: 0.0070 - accuracy: 0.9980
1542/1688 [==========================>...] - ETA: 0s - loss: 0.0070 - accuracy: 0.9980
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0070 - accuracy: 0.9980
1562/1688 [==========================>...] - ETA: 0s - loss: 0.0070 - accuracy: 0.9980
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0070 - accuracy: 0.9980
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0070 - accuracy: 0.9980
1593/1688 [===========================>..] - ETA: 0s - loss: 0.0069 - accuracy: 0.9981
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0069 - accuracy: 0.9981
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1624/1688 [===========================>..] - ETA: 0s - loss: 0.0069 - accuracy: 0.9981
1634/1688 [============================>.] - ETA: 0s - loss: 0.0069 - accuracy: 0.9981
1644/1688 [============================>.] - ETA: 0s - loss: 0.0069 - accuracy: 0.9980
1654/1688 [============================>.] - ETA: 0s - loss: 0.0069 - accuracy: 0.9981
1664/1688 [============================>.] - ETA: 0s - loss: 0.0070 - accuracy: 0.9980
1674/1688 [============================>.] - ETA: 0s - loss: 0.0070 - accuracy: 0.9980
1684/1688 [============================>.] - ETA: 0s - loss: 0.0070 - accuracy: 0.9980
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0070 - accuracy: 0.9980 - val_loss: 0.0575 - val_accuracy: 0.9868
Epoch 3/5
1/1688 [..............................] - ETA: 8s - loss: 0.0113 - accuracy: 1.0000
12/1688 [..............................] - ETA: 8s - loss: 0.0034 - accuracy: 1.0000
22/1688 [..............................] - ETA: 8s - loss: 0.0031 - accuracy: 1.0000
32/1688 [..............................] - ETA: 8s - loss: 0.0034 - accuracy: 1.0000
42/1688 [..............................] - ETA: 8s - loss: 0.0031 - accuracy: 1.0000
52/1688 [..............................] - ETA: 8s - loss: 0.0032 - accuracy: 1.0000
62/1688 [>.............................] - ETA: 8s - loss: 0.0032 - accuracy: 1.0000
72/1688 [>.............................] - ETA: 8s - loss: 0.0032 - accuracy: 1.0000
82/1688 [>.............................] - ETA: 8s - loss: 0.0031 - accuracy: 1.0000
92/1688 [>.............................] - ETA: 8s - loss: 0.0033 - accuracy: 1.0000
102/1688 [>.............................] - ETA: 8s - loss: 0.0035 - accuracy: 1.0000
112/1688 [>.............................] - ETA: 8s - loss: 0.0040 - accuracy: 0.9997
122/1688 [=>............................] - ETA: 7s - loss: 0.0039 - accuracy: 0.9997
132/1688 [=>............................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9998
142/1688 [=>............................] - ETA: 7s - loss: 0.0038 - accuracy: 0.9998
152/1688 [=>............................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9996
162/1688 [=>............................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9996
172/1688 [==>...........................] - ETA: 7s - loss: 0.0042 - accuracy: 0.9996
182/1688 [==>...........................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9997
192/1688 [==>...........................] - ETA: 7s - loss: 0.0046 - accuracy: 0.9995
203/1688 [==>...........................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9995
213/1688 [==>...........................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9996
223/1688 [==>...........................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9996
233/1688 [===>..........................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9996
244/1688 [===>..........................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9994
255/1688 [===>..........................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9994
265/1688 [===>..........................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9994
275/1688 [===>..........................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9994
285/1688 [====>.........................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9995
295/1688 [====>.........................] - ETA: 7s - loss: 0.0050 - accuracy: 0.9993
305/1688 [====>.........................] - ETA: 7s - loss: 0.0050 - accuracy: 0.9993
315/1688 [====>.........................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9993
325/1688 [====>.........................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9993
335/1688 [====>.........................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9993
345/1688 [=====>........................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9994
355/1688 [=====>........................] - ETA: 6s - loss: 0.0050 - accuracy: 0.9993
365/1688 [=====>........................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9992
376/1688 [=====>........................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9992
386/1688 [=====>........................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9992
397/1688 [======>.......................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9992
407/1688 [======>.......................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9992
417/1688 [======>.......................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9991
427/1688 [======>.......................] - ETA: 6s - loss: 0.0052 - accuracy: 0.9991
437/1688 [======>.......................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9991
447/1688 [======>.......................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9991
457/1688 [=======>......................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9990
468/1688 [=======>......................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9991
478/1688 [=======>......................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9991
488/1688 [=======>......................] - ETA: 6s - loss: 0.0053 - accuracy: 0.9991
498/1688 [=======>......................] - ETA: 6s - loss: 0.0054 - accuracy: 0.9991
508/1688 [========>.....................] - ETA: 5s - loss: 0.0054 - accuracy: 0.9991
518/1688 [========>.....................] - ETA: 5s - loss: 0.0054 - accuracy: 0.9991
528/1688 [========>.....................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9990
538/1688 [========>.....................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9990
548/1688 [========>.....................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9990
558/1688 [========>.....................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9990
568/1688 [=========>....................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9990
578/1688 [=========>....................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9989
588/1688 [=========>....................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9989
598/1688 [=========>....................] - ETA: 5s - loss: 0.0056 - accuracy: 0.9989
608/1688 [=========>....................] - ETA: 5s - loss: 0.0055 - accuracy: 0.9989
618/1688 [=========>....................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9988
628/1688 [==========>...................] - ETA: 5s - loss: 0.0058 - accuracy: 0.9988
638/1688 [==========>...................] - ETA: 5s - loss: 0.0058 - accuracy: 0.9987
648/1688 [==========>...................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9987
658/1688 [==========>...................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9987
669/1688 [==========>...................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9986
679/1688 [===========>..................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9986
689/1688 [===========>..................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9986
699/1688 [===========>..................] - ETA: 5s - loss: 0.0057 - accuracy: 0.9986
710/1688 [===========>..................] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
720/1688 [===========>..................] - ETA: 4s - loss: 0.0057 - accuracy: 0.9987
730/1688 [===========>..................] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
740/1688 [============>.................] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
750/1688 [============>.................] - ETA: 4s - loss: 0.0058 - accuracy: 0.9986
760/1688 [============>.................] - ETA: 4s - loss: 0.0058 - accuracy: 0.9986
771/1688 [============>.................] - ETA: 4s - loss: 0.0058 - accuracy: 0.9986
781/1688 [============>.................] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
791/1688 [=============>................] - ETA: 4s - loss: 0.0057 - accuracy: 0.9987
801/1688 [=============>................] - ETA: 4s - loss: 0.0057 - accuracy: 0.9986
811/1688 [=============>................] - ETA: 4s - loss: 0.0057 - accuracy: 0.9987
821/1688 [=============>................] - ETA: 4s - loss: 0.0056 - accuracy: 0.9987
831/1688 [=============>................] - ETA: 4s - loss: 0.0056 - accuracy: 0.9987
841/1688 [=============>................] - ETA: 4s - loss: 0.0055 - accuracy: 0.9987
851/1688 [==============>...............] - ETA: 4s - loss: 0.0055 - accuracy: 0.9987
861/1688 [==============>...............] - ETA: 4s - loss: 0.0056 - accuracy: 0.9986
871/1688 [==============>...............] - ETA: 4s - loss: 0.0056 - accuracy: 0.9986
881/1688 [==============>...............] - ETA: 4s - loss: 0.0056 - accuracy: 0.9987
891/1688 [==============>...............] - ETA: 4s - loss: 0.0056 - accuracy: 0.9986
901/1688 [===============>..............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
911/1688 [===============>..............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
921/1688 [===============>..............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
931/1688 [===============>..............] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
941/1688 [===============>..............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
951/1688 [===============>..............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9987
961/1688 [================>.............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
971/1688 [================>.............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
981/1688 [================>.............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9987
991/1688 [================>.............] - ETA: 3s - loss: 0.0054 - accuracy: 0.9987
1001/1688 [================>.............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9987
1011/1688 [================>.............] - ETA: 3s - loss: 0.0054 - accuracy: 0.9987
1021/1688 [=================>............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
1031/1688 [=================>............] - ETA: 3s - loss: 0.0056 - accuracy: 0.9986
1042/1688 [=================>............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
1052/1688 [=================>............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
1062/1688 [=================>............] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
1072/1688 [==================>...........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
1082/1688 [==================>...........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
1092/1688 [==================>...........] - ETA: 3s - loss: 0.0055 - accuracy: 0.9986
1102/1688 [==================>...........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9986
1112/1688 [==================>...........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9985
1122/1688 [==================>...........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9986
1132/1688 [===================>..........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9986
1142/1688 [===================>..........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9986
1152/1688 [===================>..........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9986
1162/1688 [===================>..........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9985
1172/1688 [===================>..........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9985
1182/1688 [====================>.........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9985
1192/1688 [====================>.........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9985
1202/1688 [====================>.........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9985
1212/1688 [====================>.........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9985
1223/1688 [====================>.........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9985
1233/1688 [====================>.........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9986
1243/1688 [=====================>........] - ETA: 2s - loss: 0.0057 - accuracy: 0.9986
1253/1688 [=====================>........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9986
1263/1688 [=====================>........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9986
1273/1688 [=====================>........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9986
1284/1688 [=====================>........] - ETA: 2s - loss: 0.0056 - accuracy: 0.9985
1294/1688 [=====================>........] - ETA: 1s - loss: 0.0056 - accuracy: 0.9986
1304/1688 [======================>.......] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1314/1688 [======================>.......] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1324/1688 [======================>.......] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1335/1688 [======================>.......] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1345/1688 [======================>.......] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1355/1688 [=======================>......] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1366/1688 [=======================>......] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1376/1688 [=======================>......] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1386/1688 [=======================>......] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1396/1688 [=======================>......] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1406/1688 [=======================>......] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1417/1688 [========================>.....] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1428/1688 [========================>.....] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1439/1688 [========================>.....] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1449/1688 [========================>.....] - ETA: 1s - loss: 0.0057 - accuracy: 0.9985
1459/1688 [========================>.....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9985
1469/1688 [=========================>....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9985
1479/1688 [=========================>....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9985
1489/1688 [=========================>....] - ETA: 1s - loss: 0.0056 - accuracy: 0.9986
1499/1688 [=========================>....] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1509/1688 [=========================>....] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1519/1688 [=========================>....] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1529/1688 [==========================>...] - ETA: 0s - loss: 0.0055 - accuracy: 0.9985
1539/1688 [==========================>...] - ETA: 0s - loss: 0.0055 - accuracy: 0.9986
1549/1688 [==========================>...] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1559/1688 [==========================>...] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1569/1688 [==========================>...] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1580/1688 [===========================>..] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1590/1688 [===========================>..] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1600/1688 [===========================>..] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1610/1688 [===========================>..] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1620/1688 [===========================>..] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1630/1688 [===========================>..] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1640/1688 [============================>.] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1650/1688 [============================>.] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1660/1688 [============================>.] - ETA: 0s - loss: 0.0055 - accuracy: 0.9985
1670/1688 [============================>.] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1680/1688 [============================>.] - ETA: 0s - loss: 0.0056 - accuracy: 0.9985
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0056 - accuracy: 0.9985 - val_loss: 0.0562 - val_accuracy: 0.9882
Epoch 4/5
1/1688 [..............................] - ETA: 8s - loss: 0.0022 - accuracy: 1.0000
11/1688 [..............................] - ETA: 8s - loss: 0.0060 - accuracy: 0.9972
22/1688 [..............................] - ETA: 8s - loss: 0.0043 - accuracy: 0.9986
32/1688 [..............................] - ETA: 8s - loss: 0.0042 - accuracy: 0.9990
42/1688 [..............................] - ETA: 8s - loss: 0.0040 - accuracy: 0.9993
52/1688 [..............................] - ETA: 8s - loss: 0.0037 - accuracy: 0.9994
62/1688 [>.............................] - ETA: 8s - loss: 0.0040 - accuracy: 0.9995
72/1688 [>.............................] - ETA: 8s - loss: 0.0036 - accuracy: 0.9996
82/1688 [>.............................] - ETA: 8s - loss: 0.0037 - accuracy: 0.9996
93/1688 [>.............................] - ETA: 8s - loss: 0.0045 - accuracy: 0.9993
103/1688 [>.............................] - ETA: 7s - loss: 0.0053 - accuracy: 0.9991
113/1688 [=>............................] - ETA: 7s - loss: 0.0050 - accuracy: 0.9992
124/1688 [=>............................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9992
134/1688 [=>............................] - ETA: 7s - loss: 0.0046 - accuracy: 0.9993
144/1688 [=>............................] - ETA: 7s - loss: 0.0046 - accuracy: 0.9993
155/1688 [=>............................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9994
165/1688 [=>............................] - ETA: 7s - loss: 0.0046 - accuracy: 0.9992
175/1688 [==>...........................] - ETA: 7s - loss: 0.0047 - accuracy: 0.9991
185/1688 [==>...........................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9990
195/1688 [==>...........................] - ETA: 7s - loss: 0.0047 - accuracy: 0.9990
205/1688 [==>...........................] - ETA: 7s - loss: 0.0047 - accuracy: 0.9991
216/1688 [==>...........................] - ETA: 7s - loss: 0.0048 - accuracy: 0.9990
226/1688 [===>..........................] - ETA: 7s - loss: 0.0047 - accuracy: 0.9990
237/1688 [===>..........................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9991
247/1688 [===>..........................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9991
257/1688 [===>..........................] - ETA: 7s - loss: 0.0045 - accuracy: 0.9990
267/1688 [===>..........................] - ETA: 7s - loss: 0.0044 - accuracy: 0.9991
277/1688 [===>..........................] - ETA: 7s - loss: 0.0043 - accuracy: 0.9991
288/1688 [====>.........................] - ETA: 7s - loss: 0.0046 - accuracy: 0.9990
298/1688 [====>.........................] - ETA: 7s - loss: 0.0049 - accuracy: 0.9988
308/1688 [====>.........................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9989
318/1688 [====>.........................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9988
329/1688 [====>.........................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9989
339/1688 [=====>........................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9988
349/1688 [=====>........................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9988
359/1688 [=====>........................] - ETA: 6s - loss: 0.0049 - accuracy: 0.9988
369/1688 [=====>........................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9988
380/1688 [=====>........................] - ETA: 6s - loss: 0.0048 - accuracy: 0.9988
390/1688 [=====>........................] - ETA: 6s - loss: 0.0047 - accuracy: 0.9989
400/1688 [======>.......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9989
410/1688 [======>.......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9989
420/1688 [======>.......................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9990
430/1688 [======>.......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9989
440/1688 [======>.......................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9989
450/1688 [======>.......................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9990
460/1688 [=======>......................] - ETA: 6s - loss: 0.0044 - accuracy: 0.9990
470/1688 [=======>......................] - ETA: 6s - loss: 0.0045 - accuracy: 0.9990
480/1688 [=======>......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9989
490/1688 [=======>......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9989
500/1688 [=======>......................] - ETA: 6s - loss: 0.0046 - accuracy: 0.9989
510/1688 [========>.....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9990
520/1688 [========>.....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9990
530/1688 [========>.....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9989
540/1688 [========>.....................] - ETA: 5s - loss: 0.0046 - accuracy: 0.9990
550/1688 [========>.....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9990
560/1688 [========>.....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9989
570/1688 [=========>....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9990
580/1688 [=========>....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9990
590/1688 [=========>....................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9990
600/1688 [=========>....................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9990
610/1688 [=========>....................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9990
620/1688 [==========>...................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9990
630/1688 [==========>...................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9991
640/1688 [==========>...................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9991
651/1688 [==========>...................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9990
661/1688 [==========>...................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9991
671/1688 [==========>...................] - ETA: 5s - loss: 0.0045 - accuracy: 0.9990
681/1688 [===========>..................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9990
691/1688 [===========>..................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9991
701/1688 [===========>..................] - ETA: 5s - loss: 0.0044 - accuracy: 0.9990
711/1688 [===========>..................] - ETA: 4s - loss: 0.0044 - accuracy: 0.9990
722/1688 [===========>..................] - ETA: 4s - loss: 0.0044 - accuracy: 0.9990
733/1688 [============>.................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9991
743/1688 [============>.................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9991
753/1688 [============>.................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9991
763/1688 [============>.................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9991
773/1688 [============>.................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9991
783/1688 [============>.................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9991
793/1688 [=============>................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9991
803/1688 [=============>................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9991
813/1688 [=============>................] - ETA: 4s - loss: 0.0044 - accuracy: 0.9990
824/1688 [=============>................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9991
835/1688 [=============>................] - ETA: 4s - loss: 0.0043 - accuracy: 0.9991
845/1688 [==============>...............] - ETA: 4s - loss: 0.0043 - accuracy: 0.9990
855/1688 [==============>...............] - ETA: 4s - loss: 0.0043 - accuracy: 0.9990
865/1688 [==============>...............] - ETA: 4s - loss: 0.0043 - accuracy: 0.9991
876/1688 [==============>...............] - ETA: 4s - loss: 0.0043 - accuracy: 0.9990
886/1688 [==============>...............] - ETA: 4s - loss: 0.0043 - accuracy: 0.9990
896/1688 [==============>...............] - ETA: 4s - loss: 0.0043 - accuracy: 0.9990
906/1688 [===============>..............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9990
917/1688 [===============>..............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9990
927/1688 [===============>..............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
937/1688 [===============>..............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9990
947/1688 [===============>..............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9990
957/1688 [================>.............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9991
967/1688 [================>.............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
977/1688 [================>.............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
987/1688 [================>.............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9991
998/1688 [================>.............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9991
1009/1688 [================>.............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9990
1019/1688 [=================>............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9990
1029/1688 [=================>............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9991
1039/1688 [=================>............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9991
1049/1688 [=================>............] - ETA: 3s - loss: 0.0043 - accuracy: 0.9991
1059/1688 [=================>............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1070/1688 [==================>...........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1080/1688 [==================>...........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1090/1688 [==================>...........] - ETA: 3s - loss: 0.0042 - accuracy: 0.9991
1100/1688 [==================>...........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9991
1110/1688 [==================>...........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9991
1120/1688 [==================>...........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9991
1130/1688 [===================>..........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9991
1140/1688 [===================>..........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9990
1151/1688 [===================>..........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9990
1161/1688 [===================>..........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9990
1171/1688 [===================>..........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9990
1181/1688 [===================>..........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9990
1191/1688 [====================>.........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9991
1201/1688 [====================>.........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9991
1211/1688 [====================>.........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9991
1221/1688 [====================>.........] - ETA: 2s - loss: 0.0042 - accuracy: 0.9991
1231/1688 [====================>.........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9990
1241/1688 [=====================>........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9990
1251/1688 [=====================>........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9990
1261/1688 [=====================>........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9990
1271/1688 [=====================>........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9990
1281/1688 [=====================>........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9990
1291/1688 [=====================>........] - ETA: 2s - loss: 0.0043 - accuracy: 0.9991
1301/1688 [======================>.......] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1311/1688 [======================>.......] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1321/1688 [======================>.......] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1331/1688 [======================>.......] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1341/1688 [======================>.......] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1351/1688 [=======================>......] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1361/1688 [=======================>......] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1372/1688 [=======================>......] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1382/1688 [=======================>......] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1392/1688 [=======================>......] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1402/1688 [=======================>......] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1412/1688 [========================>.....] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1422/1688 [========================>.....] - ETA: 1s - loss: 0.0042 - accuracy: 0.9991
1432/1688 [========================>.....] - ETA: 1s - loss: 0.0043 - accuracy: 0.9991
1442/1688 [========================>.....] - ETA: 1s - loss: 0.0043 - accuracy: 0.9990
1452/1688 [========================>.....] - ETA: 1s - loss: 0.0043 - accuracy: 0.9991
1462/1688 [========================>.....] - ETA: 1s - loss: 0.0043 - accuracy: 0.9991
1472/1688 [=========================>....] - ETA: 1s - loss: 0.0043 - accuracy: 0.9991
1482/1688 [=========================>....] - ETA: 1s - loss: 0.0042 - accuracy: 0.9991
1492/1688 [=========================>....] - ETA: 0s - loss: 0.0042 - accuracy: 0.9991
1502/1688 [=========================>....] - ETA: 0s - loss: 0.0042 - accuracy: 0.9991
1512/1688 [=========================>....] - ETA: 0s - loss: 0.0042 - accuracy: 0.9991
1522/1688 [==========================>...] - ETA: 0s - loss: 0.0042 - accuracy: 0.9991
1532/1688 [==========================>...] - ETA: 0s - loss: 0.0043 - accuracy: 0.9991
1543/1688 [==========================>...] - ETA: 0s - loss: 0.0043 - accuracy: 0.9991
1553/1688 [==========================>...] - ETA: 0s - loss: 0.0043 - accuracy: 0.9991
1563/1688 [==========================>...] - ETA: 0s - loss: 0.0043 - accuracy: 0.9991
1573/1688 [==========================>...] - ETA: 0s - loss: 0.0043 - accuracy: 0.9991
1583/1688 [===========================>..] - ETA: 0s - loss: 0.0043 - accuracy: 0.9991
1594/1688 [===========================>..] - ETA: 0s - loss: 0.0043 - accuracy: 0.9991
1604/1688 [===========================>..] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1614/1688 [===========================>..] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1625/1688 [===========================>..] - ETA: 0s - loss: 0.0044 - accuracy: 0.9991
1635/1688 [============================>.] - ETA: 0s - loss: 0.0045 - accuracy: 0.9990
1645/1688 [============================>.] - ETA: 0s - loss: 0.0045 - accuracy: 0.9990
1656/1688 [============================>.] - ETA: 0s - loss: 0.0045 - accuracy: 0.9990
1666/1688 [============================>.] - ETA: 0s - loss: 0.0045 - accuracy: 0.9990
1676/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1686/1688 [============================>.] - ETA: 0s - loss: 0.0046 - accuracy: 0.9990
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0046 - accuracy: 0.9990 - val_loss: 0.0617 - val_accuracy: 0.9868
Epoch 5/5
1/1688 [..............................] - ETA: 8s - loss: 0.0097 - accuracy: 1.0000
11/1688 [..............................] - ETA: 8s - loss: 0.0026 - accuracy: 1.0000
22/1688 [..............................] - ETA: 8s - loss: 0.0027 - accuracy: 1.0000
33/1688 [..............................] - ETA: 8s - loss: 0.0029 - accuracy: 1.0000
43/1688 [..............................] - ETA: 8s - loss: 0.0027 - accuracy: 1.0000
53/1688 [..............................] - ETA: 8s - loss: 0.0026 - accuracy: 1.0000
63/1688 [>.............................] - ETA: 8s - loss: 0.0033 - accuracy: 1.0000
73/1688 [>.............................] - ETA: 8s - loss: 0.0032 - accuracy: 1.0000
83/1688 [>.............................] - ETA: 8s - loss: 0.0029 - accuracy: 1.0000
93/1688 [>.............................] - ETA: 8s - loss: 0.0032 - accuracy: 0.9997
104/1688 [>.............................] - ETA: 7s - loss: 0.0031 - accuracy: 0.9997
114/1688 [=>............................] - ETA: 7s - loss: 0.0030 - accuracy: 0.9997
125/1688 [=>............................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9995
135/1688 [=>............................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9993
145/1688 [=>............................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9994
155/1688 [=>............................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9994
165/1688 [=>............................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9994
175/1688 [==>...........................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9995
185/1688 [==>...........................] - ETA: 7s - loss: 0.0033 - accuracy: 0.9995
196/1688 [==>...........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9994
206/1688 [==>...........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9994
216/1688 [==>...........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9994
226/1688 [===>..........................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9993
236/1688 [===>..........................] - ETA: 7s - loss: 0.0037 - accuracy: 0.9992
246/1688 [===>..........................] - ETA: 7s - loss: 0.0036 - accuracy: 0.9992
256/1688 [===>..........................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9993
266/1688 [===>..........................] - ETA: 7s - loss: 0.0034 - accuracy: 0.9993
276/1688 [===>..........................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9992
286/1688 [====>.........................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9992
296/1688 [====>.........................] - ETA: 7s - loss: 0.0035 - accuracy: 0.9993
306/1688 [====>.........................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9992
316/1688 [====>.........................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9992
326/1688 [====>.........................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9991
336/1688 [====>.........................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9992
347/1688 [=====>........................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9992
358/1688 [=====>........................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9992
368/1688 [=====>........................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9992
378/1688 [=====>........................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9992
389/1688 [=====>........................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9992
399/1688 [======>.......................] - ETA: 6s - loss: 0.0035 - accuracy: 0.9992
409/1688 [======>.......................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9992
419/1688 [======>.......................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9992
429/1688 [======>.......................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9992
439/1688 [======>.......................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9992
450/1688 [======>.......................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9992
460/1688 [=======>......................] - ETA: 6s - loss: 0.0036 - accuracy: 0.9992
471/1688 [=======>......................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9991
481/1688 [=======>......................] - ETA: 6s - loss: 0.0038 - accuracy: 0.9991
491/1688 [=======>......................] - ETA: 6s - loss: 0.0037 - accuracy: 0.9991
501/1688 [=======>......................] - ETA: 5s - loss: 0.0038 - accuracy: 0.9991
511/1688 [========>.....................] - ETA: 5s - loss: 0.0038 - accuracy: 0.9990
521/1688 [========>.....................] - ETA: 5s - loss: 0.0038 - accuracy: 0.9990
531/1688 [========>.....................] - ETA: 5s - loss: 0.0038 - accuracy: 0.9990
541/1688 [========>.....................] - ETA: 5s - loss: 0.0037 - accuracy: 0.9990
551/1688 [========>.....................] - ETA: 5s - loss: 0.0037 - accuracy: 0.9990
561/1688 [========>.....................] - ETA: 5s - loss: 0.0037 - accuracy: 0.9991
571/1688 [=========>....................] - ETA: 5s - loss: 0.0037 - accuracy: 0.9990
581/1688 [=========>....................] - ETA: 5s - loss: 0.0037 - accuracy: 0.9990
591/1688 [=========>....................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9990
601/1688 [=========>....................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9990
611/1688 [=========>....................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9990
622/1688 [==========>...................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9990
632/1688 [==========>...................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
642/1688 [==========>...................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
652/1688 [==========>...................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
662/1688 [==========>...................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
672/1688 [==========>...................] - ETA: 5s - loss: 0.0039 - accuracy: 0.9991
682/1688 [===========>..................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9990
692/1688 [===========>..................] - ETA: 5s - loss: 0.0040 - accuracy: 0.9991
702/1688 [===========>..................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9990
712/1688 [===========>..................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9990
722/1688 [===========>..................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9990
732/1688 [============>.................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9990
742/1688 [============>.................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
752/1688 [============>.................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
762/1688 [============>.................] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
772/1688 [============>.................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9990
782/1688 [============>.................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9990
793/1688 [=============>................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
803/1688 [=============>................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
813/1688 [=============>................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
823/1688 [=============>................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
833/1688 [=============>................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
843/1688 [=============>................] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
853/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
863/1688 [==============>...............] - ETA: 4s - loss: 0.0040 - accuracy: 0.9991
873/1688 [==============>...............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
883/1688 [==============>...............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
893/1688 [==============>...............] - ETA: 4s - loss: 0.0041 - accuracy: 0.9990
903/1688 [===============>..............] - ETA: 3s - loss: 0.0042 - accuracy: 0.9990
913/1688 [===============>..............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
923/1688 [===============>..............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
934/1688 [===============>..............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
944/1688 [===============>..............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
954/1688 [===============>..............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
964/1688 [================>.............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
974/1688 [================>.............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
984/1688 [================>.............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
994/1688 [================>.............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9991
1004/1688 [================>.............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
1014/1688 [=================>............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
1024/1688 [=================>............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
1034/1688 [=================>............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
1044/1688 [=================>............] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
1054/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
1064/1688 [=================>............] - ETA: 3s - loss: 0.0040 - accuracy: 0.9991
1074/1688 [==================>...........] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
1084/1688 [==================>...........] - ETA: 3s - loss: 0.0041 - accuracy: 0.9990
1095/1688 [==================>...........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9990
1105/1688 [==================>...........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9990
1115/1688 [==================>...........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1125/1688 [==================>...........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1135/1688 [===================>..........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1145/1688 [===================>..........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1155/1688 [===================>..........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1165/1688 [===================>..........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9991
1175/1688 [===================>..........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1185/1688 [====================>.........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9990
1195/1688 [====================>.........] - ETA: 2s - loss: 0.0041 - accuracy: 0.9990
1205/1688 [====================>.........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1215/1688 [====================>.........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1226/1688 [====================>.........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1237/1688 [====================>.........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1247/1688 [=====================>........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1258/1688 [=====================>........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1268/1688 [=====================>........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1278/1688 [=====================>........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1288/1688 [=====================>........] - ETA: 2s - loss: 0.0040 - accuracy: 0.9990
1298/1688 [======================>.......] - ETA: 1s - loss: 0.0040 - accuracy: 0.9990
1308/1688 [======================>.......] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1318/1688 [======================>.......] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1328/1688 [======================>.......] - ETA: 1s - loss: 0.0040 - accuracy: 0.9990
1338/1688 [======================>.......] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1348/1688 [======================>.......] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1358/1688 [=======================>......] - ETA: 1s - loss: 0.0042 - accuracy: 0.9989
1368/1688 [=======================>......] - ETA: 1s - loss: 0.0042 - accuracy: 0.9989
1378/1688 [=======================>......] - ETA: 1s - loss: 0.0042 - accuracy: 0.9990
1388/1688 [=======================>......] - ETA: 1s - loss: 0.0042 - accuracy: 0.9989
1398/1688 [=======================>......] - ETA: 1s - loss: 0.0042 - accuracy: 0.9989
1409/1688 [========================>.....] - ETA: 1s - loss: 0.0042 - accuracy: 0.9989
1419/1688 [========================>.....] - ETA: 1s - loss: 0.0042 - accuracy: 0.9989
1429/1688 [========================>.....] - ETA: 1s - loss: 0.0042 - accuracy: 0.9990
1439/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1449/1688 [========================>.....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1460/1688 [========================>.....] - ETA: 1s - loss: 0.0042 - accuracy: 0.9990
1470/1688 [=========================>....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9990
1480/1688 [=========================>....] - ETA: 1s - loss: 0.0041 - accuracy: 0.9989
1490/1688 [=========================>....] - ETA: 1s - loss: 0.0042 - accuracy: 0.9989
1501/1688 [=========================>....] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1511/1688 [=========================>....] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1521/1688 [==========================>...] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1532/1688 [==========================>...] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1542/1688 [==========================>...] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1552/1688 [==========================>...] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1562/1688 [==========================>...] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1572/1688 [==========================>...] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1582/1688 [===========================>..] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1592/1688 [===========================>..] - ETA: 0s - loss: 0.0041 - accuracy: 0.9989
1602/1688 [===========================>..] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1612/1688 [===========================>..] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1622/1688 [===========================>..] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1632/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1642/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1652/1688 [============================>.] - ETA: 0s - loss: 0.0043 - accuracy: 0.9989
1662/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1672/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1682/1688 [============================>.] - ETA: 0s - loss: 0.0042 - accuracy: 0.9989
1688/1688 [==============================] - 9s 5ms/step - loss: 0.0042 - accuracy: 0.9989 - val_loss: 0.0594 - val_accuracy: 0.9875
<keras.src.callbacks.History object at 0x7f4db8ca0050>
score = model_quantized.evaluate(x_test, y_test, verbose=0)[1]
print('Test accuracy after fine tuning:', score)
Test accuracy after fine tuning: 0.9878000020980835
3. Convert
3.1 Convert to Akida model
When the quantized model produces satisfactory performance, it can be converted to the native Akida format. The convert function returns a model in Akida format ready for inference.
As with Keras, the summary() method provides a textual representation of the Akida model.
from cnn2snn import convert
model_akida = convert(model_quantized)
model_akida.summary()
WARNING:tensorflow:5 out of the last 5 calls to <function NonTrackVariable.set_var at 0x7f4cd41f0540> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
WARNING:tensorflow:6 out of the last 6 calls to <function NonTrackVariable.set_var at 0x7f4cd41f1bc0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
Model Summary
______________________________________________
Input shape Output shape Sequences Layers
==============================================
[28, 28, 1] [1, 1, 10] 1 5
______________________________________________
__________________________________________________________________
Layer (type) Output shape Kernel shape
=============== SW/conv2d-dequantizer_2 (Software) ===============
conv2d (InputConv2D) [13, 13, 32] (3, 3, 1, 32)
__________________________________________________________________
depthwise_conv2d (DepthwiseConv2D) [7, 7, 32] (3, 3, 32, 1)
__________________________________________________________________
conv2d_1 (Conv2D) [7, 7, 64] (1, 1, 32, 64)
__________________________________________________________________
dense (Dense1D) [1, 1, 10] (3136, 10)
__________________________________________________________________
dequantizer_2 (Dequantizer) [1, 1, 10] N/A
__________________________________________________________________
3.2. Check performance
accuracy = model_akida.evaluate(x_test, y_test)
print('Test accuracy after conversion:', accuracy)
# For non-regression purposes
assert accuracy > 0.96
Test accuracy after conversion: 0.9850999712944031
3.3 Show predictions for a single image
Display one of the test images, such as the first image in the dataset from above, to visualize the output of the model.
# Test a single example
sample_image = 0
image = x_test[sample_image]
outputs = model_akida.predict(image.reshape(1, 28, 28, 1))
print('Input Label: %i' % y_test[sample_image])
f, axarr = plt.subplots(1, 2)
axarr[0].imshow(x_test[sample_image].reshape((28, 28)), cmap=cm.Greys_r)
axarr[0].set_title('Class %d' % y_test[sample_image])
axarr[1].bar(range(10), outputs.squeeze())
axarr[1].set_xticks(range(10))
plt.show()
print(outputs.squeeze())
Input Label: 7
[-16.959606 -12.840047 -5.162279 -2.1850464 -20.3723
-10.409038 -31.650877 12.42151 -7.6336327 0.18118115]
Consider the output from the model above. As is typical in backprop-trained models, the final layer is a Dense layer with one neuron for each of the 10 classes in the dataset. The goal of training is to maximize the response of the neuron corresponding to the label of each training sample while minimizing the responses of the other neurons.
In the bar chart above, you can see the outputs from all 10 neurons. It is easy to see that neuron 7 responds much more strongly than the others. The first sample is indeed a number 7.
Total running time of the script: (2 minutes 6.144 seconds)