Note
Go to the end to download the full example code
5. Hyperparameter Optimization
import os
import math
import numpy as np
from skopt.plots import plot_objective
import matplotlib.pyplot as plt
plt.rcParams["font.family"] = "Times New Roman"
SEP = os.sep
from typing import Union
from ai4water import Model
from ai4water.models import MLP
from ai4water.utils.utils import jsonize, TrainTestSplit, dateandtime_now
from ai4water.hyperopt import Categorical, Real, Integer, HyperOpt
from SeqMetrics import RegressionMetrics
from utils import get_dataset, evaluate_model
dataset, _, _ = get_dataset(encoding="ohe")
X_train, y_train = dataset.training_data()
X_test, y_test = dataset.test_data()
***** Training *****
input_x shape: (1059, 74)
target shape: (1059, 1)
***** Test *****
input_x shape: (455, 74)
target shape: (455, 1)
Performance with default hyperparameters
First, we will train the hyperparameters with default parameters
model = Model(
model=MLP(),
epochs=400,
input_features=dataset.input_features,
output_features=dataset.output_features
)
building DL model for
regression problem using Model
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 74)] 0
_________________________________________________________________
Dense_0 (Dense) (None, 32) 2400
_________________________________________________________________
Flatten (Flatten) (None, 32) 0
_________________________________________________________________
Dense_out (Dense) (None, 1) 33
=================================================================
Total params: 2,433
Trainable params: 2,433
Non-trainable params: 0
_________________________________________________________________
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
model.fit(X_train, y_train, validation_data=(X_test, y_test))
Epoch 1/400
assigning name input_1 to IteratorGetNext:0 with shape (None, 74)
assigning name input_1 to IteratorGetNext:0 with shape (None, 74)
1/34 [..............................] - ETA: 8s - loss: 594353.6250assigning name input_1 to IteratorGetNext:0 with shape (None, 74)
34/34 [==============================] - 0s 4ms/step - loss: 220031.2031 - val_loss: 109499.8672
Epoch 2/400
1/34 [..............................] - ETA: 0s - loss: 92746.7734
34/34 [==============================] - 0s 2ms/step - loss: 122686.9844 - val_loss: 74376.9688
Epoch 3/400
1/34 [..............................] - ETA: 0s - loss: 175867.9375
34/34 [==============================] - 0s 2ms/step - loss: 93360.4688 - val_loss: 65948.4453
Epoch 4/400
1/34 [..............................] - ETA: 0s - loss: 76330.1328
34/34 [==============================] - 0s 2ms/step - loss: 84533.7344 - val_loss: 63899.1289
Epoch 5/400
1/34 [..............................] - ETA: 0s - loss: 69069.4297
34/34 [==============================] - 0s 2ms/step - loss: 83068.5703 - val_loss: 62301.5781
Epoch 6/400
1/34 [..............................] - ETA: 0s - loss: 232121.3438
34/34 [==============================] - 0s 2ms/step - loss: 80692.7734 - val_loss: 62393.5586
Epoch 7/400
1/34 [..............................] - ETA: 0s - loss: 58747.9375
34/34 [==============================] - 0s 2ms/step - loss: 79838.4766 - val_loss: 64954.0078
Epoch 8/400
1/34 [..............................] - ETA: 0s - loss: 50394.2227
34/34 [==============================] - 0s 2ms/step - loss: 78904.3438 - val_loss: 62271.5000
Epoch 9/400
1/34 [..............................] - ETA: 0s - loss: 116293.5312
34/34 [==============================] - 0s 2ms/step - loss: 77778.1094 - val_loss: 60098.1523
Epoch 10/400
1/34 [..............................] - ETA: 0s - loss: 81720.2500
34/34 [==============================] - 0s 2ms/step - loss: 77358.8828 - val_loss: 61396.7070
Epoch 11/400
1/34 [..............................] - ETA: 0s - loss: 54145.7305
34/34 [==============================] - 0s 2ms/step - loss: 76396.3906 - val_loss: 59466.3867
Epoch 12/400
1/34 [..............................] - ETA: 0s - loss: 51080.8125
34/34 [==============================] - 0s 2ms/step - loss: 76687.2578 - val_loss: 62287.5430
Epoch 13/400
1/34 [..............................] - ETA: 0s - loss: 79539.1562
34/34 [==============================] - 0s 2ms/step - loss: 76035.9609 - val_loss: 60614.4648
Epoch 14/400
1/34 [..............................] - ETA: 0s - loss: 19492.3438
34/34 [==============================] - 0s 2ms/step - loss: 75839.3750 - val_loss: 58445.6562
Epoch 15/400
1/34 [..............................] - ETA: 0s - loss: 100351.2812
34/34 [==============================] - 0s 2ms/step - loss: 75140.5859 - val_loss: 60460.3164
Epoch 16/400
1/34 [..............................] - ETA: 0s - loss: 108643.9844
34/34 [==============================] - 0s 2ms/step - loss: 74785.5234 - val_loss: 58930.9688
Epoch 17/400
1/34 [..............................] - ETA: 0s - loss: 13270.6348
34/34 [==============================] - 0s 2ms/step - loss: 74509.6016 - val_loss: 59779.1133
Epoch 18/400
1/34 [..............................] - ETA: 0s - loss: 75653.8047
34/34 [==============================] - 0s 2ms/step - loss: 75268.6719 - val_loss: 57351.8672
Epoch 19/400
1/34 [..............................] - ETA: 0s - loss: 37618.2500
34/34 [==============================] - 0s 2ms/step - loss: 74292.9766 - val_loss: 61807.7578
Epoch 20/400
1/34 [..............................] - ETA: 0s - loss: 99349.6250
34/34 [==============================] - 0s 2ms/step - loss: 74669.7109 - val_loss: 60761.7344
Epoch 21/400
1/34 [..............................] - ETA: 0s - loss: 31310.1602
34/34 [==============================] - 0s 2ms/step - loss: 75348.3359 - val_loss: 62518.2773
Epoch 22/400
1/34 [..............................] - ETA: 0s - loss: 39910.5078
34/34 [==============================] - 0s 2ms/step - loss: 75460.6875 - val_loss: 56282.1445
Epoch 23/400
1/34 [..............................] - ETA: 0s - loss: 103756.3750
34/34 [==============================] - 0s 2ms/step - loss: 74416.5781 - val_loss: 58334.3555
Epoch 24/400
1/34 [..............................] - ETA: 0s - loss: 59303.0469
34/34 [==============================] - 0s 2ms/step - loss: 73181.4297 - val_loss: 60044.4258
Epoch 25/400
1/34 [..............................] - ETA: 0s - loss: 107259.6719
34/34 [==============================] - 0s 2ms/step - loss: 76159.7344 - val_loss: 55868.3516
Epoch 26/400
1/34 [..............................] - ETA: 0s - loss: 22389.6016
34/34 [==============================] - 0s 2ms/step - loss: 73037.7109 - val_loss: 59626.2305
Epoch 27/400
1/34 [..............................] - ETA: 0s - loss: 113570.0781
34/34 [==============================] - 0s 2ms/step - loss: 73335.6797 - val_loss: 57343.5117
Epoch 28/400
1/34 [..............................] - ETA: 0s - loss: 76173.6875
34/34 [==============================] - 0s 2ms/step - loss: 74575.0234 - val_loss: 59653.6250
Epoch 29/400
1/34 [..............................] - ETA: 0s - loss: 59996.4648
34/34 [==============================] - 0s 2ms/step - loss: 72716.3906 - val_loss: 58490.4648
Epoch 30/400
1/34 [..............................] - ETA: 0s - loss: 12744.5908
34/34 [==============================] - 0s 2ms/step - loss: 72152.8984 - val_loss: 57218.9492
Epoch 31/400
1/34 [..............................] - ETA: 0s - loss: 77523.0781
34/34 [==============================] - 0s 2ms/step - loss: 73044.3906 - val_loss: 57195.6836
Epoch 32/400
1/34 [..............................] - ETA: 0s - loss: 154005.7500
34/34 [==============================] - 0s 2ms/step - loss: 71937.1406 - val_loss: 55871.8516
Epoch 33/400
1/34 [..............................] - ETA: 0s - loss: 66685.2188
34/34 [==============================] - 0s 2ms/step - loss: 72165.1094 - val_loss: 57469.9180
Epoch 34/400
1/34 [..............................] - ETA: 0s - loss: 93428.2500
34/34 [==============================] - 0s 2ms/step - loss: 72826.9141 - val_loss: 55273.0547
Epoch 35/400
1/34 [..............................] - ETA: 0s - loss: 69641.5312
34/34 [==============================] - 0s 2ms/step - loss: 72006.0234 - val_loss: 55386.9727
Epoch 36/400
1/34 [..............................] - ETA: 0s - loss: 96625.0625
34/34 [==============================] - 0s 2ms/step - loss: 71660.4297 - val_loss: 55451.7891
Epoch 37/400
1/34 [..............................] - ETA: 0s - loss: 33045.6875
34/34 [==============================] - 0s 2ms/step - loss: 71381.9609 - val_loss: 54399.5391
Epoch 38/400
1/34 [..............................] - ETA: 0s - loss: 20458.1602
34/34 [==============================] - 0s 2ms/step - loss: 72666.8438 - val_loss: 56407.4961
Epoch 39/400
1/34 [..............................] - ETA: 0s - loss: 44042.4531
34/34 [==============================] - 0s 2ms/step - loss: 70929.1250 - val_loss: 57980.3281
Epoch 40/400
1/34 [..............................] - ETA: 0s - loss: 60959.9609
34/34 [==============================] - 0s 2ms/step - loss: 71346.0469 - val_loss: 55472.7383
Epoch 41/400
1/34 [..............................] - ETA: 0s - loss: 32515.0234
34/34 [==============================] - 0s 2ms/step - loss: 70969.7188 - val_loss: 54659.5312
Epoch 42/400
1/34 [..............................] - ETA: 0s - loss: 91697.3359
34/34 [==============================] - 0s 2ms/step - loss: 71482.6328 - val_loss: 55348.3477
Epoch 43/400
1/34 [..............................] - ETA: 0s - loss: 63806.5586
34/34 [==============================] - 0s 2ms/step - loss: 70515.3281 - val_loss: 54678.7852
Epoch 44/400
1/34 [..............................] - ETA: 0s - loss: 46455.3906
34/34 [==============================] - 0s 2ms/step - loss: 70524.5781 - val_loss: 55914.8047
Epoch 45/400
1/34 [..............................] - ETA: 0s - loss: 125174.7266
34/34 [==============================] - 0s 2ms/step - loss: 70321.4609 - val_loss: 59371.7656
Epoch 46/400
1/34 [..............................] - ETA: 0s - loss: 59366.2695
34/34 [==============================] - 0s 2ms/step - loss: 73356.3672 - val_loss: 55960.2227
Epoch 47/400
1/34 [..............................] - ETA: 0s - loss: 76136.2266
34/34 [==============================] - 0s 2ms/step - loss: 70540.4141 - val_loss: 58880.8828
Epoch 48/400
1/34 [..............................] - ETA: 0s - loss: 68902.2656
34/34 [==============================] - 0s 2ms/step - loss: 70223.0156 - val_loss: 54081.6562
Epoch 49/400
1/34 [..............................] - ETA: 0s - loss: 70204.0781
34/34 [==============================] - 0s 2ms/step - loss: 70459.2578 - val_loss: 57304.4805
Epoch 50/400
1/34 [..............................] - ETA: 0s - loss: 17844.4082
34/34 [==============================] - 0s 2ms/step - loss: 70054.2578 - val_loss: 55815.4023
Epoch 51/400
1/34 [..............................] - ETA: 0s - loss: 13719.8232
34/34 [==============================] - 0s 2ms/step - loss: 70024.2344 - val_loss: 54377.0859
Epoch 52/400
1/34 [..............................] - ETA: 0s - loss: 83933.9844
34/34 [==============================] - 0s 2ms/step - loss: 70971.3125 - val_loss: 54300.2148
Epoch 53/400
1/34 [..............................] - ETA: 0s - loss: 90039.7031
34/34 [==============================] - 0s 2ms/step - loss: 70280.1172 - val_loss: 52971.4844
Epoch 54/400
1/34 [..............................] - ETA: 0s - loss: 43619.7148
34/34 [==============================] - 0s 2ms/step - loss: 69655.0078 - val_loss: 53531.7422
Epoch 55/400
1/34 [..............................] - ETA: 0s - loss: 34955.9766
34/34 [==============================] - 0s 2ms/step - loss: 69091.2109 - val_loss: 56506.6914
Epoch 56/400
1/34 [..............................] - ETA: 0s - loss: 12568.5020
34/34 [==============================] - 0s 2ms/step - loss: 68939.5781 - val_loss: 54854.0703
Epoch 57/400
1/34 [..............................] - ETA: 0s - loss: 46707.4531
34/34 [==============================] - 0s 2ms/step - loss: 68785.3125 - val_loss: 55441.5781
Epoch 58/400
1/34 [..............................] - ETA: 0s - loss: 103703.0156
34/34 [==============================] - 0s 2ms/step - loss: 69147.4453 - val_loss: 56024.6875
Epoch 59/400
1/34 [..............................] - ETA: 0s - loss: 51716.7852
34/34 [==============================] - 0s 2ms/step - loss: 69474.7266 - val_loss: 53320.8125
Epoch 60/400
1/34 [..............................] - ETA: 0s - loss: 58603.5156
34/34 [==============================] - 0s 2ms/step - loss: 68383.9062 - val_loss: 56464.7812
Epoch 61/400
1/34 [..............................] - ETA: 0s - loss: 135864.1562
34/34 [==============================] - 0s 2ms/step - loss: 68440.1875 - val_loss: 52839.3633
Epoch 62/400
1/34 [..............................] - ETA: 0s - loss: 155404.4219
34/34 [==============================] - 0s 2ms/step - loss: 69365.1953 - val_loss: 53039.5312
Epoch 63/400
1/34 [..............................] - ETA: 0s - loss: 143151.6094
34/34 [==============================] - 0s 2ms/step - loss: 68778.1094 - val_loss: 54265.6016
Epoch 64/400
1/34 [..............................] - ETA: 0s - loss: 37743.6211
34/34 [==============================] - 0s 2ms/step - loss: 67907.9922 - val_loss: 53837.7617
Epoch 65/400
1/34 [..............................] - ETA: 0s - loss: 131854.8750
34/34 [==============================] - 0s 2ms/step - loss: 68352.3906 - val_loss: 58186.4141
Epoch 66/400
1/34 [..............................] - ETA: 0s - loss: 118154.2188
34/34 [==============================] - 0s 2ms/step - loss: 70748.2266 - val_loss: 53215.4375
Epoch 67/400
1/34 [..............................] - ETA: 0s - loss: 67380.6406
34/34 [==============================] - 0s 2ms/step - loss: 68133.0781 - val_loss: 54116.4961
Epoch 68/400
1/34 [..............................] - ETA: 0s - loss: 57728.9844
34/34 [==============================] - 0s 2ms/step - loss: 67826.9609 - val_loss: 54718.1289
Epoch 69/400
1/34 [..............................] - ETA: 0s - loss: 113164.0781
34/34 [==============================] - 0s 2ms/step - loss: 67594.7812 - val_loss: 52630.1758
Epoch 70/400
1/34 [..............................] - ETA: 0s - loss: 73203.8281
34/34 [==============================] - 0s 2ms/step - loss: 67963.2422 - val_loss: 52420.8008
Epoch 71/400
1/34 [..............................] - ETA: 0s - loss: 105445.1719
34/34 [==============================] - 0s 2ms/step - loss: 67173.5000 - val_loss: 55642.0742
Epoch 72/400
1/34 [..............................] - ETA: 0s - loss: 61681.2305
34/34 [==============================] - 0s 2ms/step - loss: 67456.7656 - val_loss: 51778.6016
Epoch 73/400
1/34 [..............................] - ETA: 0s - loss: 15637.8965
34/34 [==============================] - 0s 2ms/step - loss: 67683.4766 - val_loss: 53113.4141
Epoch 74/400
1/34 [..............................] - ETA: 0s - loss: 15992.3496
34/34 [==============================] - 0s 2ms/step - loss: 67504.0078 - val_loss: 51365.4414
Epoch 75/400
1/34 [..............................] - ETA: 0s - loss: 58950.6836
34/34 [==============================] - 0s 2ms/step - loss: 67819.2969 - val_loss: 51841.7617
Epoch 76/400
1/34 [..............................] - ETA: 0s - loss: 92996.2891
34/34 [==============================] - 0s 2ms/step - loss: 67828.0000 - val_loss: 53307.0781
Epoch 77/400
1/34 [..............................] - ETA: 0s - loss: 135267.5312
34/34 [==============================] - 0s 2ms/step - loss: 68517.0938 - val_loss: 55520.1758
Epoch 78/400
1/34 [..............................] - ETA: 0s - loss: 64995.9922
34/34 [==============================] - 0s 2ms/step - loss: 69950.5859 - val_loss: 51943.5898
Epoch 79/400
1/34 [..............................] - ETA: 0s - loss: 98838.0781
34/34 [==============================] - 0s 2ms/step - loss: 67504.9688 - val_loss: 55704.3516
Epoch 80/400
1/34 [..............................] - ETA: 0s - loss: 48094.9297
34/34 [==============================] - 0s 2ms/step - loss: 67065.4766 - val_loss: 54460.5898
Epoch 81/400
1/34 [..............................] - ETA: 0s - loss: 41920.5625
34/34 [==============================] - 0s 2ms/step - loss: 66645.5234 - val_loss: 54305.2891
Epoch 82/400
1/34 [..............................] - ETA: 0s - loss: 54806.8125
34/34 [==============================] - 0s 2ms/step - loss: 66948.8438 - val_loss: 54715.3633
Epoch 83/400
1/34 [..............................] - ETA: 0s - loss: 48681.4336
34/34 [==============================] - 0s 2ms/step - loss: 67520.0703 - val_loss: 52810.9258
Epoch 84/400
1/34 [..............................] - ETA: 0s - loss: 16537.4570
34/34 [==============================] - 0s 2ms/step - loss: 66503.7656 - val_loss: 53558.4766
Epoch 85/400
1/34 [..............................] - ETA: 0s - loss: 75363.0781
34/34 [==============================] - 0s 2ms/step - loss: 66379.0859 - val_loss: 52588.6367
Epoch 86/400
1/34 [..............................] - ETA: 0s - loss: 24346.2754
34/34 [==============================] - 0s 2ms/step - loss: 66310.5234 - val_loss: 50763.3633
Epoch 87/400
1/34 [..............................] - ETA: 0s - loss: 95344.7344
34/34 [==============================] - 0s 2ms/step - loss: 66262.8047 - val_loss: 51932.1758
Epoch 88/400
1/34 [..............................] - ETA: 0s - loss: 37343.4492
34/34 [==============================] - 0s 2ms/step - loss: 65929.5469 - val_loss: 51070.9102
Epoch 89/400
1/34 [..............................] - ETA: 0s - loss: 58360.2109
34/34 [==============================] - 0s 2ms/step - loss: 67933.0703 - val_loss: 51254.4219
Epoch 90/400
1/34 [..............................] - ETA: 0s - loss: 68260.0312
34/34 [==============================] - 0s 2ms/step - loss: 66468.4531 - val_loss: 52137.3125
Epoch 91/400
1/34 [..............................] - ETA: 0s - loss: 22601.6875
34/34 [==============================] - 0s 2ms/step - loss: 65456.5586 - val_loss: 52903.0195
Epoch 92/400
1/34 [..............................] - ETA: 0s - loss: 36261.6094
34/34 [==============================] - 0s 2ms/step - loss: 65526.7305 - val_loss: 54411.4336
Epoch 93/400
1/34 [..............................] - ETA: 0s - loss: 67572.0625
34/34 [==============================] - 0s 2ms/step - loss: 65631.4688 - val_loss: 52043.1094
Epoch 94/400
1/34 [..............................] - ETA: 0s - loss: 134719.7656
34/34 [==============================] - 0s 2ms/step - loss: 66223.1016 - val_loss: 54195.0977
Epoch 95/400
1/34 [..............................] - ETA: 0s - loss: 93847.8750
34/34 [==============================] - 0s 2ms/step - loss: 65877.0859 - val_loss: 55679.4727
Epoch 96/400
1/34 [..............................] - ETA: 0s - loss: 78665.3047
34/34 [==============================] - 0s 2ms/step - loss: 66619.5859 - val_loss: 51193.4609
Epoch 97/400
1/34 [..............................] - ETA: 0s - loss: 59887.3438
34/34 [==============================] - 0s 2ms/step - loss: 65628.5234 - val_loss: 49442.4492
Epoch 98/400
1/34 [..............................] - ETA: 0s - loss: 90874.1562
34/34 [==============================] - 0s 2ms/step - loss: 65061.4609 - val_loss: 51295.9414
Epoch 99/400
1/34 [..............................] - ETA: 0s - loss: 135871.1094
34/34 [==============================] - 0s 2ms/step - loss: 64839.2734 - val_loss: 52130.2148
Epoch 100/400
1/34 [..............................] - ETA: 0s - loss: 117404.2891
34/34 [==============================] - 0s 2ms/step - loss: 64812.9258 - val_loss: 50666.6055
Epoch 101/400
1/34 [..............................] - ETA: 0s - loss: 54243.2773
34/34 [==============================] - 0s 2ms/step - loss: 64718.7461 - val_loss: 50096.6914
Epoch 102/400
1/34 [..............................] - ETA: 0s - loss: 46695.3672
34/34 [==============================] - 0s 2ms/step - loss: 64883.0508 - val_loss: 50415.4531
Epoch 103/400
1/34 [..............................] - ETA: 0s - loss: 161809.4375
34/34 [==============================] - 0s 2ms/step - loss: 64775.6680 - val_loss: 50525.4609
Epoch 104/400
1/34 [..............................] - ETA: 0s - loss: 34294.9453
34/34 [==============================] - 0s 2ms/step - loss: 64833.8594 - val_loss: 49355.1133
Epoch 105/400
1/34 [..............................] - ETA: 0s - loss: 65461.4570
34/34 [==============================] - 0s 2ms/step - loss: 65069.8711 - val_loss: 49780.0898
Epoch 106/400
1/34 [..............................] - ETA: 0s - loss: 51766.0391
34/34 [==============================] - 0s 2ms/step - loss: 63900.8438 - val_loss: 52101.2188
Epoch 107/400
1/34 [..............................] - ETA: 0s - loss: 52772.4219
34/34 [==============================] - 0s 2ms/step - loss: 64197.3242 - val_loss: 51354.0117
Epoch 108/400
1/34 [..............................] - ETA: 0s - loss: 84164.9375
34/34 [==============================] - 0s 2ms/step - loss: 64915.6641 - val_loss: 52740.7812
Epoch 109/400
1/34 [..............................] - ETA: 0s - loss: 122616.0156
34/34 [==============================] - 0s 2ms/step - loss: 63868.2461 - val_loss: 50301.1250
Epoch 110/400
1/34 [..............................] - ETA: 0s - loss: 62364.5859
34/34 [==============================] - 0s 2ms/step - loss: 64552.8242 - val_loss: 48833.6484
Epoch 111/400
1/34 [..............................] - ETA: 0s - loss: 89267.9688
34/34 [==============================] - 0s 2ms/step - loss: 63611.9219 - val_loss: 52004.8672
Epoch 112/400
1/34 [..............................] - ETA: 0s - loss: 38863.0586
34/34 [==============================] - 0s 2ms/step - loss: 63560.4062 - val_loss: 52638.3828
Epoch 113/400
1/34 [..............................] - ETA: 0s - loss: 23559.6406
34/34 [==============================] - 0s 2ms/step - loss: 66626.5078 - val_loss: 52616.7422
Epoch 114/400
1/34 [..............................] - ETA: 0s - loss: 43429.3867
34/34 [==============================] - 0s 2ms/step - loss: 64195.6328 - val_loss: 50558.1289
Epoch 115/400
1/34 [..............................] - ETA: 0s - loss: 81985.0156
34/34 [==============================] - 0s 2ms/step - loss: 64369.6484 - val_loss: 51924.3867
Epoch 116/400
1/34 [..............................] - ETA: 0s - loss: 39584.8555
34/34 [==============================] - 0s 2ms/step - loss: 63428.7969 - val_loss: 51857.0625
Epoch 117/400
1/34 [..............................] - ETA: 0s - loss: 110020.5781
34/34 [==============================] - 0s 2ms/step - loss: 64432.0820 - val_loss: 55764.9844
Epoch 118/400
1/34 [..............................] - ETA: 0s - loss: 87823.6641
34/34 [==============================] - 0s 2ms/step - loss: 64376.8320 - val_loss: 50626.7070
Epoch 119/400
1/34 [..............................] - ETA: 0s - loss: 74997.0781
34/34 [==============================] - 0s 2ms/step - loss: 63013.2695 - val_loss: 48957.5156
Epoch 120/400
1/34 [..............................] - ETA: 0s - loss: 122956.0234
34/34 [==============================] - 0s 2ms/step - loss: 62516.6367 - val_loss: 48372.0195
Epoch 121/400
1/34 [..............................] - ETA: 0s - loss: 81875.4375
34/34 [==============================] - 0s 2ms/step - loss: 62615.2031 - val_loss: 50119.2344
Epoch 122/400
1/34 [..............................] - ETA: 0s - loss: 7458.0493
34/34 [==============================] - 0s 2ms/step - loss: 63184.4414 - val_loss: 51413.0781
Epoch 123/400
1/34 [..............................] - ETA: 0s - loss: 22857.1367
34/34 [==============================] - 0s 2ms/step - loss: 62542.3516 - val_loss: 50117.9219
Epoch 124/400
1/34 [..............................] - ETA: 0s - loss: 20394.2773
34/34 [==============================] - 0s 2ms/step - loss: 62131.1211 - val_loss: 49985.0977
Epoch 125/400
1/34 [..............................] - ETA: 0s - loss: 54008.3281
34/34 [==============================] - 0s 2ms/step - loss: 62033.1914 - val_loss: 49657.6016
Epoch 126/400
1/34 [..............................] - ETA: 0s - loss: 35047.0742
34/34 [==============================] - 0s 2ms/step - loss: 62491.0000 - val_loss: 48228.2344
Epoch 127/400
1/34 [..............................] - ETA: 0s - loss: 58548.2734
34/34 [==============================] - 0s 2ms/step - loss: 62903.9531 - val_loss: 48060.8633
Epoch 128/400
1/34 [..............................] - ETA: 0s - loss: 24760.8379
34/34 [==============================] - 0s 2ms/step - loss: 63339.2070 - val_loss: 47760.3281
Epoch 129/400
1/34 [..............................] - ETA: 0s - loss: 23762.9277
34/34 [==============================] - 0s 2ms/step - loss: 62573.2852 - val_loss: 48454.6289
Epoch 130/400
1/34 [..............................] - ETA: 0s - loss: 94140.4141
34/34 [==============================] - 0s 2ms/step - loss: 61511.5430 - val_loss: 53583.2383
Epoch 131/400
1/34 [..............................] - ETA: 0s - loss: 53456.0156
34/34 [==============================] - 0s 2ms/step - loss: 61149.2188 - val_loss: 48293.4414
Epoch 132/400
1/34 [..............................] - ETA: 0s - loss: 92209.1719
34/34 [==============================] - 0s 2ms/step - loss: 61031.4805 - val_loss: 51940.6445
Epoch 133/400
1/34 [..............................] - ETA: 0s - loss: 39310.6680
34/34 [==============================] - 0s 2ms/step - loss: 61470.2734 - val_loss: 51096.0078
Epoch 134/400
1/34 [..............................] - ETA: 0s - loss: 10824.7744
34/34 [==============================] - 0s 2ms/step - loss: 61186.4102 - val_loss: 51218.4492
Epoch 135/400
1/34 [..............................] - ETA: 0s - loss: 70173.3203
34/34 [==============================] - 0s 2ms/step - loss: 61155.9141 - val_loss: 48475.1602
Epoch 136/400
1/34 [..............................] - ETA: 0s - loss: 78477.5781
34/34 [==============================] - 0s 2ms/step - loss: 60329.6758 - val_loss: 47027.3359
Epoch 137/400
1/34 [..............................] - ETA: 0s - loss: 59927.1289
34/34 [==============================] - 0s 2ms/step - loss: 61261.3516 - val_loss: 48242.1250
Epoch 138/400
1/34 [..............................] - ETA: 0s - loss: 109385.9062
34/34 [==============================] - 0s 2ms/step - loss: 60504.8555 - val_loss: 49602.4531
Epoch 139/400
1/34 [..............................] - ETA: 0s - loss: 37667.8203
34/34 [==============================] - 0s 2ms/step - loss: 60884.8672 - val_loss: 52371.6562
Epoch 140/400
1/34 [..............................] - ETA: 0s - loss: 21938.5547
34/34 [==============================] - 0s 2ms/step - loss: 62099.9414 - val_loss: 52356.8047
Epoch 141/400
1/34 [..............................] - ETA: 0s - loss: 47290.7500
34/34 [==============================] - 0s 2ms/step - loss: 61165.4297 - val_loss: 50812.8555
Epoch 142/400
1/34 [..............................] - ETA: 0s - loss: 45268.9531
34/34 [==============================] - 0s 2ms/step - loss: 60447.8086 - val_loss: 48570.1250
Epoch 143/400
1/34 [..............................] - ETA: 0s - loss: 64783.2188
34/34 [==============================] - 0s 2ms/step - loss: 59614.2773 - val_loss: 47462.7852
Epoch 144/400
1/34 [..............................] - ETA: 0s - loss: 77763.3203
34/34 [==============================] - 0s 2ms/step - loss: 59693.9297 - val_loss: 51982.9062
Epoch 145/400
1/34 [..............................] - ETA: 0s - loss: 86018.7031
34/34 [==============================] - 0s 2ms/step - loss: 59746.8008 - val_loss: 47239.6250
Epoch 146/400
1/34 [..............................] - ETA: 0s - loss: 168836.4688
34/34 [==============================] - 0s 2ms/step - loss: 59065.7969 - val_loss: 48898.7070
Epoch 147/400
1/34 [..............................] - ETA: 0s - loss: 130363.4688
34/34 [==============================] - 0s 2ms/step - loss: 62036.8672 - val_loss: 48724.7305
Epoch 148/400
1/34 [..............................] - ETA: 0s - loss: 89867.5781
34/34 [==============================] - 0s 2ms/step - loss: 59544.5156 - val_loss: 46834.8086
Epoch 149/400
1/34 [..............................] - ETA: 0s - loss: 38477.5820
34/34 [==============================] - 0s 2ms/step - loss: 63840.5000 - val_loss: 45500.6680
Epoch 150/400
1/34 [..............................] - ETA: 0s - loss: 73927.2734
34/34 [==============================] - 0s 2ms/step - loss: 60646.2305 - val_loss: 45654.3125
Epoch 151/400
1/34 [..............................] - ETA: 0s - loss: 75286.2266
34/34 [==============================] - 0s 2ms/step - loss: 60151.3320 - val_loss: 45627.4883
Epoch 152/400
1/34 [..............................] - ETA: 0s - loss: 36630.7148
34/34 [==============================] - 0s 2ms/step - loss: 60368.6836 - val_loss: 49608.2305
Epoch 153/400
1/34 [..............................] - ETA: 0s - loss: 20796.9336
34/34 [==============================] - 0s 2ms/step - loss: 58812.5039 - val_loss: 45117.9727
Epoch 154/400
1/34 [..............................] - ETA: 0s - loss: 198811.9219
34/34 [==============================] - 0s 2ms/step - loss: 58764.3125 - val_loss: 45173.2891
Epoch 155/400
1/34 [..............................] - ETA: 0s - loss: 53345.1719
34/34 [==============================] - 0s 2ms/step - loss: 58350.8242 - val_loss: 46841.2930
Epoch 156/400
1/34 [..............................] - ETA: 0s - loss: 34276.7969
34/34 [==============================] - 0s 2ms/step - loss: 57834.8125 - val_loss: 46364.5977
Epoch 157/400
1/34 [..............................] - ETA: 0s - loss: 36103.0039
34/34 [==============================] - 0s 2ms/step - loss: 58518.1758 - val_loss: 46701.7852
Epoch 158/400
1/34 [..............................] - ETA: 0s - loss: 41711.2578
34/34 [==============================] - 0s 2ms/step - loss: 57264.3203 - val_loss: 44808.4258
Epoch 159/400
1/34 [..............................] - ETA: 0s - loss: 17407.1309
34/34 [==============================] - 0s 2ms/step - loss: 60555.3008 - val_loss: 47361.7227
Epoch 160/400
1/34 [..............................] - ETA: 0s - loss: 34634.8164
34/34 [==============================] - 0s 2ms/step - loss: 60804.6719 - val_loss: 48245.3633
Epoch 161/400
1/34 [..............................] - ETA: 0s - loss: 31306.2695
34/34 [==============================] - 0s 2ms/step - loss: 57629.1523 - val_loss: 46448.0586
Epoch 162/400
1/34 [..............................] - ETA: 0s - loss: 68235.3125
34/34 [==============================] - 0s 2ms/step - loss: 57113.7422 - val_loss: 46304.6016
Epoch 163/400
1/34 [..............................] - ETA: 0s - loss: 49075.8906
34/34 [==============================] - 0s 2ms/step - loss: 58902.1289 - val_loss: 49608.7617
Epoch 164/400
1/34 [..............................] - ETA: 0s - loss: 43686.2422
34/34 [==============================] - 0s 2ms/step - loss: 56848.5156 - val_loss: 47035.7656
Epoch 165/400
1/34 [..............................] - ETA: 0s - loss: 29437.0742
34/34 [==============================] - 0s 2ms/step - loss: 56524.2500 - val_loss: 43995.8281
Epoch 166/400
1/34 [..............................] - ETA: 0s - loss: 74801.0469
34/34 [==============================] - 0s 2ms/step - loss: 56266.9844 - val_loss: 45040.5391
Epoch 167/400
1/34 [..............................] - ETA: 0s - loss: 29212.8945
34/34 [==============================] - 0s 2ms/step - loss: 56648.5664 - val_loss: 47943.5820
Epoch 168/400
1/34 [..............................] - ETA: 0s - loss: 18251.6914
34/34 [==============================] - 0s 2ms/step - loss: 56173.6328 - val_loss: 42685.2383
Epoch 169/400
1/34 [..............................] - ETA: 0s - loss: 88790.4297
34/34 [==============================] - 0s 2ms/step - loss: 58056.2773 - val_loss: 46633.2578
Epoch 170/400
1/34 [..............................] - ETA: 0s - loss: 48586.4219
34/34 [==============================] - 0s 2ms/step - loss: 55945.7031 - val_loss: 45603.4453
Epoch 171/400
1/34 [..............................] - ETA: 0s - loss: 52076.4062
34/34 [==============================] - 0s 2ms/step - loss: 55853.3438 - val_loss: 44387.0586
Epoch 172/400
1/34 [..............................] - ETA: 0s - loss: 32178.9102
34/34 [==============================] - 0s 2ms/step - loss: 55687.5547 - val_loss: 46454.5391
Epoch 173/400
1/34 [..............................] - ETA: 0s - loss: 52827.7500
34/34 [==============================] - 0s 2ms/step - loss: 55993.4180 - val_loss: 45397.9688
Epoch 174/400
1/34 [..............................] - ETA: 0s - loss: 44013.5195
34/34 [==============================] - 0s 2ms/step - loss: 55407.0625 - val_loss: 43343.1172
Epoch 175/400
1/34 [..............................] - ETA: 0s - loss: 42998.8750
34/34 [==============================] - 0s 2ms/step - loss: 56156.1953 - val_loss: 42425.7695
Epoch 176/400
1/34 [..............................] - ETA: 0s - loss: 69396.3281
34/34 [==============================] - 0s 2ms/step - loss: 54927.4102 - val_loss: 43775.8398
Epoch 177/400
1/34 [..............................] - ETA: 0s - loss: 49724.5781
34/34 [==============================] - 0s 2ms/step - loss: 54593.0391 - val_loss: 42032.2539
Epoch 178/400
1/34 [..............................] - ETA: 0s - loss: 93857.5938
34/34 [==============================] - 0s 2ms/step - loss: 54641.0312 - val_loss: 42486.7148
Epoch 179/400
1/34 [..............................] - ETA: 0s - loss: 16766.8418
34/34 [==============================] - 0s 2ms/step - loss: 56595.6289 - val_loss: 45445.3750
Epoch 180/400
1/34 [..............................] - ETA: 0s - loss: 88317.0938
34/34 [==============================] - 0s 2ms/step - loss: 55517.0820 - val_loss: 44927.9922
Epoch 181/400
1/34 [..............................] - ETA: 0s - loss: 38976.2539
34/34 [==============================] - 0s 2ms/step - loss: 54263.5195 - val_loss: 41849.1875
Epoch 182/400
1/34 [..............................] - ETA: 0s - loss: 36227.8945
34/34 [==============================] - 0s 2ms/step - loss: 54351.2852 - val_loss: 44139.3008
Epoch 183/400
1/34 [..............................] - ETA: 0s - loss: 86596.3438
34/34 [==============================] - 0s 2ms/step - loss: 54427.3359 - val_loss: 45041.2617
Epoch 184/400
1/34 [..............................] - ETA: 0s - loss: 30247.2461
34/34 [==============================] - 0s 2ms/step - loss: 54931.0625 - val_loss: 47604.9414
Epoch 185/400
1/34 [..............................] - ETA: 0s - loss: 107818.3594
34/34 [==============================] - 0s 2ms/step - loss: 54397.6992 - val_loss: 50087.1914
Epoch 186/400
1/34 [..............................] - ETA: 0s - loss: 45338.7422
34/34 [==============================] - 0s 2ms/step - loss: 54863.1367 - val_loss: 48200.0742
Epoch 187/400
1/34 [..............................] - ETA: 0s - loss: 46097.4414
34/34 [==============================] - 0s 2ms/step - loss: 53996.0859 - val_loss: 45454.2227
Epoch 188/400
1/34 [..............................] - ETA: 0s - loss: 65773.1406
34/34 [==============================] - 0s 2ms/step - loss: 54426.1172 - val_loss: 41863.1328
Epoch 189/400
1/34 [..............................] - ETA: 0s - loss: 12427.6504
34/34 [==============================] - 0s 2ms/step - loss: 53604.9414 - val_loss: 41774.8281
Epoch 190/400
1/34 [..............................] - ETA: 0s - loss: 31151.6602
34/34 [==============================] - 0s 2ms/step - loss: 53305.8906 - val_loss: 43737.2148
Epoch 191/400
1/34 [..............................] - ETA: 0s - loss: 37769.1875
34/34 [==============================] - 0s 2ms/step - loss: 53581.0781 - val_loss: 41135.1289
Epoch 192/400
1/34 [..............................] - ETA: 0s - loss: 17795.5586
34/34 [==============================] - 0s 2ms/step - loss: 52355.5117 - val_loss: 40978.0273
Epoch 193/400
1/34 [..............................] - ETA: 0s - loss: 15923.0947
34/34 [==============================] - 0s 2ms/step - loss: 53994.1133 - val_loss: 40859.9219
Epoch 194/400
1/34 [..............................] - ETA: 0s - loss: 32320.8906
34/34 [==============================] - 0s 2ms/step - loss: 52029.6758 - val_loss: 44364.9180
Epoch 195/400
1/34 [..............................] - ETA: 0s - loss: 180996.8125
34/34 [==============================] - 0s 2ms/step - loss: 57370.6641 - val_loss: 42258.3633
Epoch 196/400
1/34 [..............................] - ETA: 0s - loss: 21005.3301
34/34 [==============================] - 0s 2ms/step - loss: 51714.6484 - val_loss: 42926.4648
Epoch 197/400
1/34 [..............................] - ETA: 0s - loss: 49281.2695
34/34 [==============================] - 0s 2ms/step - loss: 51588.6094 - val_loss: 43353.3633
Epoch 198/400
1/34 [..............................] - ETA: 0s - loss: 43991.3125
34/34 [==============================] - 0s 2ms/step - loss: 52490.4180 - val_loss: 46226.6953
Epoch 199/400
1/34 [..............................] - ETA: 0s - loss: 23392.8613
34/34 [==============================] - 0s 2ms/step - loss: 53584.6328 - val_loss: 43238.6680
Epoch 200/400
1/34 [..............................] - ETA: 0s - loss: 49115.7812
34/34 [==============================] - 0s 2ms/step - loss: 51811.3672 - val_loss: 41307.1680
Epoch 201/400
1/34 [..............................] - ETA: 0s - loss: 88825.6172
34/34 [==============================] - 0s 2ms/step - loss: 50814.7188 - val_loss: 52256.9609
Epoch 202/400
1/34 [..............................] - ETA: 0s - loss: 135647.2031
34/34 [==============================] - 0s 2ms/step - loss: 51984.7266 - val_loss: 39454.8359
Epoch 203/400
1/34 [..............................] - ETA: 0s - loss: 24046.6914
34/34 [==============================] - 0s 2ms/step - loss: 55391.1953 - val_loss: 40949.5391
Epoch 204/400
1/34 [..............................] - ETA: 0s - loss: 17325.2754
34/34 [==============================] - 0s 2ms/step - loss: 50669.8164 - val_loss: 41326.7031
Epoch 205/400
1/34 [..............................] - ETA: 0s - loss: 53453.6953
34/34 [==============================] - 0s 2ms/step - loss: 50802.2383 - val_loss: 40005.8203
Epoch 206/400
1/34 [..............................] - ETA: 0s - loss: 45577.2070
34/34 [==============================] - 0s 2ms/step - loss: 51687.2500 - val_loss: 42370.3984
Epoch 207/400
1/34 [..............................] - ETA: 0s - loss: 49919.7227
34/34 [==============================] - 0s 2ms/step - loss: 50013.2695 - val_loss: 44889.6016
Epoch 208/400
1/34 [..............................] - ETA: 0s - loss: 39908.0117
34/34 [==============================] - 0s 2ms/step - loss: 51664.5781 - val_loss: 40849.6094
Epoch 209/400
1/34 [..............................] - ETA: 0s - loss: 26023.6289
34/34 [==============================] - 0s 2ms/step - loss: 51044.5391 - val_loss: 46993.7227
Epoch 210/400
1/34 [..............................] - ETA: 0s - loss: 36539.5000
34/34 [==============================] - 0s 2ms/step - loss: 49946.6523 - val_loss: 38709.2969
Epoch 211/400
1/34 [..............................] - ETA: 0s - loss: 87264.2344
34/34 [==============================] - 0s 2ms/step - loss: 50191.5977 - val_loss: 38606.6016
Epoch 212/400
1/34 [..............................] - ETA: 0s - loss: 61370.4883
34/34 [==============================] - 0s 2ms/step - loss: 50464.3984 - val_loss: 37920.2031
Epoch 213/400
1/34 [..............................] - ETA: 0s - loss: 7449.0337
34/34 [==============================] - 0s 2ms/step - loss: 50477.8359 - val_loss: 38044.9180
Epoch 214/400
1/34 [..............................] - ETA: 0s - loss: 72013.1016
34/34 [==============================] - 0s 2ms/step - loss: 48623.5547 - val_loss: 38480.5625
Epoch 215/400
1/34 [..............................] - ETA: 0s - loss: 84033.6641
34/34 [==============================] - 0s 2ms/step - loss: 55766.1016 - val_loss: 39739.8125
Epoch 216/400
1/34 [..............................] - ETA: 0s - loss: 24019.1973
34/34 [==============================] - 0s 2ms/step - loss: 50606.7695 - val_loss: 39020.7734
Epoch 217/400
1/34 [..............................] - ETA: 0s - loss: 12043.1133
34/34 [==============================] - 0s 2ms/step - loss: 49272.7266 - val_loss: 37962.4102
Epoch 218/400
1/34 [..............................] - ETA: 0s - loss: 61337.3047
34/34 [==============================] - 0s 2ms/step - loss: 48130.6836 - val_loss: 41188.9805
Epoch 219/400
1/34 [..............................] - ETA: 0s - loss: 14691.6621
34/34 [==============================] - 0s 2ms/step - loss: 47893.9102 - val_loss: 37759.4414
Epoch 220/400
1/34 [..............................] - ETA: 0s - loss: 62410.1992
34/34 [==============================] - 0s 2ms/step - loss: 48676.2852 - val_loss: 37321.1172
Epoch 221/400
1/34 [..............................] - ETA: 0s - loss: 7615.6538
34/34 [==============================] - 0s 2ms/step - loss: 48533.5625 - val_loss: 38875.2734
Epoch 222/400
1/34 [..............................] - ETA: 0s - loss: 113206.8672
34/34 [==============================] - 0s 2ms/step - loss: 48021.6133 - val_loss: 37226.5352
Epoch 223/400
1/34 [..............................] - ETA: 0s - loss: 31874.5977
34/34 [==============================] - 0s 2ms/step - loss: 49176.5273 - val_loss: 41093.0156
Epoch 224/400
1/34 [..............................] - ETA: 0s - loss: 26028.0488
34/34 [==============================] - 0s 2ms/step - loss: 49217.5742 - val_loss: 43985.8750
Epoch 225/400
1/34 [..............................] - ETA: 0s - loss: 12910.4863
34/34 [==============================] - 0s 2ms/step - loss: 47757.8711 - val_loss: 44336.0469
Epoch 226/400
1/34 [..............................] - ETA: 0s - loss: 81828.1562
34/34 [==============================] - 0s 2ms/step - loss: 48452.6875 - val_loss: 41933.0977
Epoch 227/400
1/34 [..............................] - ETA: 0s - loss: 44897.2266
34/34 [==============================] - 0s 2ms/step - loss: 47200.0820 - val_loss: 40557.3750
Epoch 228/400
1/34 [..............................] - ETA: 0s - loss: 75957.0859
34/34 [==============================] - 0s 2ms/step - loss: 47084.6641 - val_loss: 36734.3438
Epoch 229/400
1/34 [..............................] - ETA: 0s - loss: 137891.8438
34/34 [==============================] - 0s 2ms/step - loss: 53843.5312 - val_loss: 36127.6172
Epoch 230/400
1/34 [..............................] - ETA: 0s - loss: 33239.8086
34/34 [==============================] - 0s 2ms/step - loss: 46190.6016 - val_loss: 37374.3203
Epoch 231/400
1/34 [..............................] - ETA: 0s - loss: 26260.7031
34/34 [==============================] - 0s 2ms/step - loss: 47224.6367 - val_loss: 36148.0820
Epoch 232/400
1/34 [..............................] - ETA: 0s - loss: 92265.7031
34/34 [==============================] - 0s 2ms/step - loss: 47665.7109 - val_loss: 37130.7734
Epoch 233/400
1/34 [..............................] - ETA: 0s - loss: 31480.0352
34/34 [==============================] - 0s 2ms/step - loss: 46772.5273 - val_loss: 37990.3281
Epoch 234/400
1/34 [..............................] - ETA: 0s - loss: 51004.0859
34/34 [==============================] - 0s 2ms/step - loss: 46652.6328 - val_loss: 41975.7500
Epoch 235/400
1/34 [..............................] - ETA: 0s - loss: 31201.2207
34/34 [==============================] - 0s 2ms/step - loss: 46875.0469 - val_loss: 36769.4453
Epoch 236/400
1/34 [..............................] - ETA: 0s - loss: 58107.6172
34/34 [==============================] - 0s 2ms/step - loss: 46012.3672 - val_loss: 38201.7695
Epoch 237/400
1/34 [..............................] - ETA: 0s - loss: 51223.1094
34/34 [==============================] - 0s 2ms/step - loss: 45487.1875 - val_loss: 37526.5742
Epoch 238/400
1/34 [..............................] - ETA: 0s - loss: 50905.8438
34/34 [==============================] - 0s 2ms/step - loss: 46044.0586 - val_loss: 44319.0234
Epoch 239/400
1/34 [..............................] - ETA: 0s - loss: 30112.4766
34/34 [==============================] - 0s 2ms/step - loss: 46693.1875 - val_loss: 39879.5234
Epoch 240/400
1/34 [..............................] - ETA: 0s - loss: 27212.2656
34/34 [==============================] - 0s 2ms/step - loss: 46226.3359 - val_loss: 35931.4883
Epoch 241/400
1/34 [..............................] - ETA: 0s - loss: 59471.6016
34/34 [==============================] - 0s 2ms/step - loss: 44532.1445 - val_loss: 36386.3555
Epoch 242/400
1/34 [..............................] - ETA: 0s - loss: 40058.5820
34/34 [==============================] - 0s 2ms/step - loss: 45752.4102 - val_loss: 36723.7227
Epoch 243/400
1/34 [..............................] - ETA: 0s - loss: 10260.9219
34/34 [==============================] - 0s 2ms/step - loss: 46094.5625 - val_loss: 35194.7617
Epoch 244/400
1/34 [..............................] - ETA: 0s - loss: 26404.2461
34/34 [==============================] - 0s 2ms/step - loss: 45346.0078 - val_loss: 36326.0859
Epoch 245/400
1/34 [..............................] - ETA: 0s - loss: 107225.7344
34/34 [==============================] - 0s 2ms/step - loss: 44605.9297 - val_loss: 35321.8242
Epoch 246/400
1/34 [..............................] - ETA: 0s - loss: 10602.7773
34/34 [==============================] - 0s 2ms/step - loss: 45992.2695 - val_loss: 35846.2656
Epoch 247/400
1/34 [..............................] - ETA: 0s - loss: 22512.2207
34/34 [==============================] - 0s 2ms/step - loss: 44545.8672 - val_loss: 35807.4648
Epoch 248/400
1/34 [..............................] - ETA: 0s - loss: 55753.9180
34/34 [==============================] - 0s 2ms/step - loss: 45781.8164 - val_loss: 39415.7461
Epoch 249/400
1/34 [..............................] - ETA: 0s - loss: 32846.4453
34/34 [==============================] - 0s 2ms/step - loss: 44321.3047 - val_loss: 36878.8008
Epoch 250/400
1/34 [..............................] - ETA: 0s - loss: 48016.9297
34/34 [==============================] - 0s 2ms/step - loss: 44340.5977 - val_loss: 40060.4336
Epoch 251/400
1/34 [..............................] - ETA: 0s - loss: 36504.9648
34/34 [==============================] - 0s 2ms/step - loss: 44060.0938 - val_loss: 34177.1680
Epoch 252/400
1/34 [..............................] - ETA: 0s - loss: 3874.6797
34/34 [==============================] - 0s 2ms/step - loss: 44115.4883 - val_loss: 35039.7070
Epoch 253/400
1/34 [..............................] - ETA: 0s - loss: 27885.0293
34/34 [==============================] - 0s 2ms/step - loss: 47199.5273 - val_loss: 33911.6016
Epoch 254/400
1/34 [..............................] - ETA: 0s - loss: 38428.1016
34/34 [==============================] - 0s 2ms/step - loss: 43286.0352 - val_loss: 36030.5742
Epoch 255/400
1/34 [..............................] - ETA: 0s - loss: 28620.9473
34/34 [==============================] - 0s 2ms/step - loss: 44043.3633 - val_loss: 34437.1484
Epoch 256/400
1/34 [..............................] - ETA: 0s - loss: 6801.8896
34/34 [==============================] - 0s 2ms/step - loss: 42829.7383 - val_loss: 34782.2070
Epoch 257/400
1/34 [..............................] - ETA: 0s - loss: 24377.6289
34/34 [==============================] - 0s 2ms/step - loss: 43665.8984 - val_loss: 34734.1094
Epoch 258/400
1/34 [..............................] - ETA: 0s - loss: 21182.5859
34/34 [==============================] - 0s 2ms/step - loss: 43113.3828 - val_loss: 33876.8711
Epoch 259/400
1/34 [..............................] - ETA: 0s - loss: 24638.8164
34/34 [==============================] - 0s 2ms/step - loss: 44107.8555 - val_loss: 35902.8984
Epoch 260/400
1/34 [..............................] - ETA: 0s - loss: 37557.0977
34/34 [==============================] - 0s 2ms/step - loss: 43202.0703 - val_loss: 33210.6406
Epoch 261/400
1/34 [..............................] - ETA: 0s - loss: 53183.6016
34/34 [==============================] - 0s 2ms/step - loss: 42835.4766 - val_loss: 34070.8555
Epoch 262/400
1/34 [..............................] - ETA: 0s - loss: 26114.1699
34/34 [==============================] - 0s 2ms/step - loss: 42690.2734 - val_loss: 36828.6523
Epoch 263/400
1/34 [..............................] - ETA: 0s - loss: 99595.4766
34/34 [==============================] - 0s 2ms/step - loss: 42239.2266 - val_loss: 35204.2539
Epoch 264/400
1/34 [..............................] - ETA: 0s - loss: 81782.3750
34/34 [==============================] - 0s 2ms/step - loss: 42511.6758 - val_loss: 34368.8164
Epoch 265/400
1/34 [..............................] - ETA: 0s - loss: 66054.9844
34/34 [==============================] - 0s 2ms/step - loss: 44218.7617 - val_loss: 37084.4180
Epoch 266/400
1/34 [..............................] - ETA: 0s - loss: 96314.0000
34/34 [==============================] - 0s 2ms/step - loss: 42422.3398 - val_loss: 37356.6992
Epoch 267/400
1/34 [..............................] - ETA: 0s - loss: 16599.7559
34/34 [==============================] - 0s 2ms/step - loss: 42778.5586 - val_loss: 33537.9336
Epoch 268/400
1/34 [..............................] - ETA: 0s - loss: 107319.6172
34/34 [==============================] - 0s 2ms/step - loss: 42034.6055 - val_loss: 33651.9844
Epoch 269/400
1/34 [..............................] - ETA: 0s - loss: 103499.4453
34/34 [==============================] - 0s 2ms/step - loss: 41371.2695 - val_loss: 36496.8086
Epoch 270/400
1/34 [..............................] - ETA: 0s - loss: 35666.1016
34/34 [==============================] - 0s 2ms/step - loss: 41954.9922 - val_loss: 32988.3125
Epoch 271/400
1/34 [..............................] - ETA: 0s - loss: 27345.6055
34/34 [==============================] - 0s 2ms/step - loss: 42228.8711 - val_loss: 33220.3203
Epoch 272/400
1/34 [..............................] - ETA: 0s - loss: 24265.5215
34/34 [==============================] - 0s 2ms/step - loss: 40616.1094 - val_loss: 35205.7227
Epoch 273/400
1/34 [..............................] - ETA: 0s - loss: 29288.3008
34/34 [==============================] - 0s 2ms/step - loss: 40437.6133 - val_loss: 32078.6621
Epoch 274/400
1/34 [..............................] - ETA: 0s - loss: 30054.2305
34/34 [==============================] - 0s 2ms/step - loss: 42218.5312 - val_loss: 39847.3945
Epoch 275/400
1/34 [..............................] - ETA: 0s - loss: 46407.9297
34/34 [==============================] - 0s 2ms/step - loss: 42819.9102 - val_loss: 32852.4414
Epoch 276/400
1/34 [..............................] - ETA: 0s - loss: 71979.6562
34/34 [==============================] - 0s 2ms/step - loss: 40724.4805 - val_loss: 32689.7871
Epoch 277/400
1/34 [..............................] - ETA: 0s - loss: 16968.6211
34/34 [==============================] - 0s 2ms/step - loss: 40653.9023 - val_loss: 32423.6133
Epoch 278/400
1/34 [..............................] - ETA: 0s - loss: 41407.3477
34/34 [==============================] - 0s 2ms/step - loss: 40762.7969 - val_loss: 36235.9961
Epoch 279/400
1/34 [..............................] - ETA: 0s - loss: 8875.6865
34/34 [==============================] - 0s 2ms/step - loss: 42054.1328 - val_loss: 33237.2266
Epoch 280/400
1/34 [..............................] - ETA: 0s - loss: 25256.8418
34/34 [==============================] - 0s 2ms/step - loss: 40305.6133 - val_loss: 32585.3438
Epoch 281/400
1/34 [..............................] - ETA: 0s - loss: 49766.4297
34/34 [==============================] - 0s 2ms/step - loss: 42000.5508 - val_loss: 43203.0156
Epoch 282/400
1/34 [..............................] - ETA: 0s - loss: 34940.7266
34/34 [==============================] - 0s 2ms/step - loss: 46458.5742 - val_loss: 32234.6875
Epoch 283/400
1/34 [..............................] - ETA: 0s - loss: 16898.7969
34/34 [==============================] - 0s 2ms/step - loss: 41047.5430 - val_loss: 32688.2559
Epoch 284/400
1/34 [..............................] - ETA: 0s - loss: 11722.6475
34/34 [==============================] - 0s 2ms/step - loss: 41456.7695 - val_loss: 31807.6973
Epoch 285/400
1/34 [..............................] - ETA: 0s - loss: 32133.8457
34/34 [==============================] - 0s 2ms/step - loss: 41237.6562 - val_loss: 31051.4180
Epoch 286/400
1/34 [..............................] - ETA: 0s - loss: 45668.7695
34/34 [==============================] - 0s 2ms/step - loss: 40406.8711 - val_loss: 32326.7832
Epoch 287/400
1/34 [..............................] - ETA: 0s - loss: 28414.3555
34/34 [==============================] - 0s 2ms/step - loss: 40766.0430 - val_loss: 32364.6309
Epoch 288/400
1/34 [..............................] - ETA: 0s - loss: 50190.6445
34/34 [==============================] - 0s 2ms/step - loss: 39662.5938 - val_loss: 35303.9375
Epoch 289/400
1/34 [..............................] - ETA: 0s - loss: 71067.7578
34/34 [==============================] - 0s 2ms/step - loss: 40162.9219 - val_loss: 31188.4355
Epoch 290/400
1/34 [..............................] - ETA: 0s - loss: 8906.5293
34/34 [==============================] - 0s 2ms/step - loss: 39533.4961 - val_loss: 31886.7031
Epoch 291/400
1/34 [..............................] - ETA: 0s - loss: 28372.7148
34/34 [==============================] - 0s 2ms/step - loss: 38685.5391 - val_loss: 33221.3633
Epoch 292/400
1/34 [..............................] - ETA: 0s - loss: 18239.1289
34/34 [==============================] - 0s 2ms/step - loss: 39455.2148 - val_loss: 30915.8184
Epoch 293/400
1/34 [..............................] - ETA: 0s - loss: 34976.7031
34/34 [==============================] - 0s 2ms/step - loss: 38778.4023 - val_loss: 34432.4141
Epoch 294/400
1/34 [..............................] - ETA: 0s - loss: 16539.0312
34/34 [==============================] - 0s 2ms/step - loss: 38771.4180 - val_loss: 31660.9375
Epoch 295/400
1/34 [..............................] - ETA: 0s - loss: 35490.2500
34/34 [==============================] - 0s 2ms/step - loss: 38540.1523 - val_loss: 33061.6758
Epoch 296/400
1/34 [..............................] - ETA: 0s - loss: 38948.1602
34/34 [==============================] - 0s 2ms/step - loss: 38563.1055 - val_loss: 32138.8652
Epoch 297/400
1/34 [..............................] - ETA: 0s - loss: 32140.1191
34/34 [==============================] - 0s 2ms/step - loss: 39438.9922 - val_loss: 37327.1094
Epoch 298/400
1/34 [..............................] - ETA: 0s - loss: 70526.2188
34/34 [==============================] - 0s 2ms/step - loss: 44045.8789 - val_loss: 30594.8242
Epoch 299/400
1/34 [..............................] - ETA: 0s - loss: 3685.7852
34/34 [==============================] - 0s 2ms/step - loss: 38561.4688 - val_loss: 32735.8535
Epoch 300/400
1/34 [..............................] - ETA: 0s - loss: 29263.8906
34/34 [==============================] - 0s 2ms/step - loss: 41523.1641 - val_loss: 32681.5469
Epoch 301/400
1/34 [..............................] - ETA: 0s - loss: 10997.6982
34/34 [==============================] - 0s 2ms/step - loss: 38458.1484 - val_loss: 43777.9062
Epoch 302/400
1/34 [..............................] - ETA: 0s - loss: 9583.7070
34/34 [==============================] - 0s 2ms/step - loss: 40872.9023 - val_loss: 31480.6387
Epoch 303/400
1/34 [..............................] - ETA: 0s - loss: 38337.2539
34/34 [==============================] - 0s 2ms/step - loss: 38061.9414 - val_loss: 31182.8770
Epoch 304/400
1/34 [..............................] - ETA: 0s - loss: 77488.8125
34/34 [==============================] - 0s 2ms/step - loss: 38618.2031 - val_loss: 34351.3438
Epoch 305/400
1/34 [..............................] - ETA: 0s - loss: 24395.7012
34/34 [==============================] - 0s 2ms/step - loss: 39213.9570 - val_loss: 31693.2910
Epoch 306/400
1/34 [..............................] - ETA: 0s - loss: 50962.3594
34/34 [==============================] - 0s 2ms/step - loss: 37563.4961 - val_loss: 43491.7148
Epoch 307/400
1/34 [..............................] - ETA: 0s - loss: 31842.7539
34/34 [==============================] - 0s 2ms/step - loss: 40327.7344 - val_loss: 33174.3750
Epoch 308/400
1/34 [..............................] - ETA: 0s - loss: 32821.1055
34/34 [==============================] - 0s 2ms/step - loss: 38419.9648 - val_loss: 29657.0410
Epoch 309/400
1/34 [..............................] - ETA: 0s - loss: 118624.9844
34/34 [==============================] - 0s 2ms/step - loss: 38676.5625 - val_loss: 30299.4316
Epoch 310/400
1/34 [..............................] - ETA: 0s - loss: 29261.8730
34/34 [==============================] - 0s 2ms/step - loss: 37598.0781 - val_loss: 30300.5781
Epoch 311/400
1/34 [..............................] - ETA: 0s - loss: 39644.1367
34/34 [==============================] - 0s 2ms/step - loss: 37012.9492 - val_loss: 32743.3398
Epoch 312/400
1/34 [..............................] - ETA: 0s - loss: 30537.4219
34/34 [==============================] - 0s 2ms/step - loss: 39395.1094 - val_loss: 29654.5156
Epoch 313/400
1/34 [..............................] - ETA: 0s - loss: 5580.3652
34/34 [==============================] - 0s 2ms/step - loss: 37856.3828 - val_loss: 30711.0586
Epoch 314/400
1/34 [..............................] - ETA: 0s - loss: 48826.8242
34/34 [==============================] - 0s 2ms/step - loss: 39447.8516 - val_loss: 31216.3203
Epoch 315/400
1/34 [..............................] - ETA: 0s - loss: 31550.3789
34/34 [==============================] - 0s 2ms/step - loss: 37129.7227 - val_loss: 29242.4277
Epoch 316/400
1/34 [..............................] - ETA: 0s - loss: 19205.1211
34/34 [==============================] - 0s 2ms/step - loss: 37972.6641 - val_loss: 29693.3145
Epoch 317/400
1/34 [..............................] - ETA: 0s - loss: 44180.1875
34/34 [==============================] - 0s 2ms/step - loss: 37136.3359 - val_loss: 30734.3535
Epoch 318/400
1/34 [..............................] - ETA: 0s - loss: 41646.3477
34/34 [==============================] - 0s 2ms/step - loss: 37328.7500 - val_loss: 29467.1699
Epoch 319/400
1/34 [..............................] - ETA: 0s - loss: 103679.3203
34/34 [==============================] - 0s 2ms/step - loss: 40054.9297 - val_loss: 31777.0703
Epoch 320/400
1/34 [..............................] - ETA: 0s - loss: 17707.1484
34/34 [==============================] - 0s 2ms/step - loss: 36337.9023 - val_loss: 30549.4512
Epoch 321/400
1/34 [..............................] - ETA: 0s - loss: 34138.4531
34/34 [==============================] - 0s 2ms/step - loss: 36117.3477 - val_loss: 29978.4453
Epoch 322/400
1/34 [..............................] - ETA: 0s - loss: 3583.9756
34/34 [==============================] - 0s 2ms/step - loss: 37823.4492 - val_loss: 28882.0547
Epoch 323/400
1/34 [..............................] - ETA: 0s - loss: 9485.0713
34/34 [==============================] - 0s 2ms/step - loss: 37406.8164 - val_loss: 29100.9414
Epoch 324/400
1/34 [..............................] - ETA: 0s - loss: 13133.4844
34/34 [==============================] - 0s 2ms/step - loss: 37130.7617 - val_loss: 29965.9004
Epoch 325/400
1/34 [..............................] - ETA: 0s - loss: 63516.2695
34/34 [==============================] - 0s 2ms/step - loss: 36527.0273 - val_loss: 28842.4766
Epoch 326/400
1/34 [..............................] - ETA: 0s - loss: 53567.9609
34/34 [==============================] - 0s 2ms/step - loss: 36014.6758 - val_loss: 30407.9512
Epoch 327/400
1/34 [..............................] - ETA: 0s - loss: 25849.3242
34/34 [==============================] - 0s 2ms/step - loss: 36658.4531 - val_loss: 28890.6855
Epoch 328/400
1/34 [..............................] - ETA: 0s - loss: 54752.7266
34/34 [==============================] - 0s 2ms/step - loss: 38624.9609 - val_loss: 29550.9531
Epoch 329/400
1/34 [..............................] - ETA: 0s - loss: 24300.3438
34/34 [==============================] - 0s 2ms/step - loss: 36540.1289 - val_loss: 28660.7383
Epoch 330/400
1/34 [..............................] - ETA: 0s - loss: 11722.1699
34/34 [==============================] - 0s 2ms/step - loss: 35831.4414 - val_loss: 28884.9883
Epoch 331/400
1/34 [..............................] - ETA: 0s - loss: 17759.9297
34/34 [==============================] - 0s 2ms/step - loss: 36096.7812 - val_loss: 28421.9375
Epoch 332/400
1/34 [..............................] - ETA: 0s - loss: 16703.1523
34/34 [==============================] - 0s 2ms/step - loss: 35927.2031 - val_loss: 36218.7930
Epoch 333/400
1/34 [..............................] - ETA: 0s - loss: 15835.9043
34/34 [==============================] - 0s 2ms/step - loss: 37848.4805 - val_loss: 29188.6211
Epoch 334/400
1/34 [..............................] - ETA: 0s - loss: 21078.4648
34/34 [==============================] - 0s 2ms/step - loss: 35457.1992 - val_loss: 31355.8027
Epoch 335/400
1/34 [..............................] - ETA: 0s - loss: 18392.6641
34/34 [==============================] - 0s 2ms/step - loss: 37863.1953 - val_loss: 28760.6426
Epoch 336/400
1/34 [..............................] - ETA: 0s - loss: 21207.4883
34/34 [==============================] - 0s 2ms/step - loss: 35927.0586 - val_loss: 33929.3789
Epoch 337/400
1/34 [..............................] - ETA: 0s - loss: 11938.7471
34/34 [==============================] - 0s 2ms/step - loss: 37590.0039 - val_loss: 29232.6875
Epoch 338/400
1/34 [..............................] - ETA: 0s - loss: 26304.3047
34/34 [==============================] - 0s 2ms/step - loss: 35117.4023 - val_loss: 28556.6914
Epoch 339/400
1/34 [..............................] - ETA: 0s - loss: 39662.4336
34/34 [==============================] - 0s 2ms/step - loss: 35476.7227 - val_loss: 31629.3711
Epoch 340/400
1/34 [..............................] - ETA: 0s - loss: 18328.9531
34/34 [==============================] - 0s 2ms/step - loss: 35857.3359 - val_loss: 28521.2520
Epoch 341/400
1/34 [..............................] - ETA: 0s - loss: 34640.5703
34/34 [==============================] - 0s 2ms/step - loss: 35631.5195 - val_loss: 30901.1113
Epoch 342/400
1/34 [..............................] - ETA: 0s - loss: 94130.5078
34/34 [==============================] - 0s 2ms/step - loss: 35560.6133 - val_loss: 31538.1562
Epoch 343/400
1/34 [..............................] - ETA: 0s - loss: 27378.8125
34/34 [==============================] - 0s 2ms/step - loss: 36600.8828 - val_loss: 27963.2109
Epoch 344/400
1/34 [..............................] - ETA: 0s - loss: 31467.5957
34/34 [==============================] - 0s 2ms/step - loss: 34963.7930 - val_loss: 30803.4004
Epoch 345/400
1/34 [..............................] - ETA: 0s - loss: 20480.1445
34/34 [==============================] - 0s 2ms/step - loss: 38282.2070 - val_loss: 30111.4258
Epoch 346/400
1/34 [..............................] - ETA: 0s - loss: 32035.9062
34/34 [==============================] - 0s 2ms/step - loss: 34633.0859 - val_loss: 28833.1504
Epoch 347/400
1/34 [..............................] - ETA: 0s - loss: 27426.6094
34/34 [==============================] - 0s 2ms/step - loss: 35137.4492 - val_loss: 28341.0996
Epoch 348/400
1/34 [..............................] - ETA: 0s - loss: 23024.0078
34/34 [==============================] - 0s 2ms/step - loss: 35696.5156 - val_loss: 27948.1582
Epoch 349/400
1/34 [..............................] - ETA: 0s - loss: 109761.6484
34/34 [==============================] - 0s 2ms/step - loss: 35175.4141 - val_loss: 29633.2305
Epoch 350/400
1/34 [..............................] - ETA: 0s - loss: 76537.3359
34/34 [==============================] - 0s 2ms/step - loss: 34261.0820 - val_loss: 30118.0098
Epoch 351/400
1/34 [..............................] - ETA: 0s - loss: 6557.2158
34/34 [==============================] - 0s 2ms/step - loss: 36083.6484 - val_loss: 29874.8965
Epoch 352/400
1/34 [..............................] - ETA: 0s - loss: 13762.0254
34/34 [==============================] - 0s 2ms/step - loss: 35442.3828 - val_loss: 34109.5273
Epoch 353/400
1/34 [..............................] - ETA: 0s - loss: 55694.7539
34/34 [==============================] - 0s 2ms/step - loss: 36744.6367 - val_loss: 28705.8887
Epoch 354/400
1/34 [..............................] - ETA: 0s - loss: 51931.5742
34/34 [==============================] - 0s 2ms/step - loss: 36864.5508 - val_loss: 28510.9727
Epoch 355/400
1/34 [..............................] - ETA: 0s - loss: 30740.8594
34/34 [==============================] - 0s 2ms/step - loss: 34509.0508 - val_loss: 28608.0195
Epoch 356/400
1/34 [..............................] - ETA: 0s - loss: 69211.0469
34/34 [==============================] - 0s 2ms/step - loss: 36939.5586 - val_loss: 27761.8359
Epoch 357/400
1/34 [..............................] - ETA: 0s - loss: 48570.5859
34/34 [==============================] - 0s 2ms/step - loss: 34458.5859 - val_loss: 27711.8066
Epoch 358/400
1/34 [..............................] - ETA: 0s - loss: 6340.6201
34/34 [==============================] - 0s 2ms/step - loss: 36416.3047 - val_loss: 28679.9531
Epoch 359/400
1/34 [..............................] - ETA: 0s - loss: 33610.3867
34/34 [==============================] - 0s 2ms/step - loss: 34311.7578 - val_loss: 27675.3574
Epoch 360/400
1/34 [..............................] - ETA: 0s - loss: 62498.5234
34/34 [==============================] - 0s 2ms/step - loss: 34205.1523 - val_loss: 32899.7148
Epoch 361/400
1/34 [..............................] - ETA: 0s - loss: 48223.9375
34/34 [==============================] - 0s 2ms/step - loss: 35967.2109 - val_loss: 27639.0176
Epoch 362/400
1/34 [..............................] - ETA: 0s - loss: 3046.8867
34/34 [==============================] - 0s 2ms/step - loss: 34469.4141 - val_loss: 28922.9258
Epoch 363/400
1/34 [..............................] - ETA: 0s - loss: 10579.3828
34/34 [==============================] - 0s 2ms/step - loss: 34763.2969 - val_loss: 28131.8848
Epoch 364/400
1/34 [..............................] - ETA: 0s - loss: 27476.0723
34/34 [==============================] - 0s 2ms/step - loss: 35499.1328 - val_loss: 29935.7520
Epoch 365/400
1/34 [..............................] - ETA: 0s - loss: 20064.5918
34/34 [==============================] - 0s 2ms/step - loss: 33762.8281 - val_loss: 27634.8008
Epoch 366/400
1/34 [..............................] - ETA: 0s - loss: 31571.7656
34/34 [==============================] - 0s 2ms/step - loss: 35024.4609 - val_loss: 32233.8086
Epoch 367/400
1/34 [..............................] - ETA: 0s - loss: 105132.0312
34/34 [==============================] - 0s 2ms/step - loss: 35979.0938 - val_loss: 27669.3320
Epoch 368/400
1/34 [..............................] - ETA: 0s - loss: 74330.9609
34/34 [==============================] - 0s 2ms/step - loss: 34877.2305 - val_loss: 36072.7227
Epoch 369/400
1/34 [..............................] - ETA: 0s - loss: 59901.3672
34/34 [==============================] - 0s 2ms/step - loss: 35445.8203 - val_loss: 27180.2969
Epoch 370/400
1/34 [..............................] - ETA: 0s - loss: 25387.1855
34/34 [==============================] - 0s 2ms/step - loss: 39328.0234 - val_loss: 28627.9160
Epoch 371/400
1/34 [..............................] - ETA: 0s - loss: 23321.5312
34/34 [==============================] - 0s 2ms/step - loss: 33857.2422 - val_loss: 26907.5156
Epoch 372/400
1/34 [..............................] - ETA: 0s - loss: 64746.1484
34/34 [==============================] - 0s 2ms/step - loss: 34408.1133 - val_loss: 26843.3145
Epoch 373/400
1/34 [..............................] - ETA: 0s - loss: 7776.4189
34/34 [==============================] - 0s 2ms/step - loss: 33728.8945 - val_loss: 29160.0020
Epoch 374/400
1/34 [..............................] - ETA: 0s - loss: 68628.8203
34/34 [==============================] - 0s 2ms/step - loss: 35139.1094 - val_loss: 26991.7715
Epoch 375/400
1/34 [..............................] - ETA: 0s - loss: 33352.1484
34/34 [==============================] - 0s 2ms/step - loss: 33972.5742 - val_loss: 28032.4512
Epoch 376/400
1/34 [..............................] - ETA: 0s - loss: 5491.6274
34/34 [==============================] - 0s 2ms/step - loss: 33172.3008 - val_loss: 29234.2070
Epoch 377/400
1/34 [..............................] - ETA: 0s - loss: 11878.7852
34/34 [==============================] - 0s 2ms/step - loss: 33864.0859 - val_loss: 27888.3555
Epoch 378/400
1/34 [..............................] - ETA: 0s - loss: 10281.4072
34/34 [==============================] - 0s 2ms/step - loss: 33611.0781 - val_loss: 26585.1035
Epoch 379/400
1/34 [..............................] - ETA: 0s - loss: 74049.7891
34/34 [==============================] - 0s 2ms/step - loss: 34873.1016 - val_loss: 29354.2207
Epoch 380/400
1/34 [..............................] - ETA: 0s - loss: 49149.1328
34/34 [==============================] - 0s 2ms/step - loss: 33997.4219 - val_loss: 27468.9121
Epoch 381/400
1/34 [..............................] - ETA: 0s - loss: 9962.4160
34/34 [==============================] - 0s 2ms/step - loss: 33063.1172 - val_loss: 33149.1094
Epoch 382/400
1/34 [..............................] - ETA: 0s - loss: 17689.9121
34/34 [==============================] - 0s 2ms/step - loss: 33579.2070 - val_loss: 27255.0742
Epoch 383/400
1/34 [..............................] - ETA: 0s - loss: 19589.0605
34/34 [==============================] - 0s 2ms/step - loss: 33100.3359 - val_loss: 30167.3047
Epoch 384/400
1/34 [..............................] - ETA: 0s - loss: 90793.2734
34/34 [==============================] - 0s 2ms/step - loss: 35324.4844 - val_loss: 28338.1582
Epoch 385/400
1/34 [..............................] - ETA: 0s - loss: 17090.2344
34/34 [==============================] - 0s 2ms/step - loss: 33673.2383 - val_loss: 28661.5605
Epoch 386/400
1/34 [..............................] - ETA: 0s - loss: 56417.5664
34/34 [==============================] - 0s 2ms/step - loss: 33098.4805 - val_loss: 28149.8789
Epoch 387/400
1/34 [..............................] - ETA: 0s - loss: 17946.8926
34/34 [==============================] - 0s 2ms/step - loss: 34123.1914 - val_loss: 27005.7930
Epoch 388/400
1/34 [..............................] - ETA: 0s - loss: 4953.2012
34/34 [==============================] - 0s 2ms/step - loss: 32653.5938 - val_loss: 26387.2031
Epoch 389/400
1/34 [..............................] - ETA: 0s - loss: 37416.4375
34/34 [==============================] - 0s 2ms/step - loss: 33345.9688 - val_loss: 26281.9609
Epoch 390/400
1/34 [..............................] - ETA: 0s - loss: 5162.0942
34/34 [==============================] - 0s 2ms/step - loss: 33873.7344 - val_loss: 31518.2793
Epoch 391/400
1/34 [..............................] - ETA: 0s - loss: 8200.2441
34/34 [==============================] - 0s 2ms/step - loss: 34797.4805 - val_loss: 26932.7500
Epoch 392/400
1/34 [..............................] - ETA: 0s - loss: 10621.7793
34/34 [==============================] - 0s 2ms/step - loss: 33278.8672 - val_loss: 26260.3223
Epoch 393/400
1/34 [..............................] - ETA: 0s - loss: 18414.7832
34/34 [==============================] - 0s 2ms/step - loss: 32638.2930 - val_loss: 26214.1465
Epoch 394/400
1/34 [..............................] - ETA: 0s - loss: 24632.3516
34/34 [==============================] - 0s 2ms/step - loss: 32256.9590 - val_loss: 27498.9629
Epoch 395/400
1/34 [..............................] - ETA: 0s - loss: 42598.3359
34/34 [==============================] - 0s 2ms/step - loss: 32347.1230 - val_loss: 26433.6270
Epoch 396/400
1/34 [..............................] - ETA: 0s - loss: 6639.9619
34/34 [==============================] - 0s 2ms/step - loss: 32901.7852 - val_loss: 27824.4512
Epoch 397/400
1/34 [..............................] - ETA: 0s - loss: 49513.5000
34/34 [==============================] - 0s 2ms/step - loss: 33408.6914 - val_loss: 26089.8809
Epoch 398/400
1/34 [..............................] - ETA: 0s - loss: 18591.3730
34/34 [==============================] - 0s 2ms/step - loss: 33112.8047 - val_loss: 26793.7891
Epoch 399/400
1/34 [..............................] - ETA: 0s - loss: 19865.0645
34/34 [==============================] - 0s 2ms/step - loss: 35116.7344 - val_loss: 27228.2090
Epoch 400/400
1/34 [..............................] - ETA: 0s - loss: 13863.8867
34/34 [==============================] - 0s 2ms/step - loss: 32440.0371 - val_loss: 27346.9688
findfont: Font family ['Times New Roman'] not found. Falling back to DejaVu Sans.
********** Successfully loaded weights from weights_397_26089.88086.hdf5 file **********
<keras.callbacks.History object at 0x7f7f51a7c8d0>
Training data
train_p = model.predict(x=X_train, )
assigning name input_1 to IteratorGetNext:0 with shape (None, 74)
1/34 [..............................] - ETA: 1s
34/34 [==============================] - 0s 652us/step
evaluate_model(y_train, train_p)
mse 31713.225852754815
rmse 178.08207616926194
r2 0.8342894040523627
r2_score 0.8319832131635584
mape inf
mae 86.4258317518567
Test data
test_p = model.predict(x=X_test, )
1/15 [=>............................] - ETA: 0s
15/15 [==============================] - 0s 645us/step
evaluate_model(y_test, test_p)
mse 26089.879582292997
rmse 161.5236192706596
r2 0.8193178304233172
r2_score 0.8191718136755141
mape inf
mae 79.02385055486761
Hyperparameter Optimization
PREFIX = f"hpo_mlp_{dateandtime_now()}"
ITER = 0
Number of iterations will be 70 when running locally, it will be 40 on cloud due to computational constraints.
num_iterations = 70
MONITOR = {"mse": [], "r2_score": [], "r2": []}
seed = 1575
spliter = TrainTestSplit(seed=seed)
train_x, val_x, train_y, val_y = spliter.split_by_random(X_train, y_train)
Objective Function
def objective_fn(
prefix: str = None,
return_model: bool = False,
epochs: int = 50,
fit_on_all_data: bool = False,
seed=seed,
**suggestions
) -> Union[float, Model]:
suggestions = jsonize(suggestions)
global ITER
# build model
_model = Model(
model=MLP(units=suggestions['units'],
num_layers=suggestions['num_layers'],
activation=suggestions['activation']),
batch_size=suggestions["batch_size"],
lr=suggestions["lr"],
prefix=prefix or PREFIX,
split_random=True,
seed=seed,
epochs=epochs,
input_features=dataset.input_features,
output_features=dataset.output_features,
verbosity=0)
# train model
if fit_on_all_data:
_model.fit(X_train, y_train, validation_data=(X_test, y_test))
else:
_model.fit(train_x, train_y, validation_data=(val_x, val_y))
# evaluate model
t, p = _model.predict(val_x, val_y, return_true=True,
process_results=False)
metrics = RegressionMetrics(t, p)
val_score = metrics.mse()
for metric in MONITOR.keys():
val = getattr(metrics, metric)()
MONITOR[metric].append(val)
# here we are evaluating model with respect to mse, therefore
# we don't need to subtract it from 1.0
if not math.isfinite(val_score):
val_score = 9999
print(f"{ITER} {val_score} {seed}")
ITER += 1
if fit_on_all_data:
_model.predict(X_train, y_train)
_model.predict(X_test, y_test)
if return_model:
return _model
return val_score
Parameter Space
param_space = [
Integer(30, 100, name="units"),
Integer(1, 4, name="num_layers"),
Categorical(["relu", "elu", "tanh", "sigmoid"], name="activation"),
Real(0.00001, 0.01, name="lr"),
Categorical([4, 8, 12, 16, 24, 32, 48, 64], name="batch_size")
]
x0 = [30, 1, "relu", 0.001, 8]
optimizer = HyperOpt(
algorithm="bayes",
objective_fn=objective_fn,
param_space=param_space,
x0=x0,
num_iterations=num_iterations,
process_results=True, # we can turn it False if we want post-processing of results
opt_path=f"results{SEP}{PREFIX}"
)
We have already optimized the hyperparameters using Bayesian with 100 iterations
Therefore, we are not running optimizer.fit online. We will, instead, load
the results of optimization and plot them. If you however want to optimize
the hyperparameters, you can set OPTIMIZE to True
OPTIMIZE = False
# path where hpo results are saved.
path = os.path.join(os.getcwd(), 'results', 'hpo_mlp_20221228_132336', 'hpo_results.bin')
if OPTIMIZE:
results = optimizer.fit()
else:
optimizer.load_results(path)
postprocessing of hpo results
best_iteration = optimizer.best_iter()
print(f"optimized parameters are \n{optimizer.best_paras()} at {best_iteration}")
optimized parameters are
{'units': 82, 'num_layers': 4, 'activation': 'relu', 'lr': 0.0014524284016223636, 'batch_size': 4} at 56
if OPTIMIZE:
for key in ['mse']:
print(key, np.nanmin(MONITOR[key]), np.nanargmin(MONITOR[key]))
if OPTIMIZE:
for key in ['r2', 'r2_score']:
print(key, np.nanmax(MONITOR[key]), np.nanargmax(MONITOR[key]))
optimizer._plot_convergence()
plt.show()
optimizer._plot_evaluations()
plt.tight_layout()
plt.show()
optimizer.plot_importance()
plt.tight_layout()
plt.show()
optimizer.plot_parallel_coords(figsize=(14, 8))
plt.tight_layout()
plt.show()
_ = plot_objective(optimizer.gpmin_results)
Performance with optimized hyperparameters
model = objective_fn(prefix=f"{PREFIX}{SEP}best",
seed=seed,
return_model=True,
epochs=400,
fit_on_all_data=True,
**optimizer.best_paras())
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
assigning name input_1 to IteratorGetNext:0 with shape (None, 74)
assigning name input_1 to IteratorGetNext:0 with shape (None, 74)
assigning name input_1 to IteratorGetNext:0 with shape (None, 74)
assigning name input_1 to IteratorGetNext:0 with shape (None, 74)
0 1610.342933790242 1575
model.evaluate(X_test, y_test, metrics=['r2', 'nse'])
{'r2': 0.9798527409077847, 'nse': 0.9797774768887206}
Total running time of the script: (4 minutes 30.225 seconds)