Keras.io Tensorflow Neural Network Out of Memory on GPU Issue

I am currently recieving an OOM Error while using GridSearchCV function and grid.fit. The issue is I only recieve the OOM error when my SECOND fit starts... I've tried turning Batch Size from 100 down to 1, and even only 1 epoch still same issue. The exact error is: ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[12288,12288] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc Anyone have any idea to better handle the OOM issue other than "smaller dataset" ? I'm running GTX 1080 32GB RAM and Tensorflow is set to use GPU Not sure if I provided enough information, let me know if I'm missing anything that would help diagnose. -------- My settings are as follows: def create_model(optimizer=Adam, learn_rate=0.001): inputs = Input(shape=(image_width*image_height*3,)) outputs = Dense(image_width*image_height*3, activation='tanh' )(inputs) model = Model(inputs=inputs, outputs=outputs) model_optimizer = optimizer(lr=learn_rate) [login to view URL](optimizer=model_optimizer, loss='mse', metrics=['mae']) return model model = KerasRegressor(build_fn=create_model, verbose=1) learn_rate_list = [0.03] batch_size = [1] optimizer_list = [Adam] epochs = [1] param_grid = dict(optimizer=optimizer_list, learn_rate=learn_rate_list, epochs=epochs, batch_size=batch_size) grid = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=1, scoring='neg_mean_absolute_error', verbose=1, cv=2)