Don’t Overfit! — How to prevent Overfitting in your Deep Learning Models
Base Model To see how we can prevent overfitting, we first need to create a base model to compare the improved models to. The base model is a simple keras model with two hidden layers with 128 and 64 neurons. You can check it out here: model = keras . Sequential () model . add ( keras . layers . Dense ( 300 , activation = tf . nn . relu , input_dim = 300 )) model . add ( keras . layers . Dense ( 128 , activation = tf . nn . relu )) model . add ( keras . layers . Dense ( 64 , activation = tf . nn . relu )) model . add ( keras . layers . Dense ( 1 , activation = tf . nn . sigmoid )) model . compile ( optimizer = 'adam' , loss = 'binary_crossentropy' , metrics = [ 'accuracy' ]) history = model . fit ( df_train , train_labels , batch_size = 32 , epochs = 100 , validation_split = 0.2 , shuffle = True ) With this model we can achieve a training accuracy of over 97%, but a validation accuracy of only about 60%. In the graphic below we can se...