Multivariate LSTM RMSE value is getting very high

@shounakrockz47 wrote:

I want to predict a time series with multiple variables. I am using Keras’s LSTM class.

Here is my data set description :

I want to predict var1(t-1) and my X variables are var3(t-1) , var4(t-1) , var5(t-1) , var6(t-1) and var7(t-1).

And this is how my model configuration looks like :

{
	"data": {
		
		"columns": [            
			"var1(t-1)",
			"var3(t-1)",
			"var4(t-1)",          
			"var5(t-1)",
			"var6(t-1)",
			"var7(t-1)" 
		],
		"sequence_length": 5,
		"train_test_split": 0.80,
		"normalise": false
	},
	"training": {
		"epochs": 100,
		"batch_size":10 
	},
	"model": {
		"loss": "mse",
		"optimizer": "nadam",
		"save_dir": "saved_models_multi",
		"layers": [
			{
				"type": "lstm",
				"neurons": 200,
				"input_timesteps": 5,
				"input_dim": 5,
				"return_seq": true
			},
            {
              "type":"batch_norm"  
                
            },  
			{
				"type": "dropout",
				"rate": 0.4
			},
            
            {
				"type": "lstm",
				"neurons": 200,
				"return_seq": true
			},
            {
              "type":"batch_norm"  
                
            },
            
            {
				"type": "dropout",
				"rate": 0.4
			},
			{
				"type": "dense",
				"neurons": 50,
				"activation": "sigmoid"
			},
            
             {
              "type":"batch_norm"  
                
            },
            
            {
				"type": "dropout",
				"rate": 0.2
			},
            
            {
				"type": "lstm",
				"neurons": 200,
				"return_seq": false
			},
            {
              "type":"batch_norm"  
                
            },
            
            {
				"type": "dropout",
				"rate": 0.4
			},
            {
				"type": "dense",
				"neurons": 25,
				"activation": "sigmoid"
			},
            
             {
              "type":"batch_norm"  
                
            },
            
            {
				"type": "dropout",
				"rate": 0.2
			},

			{
				"type": "dense",
				"neurons": 1,
				"activation": "linear"
			}
		]
	}
}

This is number of parameters in each layer and output shape :

Layer (type)                 Output Shape              Param #   
=================================================================
lstm_1 (LSTM)                (None, 5, 200)            164800    
_________________________________________________________________
batch_normalization_1 (Batch (None, 5, 200)            800       
_________________________________________________________________
dropout_1 (Dropout)          (None, 5, 200)            0         
_________________________________________________________________
lstm_2 (LSTM)                (None, 5, 200)            320800    
_________________________________________________________________
batch_normalization_2 (Batch (None, 5, 200)            800       
_________________________________________________________________
dropout_2 (Dropout)          (None, 5, 200)            0         
_________________________________________________________________
dense_1 (Dense)              (None, 5, 50)             10050     
_________________________________________________________________
batch_normalization_3 (Batch (None, 5, 50)             200       
_________________________________________________________________
dropout_3 (Dropout)          (None, 5, 50)             0         
_________________________________________________________________
lstm_3 (LSTM)                (None, 200)               200800    
_________________________________________________________________
batch_normalization_4 (Batch (None, 200)               800       
_________________________________________________________________
dropout_4 (Dropout)          (None, 200)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 25)                5025      
_________________________________________________________________
batch_normalization_5 (Batch (None, 25)                100       
_________________________________________________________________
dropout_5 (Dropout)          (None, 25)                0         
_________________________________________________________________
dense_3 (Dense)              (None, 1)                 26        
=================================================================
Total params: 704,201
Trainable params: 702,851
Non-trainable params: 1,350
_________________________________________________________________
Time taken: 0:00:01.914407

But , I am seeing RMSE value for test data is getting very high.
This is my train and validation loss graph.

What am I doing wrong here ? Is my model suffering from overfitting problem ?

Please help .

Posts: 1

Participants: 1

Read full topic