When do we use Kappa Statistic as a measure of model performance vs Accuracy

@pagal_guy wrote:

Hello,

In the below code and images I am trying to solve a classification problem by gbm:
## gbm fitting:
set.seed(123)
fitControl <- trainControl(method = 'cv', number = 10, summaryFunction=defaultSummary)
Grid <- expand.grid( n.trees = seq(50,1000,50), interaction.depth = c(30), shrinkage = c(0.1),n.minobsinnode = 10)
formula <- as.factor(Recommended) ~ .
#Fit gbm model:
fit.gbm <- train(formula, data=X, method = 'gbm', trControl=fitControl,tuneGrid=Grid,metric='Kappa',maximize=FALSE)
plot(fit.gbm)

The plots for Kappa and Accuracy:
Kappa

Accuracy

The iterations yield similar results but the Kappa is very low(should be more than 0.30).
Kappa is I believe a better measure of performance than Accuracy,but here the model performs worse if I keep metric as Kappa.
Why is this??Which one should we use and when?
Can someone please help me with this??

Posts: 1

Participants: 1

Read full topic