Example-5 (Sample weights)

Without weights

In [1]:
from pycm import ConfusionMatrix
y_test = [2, 0, 2, 2, 0, 1, 1, 2, 2, 0, 1, 2]
y_pred = [0, 0, 2, 1, 0, 2, 1, 0, 2, 0, 2, 2]
In [2]:
cm1=ConfusionMatrix(y_test, y_pred)
cm1
Out[2]:
pycm.ConfusionMatrix(classes: [0, 1, 2])
In [3]:
print(cm1)
Predict          0    1    2    
Actual
0                3    0    0    

1                0    1    2    

2                2    1    3    





Overall Statistics : 

95% CI                                                           (0.30439,0.86228)
AUNP                                                             0.66667
AUNU                                                             0.69444
Bennett S                                                        0.375
CBA                                                              0.47778
Chi-Squared                                                      6.6
Chi-Squared DF                                                   4
Conditional Entropy                                              0.95915
Cramer V                                                         0.5244
Cross Entropy                                                    1.59352
Gwet AC1                                                         0.38931
Hamming Loss                                                     0.41667
Joint Entropy                                                    2.45915
KL Divergence                                                    0.09352
Kappa                                                            0.35484
Kappa 95% CI                                                     (-0.07708,0.78675)
Kappa No Prevalence                                              0.16667
Kappa Standard Error                                             0.22036
Kappa Unbiased                                                   0.34426
Lambda A                                                         0.16667
Lambda B                                                         0.42857
Mutual Information                                               0.52421
NIR                                                              0.5
Overall ACC                                                      0.58333
Overall CEN                                                      0.46381
Overall J                                                        (1.225,0.40833)
Overall MCC                                                      0.36667
Overall MCEN                                                     0.51894
Overall RACC                                                     0.35417
Overall RACCU                                                    0.36458
P-Value                                                          0.38721
PPV Macro                                                        0.56667
PPV Micro                                                        0.58333
Phi-Squared                                                      0.55
RCI                                                              0.34947
RR                                                               4.0
Reference Entropy                                                1.5
Response Entropy                                                 1.48336
SOA1(Landis & Koch)                                              Fair
SOA2(Fleiss)                                                     Poor
SOA3(Altman)                                                     Fair
SOA4(Cicchetti)                                                  Poor
Scott PI                                                         0.34426
Standard Error                                                   0.14232
TPR Macro                                                        0.61111
TPR Micro                                                        0.58333
Zero-one Loss                                                    5

Class Statistics :

Classes                                                          0                       1                       2                       
ACC(Accuracy)                                                    0.83333                 0.75                    0.58333                 
AUC(Area under the roc curve)                                    0.88889                 0.61111                 0.58333                 
AUCI(Auc value interpretation)                                   Very Good               Fair                    Poor                    
BM(Informedness or bookmaker informedness)                       0.77778                 0.22222                 0.16667                 
CEN(Confusion entropy)                                           0.25                    0.49658                 0.60442                 
DOR(Diagnostic odds ratio)                                       None                    4.0                     2.0                     
DP(Discriminant power)                                           None                    0.33193                 0.16597                 
DPI(Discriminant power interpretation)                           None                    Poor                    Poor                    
ERR(Error rate)                                                  0.16667                 0.25                    0.41667                 
F0.5(F0.5 score)                                                 0.65217                 0.45455                 0.57692                 
F1(F1 score - harmonic mean of precision and sensitivity)        0.75                    0.4                     0.54545                 
F2(F2 score)                                                     0.88235                 0.35714                 0.51724                 
FDR(False discovery rate)                                        0.4                     0.5                     0.4                     
FN(False negative/miss/type 2 error)                             0                       2                       3                       
FNR(Miss rate or false negative rate)                            0.0                     0.66667                 0.5                     
FOR(False omission rate)                                         0.0                     0.2                     0.42857                 
FP(False positive/type 1 error/false alarm)                      2                       1                       2                       
FPR(Fall-out or false positive rate)                             0.22222                 0.11111                 0.33333                 
G(G-measure geometric mean of precision and sensitivity)         0.7746                  0.40825                 0.54772                 
GI(Gini index)                                                   0.77778                 0.22222                 0.16667                 
IS(Information score)                                            1.26303                 1.0                     0.26303                 
J(Jaccard index)                                                 0.6                     0.25                    0.375                   
LS(Lift score)                                                   2.4                     2.0                     1.2                     
MCC(Matthews correlation coefficient)                            0.68313                 0.2582                  0.16903                 
MCEN(Modified confusion entropy)                                 0.26439                 0.5                     0.6875                  
MK(Markedness)                                                   0.6                     0.3                     0.17143                 
N(Condition negative)                                            9                       9                       6                       
NLR(Negative likelihood ratio)                                   0.0                     0.75                    0.75                    
NPV(Negative predictive value)                                   1.0                     0.8                     0.57143                 
P(Condition positive or support)                                 3                       3                       6                       
PLR(Positive likelihood ratio)                                   4.5                     3.0                     1.5                     
PLRI(Positive likelihood ratio interpretation)                   Poor                    Poor                    Poor                    
POP(Population)                                                  12                      12                      12                      
PPV(Precision or positive predictive value)                      0.6                     0.5                     0.6                     
PRE(Prevalence)                                                  0.25                    0.25                    0.5                     
RACC(Random accuracy)                                            0.10417                 0.04167                 0.20833                 
RACCU(Random accuracy unbiased)                                  0.11111                 0.0434                  0.21007                 
TN(True negative/correct rejection)                              7                       8                       4                       
TNR(Specificity or true negative rate)                           0.77778                 0.88889                 0.66667                 
TON(Test outcome negative)                                       7                       10                      7                       
TOP(Test outcome positive)                                       5                       2                       5                       
TP(True positive/hit)                                            3                       1                       3                       
TPR(Sensitivity, recall, hit rate, or true positive rate)        1.0                     0.33333                 0.5                     
Y(Youden index)                                                  0.77778                 0.22222                 0.16667                 
dInd(Distance index)                                             0.22222                 0.67586                 0.60093                 
sInd(Similarity index)                                           0.84287                 0.52209                 0.57508                 

With random weights

In [4]:
from random import randint
weights = [randint(1,10) for i in range(len(y_test))]
weights[2]*=9
In [5]:
cm2=ConfusionMatrix(y_test, y_pred, sample_weight = weights)
cm2
Out[5]:
pycm.ConfusionMatrix(classes: [0, 1, 2])
In [6]:
print(cm2)
Predict          0     1     2     
Actual
0                26    0     0     

1                0     7     7     

2                17    8     13    





Overall Statistics : 

95% CI                                                           (0.48058,0.6989)
AUNP                                                             0.68654
AUNU                                                             0.70253
Bennett S                                                        0.38462
CBA                                                              0.47114
Chi-Squared                                                      40.91102
Chi-Squared DF                                                   4
Conditional Entropy                                              0.92088
Cramer V                                                         0.5121
Cross Entropy                                                    1.66986
Gwet AC1                                                         0.4004
Hamming Loss                                                     0.41026
Joint Entropy                                                    2.39942
KL Divergence                                                    0.19132
Kappa                                                            0.37538
Kappa 95% CI                                                     (0.20918,0.54158)
Kappa No Prevalence                                              0.17949
Kappa Standard Error                                             0.0848
Kappa Unbiased                                                   0.35042
Lambda A                                                         0.225
Lambda B                                                         0.2
Mutual Information                                               0.51361
NIR                                                              0.48718
Overall ACC                                                      0.58974
Overall CEN                                                      0.44197
Overall J                                                        (1.21172,0.40391)
Overall MCC                                                      0.40671
Overall MCEN                                                     0.49364
Overall RACC                                                     0.3432
Overall RACCU                                                    0.36843
P-Value                                                          0.04451
PPV Macro                                                        0.57377
PPV Micro                                                        0.58974
Phi-Squared                                                      0.5245
RCI                                                              0.34738
RR                                                               26.0
Reference Entropy                                                1.47853
Response Entropy                                                 1.43449
SOA1(Landis & Koch)                                              Fair
SOA2(Fleiss)                                                     Poor
SOA3(Altman)                                                     Fair
SOA4(Cicchetti)                                                  Poor
Scott PI                                                         0.35042
Standard Error                                                   0.05569
TPR Macro                                                        0.61404
TPR Micro                                                        0.58974
Zero-one Loss                                                    32

Class Statistics :

Classes                                                          0                       1                       2                       
ACC(Accuracy)                                                    0.78205                 0.80769                 0.58974                 
AUC(Area under the roc curve)                                    0.83654                 0.6875                  0.58355                 
AUCI(Auc value interpretation)                                   Very Good               Fair                    Poor                    
BM(Informedness or bookmaker informedness)                       0.67308                 0.375                   0.16711                 
CEN(Confusion entropy)                                           0.24897                 0.50376                 0.64066                 
DOR(Diagnostic odds ratio)                                       None                    7.0                     2.45143                 
DP(Discriminant power)                                           None                    0.46593                 0.2147                  
DPI(Discriminant power interpretation)                           None                    Poor                    Poor                    
ERR(Error rate)                                                  0.21795                 0.19231                 0.41026                 
F0.5(F0.5 score)                                                 0.65657                 0.47297                 0.55085                 
F1(F1 score - harmonic mean of precision and sensitivity)        0.75362                 0.48276                 0.44828                 
F2(F2 score)                                                     0.88435                 0.49296                 0.37791                 
FDR(False discovery rate)                                        0.39535                 0.53333                 0.35                    
FN(False negative/miss/type 2 error)                             0                       7                       25                      
FNR(Miss rate or false negative rate)                            0.0                     0.5                     0.65789                 
FOR(False omission rate)                                         0.0                     0.11111                 0.43103                 
FP(False positive/type 1 error/false alarm)                      17                      8                       7                       
FPR(Fall-out or false positive rate)                             0.32692                 0.125                   0.175                   
G(G-measure geometric mean of precision and sensitivity)         0.77759                 0.48305                 0.47156                 
GI(Gini index)                                                   0.67308                 0.375                   0.16711                 
IS(Information score)                                            0.85914                 1.37851                 0.41599                 
J(Jaccard index)                                                 0.60465                 0.31818                 0.28889                 
LS(Lift score)                                                   1.81395                 2.6                     1.33421                 
MCC(Matthews correlation coefficient)                            0.63795                 0.36515                 0.19129                 
MCEN(Modified confusion entropy)                                 0.26465                 0.52818                 0.69557                 
MK(Markedness)                                                   0.60465                 0.35556                 0.21897                 
N(Condition negative)                                            52                      64                      40                      
NLR(Negative likelihood ratio)                                   0.0                     0.57143                 0.79745                 
NPV(Negative predictive value)                                   1.0                     0.88889                 0.56897                 
P(Condition positive or support)                                 26                      14                      38                      
PLR(Positive likelihood ratio)                                   3.05882                 4.0                     1.95489                 
PLRI(Positive likelihood ratio interpretation)                   Poor                    Poor                    Poor                    
POP(Population)                                                  78                      78                      78                      
PPV(Precision or positive predictive value)                      0.60465                 0.46667                 0.65                    
PRE(Prevalence)                                                  0.33333                 0.17949                 0.48718                 
RACC(Random accuracy)                                            0.18376                 0.03452                 0.12492                 
RACCU(Random accuracy unbiased)                                  0.19564                 0.03456                 0.13823                 
TN(True negative/correct rejection)                              35                      56                      33                      
TNR(Specificity or true negative rate)                           0.67308                 0.875                   0.825                   
TON(Test outcome negative)                                       35                      63                      58                      
TOP(Test outcome positive)                                       43                      15                      20                      
TP(True positive/hit)                                            26                      7                       13                      
TPR(Sensitivity, recall, hit rate, or true positive rate)        1.0                     0.5                     0.34211                 
Y(Youden index)                                                  0.67308                 0.375                   0.16711                 
dInd(Distance index)                                             0.32692                 0.51539                 0.68077                 
sInd(Similarity index)                                           0.76883                 0.63557                 0.51862