1. Logistic Regression

Running Weka:

in the file responsible for fitting the lmFit, I added some extra lines so that a file would be created under the name : ExpMat_nominal.csv
This file contains all of the peptides with the following order. the first 10 patients are the AR patients and the next 24 patients are the NR patients. I added a column to this file indicating whether a patient is AR or not.

Next, I opened this file in Weka explorer page.
From the classify menu, I chose the following functions and ran them on this data to find out the accuracy of predicting the group (AR /NR) based on all the peptides.

1. Logistic Regression:
a 10 fold cross validation on the data with all the peptides gives 61.76% accuracy.

Correctly Classified Instances          21               61.7647 %
Incorrectly Classified Instances        13               38.2353 %
Kappa statistic                         -0.1571
Mean absolute error                      0.3564
Root mean squared error                  0.5758
Relative absolute error                 84.6275 %
Root relative squared error            126.1897 %
Total Number of Instances               34


=== Detailed Accuracy By Class ===

               TP Rate   FP Rate   Precision   Recall  F-Measure   ROC Area  Class
                 0         0.125      0         0         0          0.579    AR
                 0.875     1          0.677     0.875     0.764      0.577    NR
Weighted Avg.    0.618     0.743      0.478     0.618     0.539      0.578

=== Confusion Matrix ===

  a  b   <-- classified as
  0 10 |  a = AR
  3 21 |  b = NR

with the file in which the subset of data for which the p value was less than 0.1 the following results were achieved
Correctly Classified Instances          22               64.7059 %
Incorrectly Classified Instances        12               35.2941 %
Kappa statistic                          0.15  
Mean absolute error                      0.3669
Root mean squared error                  0.5962
Relative absolute error                 87.1282 %
Root relative squared error            130.6554 %
Total Number of Instances               34     

=== Detailed Accuracy By Class ===

               TP Rate   FP Rate   Precision   Recall  F-Measure   ROC Area  Class
                 0.4       0.25       0.4       0.4       0.4        0.683    AR
                 0.75      0.6        0.75      0.75      0.75       0.696    NR
Weighted Avg.    0.647     0.497      0.647     0.647     0.647      0.692

=== Confusion Matrix ===

  a  b   <-- classified as
  4  6 |  a = AR
  6 18 |  b = NR


a choice of 4 peptides as given below based on my visualization technique led to:
peptides chosen:
TLAFPLTIR
HGNTDSEGIVEVK
TPDVSSALDK
VLNQELR


Correctly Classified Instances          28               82.3529 %
Incorrectly Classified Instances         6               17.6471 %
Kappa statistic                          0.575 
Mean absolute error                      0.2488
Root mean squared error                  0.3881
Relative absolute error                 59.0782 %
Root relative squared error             85.0484 %
Total Number of Instances               34     

=== Detailed Accuracy By Class ===

               TP Rate   FP Rate   Precision   Recall  F-Measure   ROC Area  Class
                 0.7       0.125      0.7       0.7       0.7        0.846    AR
                 0.875     0.3        0.875     0.875     0.875      0.846    NR
Weighted Avg.    0.824     0.249      0.824     0.824     0.824      0.846

=== Confusion Matrix ===

  a  b   <-- classified as
  7  3 |  a = AR
  3 21 |  b = NR