...

To get unbiased correct rate, the test set sample must be independent from the training set. Now we are going to load another dataset, it has 8 samples with logged intensity values on the set of genes as that of the training data set. To use a complete independent test set to get correct rate is called hold-out validation.

• Choose File>Open... to open to browse and open testSet.fmt
• Choose Tools>Predict>Run Deployed Model... from the menu
• Select 20var-3NN-Euclidean.ppb to open, click on Test button to run,  the correct rate is reported on the top of the dialog (Figure 6)

• Click Add Prediction to New Spreadsheet to generate new spreadsheet with a predicted class name in the first column, the samples (rows) whose predicted and real class name are different are highlighted (Figure 7)

• Click on Test Report will generate a report in HTML format
• Click Close to dismiss the dialog

Hold-out validation have to split the whole data into two parts -- training set and test set. Increasing the size of training set  will improve the efficiency of the fitted predicted models; increasing the size of test set will improve power of validation. When the sample size is small (here the example data is just illustrate the function), the result is not precise. Some people believe that you should have at least 100 test samples to properly measure the correct rate with a useful precision. 

Another method to get unbiased accuracy estimate in Partek Genomics Suite is to do 2-Level Cross validation. You can use the whole data set --combining both training set and test set samples, in this example is to use the 36 samples

Cross validation

Common mistakes