Runs n-fold cross validation on data with a given classifier. Given n separate labeled data sets, trains classifier using n-1 data sets, test on remaining one. Average results over all n such runs. Shows overall results in average confusion matrix. The classifier is passed in as a parameter. For this to work the classifier (clf) must follow certain conventions. The conventions are: 1) To initialize the clf ('p' is the dimension of the data): clf = clfInit( p, clfparams{:} ) 2) clf must point to 2 functions for training and applying it: clf.funTrain and clf.funFwd 3) For training the following will be called: clf = clf.funTrain( clf, X, Y ); 4) For testing the following will be called: pred = clf.funFwd( clf, Xtest ); The format for X is nxp where there are n data points and p is their dimension. The format for Y is nx1. Given data in a cell array, to string out into single array: IDX = cell2mat(permute(IDX,[2 1])); data = cell2mat(permute(data,[2 1])); For a simple, small dataset, can do leave one out clf as follows: [n,p]=size(data); IDX=mat2cell(IDX,ones(1,n),1); data=mat2cell(data,ones(1,n),p); Overall error can be calculated via: er = 1-sum(diag(CM))/sum(CM(:)) Normalized confusion matrix can be calculated via: CMn = CM ./ repmat( sum(CM,2), [1 size(CM,2)] ); USAGE CM=nfoldxval( data, IDX, clfInit, clfparams, ... [types], [ignoreT], [fname], [show] ) INPUTS data - cell array of (n x p) arrays each of n samples of dim p IDX - cell array of (n x 1) arrays each of n labels clfInit - classifier initialization function clfparams - classifier parameters types - [] cell array of string labels for types ignoreT - [] array of types to ignore {eg: [1 4 5]}. fname - [] specify a file to save CM to, as well as image show - [] will display results in figure(show) OUTPUTS CM - confusion matrix EXAMPLE load clfData; %%% 2 class nfoldxval( data, IDX, @clfLda,{'linear'}, [],[],[],1 ); % LDA nfoldxval( data, IDX, @clfKnn,{4},[],[],[],2 ); % 4 kNN nfoldxval( data, IDX, @clfSvm,{'poly',2},[],[],[],3 ); % poly SVM nfoldxval( data, IDX, @clfSvm,{'rbf',2^-12},[],[],[],4 ); % rbf SVM nfoldxval( data, IDX, @clfDecTree,{},[],[],[],5 ); % dec. tree %%% multiclass clfparams = {@clfSvm,{'rbf',2^-12},nclasses}; nfoldxval( data, IDX, @clfEcoc,clfparams,[],[],[],6 ); % ECOC See also CLFKNN, CLFLDA, CLFSVM, CLFECOC, CLFDECTREE, DEMOCLASSIFY