Runs n-fold cross validation with a given classifier. Given n separate labeled data sets, trains classifier using n-1 data sets, test on remaining one. Average results over all n such runs. Shows overal results in one big confusion matrix and displays the results. The classifier is passed in as a parameter. For this to work the classifier must follow certain conventions. The conventions are as follows: 1) The following must initialize the model (nin is the dim of the data): model = func_netinit( nin, netparams{:} ) 2) The crated model must have point to 2 function for training and applying it: model.fun_train and model.fun_fwd 3) For training the following will be called: net = fun_train( net, X, Y ); 4) For testing the following will be called: pred = fun_fwd( net, Xtest ); The format for X is nxp where n is the number of data points and p is their dimension. The format for Y is nx1. Example of a classifier is: func_netinit = @clf_lda Given data in a cell array format, it might be useful to string it out into a single array: IDX = cell2mat( permute( IDX, [2 1] ) ); data = cell2mat( permute( data, [2 1] ) ); Overall error can be calculated via: er = 1-sum(diag(CM))/sum(CM(:)) Normalized confusion matrix can be calculated via: CMn = CM ./ repmat( sum(CM,2), [1 size(CM,2)] ); For a simple, small dataset, can do the following to do leave one out classification: [n,p]=size(data); IDX=mat2cell(IDX,ones(1,n),1); data=mat2cell(data,ones(1,n),p); INPUTS data - cell array of (ni x p) arrays of ni samples of dim p IDX - cell array of (ni x 1) arrays of ni labels func_netinit- classifier init (see above) netparams - classifier parameters (see above) types - [optional] cell array of string labels for types disctypes - [optional] array of types we aren't interested in {eg: [1 4 5]}. fname - [optional] specify a file to save CM to, as well as image show - [optional] will display results in figure(show) OUTPUTS CM - confusion matrix EXAMPLE load clf_data; nfoldxval( data, IDX, @clf_lda,{'linear'}, [],[],[],1 ); % LDA nfoldxval( data, IDX, @clf_knn,{4},[],[],[],2 ); % 4 k nearest neighbor nfoldxval( data, IDX, @clf_svm,{'poly',2},[],[],[],3 ); % polynomial SVM nfoldxval( data, IDX, @clf_svm,{'rbf',2^-12},[],[],[],4 ); % rbf SVM nfoldxval( data, IDX, @clf_dectree,{},[],[],[],1 ); % decision tree % for multi-class data nfoldxval( data, IDX, @clf_ecoc,{@clf_svm,{'rbf',2^-12},nclasses},[],[],[],4 ); % ECOC DATESTAMP 29-Sep-2005 2:00pm See also CLF_LDA, CLF_KNN, CLF_SVM, CLF_ECOC