Building a Robust Classifier Based on Linear Discriminant Analysis
M. J. Jamnejad†*, J. D Hamid‡, & R. Ghader†
†Department of Computer Engineering, Beyza Branch, Islamic Azad University, Beyza, Iran,*Email: [email protected]‡School of Mechanical Engineering, Universiti Sains Malaysia, Perak, Malaysia
ABSTRACT: Usage of recognition systems has found many applications in almost all fields. Generally in design of multiple classifier systems, the more diverse the results of the classifiers, the more appropriate the aggregated result. While most of classification algorithms have obtained a good performance for specific problems they have not enough robustness for other problems. Combination of multiple classifiers can be considered as a general solution method for pattern recognition problems. It has been shown that combination of multiple classifiers can usually operate better than a single classifier system provided that its components are independent or their components have diverse outputs. It has been shown that the necessary diversity for the ensemble can be achieved by manipulation of dataset features, manipulation of data points in dataset, different sub-samplings of dataset, and usage of different classification algorithms. We also propose a new method of creating this diversity. We use Linear Discriminant Analysis to manipulate the data points in dataset. The ensemble created by proposed method may not always outperform any of its members, it always possesses the diversity needed for creation of an ensemble, and consequently it always outperforms the simple classifier systems.
Keywords: Classifier fusion; Heuristic retraining; Neural network ensemble; Linear discriminant analysis.