Selection of relevant and appropriate features to characterize breast patterns is of paramount importance in breast tissue representation and classification in machine learning paradigm. Feature selection based on single evaluation criterion has shown limited capability in breast tumor detection and classification due to their biases towards single criterion. In this paper, a new hybrid feature selection scheme is used to determine most relevant features for classification of benign and malignant tumors in breast ultrasound images. The proposed approach uses ten different evaluation criteria to decide the relevance of a particular feature. The existing feature selection techniques are also reviewed. A new database of 178 breast ultrasound images consisting of 88 benign and 90 malignant cases are used in experiments. The performance of the proposed approach is compared with that of existing feature selection techniques using back-propagation artificial neural network (BPANN) and support vector machine (SVM) based classifiers. The results demonstrate that proposed feature selection approach outperformed traditional methods achieving significantly higher classification accuracy of 96.6% and 94.4 % with BPANN and SVM classifiers respectively.