1、Beyond One-Class Classification,Amfeng24 March 2009,Outline,Two models for One Class ClassificationFrom One Class to Binary ClassFrom Binary Class to Multi ClassFrom Multi Class to ClusteringConclustion,Two models for One Class Classification,One Class SVM Find the optimal hyperplane to separate the
2、 target class from the origin with maximum marginSupport Vector Data Description Use the minimum hyperspere to enclose the target class,Interpretation of the above models,How to extend to Binary or Multi Classification,For imbalance data,From SVDD to Binary SVDD with negative data: B_SVDD_Neg,Object
3、ive function:,Drawback: without considering the margin between classes.,The other Version of B_SVDD_Neg,Dong, X., W. Zhaohui, et al. (2001). A new multi-class support vector machines, Systems, Man, and Cybernetics, 2001 IEEE International Conference on.,Embedding margin for B_SVDD_Neg,范炜 (2003). 支持向
4、量机算法的研究及其应用, 浙江大学. PhD.,The dual form,Notice,How to get the radius R,Must find the support vector on the hypersphere that negative lived,Does it work really?,No support vector of Negative data.,Cant calculate R,Solutions for above problem,1. Modify the coefficient of R: Biased support vector machine
5、,Chan, C.-H., K. Huang, and M.R.L.a.I. King. Biased support vector machine for relevance feedback in image retrieval. in International Joint Conference on Neural Networks 2004. Budapest, Hungary.,In order to avoid the above problem, b need to less than 1, that is,Equivalent style: Minimum Enclosing
6、and Maximum Excluding Machine,Liu, Y. and Y.F. Zheng. Minimum Enclosing and Maximum Excluding Machine for Pattern Description and Discrimination Pattern Recognition. in Proc of the 18th Int Conf on ICPR 2006.Loa Alamitos: IEEE Computer Society,2. Modify the coefficient of margin,Here,Wang, J., N. P,
7、 et al. (2005). Pattern classification via single spheres, Lecture notes in artificial intelligence.( briefly PCSS),3. Modify the coefficients of margin and R,张新峰; 刘垚巍: 广义超球面SVM研究 ,计算机研究与发展 2008.11,Generalized HyperSphere SVM(GHSSVM),Extend to Ellipsoid,Wei2007:Minimum Mahalanobis Enclosing Ellipsoi
8、d Machine for Pattern Classification:ICIC 2007,CCIS2, pp. 1176-1185,SVDD with negative data for Multi-Class:M_SVDD_Neg,Drawback: without considering the margin either .,Embedding margin for SVDD_Mulit: MSM_SVM,Pei-Yi Hao, Jung Hsien Chiang, Yen Hsiu lin:A new maximal-margin spherical-structured mult
9、i-class support vector machine, Appl Intell, 2009,30,P98-111,Dual formulation,Without the problem discussed at the former.,Illustration of the difference,How about the hypenplane model,OCSVM with negative: Binary OCSVM_Neg,Motivation: using the mean of the other class instead of the optimal point. D
10、oesnt considering margin either.,From OCSVM to Asymmetric SVM: margin embeded,Like the SVDD Multi with margin, here also describe the target class by core hyperplane, then push the negative class by maximized the margin.,S. H. Wu, K. P. Lin, C. M. Chen, M. S. Chen, Asymmetric support vector machines
11、: low false positive learning under the user tolerance, Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, 749-757, 2008.,Summarize,One Class Classification for Clustering,Support Vector Clustering(JMLR2002)Iterative strategy integrating two-stage one-
12、class SVMKernel Growth (PAMI 2005)Soft Clustering for Kernel Growth,Support Vector Clustering,Clustering boundary: same as SVDD, found the support vector to get the boundary.Clustering number: based on the adjacency matrix which components decided according to:,Ben-Hur, H. A., D., et al. (2002). Sup
13、port vector clustering Journal of Machine Learning Research 2 125-137.,The kernel width decided the clustering number,Outlier enable makes the clustering possible,Iterative strategy integrating two-stage one-class SVM,Yeh, C.-Y. and S.-J. Lee (2007). A Kernel-Based Two-Stage One-Class Support Vector
14、 Machines Algorithm. Advances in Neural Networks ISNN 2007.,Different from SVC, need to know the clustering number in advance, attribute to partition-based clustering algorithm.First stage: using OCSVM for each cluster to find the non-support vectors ;Second stage: retrain the OCSVM using those non-
15、support vector for representing each clustering accurately by the optimal hyperplane.,Illustration,Clustering assignment: each pattern is assign to the maximum projection value by:,Conclusion,One Class Classifier of SVDD and OCSVM can be used in many field, including:Binary/Multi Class for unbalance data ClusteringLarge scale problem: CVM &BVMDe-noisingInformation processingDocument classificationImage retrieval .,
