Global Model Selection via Solution Paths for Robust Support Vector Machine
Zhai, Zhou Gu, Bin Deng, Cheng Huang, Heng
Zhai, Zhou
Gu, Bin
Deng, Cheng
Huang, Heng
Author
Supervisor
Department
Machine Learning
Embargo End Date
Type
Journal article
Date
2025
License
Language
English
Collections
Research Projects
Organizational Units
Journal Issue
Abstract
Robust support vector machine (RSVM) using ramp loss provides a better generalization performance than traditional support vector machine (SVM) using hinge loss. However, the good performance of RSVM heavily depends on the proper values of regularization parameter and ramp parameter. Traditional model selection technique with gird search has extremely high computational cost especially for fine-grained search. To address this challenging problem, in this paper, we first propose solution paths of RSVM (SPRSVM) based on the concave-convex procedure (CCCP) which can track the solutions of the non-convex RSVM with respect to regularization parameter and ramp parameter respectively. Specifically, we use incremental and decremental learning algorithms to deal with the Karush-Khun-Tucker violating samples in the process of tracking the solutions. Based on the solution paths of RSVM and the piecewise linearity of model function, we can compute the error paths of RSVM and find the values of regularization parameter and ramp parameter, respectively, which corresponds to the minimum cross validation error. We prove the finite convergence of SPRSVM and analyze the computational complexity of SPRSVM. Experimental results on a variety of benchmark datasets not only verify that our SPRSVM can globally search the regularization and ramp parameters respectively, but also show a huge reduction of computational time compared with the grid search approach.
Citation
Z. Zhai, B. Gu, C. Deng and H. Huang, "Global Model Selection via Solution Paths for Robust Support Vector Machine," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 3, pp. 1331-1347, March 2025, doi: 10.1109/TPAMI.2023.3346765
Source
IEEE Transactions on Pattern Analysis and Machine Intelligence
Conference
Keywords
Cross validation, Error path, Robust support vector machines, Solution path
Subjects
Source
Publisher
IEEE