With the spread of large digital painting collections, the demand for automatic theme classification has risen in both academic and commercial fields. Many researchers have investigated various techniques to implement automatic painting classification and developed several approaches to improve classification accuracy. SURF is one of the most commonly used local invariant feature descriptors for painting classification. However, the common method of classification based on traditional SURF local feature description makes the color feature information not comprehensive. Also, the feature space is high dimensional and sparse which will result in low classification accuracy, data redundancy and a time-consuming process. To overcome this drawback, we suggest a new method of classifying painting styles with bag of words (BOW) model based on the RGB-SURF algorithm. First, split RGB channels into separate images for each painting. Then, independently calculate key points and descriptor of each RGB channel using SURF and combine descriptors of each channel to obtain overall RGB-SURF feature for each painting. This approach is to compensate the loss of color feature brought by traditional SURF descriptor which mainly treats colorful images as gray images. Next, construct the visual vocabulary by reducing the number of features through quantization of feature space using K-means clustering to obtain BOW model. The histogram described by appearance frequency of the visual words is used to represent the content of the image. As a result, every image is viewed as a bag full of visual words. Finally, establish the support vector machine (SVM) classifier based on radial basis function (RBF) with the data above for training and testing. Compared to classification method based on the traditional SURF feature descriptor and other painting classification approaches, BOW method based on RGB-SURF algorithm shows the highest average classification accuracy, which means this method produces the most consistent automatic classification results with manual classification.