Sketch Recognition Based on Deformable Convolutional Network

  • yao shi School of Computer Science and Technology, China University of Mining and Technology,China
  • ke wang School of Computer Science and Technology, China University of Mining and Technology,China
Keywords: deep learning, deformable convolution, sketch, residual network, convolutional neural network


Sketch contains various poses of objects and exaggerated strokes due to various painting styles of artists. Traditional method could not achieve high accuracy on recognizing sketch. To solve this problem, this method proposed deformable convolutional neutral network to enhance the ability of the model on recognizing transforms on sketch objects. We proposed a improved deformable convolutional neural network for recognizing sketch. First, we add convolutional kernel on original convolutional layer to construct deformable convolutional layer, which can learn offset and fine-tune factor. Second, we replace the first layer of the ResNet18 network to learn the detailed features of the sketch. Third, we replace the last layer of the ResNet18 with the full connection layer which has an output of 250 dimensions. We verify our method on the TU-Berlin sketch dataset and achieve accuracy of 79.1%.


Download data is not yet available.


Eitz M , Hays J , Alexa M . How do humans sketch objects?[J]. ACM Transactions on Graphics, 2012, 31(4):1-10.

Lecun, Y, Bottou, L, Bengio, Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998 , 86(11):2278-2324.

Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]//Advances in neural information processing systems. 2012: 1097-1105.

Ren, Shaoqing, He, Kaiming, Girshick, Ross,等. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2015, 39(6):1137-1149.

Yang Y, Hospedales T M. Deep neural networks for sketch recognition[J]. arXiv preprint arXiv:1501.07873, 2015, 1(2): 3.

Yu Q, Yang Y, Song Y Z, et al. Sketch-a-net that beats humans[J]. arXiv preprint arXiv:1501.07873, 2015.

Zhang H, Liu S, Zhang C, et al. Sketchnet: Sketch classification with web images[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 1105-1113.

Yu Q , Yang Y , Liu F , et al. Sketch-a-Net: A Deep Neural Network that Beats Humans[J]. International Journal of Computer Vision, 2017, 122(3):411-425.

于美玉, 吴昊, 郭晓燕, et al. 基于时序特征的草图识别方法[J]. 计算机科学, 2018, 45(S2):208-212.

Zhao P , Liu Y , Lu Y , et al. A sketch recognition method based on transfer deep learning with the fusion of multi-granular sketches[J]. Multimedia Tools and Applications, 2019, 78(24):35179-35193.

Zhang H, She P, Liu Y, et al. Learning structural representations via dynamic object landmarks discovery for sketch recognition and retrieval[J]. IEEE Transactions on Image Processing, 2019, 28(9): 4486-4499.

赵鹏, 冯晨成, 韩莉, et al. 融合深度学习和语义树的草图识别方法[J]. 模式识别与人工智能, 2019(4).

Dai J, Qi H, Xiong Y, et al. Deformable convolutional networks[C]//Proceedings of the IEEE international conference on computer vision. 2017: 764-773.

Zhu X, Hu H, Lin S, et al. Deformable convnets v2: More deformable, better results[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019: 9308-9316.

He K, Zhang X, Ren S, et al. Identity mappings in deep residual networks[C]//European conference on computer vision. Springer, Cham, 2016: 630-645.

How to Cite
shi, yao, & wang, ke. (2020). Sketch Recognition Based on Deformable Convolutional Network. IJRDO -Journal of Computer Science Engineering, 6(2), 20-29.