引用本文

王松涛, 周真, 曲寒冰, 李彬. RGB-D图像的贝叶斯显著性检测. 自动化学报, 2017, 43(10): 1810-1828. doi: 10.16383/j.aas.2017.e160141

Wang Songtao, Zhou Zhen, Qu Hanbing, Li Bin. Bayesian Saliency Detection for RGB-D Images. ACTA AUTOMATICA SINICA, 2017, 43(10): 1810-1828. doi: 10.16383/j.aas.2017.e160141

http://www.aas.net.cn/cn/article/doi/10.16383/j.aas.2017.e160141

关键词

多尺度超像素分割，判别混合分量朴素贝叶斯模型，显著性检测，深度特征图，RGB-D图像 

摘要

本文提出了一种基于贝叶斯框架融合颜色和深度对比特征的RGB-D图像显著性检测模型.基于空间先验的超像素对比计算得到深度特征，并通过高斯分布近似深度对比特征概率密度建模深度显著图.类似于深度显著性计算，采用高斯分布计算多尺度超像素低层对比特征得到颜色显著图.假设在给定显著类别下颜色和深度对比特征条件独立，依据贝叶斯定理，由深度显著概率和颜色显著概率得到RGB-D图像显著性后验概率，并采用判别混合分量朴素贝叶斯（DMNB）模型进行计算，其中DMNB模型中的高斯分布参数由变分最大期望算法进行估计.在RGB-D图像显著性检测公开数据集的实验结果表明提出的模型优于现有的方法.

文章导读

Saliency detection is the problem of identifying the points that attract the visual attention of human beings. Callet et al. introduced the concepts of overt and covert visual attention and the concepts of bottom-up and top-down processing [1]. Visual attention selectively processes important visual information by filtering out less important information and is an important characteristic of the human visual system (HVS) for visual information processing. Visual attention is one of the most important mechanisms that are deployed in the HVS to cope with large amounts of visual information and reduce the complexity of scene analysis. Visual attention models have been successfully applied in many domains, including multimedia delivery, visual retargeting, quality assessment of images and videos, medical imaging, and 3D image applications [1].

Borji et al. provided an excellent overview of the current state-of-the-art 2D visual attention modeling and included a taxonomy of models (cognitive, Bayesian, decision theoretic, information theoretical, graphical, spectral analysis, pattern classification, and more) [2]. Many saliency measures have emerged that simulate the HVS, which tends to find the most informative regions in 2D scenes [3]-[10]. However, most saliency models disregard the fact that the HVS operates in 3D environments and these models can thus investigate only from 2D images. Eye fixation data are captured while looking at 2D scenes, but depth cues provide additional important information about content in the visual field and therefore can also be considered relevant features for saliency detection. The stereoscopic content carries important additional binocular cues for enhancing human depth perception [11], [12]. Today, with the development of 3D display technologies and devices, there are various emerging applications for 3D multimedia, such as 3D video retargeting [13], 3D video quality assessment [14], [15], 3D ultrasound images processing [16], [17] and so forth. Overall, the emerging demand for visual attention-based applications for 3D multimedia has increased the need for computational saliency detection models for 3D multimedia content. In contrast to saliency detection for 2D images, the depth factor must be considered when performing saliency detection for RGB-D images. Therefore, two important challenges when designing 3D saliency models are how to estimate the saliency from depth cues and how to combine the saliency from depth features with those of other 2D low-level features.

In this paper, we propose a new computational saliency detection model for RGB-D images that considers both color-and depth-based contrast features within a Bayesian framework. The main contributions of our approach consist of two aspects: 1) to estimate saliency from depth cues, we propose the creation of depth feature maps based on superpixel contrast computation with spatial priors and model the depth saliency map by approximating the density of depth-based contrast features using a Gaussian distribution, and 2) by assuming that color-based and depth-based features are conditionally independent given the classes, the discriminative mixed-membership naive Bayes (DMNB) model is used to calculate the final saliency map by applying Bayes' theorem.

The remainder of this paper is organized as follows. Section 2 introduces the related work in the literature. In Section 3, the proposed model is described in detail. Section 4 provides the experimental results on eye tracking databases. The final section concludes the paper.

图 1  The flowchart of the proposed model. The framework of our model consists of two stages: the training stage shown in the left part of the figure and the testing stage shown in the right part of the figure. In this work, we perform experiments based on the EyMIR dataset in [32], NUS dataset in [11], NLPR dataset in [29] and NJU-DS400 dataset in [31].

图 2  Visual samples for superpixel segmentation of RGB-D images with S=40. Rows 1-4: comparative results on the EyMIR dataset, NUS dataset, NLPR dataset and NJU-DS400 dataset, respectively.

图 3  Visual illustration for the saliency measure based on manifold ranking, where patches from corners of images marked as red is defined as pseudo-background.

In this study, we proposed a saliency detection model for RGB-D images that considers both color-and depth-based contrast features within a Bayesian framework. The experiments verify that the proposed model's depth-produced saliency can serve as a helpful complement to the existing color-based saliency models. Compared with other competing 3D models, the experimental results based on four recent eye tracking databases show that the performance of the proposed saliency detection model is promising. We hope that our work is helpful in stimulating further research in the area of 3D saliency detection.

作者简介

Zhen Zhou

received the M.S.and Ph.D.degrees from Harbin University of Science and Technology (HUST), Harbin, China, in 1991 and 2005, respectively.Currently, he is a Professor at HUST and is the Director in measurement and control technology and communication engineering of HUST.His research interests include reliability engineering technology and biological information detection.E-mail:zhzh49@126.com

Hanbing Qu 

received the M.S.and Ph.D.degrees from Harbin Institute of Technology (HIT) and the Institute of Automation, Chinese Academy of Sciences (CASIA), in 2003 and 2007, respectively.Currently, He is an Associate Professor at Beijing Institute of New Technology Applications and is the Director of the Key Laboratory of Pattern Recognition, Beijing Academy of Science and Technology (BJAST).He is also a committee member of the Intelligent Automation committee of Chinese Association of Automation (IACAA).His research interests include biometrics, machine learning, pattern recognition, and computer vision.E-mail:quhanbing@gmail.com

Bin Li 

received the MSc and Ph.D.degrees in computer science from Harbin Institute of Technology (HIT), Harbin, China, in 2000 and 2006, respectively.From 2006 to 2008, he worked at the School of computer Science and Technology, HIT, as a Lecturer.He is currently an Associate Professor and Deputy Director of Beijing Institute of New Technology Applications.His research interests include signal processing, pattern recognition, and biometrics.E-mail:lbn hit@sina.com

Songtao Wang 

received the M.S.degree from Harbin University Of Science and Technology (HUST) in 2009, and is currently a Ph.D.candidate at the Higher Educational Key Laboratory for Measuring and Control Technology and Instrumentations of Heilongjiang Province, HUST.He is also a Research Assistant at the Beijing Institute of New Technology Applications (BIONTA) and the Key Laboratory of Pattern Recognition, Beijing Academy of Science and Technology (BJAST).His research interests include pattern recognition and computer vision, especially the visual saliency detection in surveillance scenarios.Corresponding author of this paper. E-mail:wangsongtao1983@163.com