CMPUT414 Lab Excercise 1 Due date Jan 25, 2020 1 Preliminary Technologies of Background Subtraction 80% • Reading and Displaying Image Frames: The following code can be used to read and display the image sequence of the video “highway”. Please understand it and run it on your computer. A screen shot of the computer will be enough. 1 c l e a r a l l 2 c l o s e a l l 3 c l c 4 5 im pa = ’ . / d a t a / highway / i n p u t / ’ ; 6 i m f t = ’ j p g ’ ; 7 8 [ f i l e s d a t a ] = l o a d D a t a p l u s ( im pa , i m f t ) ; 9 d a t a = do ub l e ( d a t a ) ; 10 11 [ row im column im b y t e i m f r a m e s i m ] = s i z e ( d a t a ) ; 12 13 f o r i = 1 : f r a m e s i m 14 im = d a t a ( : , : , : , i ) ; 15 16 d i s p l a y M a t r i x I m a g e ( i , 1 , 2 , im , im ) 17 % d i s p l a y M a t r i x I m a g e ( i , 1 , 3 , im , im , im ) 18 % d i s p l a y M a t r i x I m a g e ( i , 2 , 2 , im , im , im , im ) 19 20 end • Capturing Background Image: A easy method for capturing background im- age is averaging the value of pixels over time. Mathematically, it is shown as follows: Ibk(x,y) = 1N N∑ t=1 It(x,y) (1) where It(x,y) denotes the value of the pixel lo- cated at (x,y) in frames t and N is the number of frames in the entire video sequence. Write a Matlab program to find the background image of video “highway” according to Eq. 1. If you do every thing correctly, the image should look exactly like the image on right. 1 • Subtracting Background: Now you have the background image and frames from the video; subtract them to show the result of background subtraction. The subtraction could be a minus operation, which is shown as follows: Pt(x,y) = |It(x,y)−Ibk(x,y)| (2) where Ibk(x,y) is the background image captured by Eq. 1 and Pt(x,y) is the proximity image of frame t. Write a Matlab program to compute the proximity image of Frame 1000 by Eq. 2. If you do every thing correctly, the result should look like following: Figure 1: From left to right: Frame of 1000, background image and proximity image. • Thresholding: The proximity image is compared with a threshold value T , which is a user parameter, for binary classification of background subtraction. Once the proximity value is smaller than the T , the corresponding pixel is labelled as background (black point), otherwise, it is labelled as foreground (white point). Mathematically, it can be represented as: M(x,y) = g(Pt(x,y),T ) g(Pt(x,y),T ) = { 255, Pt(x,y) > T 0, otherwise , whereM(x,y) denotes the binary mask of the background image. Write a Matlab pro- gram to compute binary mask. If you do every thing correctly, the binary mask should look like Fig. 2. Figure 2: The binary mask following background subtraction. 2 Background Subtraction 20% You already have a good understanding of background subtraction. Unfortunately, the per- formance of the binary mask above is not very good, there are some noise points on the plant in the video. Therefore, the task of this section is improving the accuracy, you can use any method you like. Once you achieve over 90% in Fm (F-Measure) value for Frame of 1000, the frame shown in the Fig. 1, you will get full marks for this section. You can check this webpage (http://jacarini.dinf.usherbrooke.ca/resultEvaluation/) to understand how to calculate the Fm value, and there is also an example to capture the Fm value of the binary mask shown in Fig. 2. 1 c l e a r a l l 2 c l o s e a l l 3 c l c 4 5 g t p a = ’ . / d a t a / highway / g r o u n d t r u t h / ’ ; 6 g t f t = ’ png ’ ; 7 8 9 [ f i l e s d a t a ] = l o a d D a t a p l u s ( g t p a , g t f t ) ; 10 11 gt im = d a t a ( : , : , : , 1000) ; 12 fgim = d ou b le ( imread ( ’ . / fg im . png ’ ) ) ; 13 14 15 [ TP FP FN TN] = e v a l u a t i o n e n t r y ( fgim , g t im ) ; 16 17 Re = TP / ( TP + FN) 18 Pr = TP / ( TP + FP ) 19 Fm = (2* Pr *Re ) / ( Pr + Re ) The following papers may inspire you on how to improve accuracy. Good luck! • Heikkila, Marko, and Matti Pietikainen. ”A texture-based method for modeling the background and detect- ing moving objects.” IEEE transactions on pattern analysis and machine intelligence 28.4 (2006): 657-662. • Liao, Shengcai, et al. ”Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes.” Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. IEEE, 2010. • Lin, Liang, et al. ”Complex background subtraction by pursuing dynamic spatio-temporal models.” IEEE Trans. Image Processing 23.7 (2014): 3191-3202. • Wang, Yi, Zhiming Luo, and Pierre-Marc Jodoin. ”Interactive deep learning method for segmenting moving objects.” Pattern Recognition Letters 96 (2017): 66-75. • Babaee, Mohammadreza, Duc Tung Dinh, and Gerhard Rigoll. ”A deep convolutional neural network for video sequence background subtraction.” Pattern Recognition 76 (2018): 635-649. • https://scholar.google.ca/scholar?hl=en&as_sdt=0%2C5&q=background+subtraction&btnG=
