- Task: segment an image into two types of regions as
below
- a region corresponds to natural objects such as trees, flowers, grass, sky, clouds.
- a region corresponds to artificial/man-made objects such as cars, chairs, buildings, roads.
- Techniques
- Use a hidden Markov model (HMM) or Markov random field to model the relations among pixels in an image; use Gaussian mixture to model the marginal distribution of pixels; use the belief propagation algorithm to learn parameters in HMM.
- Use Kanade's rules to analyze an image
- T. Kanade, "Recovery of the Three-Dimensional Shape of an Object from a Single View," Artificial Intelligence, Vol. 17, 1981, pp. 409 - 460
- Shomar, W.J., Seetharaman, G., Young, T.Y., An Expert System For Recovering 3D Shape And Orientation From A Single View, CVIP(92), pp. 459-515.
- Carnell Hampton and Tim Persons and Chris Wyatt and Yaochun Zhang, "Survey of Image Segmentation," 1998 (a brief survey, not thorough)
- W. Skarbek and A. Koschan. "Colour image segmentation - a survey," Technical report, Technical University Berlin, 1994. (a thorough survey but not up-to-date)
- Watershed transform for image segmentation
A review of Watershed algorithms can be found at : http://www.cs.rug.nl/~roe/publications/parwshed.pdf @Article{RoeMei00, author = "Roerdink and Meijster", title = "The Watershed Transform: Definitions, Algorithms and Parallelization Strategies", journal = "FUNDINF: Fundamenta Informatica", volume = "41", publisher = "IOS Press", year = "2000", } - Level set methods for image segmentation:
http://en.wikipedia.org/wiki/Level_set_method
- Advantage: segmentation performance is good for medical images such as CT and MRI since medical images usually have low degree of texture.
- Disadvantage: segmentation performance is poor for EO images (captured by cameras operating in the visible band) since EO images may have rich texture.
- Kernel-based method
- Mean shift method
- Feature-based method (point features)
- Kanade-Lucas-Tomasi (KLT) feature-based tracking algorithm
- Silhouette-based method
- Reference:
- A. Yilmaz, O. Javed and M. Shah, "Object Tracking: A Survey," ACM Journal of Computing Surveys, Vol. 38, No. 4, 2006
- VIVID Tracking Evaluation Web Site: http://www.vividevaluation.ri.cmu.edu/main.html
- ViPER The Video Performance Evaluation Resource:
http://viper-toolkit.sourceforge.net/
- Align images to the same world coordinate system.
- Image georegistration:
- Georegistration is the process of adjusting one drawing or image to the geographic location of a "known good" reference drawing, image, surface or map. Refer to http://exchange.manifold.net/manifold/manuals/5_userman/mfd50Georegistration.htm
- E.g., register a given image (say, a UF campus image) to Google Earth
- Methods:
- Kanade-Lucas-Tomasi (KLT) feature-based algorithm
- Ransac (RANdom SAmple Consensus): outlier removal
- Reference:
- Zitova, B.[Barbara], Flusser, J.[Jan], "Image Registration Methods: A Survey," Image and Vision Computing, Vol. 21, No. 11, October 2003, pp. 977-1000.
- A. Ardeshir Goshtasby, 2-D and 3-D Image Registration, Wiley Press, April. 2005. [ebook on NetLibrary]
- Darko Skerl , Bostjan Likar , J Michael Fitzpatrick , Franjo Pernus,
"Comparative evaluation of similarity measures for the rigid registration of
multi-modal head images,"
Physics in Medicine and Biology, volume={52}, number={18},
pages={5587-601}, year={2007},
Image registrations that are based on similarity measures simply adjust the parameters of an appropriate spatial transformation model until the similarity measure reaches an optimum. The numerous similarity measures that have been proposed in the past are differently sensitive to imaging modality, image content and differences in the image content, selection of the floating and target image, partial image overlap, etc. In this paper, we evaluate and compare 12 similarity measures for the rigid registration. To study the impact of different imaging modalities on the behavior of similarity measures, we have used 16 CT/MR and 6 PET/MR image pairs with known 'gold standard' registrations. The results for the PET/MR registration and for the registration of CT to both rectified and unrectified MR images indicate that mutual information, normalized mutual information and the entropy correlation coefficient are the most accurate similarity measures and have the smallest risk of being trapped in a local optimum. The results of an experiment on the impact of exchanging the floating and target image indicate that, especially in MR/PET registrations, the behavior of some similarity measures, such as mutual information, significantly depends on which image is the floating and which is the target.
-
IEEE Trans Med Imaging. 2006 Jun ;25 (6):779-91 16768242 A protocol for evaluation of similarity measures for rigid registration. [My paper] Int J Radiat Oncol Biol Phys. 2006 Jul 1;65 (3):943-53 16751077 Evaluation of similarity measures for reconstruction-based registration in image-guided radiotherapy and surgery. [My paper] Phys Med Biol. 2007 May 21;52 (10):2865-78 17473356 Automated generation of curved planar reformations from MR images of the spine. [My paper] Med Image Comput Comput Assist Interv Int Conf Med Image Comput Comput Assist Interv. 2006 ;9 (Pt 2):135-43 17354765 Generation of curved planar reformations from magnetic resonance images of the spine. [My paper] IEEE Trans Med Imaging. 2007 Mar ;26 (3):405-21 17354645 A review of methods for correction of intensity inhomogeneity in MRI. [My paper] Conf Proc IEEE Eng Med Biol Soc. 2005 ;5 :5120-3 17281399 Spine-based coordinate system. [My paper] Med Image Comput Comput Assist Interv Int Conf Med Image Comput Comput Assist Interv. 2005 ;8 (Pt 2):231-8 16685964 Reconstruction-based 3D/2D image registration. [My paper] IEEE Trans Med Imaging. 2006 Jan ;25 (1):17-27 16398411 3-D/2-D registration by integrating 2-D information in 3-D. [My paper] Comput Aided Surg. 2004 ;9 (4):137-44 16192053 "Gold standard" data for evaluation and comparison of 3D/2D registration methods. [My paper] Phys Med Biol. 2005 Oct 7;50:4527-40 16177487 Automated curved planar reformation of 3D spine images. [My paper]
- Using machine learning or statistical inference techniques to detect whether there is significant difference between two images
- Applications:
- In remote sensing, change detection can help assessing damage from natural disaster.
- In video surveillance, change detection can help detecting suspicious activities (activity monitoring).
- In medical diagnosis, change detection can help detect diseases.
- Reference:
- R. J. Radke, et al., "Image Change Detection Algorithms: A Systematic Survey," IEEE Trans. Image Processing, vol. 14 , no. 3, pp. 294-307, March 2005.
- Data-independent approach or transform-based
- Fourier transform
- Wavelet transform
- Ripplet transform
- Curvelet transform
- Contourlet transform
- Ridgelet transform
- Bandelet
- Wedgelet
- Chirplet
- Shearlet
- Noiselet
- Wavelet packet
- Over-complete wavelet
- Data-dependent approach:
- Scale-invariant feature transform: http://en.wikipedia.org/wiki/Scale-invariant_feature_transform
- Manifold learning:
http://www.math.umn.edu/~wittman/mani/
- Multi-Dimensional Scaling (MDS)
- Principal Component Analysis (PCA)
- ISOMAP: The ISOMAP method builds a graph from the K nearest neighbors and then performs MDS.
- Locally Linear Embedding (LLE): LLE builds a graph from the K nearest neighbors, computes weights assuming that neighborhoods are locally planar, and takes the top d eigenvectors. LLE was proposed by Sam Roweis & Lawrence Saul.
- Hessian LLE: The Hessian LLE method adapts the weights in LLE to minimize the Hessian operator. Like LLE, it requires careful setting of the parameter K. The main advantage of Hessian LLE is the only method designed for non-convex data sets, although the eigen-solver used seems to crash on some data sets.
- Laplacian: The Laplacian eigenmap method builds a graph from the K nearest neighbors and computes weights to minimize the norm of the gradient in the least squares sense.
- Diffusion Map: The Diffusion Map uses a Gaussian kernel of width Sigma to construct a weighted graph and normalizes the operator with the parameter Alpha. This method was developed by R. Coifman and S. Lafon.
- Local Tangent Space Alignment (LTSA): LTSA uses the K nearest neighbors to form a graph and approximate local tangent spaces for each neighborhood.
- References:
-
R. Kohavi and G. H. John, "Wrappers for feature subset selection," Artificial Intelligence, vol. 97, no. 1-2, pp. 273-324, 1997.
-
Feature Extraction Survey, Proceedings of IEEE, 1969
-
- Given a low-quality video sequence, recognize a human face, which consists of less than 50 pixels in a CIF image.
- Need to address occlusion, lighting variation, pose change
- Use feature extraction, diversity combining, super-resolution
- References:
- Andrea Cavailaro, "Privacy in Video Surveillance," IEEE Signal Processing Magazine, Volume: 24, Issue: 2, On page(s): 168-169, March 2007.
- Jennifer Granick, 2006, May 24 “Security versus privacy: The rematch,” Wired News, [Online]. Available: http://www.wired.com/news/columns/ 0,70971-0.html
- J. Kumagai and S. Cherry, “Sensors and sensibility,” IEEE Spectr., vol. 41, no. 7, pp. 22–26, July 2004.
- R. Gross, L. Sweeney, F. de la Torre, and S. Baker. “Model-based face de-identification,” in Proc. IEEE Computer Vision and Pattern Recognition Workshop on Privacy Research in Vision, p. 161, June 2006.
- A. Senior, S. Pankati, A. Hampapur, L. Brown, Y.-L. Tian, and A. Ekin. “Enabling video surveillance privacy through computer vision,” IEEE Security Privacy, vol. 3, no. 3, pp. 50–57, May-June 2005.
- F. Dufaux and T. Ebrahimi, “Scrambling for video surveillance with privacy,” in Proc. IEEE Computer Vision and Pattern Recognition Workshop on Privacy Research in Vision, p. 160, June 2006.
- A. Cavallaro, “Adding privacy constraints to video-based applications.” in Proc. European Workshop on the Integration of Knowledge, Semantics and Digital Media Technology, Nov. 2004, pp. 257-264.
- Study the human visual system and explain the
phenomenon of Marilyn Monroe--Albert Einstein illusion as below:
- Marilyn Monroe and Albert Einstein embed themselves together in this illusion. You will see Albert Einstein at this reading distance but move back a few feet away and you will see Marilyn Monroe.
- How was this image created?
- Apply low pass filter to the image of Albert Einstein and obtain image A1; apply high pass filter to the image of Marilyn Monroe and obtain image A2; A1+A2 is the resulting image.
- Explain Angry Illusion
- Some explanations: Human eyes can see the details of an image at a close distance but only see a low-resolution image at a far distance (as if the image is low-pass filtered).
- blocking effect
- ringing effect
- the mosquito effect
- MC mismatch
- blurring
- color bleeding
References:
| A survey of hybrid MC/DPCM/DCT video coding distortions | ||
| Source | Signal
Processing
archive Volume 70 , Issue 3 (November 1998) table of contents Special issue on image and video quality metrics
Pages: 247 - 278
Year of Publication: 1998
ISSN:0165-1684
|
|
| Authors | ||
| Publisher |
Elsevier North-Holland, Inc. Amsterdam, The Netherlands, The
Netherlands
|
|