CLAN Photo Presenter
A Multi-modal Summarization Tool for Image Collections
Michal Batko ▪ Petra Budikova ▪ Petr Elias ▪ Pavel Zezula
Motivation
 Amount of images produced is huge
 Summarization tools are needed
 Available for visual-only summaries
 Difficult to index and search
Objectives
 Automatic multi-modal image summarization
 Various algorithms for visual clustering
 Automatic keyword annotation
 Summary annotation for each cluster
 Intuitive, user-friendly interface
 Results presented as a web page
Sample output – Barcelona
Collection description 426 high-resolution photos from a
trip to Barcelona
Clustering method Distinct kNN clustering – Centroid,
25 runs, Davies Bouldin Index
Postprocessing Singleton merging
Annotation method 2-union annotation
Processing time 32 minutes (costly preprocessing of
large photos – 26 minutes)
CLAN Photo Presenter Application
 Desktop application written in Java
 Visual clustering methods based on MPEG7 global visual descriptors
 k-Means, Bisecting k-Means, Distinct kNN
 Repeated clustering to increase the quality
 Quality measures: Davies Bouldin Index, Silhouette Coefficient, Dunn Index
 Postprocessing methods to compensate for clustering anomalies
 Single-member clusters removal, outliers merging
 Annotation of clusters
 Selection of cluster representatives, MUFIN Annotation Web Service
Processing times
http://disa.fi.muni.cz/clan
Resize images
Extract MPEG7 descriptors
Run clustering algorithm
Params: number of clusters, clustering algorithm,
number of repetitions, cluster optimality measure,
postprocessing options
Retrieve annotations for cluster representants
Params: number of images from each
cluster to be annotated
External web service: MUFIN image annotation
Create the web presentation
Plants, flower, garden,
summer, blue , blossom, …
Preprocessing
My trip to Amsterdam
Flowers, buildings, people, cheese
Clustering
Annotation
Presentation
Solution
Laboratory of Data Intensive Systems and Applications, Masaryk University, Czech Republic
ICMR 2014, Glasgow, United Kingdom
Collection size Clustering method # of clusters Preprocessing
time [s]
Clustering time [s]
(25 repetitions)
Annotation time [s]
(5-union)
Overall costs
[min]
160/500/1000 Bisecting k-means 16/20/30 76/214/471 12/138/610 229/299/418 5.3/10.9/25.0
160/500/1000 K-means 16/20/30 76/214/471 3/18/69 229/299/418 5.1/8.9/16.0
160/500/1000 Distinct kNN 25/50/100 76/214/471 3/20/82 373/696/1428 7.5/15.5/33.0