Download - Query expansion based on visual content new

Transcript
Page 1: Query expansion based on visual content new

1

Query Expansion Based on Visual Content

Lazaros T. Tsochatzidis

Athanasios Ch. Kapoutsis

Nikolaos I. Dourvas

Savvas A. Chatzichristofis

Yiannis S. Boutalis

Monday, May 1, 2023

Department of Electrical and Computer Engineering Democritus University of Thrace

Page 2: Query expansion based on visual content new

Monday, May 1, 2023 2

How the today’s web search engines work?

Keywords (from the user)Metadata (keywords, tags, any other

information)

compare

Is everything working fine?

Page 3: Query expansion based on visual content new

Monday, May 1, 2023 3

The Problems

Ordinary image search is based on metadata, sometimes irrelative or missing

Very difficult to describe(using textual query) the image exactly as it is

Page 4: Query expansion based on visual content new

Monday, May 1, 2023 4

Our goal… is to reduce the sematic gap between “What” the user wants to find and “How” he describes it.

Page 5: Query expansion based on visual content new

Monday, May 1, 2023 5

How does TsoKaDo work?

Page 6: Query expansion based on visual content new

Monday, May 1, 2023 6

CEDD(Color and Edge Directivity Descriptor)

Page 7: Query expansion based on visual content new

Monday, May 1, 2023 7

What is it?

Belongs to the family of Compact Composite Descriptors (CCD’s)

Describes color and texture of an image.Used for searching and retrieving images

from multimedia databases.

WHY?The results are very effective.It is very light and fast as it used only 54

bytes per image

Page 8: Query expansion based on visual content new

Monday, May 1, 2023 8

Function of CEDD For the color:2 fuzzy systems map the color of the

image in a 24-color palette.

For the texture:Using a fuzzy version of the five digital

filters proposed by MPEG-7 Edge Histogram Descriptor

Page 9: Query expansion based on visual content new

Monday, May 1, 2023 9

Classification

Page 10: Query expansion based on visual content new

Monday, May 1, 2023 10

ClassificationSGONGThe Self Growing and Self Organized Neural Gas Network (SGONG) Innovative neural classifier Dynamically defines the number of classes

needed

K-means N observations into k clusters Criterion: Each observation belongs to the

cluster with the nearest mean.

Page 11: Query expansion based on visual content new

Monday, May 1, 2023 11

The proposed tags

Page 12: Query expansion based on visual content new

Monday, May 1, 2023 12

Semantic similaritySemantic similarityThe definition of it relates to

computing the conceptual similarity between terms that are not lexicography similar

Why we use it?It reduces the amount of proposed

tags and gives more corresponding results.

Page 13: Query expansion based on visual content new

Monday, May 1, 2023 13

WORDNETA lexical database which provides a large

repository of English lexical terms.

How it works?Parts Of Speech (POS) →

noun,verb,adjective, adverb.Establishes connection between POS and

groups them into “synsets”.“Sense” → A group of synset. .Represents

different meaning of the same term.

Page 14: Query expansion based on visual content new

Monday, May 1, 2023 14

The “Path length” measurement

Wu and Palmer method

Page 15: Query expansion based on visual content new

Monday, May 1, 2023 15

Tsokado in action

Page 16: Query expansion based on visual content new

Monday, May 1, 2023 16

Problems And ObstaclesCEDD is based only in color and texture

of the image

Noisy tags at Flickr (human imported): Irrelevant tags

Time consuming when using Wordnet.

Page 17: Query expansion based on visual content new

Monday, May 1, 2023 17

Solutions and future workUse of Visual Words instead of CEDD

for better semantic classification

Fetching of tags from more reliable sources (e.g. Wikipedia)

Use of EuroWordnet, a multilanguage version.

Page 18: Query expansion based on visual content new

Monday, May 1, 2023 18

Are there any ?'s

Please visit:http://tsokado.nonrelevant.net