Research Interests

Multimedia Analysis and Retrieval

Development of multimedia search engines. Design and development of multimedia search engines for image and video. Extraction of low level visual features to support visual search by similarity. High level feature (concept) extraction from multimedia based on supervised machine learning techniques. Applications on general purpose content (news, movies, documentaries), as well as on special content such as digitized art (paintings, sculptures) and patent images (technical drawings, circuits). The developed search engines have been evaluated in well-known evaluation contests such as TRECVID and VideOlympics.

 

Ontologies and the Semantic Web

Ontology construction and semantic ontology-based retrieval: Development of a semantic representation schema for patents based on ontologies.The patent ontology framework is available here. Development of a semantic infrastructure for cultural content and a hybrid ontology and visual-based retrieval model for cultural heritage multimedia collections

 

Human Computer Interaction in Multimedia Retrieval

Implicit User Feedback Analysis during Interative Video Retrieval: Recording of user implicit feedback expressed as clicktrhoughs and gaze movements in video retrieval tasks. Construction of affinity graphs by considering retrieval subsessions using aggregated user clicks. Machine learning-based combination of past user interaction data with visual low level descriptors with a view to optimizing visual search by example. Identification of user interest in the context of a submitted query based on gaze movements by generating fixation and pupil dilation-based descriptors and using SVMs.

 

Patent Analysis and Search

Patent Analysis: Development of a semantic representation schema for patents based on ontologies.The patent ontology framework is available here.

Patent Image Analysis and Search: Development of image retrieval techniques for patent figures based on low level and high level feature extraction. Extraction of optimized low level features for complex binary images (drawings, flowcharts) and semantic indexing based on high level concepts. The concept for patent images are based on the IPC classification schema and the extraction techniques rely upon supervised machine learning. These techniques are integrated on the innovative patent image search PatMedia.

 

Environmental Applications

Discovery of Environmental websites: Development of domain-specific search techniques for the discovery of environmental nodes (websites and service providers).

Indexing and Retrieval of Environmental data: Development of indexing and retrieval techniques of environmental website and measurements based on OGC standards. More information on our recent implementation based on Sensor Observation Service (SOS) is available here.

Extraction of Environmental Data from Multimedia Resources: Development of image analysis techniques for processing and analysis of heatmaps in order to convert images into numerical data.

 

Learning Applications

Development of multimedia authoring applications to support creation and presentation of e-learning and t-learning courses.

 

Domain-specific Search

Development of domain-specific search techniques and tools: Development of domain-specific search techniques for the environmental and the security domain based on focused crawling and on filtering results of general purpose search engines.