Information Retrieval

The Information Retrieval group is a research group within the Information, Data and Analysis Section. The group has a long and strong research history in the process of information retrieval as a whole from theoretical modelling of the retrieval process to large-scale text retrieval systems building and to the interactive evaluation of multimedia information retrieval systems. The research interests of the group include:

- Theoretical modelling of IR systems
- Probabilistic retrieval
- Web information retrieval
- Implementation of large-scale IR systems
- Multimedia (Image, Video, Audio) information retrieval
- Intranet/Enterprise and Blog search
- Distributed and Peer-to-Peer retrieval
- User Modelling and the development of novel adaptive interaction techniques
- Evaluation of IR systems
- Text mining and knowledge discovery
- Multilingual information retrieval
- Semantic Web and information retrieval

The group maintains strong links with researchers in Machine Learning and Human-Computer Interaction, as well as with industry through knowledge and technology transfer

Current Projects:

  • SMART(Search engine for MultimediA environment geRenated contenT) is a research project funded by the European Commission's FP7 (number 287583).
  • CROSS is is an EPSRC funded project (number EP/J020664/1). ls with finding novel events in streams such as Twitter, Wikipedia or Newswire, in real time. 

Past Projects

  • RAENG Fellowship
    Foundations research in information retrieval inspired by quantum theory.
    Prof C.J. van Rijsbergen, 2007-2012.

Academic Staff: Prof Joemon Jose,  Prof Iadh Ounis, Dr Craig Macdonald, Dr Jeff Dalton

Research Assistants and Research Students: Eugene Kharitonov,Mr Richard Mccreadie, Graham McDonald,Philip McParlane,Fajie Yuan, Colin Wilkie, David Maxwell, Long Chen, Rami Alkawaldeh, Andrew McMinn, Haitao Yu, Jorge Gonzalez-Paule, Xiao Yang, Anjie Fang, Stuart Mackie, Jarana Manotumruksa.

  • Theoretical development of probabilistic and logic-based models
  • multimedia IR systems
  • information analysis and access across media
  • evaluation and usability of IR systems
  • data mining of large data sets
  • Web information retrieval
  • citation/link analysis
  • implementation and evaluation of large-scale IR systems
  • performance prediction and optimisation
  • information retrieval in context
  • multilingual retrieval
  • interaction techniques based on implicit relevance feedback and summarisation
  • adaptive information retrieval
  • Intranet/Enterprise and Blog search

This Week’s EventsAll Upcoming EventsPast EventsWebapp

This Week’s Events

There are no events scheduled for this week

Upcoming Events

There are no upcoming events

Past Events

IR Seminar: Recommendations in a Marketplace: Personalizing Explainable Recommendations with Multi-objective Contextual Bandits (08 April, 2019)

Speaker: Rishabh Mehrotra

In recent years, two sided marketplaces have emerged as viable business models in many real world applications (e.g. Amazon, AirBnb, Spotify, YouTube), wherein the platforms have customers not only on the demand side (e.g. users), but also on the supply side (e.g. retailer, artists). Such multi-sided marketplace involves interaction between multiple stakeholders among which there are different individuals with assorted needs. While traditional recommender systems focused specifically towards increasing consumer satisfaction by providing relevant content to the consumers, two-sided marketplaces face an interesting problem of optimizing their models for supplier preferences, and visibility.

In this talk, we begin by describing a contextual bandit model developed for serving explainable music recommendations to users and showcase the need for explicitly considering supplier-centric objectives during optimization. To jointly optimize the objectives of the different marketplace constituents, we present a multi-objective contextual bandit model aimed at maximizing long-term vectorial rewards across different competing objectives. Finally, we discuss theoretical performance guarantees as well as experimental results with historical log data and tests with live production traffic in a large-scale music recommendation service.

Rishabh Mehrotra is a Research Scientist at Spotify Research in London. He obtained his PhD in the field of Machine Learning and Information Retrieval from University College London where he was partially supported by a Google Research Award. His PhD research focused on inference of search tasks from query logs and their applications. His current research focuses on bandit based recommendations, counterfactual analysis and experimentation. Some of his recent work has been published at top conferences including WWW, SIGIR, NAACL, CIKM, RecSys and WSDM. He has co-taught a number of tutorials at leading conferences (WWW & CIKM) & was recently invited to teach a course on "Learning from User Interactions" at a number of summer schools including Russian Summer School on Information Retrieval and the ACM SIGKDD Africa Summer School on Machine Learning for Search.

IR seminar: Unbiased Learning to Rank from User Interactions (01 April, 2019)

Speaker: Harrie Oosterhuis

Learning to rank provides methods for optimizing ranking systems, enabling effective search and recommendation systems. Traditionally, these methods relied on annotated datasets i.e. relevance labels query-document pairs provide by human judges. Over the years, the limitations of such datasets have become apparent. Recently attention has mostly shifted to methods that learn from user interactions, as they more closely indicate user preferences. However, user interactions contain large amounts of noise and bias, learning from them while naively ignoring biases can lead to detrimental results. Consequently, the current focus is on unbiased methods that can reliably learn from user interactions. In this talk I will contrast the two main approaches to unbiased learning to rank: counterfactual learning and online learning, and discuss the most recent methods from the field.
Harrie Oosterhuis ( is a 3rd year PhD student under supervision of Prof. dr. Maarten de Rijke at the University of Amsterdam. His main topic is learning to rank from user behaviour and he has publications at major IR conferences including CIKM, SIGIR, ECIR and WSDM. In addition he has completed multiple internships at Google Research & Brain in California, and worked as a visiting student at RMIT university in Melbourne during his PhD.

Post-CHIIR IR Seminar (15 March, 2019)

Speaker: Jaime Arguello and Adam Roegiest

This week have have a special post-CHIIR edition IR seminar talk(s).  We have two speakers from North America who will be speaking this Friday afternoon.
Talk 1:Understanding How Cognitive Abilities Influence Search Behaviors and Outcomes by Jaime Arguello at the University of North Carolina at Chapel Hill
Talk 2:Total Recall and Beyond: Real-world world experience in the legal domain
by Adam Roegiest, a Research Scientist at Kira Systems
When: 3-4pm, Friday March 14th
Where: SAWB 422
Event link
Details of both talks are below
Title: Understanding How Cognitive Abilities Influence Search Behaviors and Outcomes
Traditionally, personalization in IR has meant predicting which results to return based on a user's query and interest profile.  However, personalization in IR should also consider how to display results based on a user's cognitive abilities.  In this talk, I will summarize several studies that have investigated the effects of different cognitive abilities on search behaviors (e.g., how easily users find relevant results on a SERP) and search outcomes (e.g., users' perceptions of workload and engagement).  In these studies, we have considered cognitive abilities such as perceptual speed, working memory, and inhibitory attention control.  Additionally, we have considered how cognitive abilities interact with other factors such as different SERP layouts and search task types.  For example, does perceptual speed have a stronger influence for SERPs that are more visually complex? I will discuss challenges faced in conducting these studies and implications for designing systems that are well-suited to users' individual cognitive abilities.
Bio: Jaime Arguello is an Associate Professor at the School of Information and Library Science at the University of North Carolina at Chapel Hill.  Jaime received his Ph.D. from the Language Technologies Institute at Carnegie Mellon University in 2011.  Since then, his research has focused on a wide range of areas, including aggregated/federated search, voice query reformulation, understanding search behaviors during complex tasks, developing search assistance tools for complex tasks, and (more closely related to this talk) understanding the effects of different cognitive abilities on search behaviors and outcomes.  He has received Best Paper Awards at ECIR 2017, IIiX 2014, ECIR 2011, and SIGIR 2009.  His current research is supported by two NSF grants.  Since 2015, Jaime has chaired the SIGIR Travel Awards Program, which helps support about 160 students per year to attend SIGIR-sponsored conferences.
Title: Total Recall and Beyond: Real-world world experience in the legal domain
In this talk, I will discuss the benefits and drawbacks to working with research real-world problems. This begins with a discussion of my work in coordinating the TREC Total Recall track and subsequent investigations. Following this, I will discuss my experiences in developing features to aid in performing due diligence. Tying these experiences together are the focus on the legal domain and need to make results accessible to non-experts and covers both system evaluations and several user studies. 
Bio: Adam Roegiest is a Research Scientist at Kira Systems, where he spends time developing machine learning algorithms to aid lawyers perform due diligence. As part of this work, he collaborates with designers and legal professionals to ensure that these algorithms and their results are accessible to non-experts. Prior to working at Kira Systems, Adam completed his PhD at the University of Waterloo where studied the design and evaluation of high-recall systems for technology-assisted review and helped coordinate the TREC Total Recall and Real-Time Summarization tracks. 

IR Seminar: Topic-centric sentiment analysis of UK parliamentary debate transcripts (25 February, 2019)

Speaker: Gavin Abercrombie

Debate transcripts from the UK House of Commons provide access to a wealth of information concerning the opinions and attitudes of politicians and their parties towards arguably the most important topics facing societies and their citizens, as well as potential insights into the democratic processes that take place within Parliament.

By applying natural language processing and machine learning methods to debate speeches, it is possible to automatically determine the attitudes and positions expressed by speakers towards the topics they discuss.

This talk will focus on research on speech-level sentiment analysis and opinion-topic/policy detection, as well as discussing the challenges of working in this domain.


Gavin Abercrombie holds a Masters degree in IT & Cognition from the University of Copenhagen, and is currently a second-year PhD student at the School of Computer Science, University of Manchester. His research interests include natural language understanding and computational social science.

Challenges and Opportunities at the Intersection of the Computing and Social Sciences (21 February, 2019)

Speaker: Multiple speakers

The workshop aims to bring together social, political and computer scientists to discuss the challenges and opportunities when studying political events and campaigns especially on & through social media. Speakers include UoG's Assistant VP Des McNulty, Philip Habel (USA), Zac Green (Strathclyde) and our own Anjie Fang, who will be defending his PhD this week.

IR Seminar: Measuring User Satisfaction and Engagement (10 December, 2018)

Speaker: Adam Zhou

In the online world, it is important to design user-centric applications that can engage the users and make them satisfied. In order to improve user satisfaction and engagement, one prerequisite is to find effective ways to measure them.

In this talk, I will present several efforts to measure satisfaction and engagement in the context of search and mobile app usage. Firstly, I will present my work that aims to find the best metrics (either offline or online) for various search scenarios: organic, aggregated and image search. Secondly, I will talk about various ways how mobile users engage with the apps and how to exploit them to predict their next engagement. Finally, I will very briefly cover some of our current work on conversational search.

Ke (Adam) Zhou holds dual academic and industrial appointments as an assistant professor in the school of computer science at University of Nottingham, and a Senior Research Scientist at Nokia Bell Labs. His research interests and expertise lie in web search and analytics, evaluation metrics, text mining and human computer interaction. He has published over 50 publications in reputable conferences and journals. His past researches have won the best paper award in ECIR'15 and CHIIR'16, and best paper honorable mention in SIGIR'15.

IR Seminar: Alana: Entertaining and Informative Open-domain Social Dialogue using Ontologies and Entity Linking (03 December, 2018)

Speaker: Ioannis Konstas

In this talk I will present our 2018 Alexa prize system (called ‘Alana’), an open-domain spoken dialogue system aimed at maintaining a fun, engaging and informative discussion with users. Alana consists of an ensemble of bots, combining rule-based and machine learning systems. The main highlights are (1) a neural Natural Language Understanding (NLU) pipeline; (2) a family of retrieval bots that store and deliver content interactively from heterogeneous sources (e.g., News, Wikipedia, Reddit), using traditional as well as graph-based datastores; (3) an ensemble of rule-based bots aimed at laying out a certain persona for Alana, while at the same time maintaining a coherent dialogue; (4) a profanity & abuse detection model with rule-based mitigation strategies. In the second part of the talk, I will describe an ongoing project on neural conversational agents aiming to produce coherent dialogues in human-to-human interactions. I will also illustrate our efforts on a more traditional task-based dialogue setup in the e-commerce domain exploiting several modalities (vision, knowledge base) on top of the textual input.
Yannis Konstas is a lecturer in the department of Mathematical and Computer Sciences at Heriot-Watt University, Edinburgh. His main research interests focus on the area of Natural Language Generation (NLG) with an emphasis on data-driven deep learning methods. Before that he was a postdoctoral researcher at the University of Washington (2015-17) working with Luke Zettlemoyer. He has received a BSc in Computer Science from AUEB (Greece) in 2007, and an MSc in Artificial Intelligence from the University of Edinburgh (2008). He continued his study at the University of Edinburgh and received a Ph.D. degree in 2014, under the supervision of Mirella Lapata. He has previously worked as a research assistant at the University of Glasgow (2008), and as a postdoctoral researcher at the University of Edinburgh (2014). 

IR Seminar: The Quantified Self as Testbed for Multimodal Information Retrieval (19 November, 2018)

Speaker: Frank Hopfgartner

Title: The Quantified Self as Testbed for Multimodal Information Retrieval
Abstract: Thanks to recent advances in the field of ubiquitous computing, an increasing number of people now rely on tools and apps that allow them to track specific aspects of their lives. The result of this is development is that people are now able to unobtrusively create records of their daily experiences, captured multi-modally through digital sensors and stored permanently as a personal lifelog archive.  From an information retrieval perspective, these personal archives are rather challenging due to the multimodal nature of data created. In this talk, I will provide an overview of NTCIR Lifelog, an evaluation campaign that focuses on promoting research on multimodal information retrieval. 
Bio: Frank Hopfgartner is Senior Lecturer in Data Science and Head of the Information Retrieval Research Group at University of Sheffield. His research interest is in the intersection of information and data analytics. In particular, he focuses on novel approaches to personalise information access, especially in the fields of recommender and information retrieval systems. Due to the content-rich nature of data created, he increasingly concentrates on lifelogging as a challenging use case to improve multimedia access methods.

Investigating How Conversational Search Agents Affect User’s Behaviour, Performance and Search Experience (05 November, 2018)

Speaker: Mateusz Dubiel

Voice based search systems currently do not support natural conversational interaction. Consequently, people tend to limit their use of voice search to simple navigational tasks, as more complex search tasks require more sophisticated dialogue modelling. Previous research has demonstrated that a voice based search system’s inability to preserve contextual information leads to user’s dissatisfaction and discourages further usage. In my talk I will explore how people’s search behaviour, performance and perception of usability change when interacting with a conversational search system which supports natural language interaction, as opposed to a voice based search system which does not.

Short bio:
Mateusz Dubiel is a PhD candidate in the department of Computer and information Sciences at Strathclyde University in Glasgow . His research is focused on development and evaluation of conversational search agents. Mateusz holds an MSc in Speech and Language Processing from The University of Edinburgh.

IR Seminar: Measuring the Utility of Search Engine Result Pages (08 October, 2018)

Speaker: Dr. Leif Azzopardi

Web Search Engine Result Pages (SERPs) are complex responses to queries, containing many heterogeneous result elements (web results, advertisements, and specialised ``answers'') positioned in a variety of layouts. This poses numerous challenges when trying to measure the quality of a SERP because standard measures were designed for homogeneous ranked lists.

In this talk, I will explain how we developed a means to measure the utility and cost of SERPs. 
To ground this work we adopted the C/W/L framework by Moffat et al which enables a direct comparison between different measures in the same units of measurement, i.e. expected (total) utility and cost. I argue that the extended C/W/L framework provides a clearer and more interpretable framework for measurement i.e. utility, cost (in time), and also predicted stopping rank - the latter two are both directly observable - and so the quality of the metric can be assessed by how well it predicts these observables.

Within this framework, we proposed a new measure based on Information Foraging Theory, which can account for the heterogeneity of elements, through different costs, and which naturally motivates the development of a user stopping model that adapts behaviour depending on the rate of gain. This directly connects models of how people search with how we measure search, providing a number of new dimensions in which to investigate and evaluate user behaviour and performance.We perform an analysis over 1000 popular queries issued to a major search engine, and report the aggregate utility experienced by users over time. Then in an comparison against common measures, we show that the proposed foraging based measure provides a more accurate reflection of the utility and of observed behaviours (stopping rank and time spent).

IR Seminar: Controversy Analysis and Detection (21 May, 2018)

Speaker: Shiri Dori-Hacohen

Controversy Analysis and Detection
Seeking information on a controversial topic is often a complex task. Alerting users about controversial search results can encourage critical literacy, promote healthy civic discourse and counteract the "filter bubble" effect, and therefore would be a useful feature in a search engine or browser extension. Additionally, presenting information to the user about the different stances or sides of the debate can help her navigate the landscape of search results beyond a simple "list of 10 links". Our existing work has made strides in the emerging niche of controversy analysis and detection. In our work, we've made a few conceptual and technical contributions, including: (1) Offering a computational definition of controversy and its components; (2) Improving the current state-of-the-art controversy detection in Wikipedia by employing a stacked model using a combination of link structure and similarity; and (3) the first automated approach to detecting controversy on the web, using a KNN classifier that maps from the web to similar Wikipedia articles. I also recently founded a startup aiming to bring this research & technology to practical uses. This talk will largely focus on contribution (2) above, and touch on the other aspects briefly as time allows.


This talk is based on joint work with James Allan, John Foley, Myung-ha Jang, David Jensen and Elad Yom-Tov.
Dr. Shiri Dori-Hacohen is the CEO & founder of AuCoDe. She has fifteen years of academic and industry experience, including Google and Facebook. She received her M.Sc. and B.Sc. (cum laude) at the University of Haifa in Israel and her M.S. and Ph.D. from the University of Massachusetts Amherst where she researched computational models of controversy. Dr. Dori-Hacohen is the recipient of several prestigious awards, including the 2011 Google Lime Scholarship and first place at the 2016 UMass Amherst’s Innovation Challenge. She has one daughter; identifies as a person with disabilities; and has taken an active leadership role in broadening participation in Computer Science on a local and global scale.

IR Seminar: Understanding and Leveraging the Impact of Response 1 Latency on User Behaviour in Web Search (18 May, 2018)

Speaker: Ioannis Arapakis

The interplay between the response latency of web search systems and users' search experience has only recently started to attract research attention, despite the important implications of response latency on monetisation of such systems. In this work, we carry out two complementary studies to investigate the impact of response latency on users' searching behaviour in web search engines. We first conduct a controlled user study to investigate the sensitivity of users to increasing delays in response latency. This study shows that the users of a fast search system are more sensitive to delays than the users of a slow search system. Moreover, the study finds that users are more likely to notice the response latency delays beyond a certain latency threshold, their search experience potentially being affected. We then analyse a large number of search queries obtained from Yahoo Web Search to investigate the impact of response latency on users' click behaviour. This analysis demonstrates the significant change in click behaviour as the response latency increases. We also find that certain user, context, and query attributes play a role in the way increasing response latency affects the click behaviour. To demonstrate a possible use case for our findings, we devise a machine learning framework that leverages the latency impact, together with other features, to predict whether a user will issue any clicks on web search results. As a further extension of this use case, we investigate whether this machine learning framework can be exploited to help search engines reduce their energy consumption during query processing.

Surviving the Flood of Big Data Streams (30 April, 2018)

Speaker: Richard McCreadie

Research talk abstract: The way big data is being processed is evolving from predominantly batch-based analysis of static datasets, towards to microservice-driven architectures designed to analyse big data streams. This change raises new challenges for both data systems enginers examining how to build efficient and scalable architectures/platforms; as well as for researchers and developers looking to extract value from emerging real-time streams. In this talk, I will discuss how real-time streaming data is altering the research landscape from the perspective of real-time event detection and modelling. In particular, I will cover my past and present research in this area, focusing on challenges in data systems development, event detection from real-time streams, as well as how to model information from event streams over time. I will conclude the talk with a discussion on some promising new research directions to examine in this area in the future.

Lectureship abstract: We are asking all IDA Lectureship candidates to give a 15 minute lecture, as if they were teaching Level 4 undergraduates. The topic is “Explaining the matrix factorisation (MF) approach for collaborative filtering”.

Scaling Entity Linking with Crowdsourcing (23 April, 2018)

Speaker: Dyaa Albakour

In this presentation, we first review the current state-of-the-art for the EL task and make the case for using supervised learning approaches to tackle EL. These approaches require large amounts of labelled data, which represent a bottleneck for scaling them out to cover large numbers of entities. To mitigate this, we have developed a production-ready solution to efficiently collect high-quality labelled data at scale using Active Learning and  Crowdsourcing. In particular, we will discuss the different steps and the challenges in tuning the design parameters of the crowdsourcing task. The design parameters include the qualification of the workers and UI features that help them complete the task. The tuning aims to limit the noise, reduce the cost and maximise the throughput of labelling whilst maintaining the quality of the resulting models for EL.

Signal Media is a research-led company that uses text analytics and machine learning to turn streams of unstructured text, e.g. news articles, into useful information for professional users. One of the core components of Signal’s text analytics pipeline is entity linking (EL).


Shard Effects on Effectiveness (18 April, 2018)

Speaker: Mark Sanderson

Shard Effects on Effectiveness

Studying the experimental factors that impact IR measures is often overlooked when comparing IR systems. In particular, the effects of splitting the document collection into shards has not been examined in detail. I will talk about our use of the general linear mixed model framework and present a model that encompasses the experimental factors of system, topic, shard, and their interaction effects. The model allows us to more accurately estimate significant differences between the effect of various factors. We study shards created by various methods used in prior work and better explain observations noted in prior work in a principled setting and offer new insights. Notably, I describe how we discovered that the topic*shard interaction effect is large, almost globally across all datasets, an observation that has not been recognised or measured before to our knowledge.

Prototyping Deep Learning Applications Through Knowledge Transfer (16 April, 2018)

Speaker: Nina S Dethlefs

Deep learning plays an ever increasing role in artificial intelligence and a growing number of libraries facilitate the fast development of new applications. For each new learning task, some trial and error is normally required to tune hyperparameters or find an adequate learning representation etc before a suitable prediction model can be learnt. In this talk, I explore the possibility of transferring hyperparameters (and learning representations) from one task to another based on the tasks’ similarity. The idea is to reuse previously acquired knowledge and in this way reduce time and development costs and speed up prototyping of new deep learning applications. I present a number of case studies from natural language processing and other AI tasks that show how knowledge transfer can - in some cases - lead to state-of-the-art performance on unseen tasks while substantially reducing computation time. Embedding important operations into a generalised abstract framework, e.g. a domain specific programming language, facilitates prototyping even further.

I am a Lecturer in Computer Science at the University of Hull, UK. I lead the Big Data Analytics groups and I am a member of the Computational Science group. Previously, I was a Research Fellow at the Interaction Lab at Heriot-Watt University, Edinburgh. I have a PhD in Computational Linguistics from the University of Bremen, Germany. 
My research interests are in computational intelligence and machine learning - particularly deep learning and optimisation - as well as natural language processing. I investigate how machine learning algorithms themselves can be equipped with intelligence so as to enable transfer learning across domains and learning tasks. Most of my work has been in natural language processing but I have also worked in other areas, including health informatics and human-robot interaction.

Simulating Interaction for Evaluation (09 April, 2018)

Speaker: Leif Azzopardi

Search is an inherently interactive, non-deterministic and user-dependent process. This means that there are many different possible sequences of interactions which could be taken (some ending in success and others ending in failure). Simulation provides a powerful tool for low-cost, repeatable and reproducible evaluations which explore a large range of different possibilities - and enables the analysis of IR systems, interfaces, user behaviour and user strategies. To run a simulation, a model of the user is formalised, and then used, for example, as the basis of a metric, to create a test collection, or generate interaction data. In this talk, I will give an overview of various methods that we have developed in order to: (1) create simulated test collections which enable more extensive evaluations, as well as enable the evaluation on new collections without the expense of costly user judgements, and (2) create user interaction data, which enables a range of different user strategies/behaviours to be compared and contrasted in a systematic manner.

Bio: Dr. Leif Azzopardi is a Chancellor's Fellow in Data Science and Associate Professor at the University of Strathclyde, Glasgow within Department of Computer and Information Science. He leads the Interactive Information Retrieval group within Strathclyde's iSchool. His research focuses on examining the influence and impact of search technology on people and society and is heavily underpinned by theory. He has made numerous contributions in: (i) the development of statistical language models for document, sentence, expert retrieval, (ii) the simulation and evaluation of users and their interactions, (iii) the analysis of systems and retrieval bias using retrievability theory and the (iv) the formalisation of search and search behaviour using economic theory. He has given numerous keynotes, invited talks and tutorials through out the world on retrievability, search economics, and simulation. He is co-author of the Tango with Django ( which has seen over 1.5 million visitors. And more recently he has been co-developing resources for IR research with Lucene (, while co-creating evaluation resources for Technology Assisted Reviews as part of the CLEF eHealth Track 2017. He is an honorary lecturer at the University of Glasgow (where he was previously a Senior Lecturer) and an honorary Adjunct Associate Professor at Queensland University of Technology. He received his Ph.D. in Computing Science from the University of Paisley in 2006, under the supervision of Prof. Mark Girolami and Prof. Keith van Rijsbergen. Prior to that he received a First Class Honours Degree in Information Science from the University of Newcastle, Australia, 2001.

Analyzing and Using Large-scale Web Graphs (29 March, 2018)

Speaker: Ansgar Scherp

The talk first provides an overview about my research in Data Science, namely text and data mining. Subsequently, I focus on graph data mining on the Web. I have developed a schema-level index called SchemEX in order to be able to search in large-scale web graphs. The SchemEX index can be efficiently computed in a stream-based fashion with reasonable accuracy over graphs of billions of edges. The data search engine LODatio+ (see: uses the SchemEX index to find relevant data sources. In order to quickly develop, tailor, and compare schema-level indices, I provide a novel formal, parameterized model for schema-level indices. A grant challenge is to deal with the evolution of the web graphs, specifically their schema in terms of types and properties used to describe entities. I have investigated the dynamics of entities in order to find, e. g., periodicities in the schema changes, and to use this information to predict future changes. This is important for various future data-driven applications that aim at using graph data on the web.


IR Seminar: Using Synthetic Text for Developing Content Coordination Metrics and Semantic Verification (12 March, 2018)

Speaker: Dmitri Roussinov

Recurrent neural language models allowed generating realistically looking synthetic texts, but the use of those texts for scientific purposes has been largely unexplored. I will present my work in progress and some forthcoming results on using simulated text for developing the metrics for catching coordinated content in microblogs (e.g. Twitter trolling attacks) and verifying semantic classes of words (e.g. France is a country, Gladiator is a movie, but not a country) for question answering applications. My simulation results support the conjecture that only when the metric takes the context and the properties of the repeated sequence into consideration, it is capable of separating organic and coordinated content. I will also demonstrate how those context-specific adjustments can be obtained using existing resources.


Dr. Roussinov is a Senior Lecturer in Computer and Information Sciences University of Strathclyde. He has contributed to the fields of information systems, information retrieval, natural language processing, search engines, security informatics, medical informatics, human computer interaction, databases and others. He received his doctoral degree in Information Systems from the University of Arizona (advisor H. Chen), Master’s in Economics from Indiana University, and his undergraduate in Physics and computer science from Moscow Institute of Physics and Technology.

Learning from samples of variable quality (26 February, 2018)

Speaker: Mostafa Deghani

The success of deep neural networks to date depends strongly on the availability of labeled data which is costly and not always easy to obtain. Usually, it is much easier to obtain small quantities of high-quality labeled data and large quantities of unlabeled, weak or noisy data. The problem of how to best integrate these two different sources of information during training and hot to get best of samples of variable quality is an active pursuit in the field of semi-supervised learning. In this talk, we are going to discuss some methods for training neural networks with labels with different quality.

Mostafa Dehghani is a PhD student at the University of Amsterdam working with Jaap Kamps and Maarten de Rijke. His doctorate research lies at the intersection machine learning and information retrieval, in particular employing weak supervision signals for training neural models for IR problems. He has contributed to top-tier ML and IR conferences like NIPS, ICLR, SIGIR, CIKM, WSDM, and ICTIR by publishing papers and giving tutorials and received awards at SIGIR, ICTIR, ECIR, and CLEF for some of his works. He has done internships at Google Research on search conversationalization and currently interning at Google Brain.


Speaker: Dimitrios Rafailidis

How the selections of social friends can influence user preferences in recommender systems? How can we exploit distrust relationships when generating product, movie or song recommendations? In the first part of my talk I will present my recent research in social recommender systems, and how these questions are answered to produce accurate recommendations by considering both trust and distrust relationships.

While Amazon users can rate products from different domains, such as books, toys and clothes, they do not necessarily have the same behavior when different types of products are recommended, making the widely used collaborative filtering strategy underperform. So, the main challenge is to carefully transfer the knowledge of user preferences from one domain to another by handling their different behaviors accordingly. In the second part of my talk, I will demonstrate my recently proposed algorithm for generating cross-domain recommendations and how the different user behaviors are weighted across multiple domains."

BIO: "Dimitrios Rafailidis is a postdoctoral research fellow at the Department of Computer Science at UMons in Belgium. His research interests are recommender systems and social media mining. His primary research goal is to generate personalized recommendations of massive, multimodal and streaming user data from different social media platforms, or any source that can capture user preferences. His main focus is on capturing user preference dynamics, and producing social and cross-domain recommendations. The results from this research have been published in leading peer reviewed journals, like TBD, TiiS, TOMCCAP, TMM, TCBB, TSMC, TASLP and SNAM, and highly selective conference proceedings such as RecSys, ECML/PKDD, CIKM, SIGIR, WWW and ASONAM."

IR Seminar: A Survey of Information Retrieval Approaches with Embedded Word Vectors (05 February, 2018)

Speaker: Debasis Ganguly

Standard information retrieval (IR) models are designed to work with categorical features, i.e., discrete terms. Generally speaking, documents are represented as vectors in a discrete term space facilitating the computation of pair-wise document similarities by standard vector space similarity (inverse distance) measures, such as the inner product between the vectors.
The limitations of these approaches are that: i) they assume that terms are independent; ii) they have no way of incorporating the notion of semantic distances between terms; iii) they have no way to address ‘concepts’ (combined meaning of multiple terms). To address the above limitations (and thereby, the age old problem of vocabulary mismatch for discrete terms), there has been an increasing trend in the IR research community to utilize semantic relationships between terms by embedding them within a continuous vector space over reals. The semantic relationship between the terms are then predicted by computing the distances between the words embedded as real-valued vectors. These semantic relationships are then applied to improve various IR tasks such as document ranking, query formulation, relevance feedback, end-to-end deep neural ranking models, session modeling etc.
This talk will focus on describing ways to incorporate term semantic information into standard retrieval models through the application of embedded word vectors. More specifically, we will analyze the key ideas of some recent papers on applications of word vectors for improving the effectiveness of various IR tasks, such as ad hoc ranking,
query modeling and session modeling.

IR Seminar: Natural Language Understanding in Virtual Agents for Airline Pilots. (22 January, 2018)

Speaker: Sylvain Daronnat

This presentation summarizes a 6 months master's internship that took place at Airbus (Toulouse, France) around a virtual agent research thematic for airline pilots. Our initial hypothesis was that a intent categorization system could benefit from using synthetic “natural-like” data. In order to test this hypothesis we decided, first, to create a methodology that would help us collect natural questions from end-users. Then we used the “natural” data we previously collected along with a synthetic question generator we designed in order to output synthetic questions that are as close as possible from the original ones. Lastly, we experimented on the synthetic datasets using various tools in order to put to the test our initial hypothesis. The results we obtained allowed us to open new perspectives on the natural language understanding part of the virtual agent system for airline pilots.

Short bio:
My name is Sylvain Daronnat, I'm a PhD student in computer and information sciences at Strathclyde University working on implementing new human-agent collaboration systems aboard submarines. For this project I'm also funded by Thales, a company designing electrical systems for various industries. Before my PhD, I was studying Natural Language Processing at the Grenoble Alpes University in France.

Automated Clinical Patient Health Surveillance (15 January, 2018)

Speaker: Stewart Whiting

Going beyond relevance: Incorporating effort into Information retrieval (04 December, 2017)

Speaker: Manisha Verma

Relevance lies at the core of evaluation of Information retrieval systems. However, with rapid development in search algorithms, a myriad number of search devices and increasing complexity of user information needs, we argue that relevance can no longer be the primary criteria for design and evaluation of IR systems. In this talk, I shall provide a brief overview of our work on characterizing, measuring and incorporating effort in IR.

The first half of the talk shall highlight our work on characterizing and measuring document specific effort. I shall provide a brief overview of how effort can be incorporated in information retrieval. I shall outline one important source of the mismatch between search log based evaluation and offline relevance judgments: the high degree of effort required to identify and consume relevant information in a document. I shall describe how to incorporate effort into existing learning to rank algorithms and their performance on publicly available datasets.

The second half of the talk shall focus on device-specific effort. Users have access to same information on several devices today. Our work attempts to analyze in-depth the differences between mobile and desktop. I shall give a brief overview of how judgments on both devices may differ significantly for different documents. I shall touch on the features that are useful in predicting effort across devices. Finally, I shall close the talk with some unresolved research questions and some failed attempts.

Manisha Verma is a final year Ph.D. student in Media futures group at University College London. Her primary area of research is characterizing user effort and incorporating it in retrieval and evaluation. Some of her recent work has been published at conferences such as CIKM, WSDM, ECIR, and SIGIR. Over the past few years, Manisha has worked with researchers at Google, Microsoft, and Yahoo on understanding role of user effort in retrieval. She has served as an Ambassador for postgraduate women at UCL and a co-coordinator of the Tasks Track in TREC 2015-2016 and TREC CAR Track in 2017.

IR seminar: Summarizing the Situation with Social Media Streams (27 November, 2017)

Speaker: Richard McCreadie

When a crisis hits, it is important for response agencies to quickly determine the situation on the ground, such that they can deploy the limited resources at their disposal as quickly and effectively as possible. However, during an emergency, information is difficult to come-by, as response units often need to arrive on the scene before the severity of the situation can be estimated. On the other hand, during emergencies, the general public is gravitating to social media platforms to ask for assistance and to show what they see to their friends. As such, emergency services are increasingly interested in technologies that can extract relevant information from social media during an emergency, to aid situational awareness. Meanwhile, real-time summarization is an emerging field that aims to build timeline summaries of events that are happening in the world, using news and social media streams as sensors. In this talk, I will provide an overview of what emergency services want to extract from social media, and how real-time summarization systems can help achieve this. Furthermore, I will discuss current technologies and techniques for real-time summarization that are relevant to the crisis domain, along with the challenges that are yet to be solved.


Richard McCreadie is a Research Associate at the University of Glasgow, UK. He is an information retrieval specialist, as well as developer and manager for the Terrier open source IR platform, which has been downloaded over 40,000 times since 2004. His research is focused on the interface between streaming IR and social media, tackling topics such information retrieval architectures for real-time stream processing; leveraging social media for event sensing (detecting events, extracting knowledge and summarizing those events); evaluation methodologies for streaming IR; and social media analytics, particularly when applied to security-related use-cases such as disaster management.

Richard received his Ph.D on the topic of News Vertical Search using User Generated Content in 2012 and is currently a senior researcher within the Terrier Team IR research group in Glasgow. Furthermore, he works with researchers and industry partners around the world to advance the IR field as co-chair of the streaming summarization evaluation initiatives (2014-Present) and 2018 Incident Streams emergency informatics initiative (2018) at the Text Retrieval Conference (TREC). He is active in the research community with 26 published conference papers in the areas of IR and social media, in addition to articles in longer formats, such as a book on Search in Social Media published in the highly-cited FnTIR series. Richard is also a current PC member for the top-tier conferences in the IR field (ACM CIKM, ACM SIGIR, AAAI ICWSM and ACM WSDM). 

IR Seminar: A Study of Snippet Length and Informativeness: Behaviour, Performance and User Experience (20 November, 2017)

Speaker: David Maxwell

The design and presentation of a Search Engine Results Page (SERP) has been subject to much research. With many contemporary aspects of the SERP now under scrutiny, work still remains in investigating more traditional SERP components, such as the result summary. Prior studies have examined a variety of different aspects of result summaries, but in this paper we investigate the influence of result summary length on search behaviour, performance and user experience. To this end, we designed and conducted a within-subjects experiment using the TREC AQUAINT news collection with 53 participants. Using Kullback-Leibler distance as a measure of information gain, we examined result summaries of different lengths and selected four conditions where the change in information gain was the greatest: (i) title only; (ii) title plus one snippet; (iii) title plus two snippets; and (iv) title plus four snippets. Findings show that participants broadly preferred longer result summaries, as they were perceived to be more informative. However, their performance in terms of correctly identifying relevant documents was similar across all four conditions. Furthermore, while the participants felt that longer summaries were more informative, empirical observations suggest otherwise; while participants were more likely to click on relevant items given longer summaries, they also were more likely to click on non-relevant items. This shows that longer is not necessarily better, though participants perceived that to be the case -- and second, they reveal a positive relationship between the length and informativeness of summaries and their attractiveness (i.e. clickthrough rates). These findings show that there are tensions between perception and performance when designing result summaries that need to be taken into account.

Neural Models for Information Retrieval (06 November, 2017)

Speaker: Bhaskar Mitra

Abstract: In the last few years, neural representation learning approaches have achieved very good performance on many natural language processing (NLP) tasks, such as language modelling and machine translation. This suggests that neural models may also yield significant performance improvements on information retrieval (IR) tasks, such as relevance ranking, addressing the query-document vocabulary mismatch problem by using semantic rather than lexical matching. IR tasks, however, are fundamentally different from NLP tasks leading to new challenges and opportunities for existing neural representation learning approaches for text.

In this talk, I will present my recent work on neural IR models. We begin with a discussion on learning good representations of text for retrieval. I will present visual intuitions about how different embeddings spaces capture different relationships between items, and their usefulness to different types of IR tasks. The second part of this talk is focused on the applications of deep neural architectures to the document ranking task.
Bio: Bhaskar Mitra is a Principal Applied Scientist at Microsoft AI & Research, Cambridge. He started at Bing in 2007 (then called Live Search) working on several problems related to document ranking, query formulation, entity ranking, and evaluation. His current research interests include representation learning and neural networks, and their applications to information retrieval. He co-organized multiple workshops (at SIGIR 2016 and 2017) and tutorials (at WSDM2017 and SIGIR 2017) on neural IR, and served as a guest editor for the special issue of the Information Retrieval Journal on the same topic. He is currently pursuing a doctorate at University College London under the supervision of Dr. Emine Yilmaz and Dr. David Barber.

IR Seminar: Jarana Manotumruksa (30 October, 2017)

Speaker: Jarana Manotumruksa

IR Seminar: Incorporating Positional Information and Other Domain Knowledge into a Neural IR Model (23 October, 2017)

Speaker: Andrew Yates

Retrieval models consider query-document interactions to produce a document relevance score for a given query. Traditionally, such interactions have been modelled using handcrafted statistics that generally compare term frequencies within a document and across a collection. Recently, neural models have demonstrated that they provide the instruments necessary to consider query-document interactions directly, without the need for such statistics.
In this talk, I will describe how positional term information can be incorporated into a neural IR model. The resulting model, called PACRR, performs substantially better on TREC benchmarks than previous neural approaches. This improvement can be attributed to the fact that PACRR can learn to match both ordered and unordered sequences of query terms in addition to the unigram matches considered by prior work. Using PACRR's approach to modeling query-document interactions as a foundation, I will describe how several well-known IR problems can be addressed within a neural framework; the resulting model substantially outperforms the original PACRR model. Finally, I will provide a brief look inside the PACRR model to highlight the types of positional information it uses and to investigate how such information is combined to produce a relevance score.

Context-aware and Context-Driven Applications on the Web (07 July, 2017)

Speaker: Yong Zheng

Context-awareness has been explored and applied in multiple areas, including ubiquitous computing, information retrieval and recommender systems. We may need to collect contexts in advance, so that the system can make changes by adapting to these dynamic situations. Obviously, it is much easier to collect these information from sensors in ubiquitous computing, but the process of context acquisition becomes one of the challenges in the Web applications. In this talk, we introduce the context-aware applications on the Web, especially based on the information retrieval and recommender systems. In addition, we highlight and discuss the context-driven applications that may influence the process of context collections, user interface and interactions, as well as relevant algorithms to support these novel applications.


Dr. Yong Zheng obtained his PhD degree in Computer and Information Sciences from DePaul University, USA. Currently, he is a full-time senior lecturer at Illinois Institute of Technology, USA. His research lie in user modeling, behavior analysis, human factors (user emotions and personalities), context-awareness, multi-criteria decision making, educational learning, and recommender systems. Particularly, he is one of the experts in the context-aware recommender systems. And he served as a data science consultant at NPAW (Nice People At Work), Barcelona, Spain to help them build context-aware recommendation engines. He published more than two dozens of academic papers related to his research topics. He served as publicity chair at ACM RecSys 2018 and ACM IUI 2018. He organized multiple workshops related to recommender systems. And he was invited as PC members for a number of academic conferences, such as WWW, ACM RecSys, ACM UMAP, ACM IUI, etc

Building relevance judgments automatically for a test collection. (19 June, 2017)

Speaker: Mireille Makary

In this talk, I will talk about my ongoing research and will present two different approaches I used to build relevance judgments (qrels) for TREC test collections without any human intervention. I will talk about an approach that involves using keyphrases extraction and another based on supervised machine learning techniques using Naïve Bayes and Support vector machines classifiers.

Bio: I am a PhD student at the University of Wolverhampton, Research Group in Computational Linguistics. My research area is in information retrieval. I am also a lecturer at Computer Science Department in the International University - Lebanon.


Effectively and Efficiently Searching Among Sensitive Content (08 June, 2017)

Speaker: Professor Douglas W. Oard

In Europe today, people have a “right to be forgotten.”  Exercising that right requires identifying each Web page that a person wishes to have removed from a search engine’s index.  In Maryland today, people have no right to record what they hear in the course of a day without the consent of every person whom they hear.  The law provides that the penalty for doing so could be as much as a year in jail for the first offence.  In many jurisdictions today, citizens have a right to request information held by their government.  Government officials who seek to sift through that information to determine which parts are releasable sometimes take so long to do so that the public purpose for which the request was originally made simply cannot be served.  In this talk I will argue that each of these problems arises from the same cause: an almost complete lack of attention to building language technologies that can proactively protect sensitive content.  I will further claim that the language technology for performing these tasks is well within the present state of the art, but that we will need to co-evolve the design of our information systems with the legislative, regulatory and normative public policy frameworks within those new capabilities would be employed.  Finally, I will illustrate the considerations that arise by describing a new project in which we are seeking to integrate protection for sensitive content into a search engine that is designed to provide public access to collections in which sensitive and non-sensitive content are intermixed and unlabelled.  

About the Speaker:

Douglas Oard is a Professor at the University of Maryland, College Park (USA), with joint appointments there in the College of Information Studies (Maryland’s iSchool) and the University of Maryland Institute for Advanced Computer Studies (UMIACS).  Dr. Oard earned his Ph.D. in Electrical Engineering from the University of Maryland.  His research interests center around the use of emerging technologies to support information seeking by end users.  Additional information is available at

Big Crisis Data - an exciting frontier for applied computing. (24 April, 2017)

Speaker: Carlos Castillo

Social media is an invaluable source of time-critical information during a crisis. However, emergency response and humanitarian relief organizations that would like to use this information struggle with an avalanche of social media messages, exceeding their capacity to process them. In this talk, we will look at how interdisciplinary research has enabled the creation of new tools for emergency managers, decision makers, and affected communities. These tools typically incorporate a combination of automatic processing and crowdsourcing. The talk will also look at ethical issues of this line of research.

Information Foraging in Environments (31 March, 2017)

Speaker: Kevin Ong

Kevin is a PhD student from ISAR Research Group at RMIT University, Australia. Kevin had previously worked on logs from National Archives UK, Peter MacCallum Cancer Institute, Westfield Group and Listcorp.

In this talk, he will talk about his work on information foraging in physical and virtual environments. The first part of his talk will be on "Understanding information foraging in physical environment - a log analysis" and the second part of his talk will be on "information foraging in virtual environments - an observational study".

Semantic Search at Bloomberg. (27 March, 2017)

Speaker: Edgar Meij


Large-scale knowledge graphs (KGs) store relationships between entities that are increasingly being used to improve the user experience in search applications. At Bloomberg we are currently in the process of rolling out our own knowledge graph and in this talk I will describe some of the semantic search applications that we aim to support. In particular, I will be discussing some of our recent papers on context-specific entity recommendations and automatically generating textual descriptions for arbitrary KG relationships.


Dr. Edgar Meij is a senior scientist at Bloomberg. Before this, he was a research scientist at Yahoo Labs and a postdoc at the University of Amsterdam, where he also obtained his PhD. His research focuses on advancing the state of the art in semantic search at Web scale, by designing entity-oriented search systems that employ knowledge graphs, entity linking, NLP, and machine learning techniques to improve the user experience, search, and recommendations. He has co-authored 50+ peer-reviewed papers and regularly teaches at the post-graduate level, including university courses, summer schools, and conference tutorials.

Assessing User Engagement in Information Retrieval Systems (20 March, 2017)

Speaker: Mengdie Zhuang


In this study, we investigated both using user actions from log files, and the results of the User Engagement Scale, both of which came from a study of people interacting with a retrieval interface containing an image collection, but with a non-purposeful task. Our results suggest that selected behaviour measures are associated with selected user perceptions of engagement  (i.e., focused attention, felt involvement, and novelty), while typical search and browse measures have no association with aesthetics and perceived usability. This is finding can lead towards a more systematic user-centered evaluation model.


Mengdie Zhuang is a PhD student from the University of Sheffield, UK. Her research focuses on evaluation metrics of Information Retrieval Systems.

Access, Search and Enrichment in Temporal Collections (06 March, 2017)

Speaker: Avishek Anand

There have been numerous efforts recently to digitize previously published content and preserving born-digital content leading to the widespread growth of large temporal text repositories. Temporal collections are continuously growing text collections which contain versions of documents spanning over long time periods and present many opportunities for historical, cultural and political analyses. Consequently there is a growing need for methods that can efficiently access, search and mine them. In this talk we deal with approaches in each of these aspects -- access, search and enrichment. First, I describe some of the access methods for searching temporal collections. Specifically, how do we index text to support temporal workloads? Secondly, I will describe retrieval models, which exploit historical information, essential in searching such collections. That is, how do we rank documents given temporal query intents? Finally, I will present some of the ongoing efforts to mine such collections for enriching Knowledge sources like Wikipedia.

Collaborative Information Retrieval. (27 February, 2017)

Speaker: Nyi Nyi Htun

Presentation of 2 papers to appear at CHIIR 2017.

Paper 1:

Title: How Can We Better Support Users with Non-Uniform Information Access in Collaborative Information Retrieval?

Abstract: The majority of research in Collaborative Information Retrieval (CIR) has assumed that collaborating team members have uniform information access. However, practice and research has shown that there may not always be uniform information access among team members, e.g. in healthcare, government, etc. To the best of our knowledge, there has not been a controlled user evaluation to measure the impact of non-uniform information access on CIR outcomes. To address this shortcoming, we conducted a controlled user evaluation using 2 non-uniform access scenarios (document removal and term blacklisting) and 1 full and uniform access scenario. Following this, a design interview was undertaken to provide interface design suggestions. Evaluation results show that neither of the 2 non-uniform access scenarios had a significant negative impact on collaborative and individual search outcomes. Design interview results suggested that awareness of team’s query history and intersecting viewed/judged documents could potentially help users share their expertise without disclosing sensitive information.

Paper 2:

Title: An Interface for Supporting Asynchronous Multi-Level Collaborative Information Retrieval

Abstract: Case studies and observations from different domains including government, healthcare and legal, have suggested that Collaborative Information Retrieval (CIR) sometimes involves people with unequal access to information. This type of scenario has been referred to as Multi-Level CIR (MLCIR). In addition to supporting collaboration, MLCIR systems must ensure that there is no unintended disclosure of sensitive information, this is an under investigated area of research. We present results of an evaluation of an interface we have designed for MLCIR scenarios. Pairs of participants used the interface under 3 different information access scenarios for a variety of search tasks. These scenarios included one CIR and two MLCIR scenarios, namely: full access (FA), document removal (DR) and term blacklisting (TR). Design interviews were conducted post evaluation to obtain qualitative feedback from participants. Evaluation results showed that our interface performed well for both DR and FA scenarios but for TR, team members with less access had a negative influence on their partner’s search performance, demonstrating insights into how different MLCIR scenarios should be supported. Design interview results showed that our interface helped the participants to reformulate their queries, understand their partner’s performance, reduce duplicated work and review their team’s search history without disclosing sensitive information.

A Comparison of Document-at-a-Time and Score-at-a-Time Query Evaluation (14 February, 2017)

Speaker: Joel Mackenzie

We present an empirical comparison between document-at-a-time (DaaT) and score-at-a-time (SaaT) document ranking strategies within a common framework. Although both strategies have been extensively explored, the literature lacks a fair, direct comparison: such a study has been difficult due to vastly different query evaluation mechanics and index organizations. Our work controls for score quantization, document processing, compression, implementation language, implementation effort, and a number of details, arriving at an empirical evaluation that fairly characterizes the performance of three specific techniques:WAND (DaaT), BMW (DaaT), and JASS (SaaT). Experiments reveal a number of interesting findings. The performance gap between WAND and BMW is not as clear as the literature suggests, and both methods are susceptible to tail queries that may take orders of magnitude longer than the median query to execute. Surprisingly, approximate query evaluation in WAND and BMW does not significantly reduce the risk of these tail queries. Overall, JASS is slightly slower than either WAND or BMW, but exhibits much lower variance in query latencies and is much less susceptible to tail query effects. Furthermore, JASS query latency is not particularly sensitive to the retrieval depth, making it an appealing solution for performance-sensitive applications where bounds on query latencies are desirable.


Joel is a PhD candidate at RMIT University, Melbourne, Australia. He works with Dr J. Shane Culpepper and Assoc Prof. Falk Scholer on efficient and effective candidate generation for multi-stage retrieval. His research interests include index efficiency, multi-stage retrieval and distributed IR.

Unsupervised Event Extraction and Storyline Generation from Text (13 February, 2017)

Speaker: Dr. Yulan He

This talk consists of two parts. In the first part, I will present our proposed Latent Event and Categorisation Model (LECM) which is an unsupervised Bayesian model for the extraction of structured representations of events from Twitter without the use of any labelled data. The extracted events are automatically clustered into coherence event type groups. The proposed framework has been evaluated on over 60 millions tweets and has achieved a precision of 70%, outperforming the state-of-the-art open event extraction system by nearly 6%. The LECM model has been extended to jointly modelling event extraction and visualisation which performs remarkably better than both the state-of-the-art event extraction method and a pipeline approach for event extraction and visualisation.

In the second part of my talk, I will present a non-parametric generative model to extract structured representations and evolution patterns of storylines simultaneously. In the model, each storyline is modelled as a joint distribution over some locations, organisations, persons, keywords and a set of topics. We further combine this model with the Chinese restaurant process so that the number of storylines can be determined automatically without human intervention. The proposed model is able to generate coherent storylines from new articles.

Yulan He is a Reader and Director of the Systems Analytics Research Institute at Aston University. She obtained her PhD degree in Spoken Language Understanding in 2004 from the University of Cambridge, UK. Prior joining Aston, she was a Senior Lecturer at the Open University, Lecturer at the University of Exeter and Lecturer at the University of Reading. Her current research interests lie in the integration of machine learning and natural language processing for text mining and social media analysis. Yulan has published over 140 papers with most appeared in high impact journals and at top conferences such as IEEE Transactions on Knowledge and Data Engineering, IEEE Intelligent Systems, KDD, CIKM, ACL, etc. She served as an Area Chair in NAACL 2016, EMNLP 2015, CCL 2015 and NLPCC 2015
and co-organised ECIR 2010 and IAPR 2007.

Applying Machine Learning to Data Exploration. (23 January, 2017)

Speaker: Charles Sutton

One of the first and most fundamental tasks in data mining is what we might call data understanding. Given a dump of data, what's in it? If modern machine learning methods are effective at finding patterns in data, then they should be effective at summarizing data sets so as to help data analysts develop a high-level understanding of them.

I'll describe several different approaches to this problem. First I'll describe a new approach to classic data mining problems, such as frequent itemset mining and frequent sequence mining, using a new principled model from probabilistic machine learning. Essentially, this casts the problem of pattern mining as one of structure learning in a probabilistic model. I'll describe an application to summarizing the usage of software libraries on Github.

A second attack to this general problem is based on cluster analysis. A good clustering can help a data analyst to explore and understand a data set, but what constitutes a good clustering may depend on domain-specific and application-specific criteria. I'll describe a new framework for interactive clustering that allows the analyst to examine a clustering and guide it in a way that is more useful for their information need.

Finally, topic modelling has proven to be a highly useful family of methods for data exploration, but it still requires a large amount of specialized effort to develop a new topic model for a specific data analysis scenario. I'll present new results on highly scalable inference for latent Dirichlet allocation based on recently proposed deep learning methods for probabilistic models.

Slides and relevant papers will be available at

The Role of Relevance in Sponsored Search. (16 January, 2017)

Speaker: Fabrizio Silvestri

Sponsored search aims at retrieving the advertisements that in the one hand meet users’ intent reflected in their search queries, and in the other hand attract user clicks to generate revenue. Advertisements are typically ranked based on their expected revenue that is computed as the product between their predicted probability of being clicked (i.e., namely clickability) and their advertiser provided bid. The relevance of an advertisement to a user query is implicitly captured by the predicted clickability of the advertisement, assuming that relevant advertisements are more likely to attract user clicks. However, this approach easily biases the ranking toward advertisements having rich click history. This may incorrectly lead to showing irrelevant advertisements whose clickability is not accurately predicted due to lack of click history. Another side effect consists of never giving a chance to new advertisements that may be highly relevant due to their lack of click history. To address this problem, we explicitly measure the relevance between an advertisement and a query without relying on the advertisement’s click history, and present different ways of leveraging this relevance to improve user search experience without reducing search engine revenue. Specifically, we propose a machine learning approach that solely relies on text-based features to measure the relevance between an advertisement and a query. We discuss how the introduced relevance can be used in four important use cases: pre-filtering of irrelevant advertisements, recovering advertisements with little history, improving clickability prediction, and re-ranking of the advertisements on the final search result page. Offline experiments using large-scale query logs and online A/B tests demonstrate the superiority of the proposed click-oblivious relevance model and the important roles that relevance plays in sponsored search.

Studies of Disputed Authorship (09 January, 2017)

Speaker: Michael P. Oakes

Automatic author identification is a branch of computational stylometry, which is the computer analysis of writing style. It is based on the idea that an author’s style can be described by a unique set of textual features, typically the frequency of use of individual words, but sometimes considering the use of higher level linguistic features. Disputed authorship studies assume that some of these features are outside the author’s conscious control, and thus provide a reliable means of discriminating between individual authors. Many studies have successfully made use of high frequency function words like “the”, “of” and “and”, which tend to have grammatical functions rather than reveal the topic of the text. Their usage is unlikely to be consciously regulated by authors, but varies substantially between authors, texts, and even individual characters in Jane Austen’s novels. Using stylometric techniques, Oakes and Pichler (2013) were able to show that the writing style of the document “Diktat für Schlick” was much more similar to that of Wittgenstein than that of other philosophers of the Vienna Circle. Michael Oakes is currently researching the authorship of “The Dark Tower”, normally attributed to C. S. Lewis.

Satisfying User Needs or Beating Baselines? Not always the same. (12 December, 2016)

Speaker: Walid Magdy

Information retrieval (IR) is mainly concerned with retrieving relevant documents to satisfy the information needs of users. Many IR tasks involving different genres and search scenarios have been studied for decades. Typically, researchers aim to improve retrieval effectiveness beyond the current “state-of-the-art”. However, revisiting the modeling of the IR task itself is often essential before seeking improvement of results. This includes reconsidering the assumed search scenario, the approach used to solve the problem, or even the conducted evaluation methodology. In this talk, some well-known IR tasks are explored to demonstrate that beating the state-of-the-art baseline is not always sufficient. Novel modeling, understanding, or approach to IR tasks could lead to significant improvements in user satisfaction compared to just improving “objective” retrieval effectiveness. The talk includes example IR tasks, such as printed document search, patent search, speech search, and social media search.

Supporting Evidence-based Medicine with Natural Language Processing (28 November, 2016)

Speaker: Dr. Mark Stevenson

The modern evidence-based approach to medicine is designed to ensure that patients are given the best possible care by basing treatment decisions on robust evidence. But the huge volume of information available to medical and health policy decision makers can make it difficult for them to decide on the best approach. Much of the current medical knowledge is stored in textual format and providing tools to help access it represents a significant opportunity for Natural Language Processing and Information Retrieval. However, automatically processing documents in this domain is not straightforward and doing so successfully requires a range of challenges to be overcome, including dealing with volume, ambiguity, complexity and inconsistency.  This talk will present a range of approaches from Natural Language Processing that support access to medical information. It will focus on three tasks: Word Sense Disambiguation, Relation Extraction and Contradiction Identification. The talk will outline the challenges faced when developing approaches for accessing information contained in medical documents, including the lack of available gold standard data to train systems. It will show how existing resources can help alleviate this problem by providing information that allows training data to be created automatically.

Human Computation for Entity-Centric Information Access (21 November, 2016)

Speaker: Dr. Gianluca Demartini

Human Computation is a novel approach used to obtain manual data processing at scale by means of crowdsourcing. In this talk we will start introducing the dynamics of crowdsourcing platforms and provide examples of their use to build hybrid human-machine information systems. We will then present ZenCrowd: a hybrid system for entity linking and data integration problems over linked data showing how the use of human intelligence at scale in combination with machine-based algorithms outperforms traditional systems. In this context, we will then discuss efficiency and effectiveness challenges of micro-task crowdsourcing platforms including spam, quality control, and job scheduling in crowdsourcing.

Analysis of the Cost and Benefits of Search Interactions (07 November, 2016)

Speaker: Dr. Leif Azzopardi

Interactive Information Retrieval (IR) systems often provide various features and functions, such as query suggestions and relevance feedback, that a user may or may not decide to use. The decision to take such an option has associated costs and may lead to some benefit. Thus, a savvy user would take decisions that maximises their net benefit. In this talk, we will go through a number of formal models which examine the costs and benefits of various decisions that users, implicitly or explicitly, make when searching. We consider and analyse the following scenarios: (i) how long a user's query should be? (ii) should the user pose a specific or vague query? (iii) should the user take a suggestion or re-formulate? (iv) when should a user employ relevance feedback? and (v) when would the "find similar" functionality be worthwhile to the user? To this end, we analyse a series of cost-benefit models exploring a variety of parameters that affect the decisions at play. Through the analyses, we are able to draw a number of insights into different decisions, provide explanations for observed behaviours and generate numerous testable hypotheses. This work not only serves as a basis for future empirical work, but also as a template for developing other cost-benefit models involving human-computer interaction.

This talk is based on the recent ICTIR 2016 paper with Guido Zuccon:

I'm an information scientist - let me in! (31 October, 2016)

Speaker: Martin White

For the last 46 years Martin has been a professional information scientist, though often in secret. Since founding Intranet Focus Ltd he has found that the awareness of research into topics such as information behaviour, information quality and information seeking in his clients is close to zero. This is especially true in information retrieval. In his presentation Martin will consider why this is the case, what the impact might be and what (if anything) should and could be done to change this situation.

The problem of quantification in Information Retrieval and on Social Networks. (17 October, 2016)

Speaker: Gianni Amati

There is a growing interest to know how fast information spreads on social networks, how many unique users are participating to an event, the leading opinion polarity in a stream. Quantifying distinct elements on a flow information is thus becoming a crucial problem in many real time information retrieval or streaming applications. We discuss the state-of-art of quantification and show that many problems can be interpreted within a common framework. We introduce a new probabilistic framework for quantification and show as examples how to count opinions in a stream and how to compute the degrees of separation of a network.

Analytics over Parallel Multi-view Data (03 October, 2016)

Speaker: Dr. Deepak Padmanabhan

Conventional unsupervised data analytics techniques have largely focused on processing datasets of single-type data, e.g., one of text, ECG, Sensor Readings and Image data. With increasing digitization, it has become common to have data objects having representations that encompass different "kinds" of information. For example, the same disease condition may be identified through EEG or fMRI data. Thus, a dataset of EEG-fMRI pairs would be considered as a parallel two-view dataset.  Datasets of text-image pairs (e.g., a description of a seashore, and an image of it) and text-text pairs (e.g., problem-solution text, or multi-language text from machine translation scenarios) are other common instances of multi-view data. The challenge in multi-view data analytics is about effectively leveraging such parallel multi-view data to perform analytics tasks such as clustering, retrieval and anomaly detection. This talk will cover some emerging trends in processing multi-view parallel data, and different paradigms for the same. In addition to looking at the different schools of techniques, and some specific techniques from each school, this talk will also be used to present some possibilities for future work in this area.


Dr. Deepak Padmanabhan is a lecturer with the Centre for Data Sciences and Scalable Computing at Queen's University Belfast. He obtained his B.Tech in Comp. Sc. and Engg. from Cochin University (Kerala, India), followed by his M.Tech and PhD, all in computer science, from Indian Institute of Technology Madras. Prior to joining Queen's, he was a researcher at IBM Research - India. He has over 40 publications across top venues in Data Mining, NLP, Databases and Information Retrieval. He co-authored a book on Operators for Similarity Search, published by Springer in 2015. He is the author on ~15 patent applications to the USPTO, including 4 granted patents. He is a recipient of the INAE Young Engineer Award 2015, and is a Senior Member of the ACM and the IEEE. His research interests include Machine Learning, Data Mining, NLP, Databases and Information Retrieval. Email:  URL:

The whole is greater than the sum of its parts: how semantic trajectories and recommendations may help tourism. (22 August, 2016)

Speaker: Dr. Chiara Renso

During the first part of this talk I will overview my recent activity in the field of mobility data mining with particular interest in the study of semantics in trajectory data and the experience with the SEEK Marie Curie project [1] recently concluded.  Then I will present two highlights of tourism recommendation works based on the idea of semantic trajectories: TripBuilder [2] and GroupFinder [3].  Tripbuilder is based on the analysis of enriched tourist trajectories extracted from Flickr photos to suggest itineraries constrained by a temporal budget and based on the travellers preferences.  The Groupfinder framework recommends a group of friends with whom to enjoy a visit to a place, balancing the friendship relations of the group members with the user individual interests in the destination location.

[2] Igo Ramalho Brilhante, José Antônio Fernandes de Macêdo, Franco Maria Nardini, Raffaele Perego,Chiara Renso. On planning sightseeing tours with TripBuilder. Inf. Process. Manage. 51(2): 1-15 (2015)
[3]  Igo Ramalho Brilhante, José Antônio Fernandes de Macêdo, Franco Maria Nardini, Raffaele Perego,Chiara Renso. Group Finder: An Item-Driven Group Formation Framework. MDM 2016: 8-17


Dr. Chiara Renso holds a PhD and M.Sc. degree in Computer Science from University of Pisa (1992, 1997).  She is permanent researcher at ISTI Institute of CNR, Italy.  Her research interests are related to spatio-temporal data mining, reasoning, data mining query languages, semantic data mining, trajectory data mining.  She has been involved in several EU projects about mobility data mining.  She has been the scientific coordinator of an FP7 Marie-Curie project on semantic trajectories knowledge discovery called SEEK (  She was also coordinator of a bilateral CNR-CNPQ Italy-Brazil project on mobility data mining with Federal University of Cearà.  She is author of more than 90 peer-reviewed publications.  She is co-editor of the book "Mobility Data: Modelling, Management, and Understanding" edited by Cambridge Press in 2013; co-editor of the special issue for Journal on Knowledge and Information system (KAIS) on Context aware data mining; co-editor of International Journal of Knowledge and Systems Science (IJKSS) on Modelling Tools for Extracting Useful Knowledge and Decision Making.  She has been co-chair of three editions of the Workshop on Semantic Aspects of Data Mining in conjunction with IEEE ICDM conference.  She is a regular reviewer of ACM CIKM, ACM KDD, ACM SIGSPATIAL and many journals on these topics.

Predicting Ad Quality for Native Advertisements (06 June, 2016)

Speaker: Dr Ke Zhou,

Native advertising is a specific form of online advertising where ads replicate the look-and-feel of their serving platform. In such context, providing a good user experience with the served ads is crucial to ensure long-term user engagement. 


In this talk, I will explore the notion of ad quality, namely the effectiveness of advertising from a user experience perspective. I will talk from both the pre-click and post-click perspective for predicting quality for native ads. With respect to pre-click ad quality, we design a learning framework to detect offensive native ads, showing that, to quantify ad quality, ad offensive user feedback rates are more reliable than the commonly used click-through rate metrics. We translate a set of user preference criteria into a set of ad quality features that we extract from the ad text, image and advertiser, and then use them to train a model able to identify offensive ads. In terms of post-click quality, we use ad landing page dwell time as our proxy and exploit various ad landing page features to predict ad landing page with high dwell time.

Efficient Web Search Diversification via Approximate Graph Coverage (25 April, 2016)

Speaker: Carsten Eickhoff

In the case of general or ambiguous Web search queries, retrieval systems rely on result set diversification techniques in order to ensure an adequate coverage of underlying topics such that the average user will find at least one of the returned documents.

In the case of general or ambiguous Web search queries, retrieval systems rely on result set diversification techniques in order to ensure an adequate coverage of underlying topics such that the average user will find at least one of the returned documents useful. Previous attempts at result set diversification employed computationally expensive analyses of document content and query intent. In this paper, we instead rely on the inherent structure of the Web graph. Drawing from the locally dense distribution of similar topics across the hyperlink graph, we cast the diversification problem as optimizing coverage of the Web graph. In order to reduce the computational burden, we rely on modern sketching techniques to obtain highly efficient yet accurate approximate solutions. Our experiments on a snapshot of Wikipedia as well as the ClueWeb'12 dataset show ranking performance and execution times competitive with the state of the art at dramatically reduced memory requirements.

Searching for better health: challenges and implications for IR (04 April, 2016)

Speaker: Dr. Guido Zuccon
A talk about why IR researchers should care about health search

In this talk I will discuss research problems and possible solutions related to helping the general public searching for health information online. I will show that although in the first instance this appears to be a domain-specific search task, research problems associated with this task have more general implications for IR and offer opportunities to develop advances that are applicable to the whole research field. In particular, in the talk I will focus on two aspects related to evaluation: (1) the inclusion of multiple dimensions of relevance in the evaluation of IR systems and (2) the modelling of query variations within the evaluation framework.

A Comparison of Primary and Secondary Relevance Judgements for Real-Life Topics (07 March, 2016)

Speaker: Dr Martin Halvey
n this talk I present a user study that examines in detail the differences between primary and secondary assessors on a set of

The notion of relevance is fundamental to the field of Information Retrieval. Within the field a generally accepted conception of relevance as inherently subjective has emerged, with an individual's assessment of relevance influenced by numerous contextual factors. In this talk I present a user study that examines in detail the differences between primary and secondary assessors on a set of "real-world" topics which were gathered specifically for the work. By gathering topics which are representative of the staff and students at a major university, at a particular point in time, we aim to explore differences between primary and secondary relevance judgements for real-life search tasks. Findings suggest that while secondary assessors may find the assessment task challenging in various ways (they generally possess less interest and knowledge in secondary topics and take longer to assess documents), agreement between primary and secondary assessors is high.  

Steps towards Profile-Based Web Site Search and Navigation (29 February, 2016)

Speaker: Prof. Udo Kruschwitz
Steps towards Profile-Based Web Site Search and Navigation

Web search in all its flavours has been the focus of research for decades with thousands of highly paid researchers competing for fame. Web site search has however attracted much less attention but is equally challenging. In fact, what makes site search (as well as intranet and enterprise search) even more interesting is that it shares some common problems with general Web search but also offers a good number of additional problems that need to be addressed in order to make search on a Web site no longer a waste of time. At previous visits to Glasgow I talked about turning the log files collected on a Web site into usable, adaptive data structures that can be used in search applications (and which we call user or cohort profiles). This time I will focus on applying these profiles to a navigation scenario and illustrate how the automatically acquired profiles provide a practical use case for combining natural language processing and information retrieval techniques (as that is what we really do at Essex).

Sentiment and Preference Guided Social Recommendation. (22 February, 2016)

Speaker: Yoke Yie Chen
In this talk, I will focus on two knowledge sources for product recommendation: product reviews and user purchase trails.

Social recommender systems harness knowledge from social media to generate recommendations. Previous works in social recommender systems use social knowledge such as social tags, social relationship (social network) and microblogs.  In this talk, I will focus on two knowledge sources for product recommendation: product reviews and user purchase trails. In particular, I will present how we exploit the sentiment expressed in product reviews and user preferences which are implicitly contained in user purchase trails as the basis for recommendation.

Recent Advances in Search Result Diversification for the Web and Social Media (17 February, 2016)

Speaker: Ismail Sengor Altingovde
I will focus on the web search result diversification problem and present our novel contributions in the field.

In this talk, I will start with a short potpourri of our most recent research, emphasis being on the topics related to the web search engines and social Web. Then, I will focus on the web search result diversification problem and present our novel contributions in three directions. Firstly, I will present how the normalizaton of query relevance scores can boost the performance of the state-of-the-art explicit diversification strategies. Secondly, I will introduce a set of new explicit diversification strategies based on the score(-based) and rank(-based) aggregation methods. As a third contribution, I will present how query performance prediction (QPP) can be utilized to weight query aspects during diversification. Finally, I will discuss how these diversification methods perform in the context of Tweet search, and how we improve them using word embeddings.

Practical and theoretical problems on the frontiers of multilingual natural language processing (16 February, 2016)

Speaker: Dr Adam Lopez
Multilingual natural language processing (NLP) has been enormously successful over the last decade, highlighted by offerings like Google translate. What is left to do?

Multilingual natural language processing (NLP) has been enormously successful over the last decade, highlighted by offerings like Google translate. What is left to do? I'll focus on two quite different, very basic problems that we don't yet know how to solve. The first is motivated by the development of new, massively-parallel hardware architectures like GPUs, which are especially tantalizing for computation-bound NLP problems, and may open up new possibilities for the application and scale of NLP. The problem is that classical NLP algorithms are inherently sequential, so harnessing the power of such processors requires complete rethinking the fundamentals of the field. The second is motivated by the fact that NLP systems often fail to correctly understand, translate, extract, or generate meaning. We're poised to make serious progress in this area using the reliable method of applying machine learning to large datasets—in this case, large quantities of natural language text annotated with explicit meaning representations, which take the form of directed acyclic graphs. The problem is that probabilities on graphs are surprisingly hard to define. I'll discuss work on both of these fronts.

Information retrieval challenges in conducting systematic reviews (08 February, 2016)

Speaker: Julie Glanville
The presentation will also describe other areas where software such as text mining and machine learning have potential to contribute to the Systematic Review process

Systematic review (SR) is a research method that seeks to provide an assessment of the state of the research evidence on a specific question.  Systematic reviews (SRs) aim to be objective, transparent and replicable and seek to minimise bias by means of extensive  searches.


The challenges of extensive searching will be summarised.  As software tools and internet interconnectivity increase, we are seeing increasing use of a range of tools within the SR process (not only for information retrieval).  This presentation will present some  of the tools we are currently using within the Cochrane SR community and UK SRs, and the challenges which remain for efficient information retrieval.  The presentation will also describe other areas where software such as text mining and machine learning have potential to contribute to the SR process.

Learning to Hash for Large Scale Image Retrieval (14 December, 2015)

Speaker: Sean Moran
In this talk I will introduce two novel data-driven models that significantly improve the retrieval effectiveness of locality sensitive hashing (LSH), a popular randomised algorithm for nearest neighbour search that permits relevant data-points to be ret

In this talk I will introduce two novel data-driven models that significantly improve the retrieval effectiveness of locality sensitive hashing (LSH), a popular randomised algorithm for nearest neighbour search that permits relevant data-points to be retrieved in constant time, independent of the database size.

To cut down the search space LSH generates similar binary hashcodes for similar data-points and uses the hashcodes to index database data-points into the buckets of a set of hashtables. At query time only those data-points that collide in the same hashtable buckets as the query are returned as candidate nearest neighbours. LSH has been successfully used for event detection in large scale streaming data such as Twitter [1] and for detecting 100,000 object classes on a single CPU [2].


The generation of similarity preserving binary hashcodes comprises two steps: projection of the data-points onto the normal vectors of a set of hyperplanes partitioning the input feature space followed by a quantisation step that uses a single threshold to binarise the resulting projections to obtain the hashcodes. In this talk I will argue that the retrieval effectiveness of LSH can be significantly improved by learning the thresholds and hyperplanes based on the distribution of the input data.


In the first part of my talk I will provide a high level introduction of LSH. I will then argue that LSH makes a set of limiting assumptions arising from its data-independence that hamper its retrieval effectiveness. This motivates the second and third parts of my talk in which I introduce two new models that address these limiting assumptions. 


Firstly, I will discuss a scalar quantisation model that can learn multiple thresholds per LSH hyperplane using a novel semi-supervised objective function [3]. Optimising this objective function results in thresholds that reduce information loss inherent in converting the real-valued projections to binary. Secondly, I will introduce a new two-step iterative model for learning the hashing hyperplanes [4]. In the first step the hashcodes of training data-points are regularised over an adjacency graph which encourages similar data-points to be assigned similar hashcodes. In the second step a set of binary classifiers are learnt so as to separate opposing bits (0,1) with maximum margin. Repeating both steps iteratively encourages the hyperplanes to evolve into positions that provide a much better bucketing of the input feature space compared to LSH.


For both algorithms I will present a set of query-by-example image retrieval results on standard image collections, demonstrating significantly improved retrieval effectiveness versus state-of-the-art hash functions, in addition to a set of interesting and previously unexpected results.

[1] Sasa Petrovic, Miles Osborne and Victor Lavrenko, Streaming First Story Detection with Application to Twitter, In NAACL'10.


[2] Thomas Dean, Mark Ruzon, Mark Segal, Jonathon Shlens, Sudheendra Vijayanarasimhan,  and Jay Yagnik, Fast, Accurate Detection of 100,000 Object Classes on a Single Machine, In CVPR'13.


[3] Sean Moran, Victor Lavrenko and Miles Osborne. Neighbourhood Preserving Quantisation for LSH, In SIGIR'13.


[4] Sean Moran and Victor Lavrenko. Graph Regularised Hashing. In ECIR'15.




Building Effective and Efficient Information Retrieval Systems (26 June, 2015)

Speaker: Jimmy Lin
Machine learning has become the tool of choice for tackling challenges in a variety of domains, including information retrieval

Machine learning has become the tool of choice for tackling challenges in a variety of domains, including information retrieval. However, most approaches focus exclusively on effectiveness---that is, the quality of system output. Yet, real-world production systems need to search billions of documents in tens of milliseconds, which means that techniques also need to be efficient (i.e., fast).  In this talk, I will discuss two approaches to building more effective and efficient information retrieval systems. The first is to directly learn ranking functions that are inherently more efficient---a thread of research dubbed "learning to efficiently rank". The second is through architectural optimizations that take advantage of modern processor architectures---by paying attention to low-level details such as cache misses and branch mispredicts. The combination of both approaches, in essence, allow us to "have our cake and eat it too" in building systems that are both fast and good.

Analysing UK Annual Report Narratives using Text Analysis and Natural Language Processing (23 February, 2015)

Speaker: Mahmoud El-Haj
In this presentation I will show the work we’ve done in our Corporate Financial Information Environment (CFIE) project.

In this presentation I will show the work we’ve done in our Corporate Financial Information Environment (CFIE) project.  The Project, funded by ESRC and ICAEW, seeks to analyse UK financial narratives, their association with financial statement information, and their informativeness for investors using Computational Linguistics, heuristic Information Extraction (IE) and Natural Language Processing (NLP).  We automatically collected and analysed a number of 14,000 UK annual reports covering a period between 2002 and 2014 for the UK largest firms listed on the London Stock Exchange. We developed software for this purpose which is available online for general use by academics.  The talk includes a demo on the software that we developed and used in our analysis: Wmatrix-import and Wmatrix.  Wmatrix-import is a web-based tool to automatically detect and parse the structure of UK annual reports; the tool provides sectioning, word frequency and readability metrics.  The output from Wmatrix-import goes as input for further NLP and corpus linguistic analysis by Wmatrix - a web based corpus annotation and retrieval tool which currently supports the analysis of small to medium sized English corpora.




CFIE Project

Compositional Data Analysis (CoDA) approaches to distance in information retrieval (20 February, 2015)

Speaker: Dr Paul Thomas
Many techniques in information retrieval produce counts from a sample

Many techniques in information retrieval produce counts from a sample, and it is common to analyse these counts as proportions of the whole—term frequencies are a familiar example.  Proportions carry only relative information and are not free to vary independently of one another: for the proportion of one term to increase, one or more others must decrease.  These constraints are hallmarks of compositional data.  While there has long been discussion in other fields of how such data should be analysed, to our knowledge, Compositional Data Analysis (CoDA) has not been considered in IR. In this work we explore compositional data in IR through the lens of distance measures, and demonstrate that common measures, naïve to compositions, have some undesirable properties which can be avoided with composition-aware measures.  As a practical example, these measures are shown to improve clustering.

Users versus Models: What observation tells us about effectiveness metrics (16 February, 2015)

Speaker: Dr. Paul Thomas
This work explores the link between users and models by analysing the assumptions and implications of a number of effectiveness metrics, and exploring how these relate to observable user behaviours

Retrieval system effectiveness can be measured in two quite different ways: by monitoring the behaviour of users and gathering data about the ease and accuracy with which they accomplish certain specified information-seeking tasks; or by using numeric effectiveness metrics to score system runs in reference to a set of relevance judgements.  In the second approach, the effectiveness metric is chosen in the belief that it predicts ease or accuracy.

This work explores that link, by analysing the assumptions and implications of a number of effectiveness metrics, and exploring how these relate to observable user behaviours.  Data recorded as part of a user study included user self-assessment of search task difficulty; gaze position; and click activity.  Our results show that user behaviour is influenced by a blend of many factors, including the extent to which relevant documents are encountered, the stage of the search process, and task difficulty.  These insights can be used to guide development of batch effectiveness metrics.

Towards Effective Retrieval of Spontaneous Conversational Spoken Content (08 January, 2015)

Speaker: Gareth J. F. Jones
Spoken content retrieval (SCR) has been the focus of various research initiatives for more then 20 years.

Spoken content retrieval (SCR) has been the focus of various research initiatives for more then 20 years. Early research focused on retrieval of clear defined spoken documents principally from the broadcast news domain. The main focus of this work was spoken document retrieval (SDR) task at TREC-6-9. The end of which saw SDR declared a largely solved problem. However, this was soon found to be a premature conclusion relating to controlled recordings of professional news content and overlooking many of the potential challenges of searching more complex spoken content. Subsequent research has focused on more challenging tasks such as search of interview recordings and semi-professional internet content.  This talk will begin by reviewing early work in SDR, explaining its successes and limitations, it will then move to outline work exploring SCR for more challenging tasks, such as identifying relevant elements in long spoken recordings such as meetings and presentations, provide a detailed analysis of the characteristics of retrieval behaviour of spoken content elements when indexed using manual and automatic transcripts, and conclude with a summary of the challenges of delivering effective SCR for complex spoken content and initial attempts to address these challenges. 

On Inverted Index Compression for Search Engine Efficiency (01 September, 2014)

Speaker: Matteo Catena

Efficient access to the inverted index data structure is a key aspect for a search engine to achieve fast response times to users’ queries. While the performance of an information retrieval (IR) system can be enhanced through the compression of its posting lists, there is little recent work in the literature that thoroughly compares and analyses the performance of modern integer compression schemes across different types of posting information (document ids, frequencies, positions). In this talk, we show the benefit of compression for different types of posting information to the space- and time-efficiency of the search engine. Comprehensive experiments have been conducted on two large, widely used document corpora and large query sets; using different modern integer compression algorithms, integrated into a modern IR system, the Terrier IR platform. While reporting the compression scheme which results in the best query response times, the presented analysis will also show the impact of compression on frequency and position posting information in Web corpora that have large volumes of anchor text.

Adaptive Interaction (02 June, 2014)

Speaker: Professor Andrew Howes
A utility maximization approach to understanding human interaction with technology

This lecture describes a theoretical framework for the behavioural sciences that holds high promise for theory-driven research and design in Human-Computer Interaction. The framework is designed to tackle the adaptive, ecological, and bounded nature of human behaviour. It is designed to help scientists and practitioners reason about why people choose to behave as they do and to explain which strategies people choose in response to utility, ecology, and cognitive information processing mechanisms. A key idea is that people choose strategies so as to maximise utility given constraints. The framework is illustrated with a number of examples including pointing, multitasking, skim- reading, online purchasing, Signal-Detection Theory and diagnosis, and the influence of reputation on purchasing decisions. Importantly, these examples span from perceptual/motor coordination, through cognition to social interaction. Finally, the lecture discusses the challenging idea that people seek to find optimal strategies and also discusses the implications for behavioral investigation in HCI.

Web-scale Semantic Ranking (16 May, 2014)

Speaker: Dr Nick Craswell
Bing Ranking Techniques

Semantic ranking models score documents based on closeness in meaning to the query rather than by just matching keywords. To implement semantic ranking at Web-scale, we have designed and deployed a new multi-level ranking systems that combines the best of inverted index and forward index technologies. I will describe this infrastructure which is currently serving many millions of users and explore several types of semantic models: translation models, syntactic pattern matching and topical matching models. Our experiments demonstrate that these semantic ranking models significantly improve relevance over our existing baseline system. This is the repeat of a WWW2014 industry track talk.

Optimized Interleaving for Retrieval Evaluation (28 April, 2014)

Speaker: Filip Radlinski

Interleaving is an online evaluation technique for comparing the relative quality of information retrieval functions by combining their result lists and tracking clicks. A sequence of such algorithms have been proposed, each being shown to address problems in earlier algorithms. In this talk, I will formalize and generalize this process, while introducing a formal model: After identifying a set of desirable properties for interleaving, I will show that an interleaving algorithm can be obtained as the solution to an optimization problem within those constraints. This approach makes explicit the parameters of the algorithm, as well as assumptions about user behavior. Further, this approach leads to an unbiased and more efficient interleaving algorithm than any previous approach, as I will show a novel log-based analysis of user search behaviour.

Composite retrieval of heterogeneous web search (24 March, 2014)

Speaker: Horatiu Bota

Traditional search systems generally present a ranked list of documents as answers to user queries. In aggregated search systems, results from different and increasingly diverse verticals (image, video, news, etc.) are returned to users. For instance, many such search engines return to users both images and web documents as answers to the query ``flower''. Aggregated search has become a very popular paradigm. In this paper, we go one step further and study a different search paradigm: composite retrieval. Rather than returning and merging results from different verticals, as is the case with aggregated search, we propose to return to users a set of "bundles", where a bundle is composed of "cohesive'' results from several verticals. For example, for the query "London Olympic'', one bundle per sport could be returned, each containing results extracted from news, videos, images, or Wikipedia. Composite retrieval can promote exploratory search in a way that helps users understand the diversity of results available for a specific query and decide what to explore in more detail. 


We proposed and evaluated a variety of approaches to construct bundles that are relevant, cohesive and diverse. We also utilize both entitiy and term as a surrogate to represent items and demonstrate their effectiveness of bridging the "mismatch" gap among heterogeneous sources. Compared with three baselines (traditional "general web only'' ranking, federated search ranking and aggregated search),  our evaluation results demonstrate significant performance improvement for a highly heterogeneous web collection.

Query Auto-completion & Composite retrieval (17 March, 2014)

Speaker: Stewart Whiting and Horatiu Bota

=Recent and Robust Query Auto-Completion by Stewart Whiting=

Query auto-completion (QAC) is a common interactive feature that assists users in formulating queries by providing completion suggestions as they type. In order for QAC to minimise the user’s cognitive and physical effort, it must: (i) suggest the user’s intended query after minimal input keystrokes, and (ii) rank the user’s intended query highly in completion suggestions. QAC must be both robust and time-sensitive – that is, able to sufficiently rank both consistently and recently popular queries in completion suggestions. Addressing this trade-off, we propose several practical completion suggestion ranking approaches, including: (i) a sliding window of query popularity evidence from the past 2-28 days, (ii) the query popularity distribution in the last N queries observed with a given prefix, and (iii) short-range query popularity prediction based on recently observed trends. Through real-time simulation experiments, we extensively investigated the parameters necessary to maximise QAC effectiveness for three openly available query log datasets with prefixes of 2-5 characters: MSN and AOL (both English), and Sogou 2008 (Chinese). Results demonstrate consistent and language-independent improvements of up to 9.2% over a non-temporal QAC baseline for all query logs with prefix lengths of 2-3 characters. Hence, this work is an important step towards more effective QAC approaches.


=Composite retrieval of heterogeneous web search by Horatiu Bota=

Traditional search systems generally present a ranked list of documents as answers to user queries. In aggregated search systems, results from different and increasingly diverse verticals (image, video, news, etc.) are returned to users. For instance, many such search engines return to users both images and web documents as answers to the query ``flower''. Aggregated search has become a very popular paradigm. In this paper, we go one step further and study a different search paradigm: composite retrieval. Rather than returning and merging results from different verticals, as is the case with aggregated search, we propose to return to users a set of "bundles", where a bundle is composed of "cohesive'' results from several verticals. For example, for the query "London Olympic'', one bundle per sport could be returned, each containing results extracted from news, videos, images, or Wikipedia. Composite retrieval can promote exploratory search in a way that helps users understand the diversity of results available for a specific query and decide what to explore in more detail. 


We proposed and evaluated a variety of approaches to construct bundles that are relevant, cohesive and diverse. We also utilize both entitiy and term as a surrogate to represent items and demonstrate their effectiveness of bridging the "mismatch" gap among heterogeneous sources. Compared with three baselines (traditional "general web only'' ranking, federated search ranking and aggregated search),  our evaluation results demonstrate significant performance improvement for a highly heterogeneous web collection.

Studying the performance of semi-structured p2p information retrieval (10 March, 2014)

Speaker: Rami Alkhawaldeh

In recent decades, retrieval systems deployed over peer-to-peer (P2P) overlay networks have been investigated as an alternative to centralised search engines. Although modern search engines provide efficient document retrieval, there are several drawbacks, including: a single point of failure, maintenance costs, privacy risks, information monopolies from search engines companies, and difficulty retrieving hidden documents in the web (i.e. the deep web). P2P information retrieval (P2PIR) systems promise an alternative distributed system to the traditional centralised search engine architecture. Users and creators of web content in such networks have full control over what information they wish to share as well as how they share it.




Researchers have been tackling several challenges to build effective P2PIR systems: (i) collection (peer) representation during indexing, (ii) peer selection during search to route queries to relevant peers and (iii) final peer result merging. Semi-structured P2P networks (i.e, a partially decentralised unstructured overlay network) offer an intermediate design that minimizes the weakness of both centralised and completely decentralised overlay networks and combines the advantages of those two topologies. So, an evaluation framework for this kind of network is necessary to compare the performance of different P2P approaches and to be a guide for developing new and more powerful approaches. In this work, we study the performance of three cluster-based semi-structured P2PIR models and explain the effectiveness of several important design considerations and parameters on retrieval performance, as well as the robustness of these types of network.


4pm @ Level 4

Inside The World’s Playlist (23 February, 2014)

Speaker: Manos Tsagkias


We describe the algorithms behind Streamwatchr, a real-time system for analyzing the music listening behavior of people around the world. Streamwatchr collects music-related tweets, extracts artists and songs, and visualises the results in two ways: (i)~currently trending songs and artists, and (ii)~newly discovered songs.


Dublin City Search: An evolution of search to incorporate city data (24 November, 2013)

Speaker: Dr Veli Bicer, IBM Research Dublin
ors, devices, social networks, governmental applications, or service networks. In such a diversity of information, answering specific information needs of city inhabitants requires holistic information retrieval techniques, capable of harnessing differen

Dr Veli Bicer is a researcher at Smarter Cities Technology Center of IBM Research in Dublin. His research interests include semantic data management, semantic search, software engineering and statistical relational learning. He obtained his PhD from Karlsruhe Institute of Technology, Karlsruhe, Germany and B.Sc. and M.Sc. degrees in computer engineering from Middle East Technical University, Ankara, 

Economic Models of Search (18 November, 2013)

Speaker: Leif Azzopardi


Predicting Screen Touches From Back-of-Device Grip Changes (14 November, 2013)

Speaker: Faizuddin Mohd Noor

We demonstrate that front-of-screen targeting on mobile phones can be predicted from back-of-device grip manipulations. Using simple, low-resolution capacitive touch sensors placed around a standard phone, we outline a machine learning approach to modelling the grip modulation and inferring front-of-screen touch targets. We experimentally demonstrate that grip is a remarkably good predictor of touch, and we can predict touch position 200ms before contact with an accuracy of 18mm.

Online Learning in Explorative Multi Period Information Retrieval (11 November, 2013)

Speaker: Marc Sloan


In Multi Period Information Retrieval we consider retrieval as a stochastic yet controllable process, the ranking action during the process continuously controls the retrieval system's dynamics and an optimal ranking policy is found in order to maximise the overall users' satisfaction. Different aspects of this process can be fixed giving rise to different search scenarios. One such application is to fix search intent and learn from a population of users over time. Here we use a multi-armed bandit algorithm and apply techniques from finance to learn optimally diverse and explorative search results for a query. We can also fix the user and dynamically model the search over multiple pages of results using relevance feedback. Likewise we are currently investigating using the same technique over session search using a Markov Decision Process.

Stopping Information Search: An fMRI Investigation (04 November, 2013)

Speaker: Eric Walden

Information search has become an increasingly important factor in people's use of information systems.  In both personal and workplace environments, advances in information technology and the availability of information have enabled people to perform far more search and access much more information for decision making than in the very recent past.  One consequence of this abundance of information has been an increasing need for people to develop better heuristic methods for stopping search, since information available for most decisions now overwhelms people's cognitive processing capabilities and in some cases is almost infinite.  Information search has been studied in much past research, and cognitive stopping rules have also been investigated.  The present research extends and expands on previous behavioral research by investigating brain activation during searching and stopping behavior using functional Magnetic Resonance Imaging (fMRI) techniques.  We asked subjects to search for information about consumer products and to stop when they believed they had enough information to make a subsequent decision about whether to purchase that product.  They performed these tasks while in an MRI machine.  Brain scans were taken that measured brain activity throughout task performance.  Results showed that different areas of the brain were active for searching and stopping, that different brain regions were used for several different self-reported stopping rules, that stopping is a neural correlate of inhibition, suggesting a generalized stopping mechanism in the brain, and that certain individual difference variables make no difference in brain regions active for stopping.  The findings extend our knowledge of information search, stopping behavior, and inhibition, contributing to both the information systems and neuroscience literatures.  Implications of our findings for theory and practice are discussed.

Towards Technically assisted Sensitivity Review of UK Digital Public Records (21 October, 2013)

Speaker: Tim Gollins

There are major difficulties involved in identifying sensitive information in digital public records. These difficulties, if not addressed, will together with the challenge of managing the risks of failing to identify sensitive documents, force government departments into the precautionary closure of large swaths of digital records. Such closures will inhibit timely, open and transparent access by citizens and others in civic society. Precautionary closures will also prevent social scientists’ and contemporary historians’ access to valuable qualitative information, and their ability to contextualise studies of emerging large scale quantitative data. Closely analogous problems exist in UK local authorities, the third sector, and in other countries which are covered by the same or similar legislation and regulation. In 2012, having conducted investigations and earlier research into this problem, and with new evidence of immediate need emerging from the 20 year rule transition process, The UK National Archives (TNA) highlighted this serious issue facing government departments in the UK Public Records system; the Abaca project is the response.


The talk will outline the role of TNA, the background to sensitivity review, the impact of the move to born digital records, the nature of the particular challenge of reviewing them for sensitivity, and the broad approach that the Abaca Project is taking.



Next Monday, 4pm at 423

Accelerating research on big datasets with Stratosphere (14 October, 2013)

Speaker: Moritz Schubotz
Stratosphere is a research project investigating new paradigms for scalable, complex analytics on massively-parallel data sets.

Stratosphere is a research project investigating new paradigms for scalable, complex analytics on massively-parallel data sets. The core concept of Stratosphere is the PACT programming model that extends MapReduce with second order functions like Match, CoGroup and Cross, which allows researchers to describe complex analytics task naturally. The result are directed acyclic that are optimized for parallel execution, by a cost based optimizer that incorporates user code properties, and executed by the Nephele Data Flow Engine. Nephele is a massively parallel data flow engine dealing with resource management, work scheduling, communication, and fault tolerance.

In the seminar session we introduce and showcase how researchers can set their working environment quickly and start doing research right away. As a proof of concept, we present how a simple java program parallelized optimized by Stratosphere obtained top results at the "exotic" Math search task at NTCIR-10. While other research groups optimized index structures and data formats and waited several hours for their indices to be build on high end hardware, we could focus on the essential program logic use basic data types and run the experiments on a heterogenous desktop cluster in several minutes.

Validity and Reliability in Cranfield-like Evaluation in Information Retrieval (23 September, 2013)

Speaker: Julián Urbano

The Cranfield paradigm to Information Retrieval evaluation has been used for half a century now as the means to compare retrieval techniques and advance the state of the art accordingly. However, this paradigm makes certain assumptions that remain a research problem in Information Retrieval and that may invalidate our experimental results.

In this talk I will approach the Cranfield paradigm as an statistical estimator of certain probability distributions that describe the final user experience. These distributions are estimated with a test collection, which actually computes system-related distributions that are assumed to be correlated with the target user-related distributions. From the point of view of validity, I will discuss the strength of that correlation and how it affects the conclusions we draw from an evaluation experiment. From the point of view of reliability, I will discuss on past and current practice to measure the reliability of test collections and review several of them accordingly.

Exploration and contextualization: towards reusable tools for the humanities. (16 September, 2013)

Speaker: Marc Bron

The introduction of new technologies, access to large electronic

cultural heritage repositories, and the availability of new

information channels continues to change the way humanities

researchers work and the questions they seek to answer. In this talk I

will discuss how the research cycle of humanities researchers has been

affected by these changes and argue for the continued development of

interactive information retrieval tools to support the research

practices of humanities researchers. Specifically, I will focus on two

phases in the humanities research cycle: the exploration phase and

contextualization phase. In the first part of the talk I discuss work

on the development and evaluation of search interfaces aimed at

supporting exploration. In the second part of the talk I will focus on

how information retrieval technology focused on identifying relations

between concepts may be used to develop applications that support


Quantum Language Models (19 August, 2013)

Speaker: Alessandro Sordoni

A joint analysis of both Vector Space and Language Models for IR

using the mathematical framework of Quantum Theory revealed how both

models allocate the space of density matrices. A density matrix is

shown to be a general representational tool capable of leveraging

capabilities of both VSM and LM representations thus paving the way

for a new generation of retrieval models. The new approach is called

Quantum Language Modeling (QLM) and has shown its efficiency and

effectiveness in modeling term dependencies for Information


Toward Models and Measures of Findability (21 July, 2013)

Speaker: Colin Wilkie
A summary of the work being undertaken on Findability

In this 10 minute talk, I will provide an overview of the project I am working on, which is about Findability, and review some of the existing models and measures of findability, before outlining the models that I have working on.

How cost affects search behaviour (21 July, 2013)

Speaker: Leif Azzopardi
Find out about how microeconomic theory predicts user behaviour...

In this talk, I will run through the work I will be presenting at SIGIR on "How cost affects search behavior". The empirical analysis is motivated and underpinned using the Search Economic Theory that I proposed at SIGIR 2011. 

[SICSA DVF] Language variation and influence in social media (15 July, 2013)

Speaker: Dr. Jacob Eisenstein
Dr. Eisenstein works on statistical natural language processing, focusing on social media analysis, discourse, and latent variable models

Languages vary by speaker and situation, and change over time.  While variation and change are inhibited in written corpora such as news text, they are endemic to social media, enabling large-scale investigation of language's social and temporal dimensions. The first part of this talk will describe a method for characterizing group-level language differences, using the Sparse Additive Generative Model (SAGE). SAGE is based on a re-parametrization of the multinomial distribution that is amenable to sparsity-inducing regularization and facilitates joint modeling across many author characteristics. The second part of the talk concerns change and influence. Using a novel dataset of geotagged word counts, we induce a network of linguistic influence between cities, aggregating across thousands of words. We then explore the demographic and geographic factors that drive spread of new words between cities. This work is in collaboration with Amr Ahmed, Brendan O'Connor, Noah A. Smith, and Eric P. Xing.

Jacob Eisenstein is an Assistant Professor in the School of Interactive Computing at Georgia Tech. He works on statistical natural language processing, focusing on social media analysis, discourse, and latent variable models. Jacob was a Postdoctoral researcher at Carnegie Mellon and the University of Illinois. He completed his Ph.D. at MIT in 2008, winning the George M. Sprowls dissertation award.


The Use of Correspondence Analysis in Information Retrieval (11 July, 2013)

Speaker: Dr Taner Dincer
This presentation will introduce the application of Correspondence Analysis in Information Retrieval

This presentation will introduce the application of Correspondence Analysis (CA) to Information Retrieval. CA is a well-established multivariate, statistical, exploratory data analysis technique. Multivariate data analysis techniques usually operate on a rectangular array of real numbers called a data matrix whose rows represent r observations (for example, r terms/words in documents) and columns represent c variables (for the example, c documents, resulting in a rxc term-by-document matrix). Multivariate data analysis refers to analyze the data in a manner that takes into account the relationships among observations and also among variables. In contrast to univariate statistics, it is concerned with the joint nature of measurements. The objective of exploratory data analysis is to explore the relationships among objects and among variables over measurements for the purpose of visual inspection. In particular, by using CA one can visually study the “Divergence From Independence” (DFI) among observations and among variables.

For Information Retrieval (IR), CA can serve three different uses: 1) As an analysis tool to visually inspect the results of information retrieval experiments, 2) As a basis to unify the probabilistic approaches to term weighting problem such as Divergence From Randomness and Language Models, and 3) As a term weighting model itself, "term weighting based on measuring divergence from independence". In this presentation, the uses of CA for these three purposes are exemplified.

A study of Information Management in the Patient Surgical Pathway in NHS Scotland (03 June, 2013)

Speaker: Matt-Mouley Bouamrane

We conducted a study of information management processes across the patient surgical pathway in NHS Scotland. While the majority of General Practitioners (GPs) consider electronic information systems as an essential and integral part of their work during the patient consultation, many were not fully satisfied with the functionalities of these systems. A majority of GPs considered that the national eReferral system streamlined referral processes. Almost all GPs reported marked variability in the quality of discharge information. Preoperative processes vary significantly across Scotland, with most services using paper based systems. There is insufficient use made of information provided through the patient electronic referral and a considerable duplication of effort with the work already performed in primary care. Three health-boards have implemented electronic preoperative information systems. These have transformed clinical practices and facilitated communication and information-sharing among the multi-disciplinary team and within the health boards. Substantial progress has been made towards improving information transfer and sharing within the surgical pathway in recent years but there remains scope for further improvements at the interface between services.

Discovering, Modeling, and Predicting Task-by-Task Behaviour of Search Engine Users (20 May, 2013)

Speaker: Salvatore Orlando

Users of web search engines are increasingly issuing queries to accomplish their daily tasks (e.g., “finding a recipe”, “booking a flight”, “read- ing online news”, etc.). In this work, we propose a two-step methodology for discovering latent tasks that users try to perform through search engines. Firstly, we identify user tasks from individual user sessions stored in query logs. In our vision, a user task is a set of possibly non-contiguous queries (within a user search session), which refer to the same need. Secondly, we discover collective tasks by aggregating similar user tasks, possibly performed by distinct users. To discover tasks, we propose to adopt clustering algorithms based on novel query similarity functions, in turn obtained by exploiting specific features, and both unsupervised and supervised learning approaches.  All the proposed solutions were evaluated on a manually-built ground-truth.

Furthermore, we introduce the the Task Relation Graph (TGR) as a representation of users' search behaviors on a task-by-task perspective, by exploiting the collective tasks obtained so far. The task-by-task behavior is captured by weighting the edges of TGR with a relatedness score computed between pairs of tasks, as mined from the query log.  We validated our approach on a concrete application, namely a task recommender system, which suggests related tasks to users on the basis of the task predictions derived from the TGR. Finally, we showed that the task recommendations generated by our technique are beyond the reach of existing query suggestion schemes, and that our solution is able to recommend tasks that user will likely perform in the near future. 


Work in collaboration with Claudio Lucchese, Gabriele Tolomei, Raffaele Perego, and Fabrizio Silvestri.



[1] C. Lucchese, S. Orlando, R. Perego, F. Silvestri, G. Tolomei. "Identifying Task-based Sessions in Search Engine Query Logs". Forth ACM Int.l Conference on Web Search and Data Mining (WSDM 2011), Hong Kong, February 9-12, 2011

[2] C. Lucchese, S. Orlando, R. Perego, F. Silvestri, G. Tolomei. "Discovering Tasks from Search Engine Query Logs", To appear on ACM Transactions on Information Systems (TOIS). 

[3] C. Lucchese, S. Orlando, R. Perego, F. Silvestri, G. Tolomei. "Modeling and Predicting the Task-by-Task Behavior of Search Engine Users". To appear in Proc. OAIR 2013, Int.l Conference in the RIAO series.

Personality Computing (13 May, 2013)

Speaker: Alessandro Vinciarelli



Personality is one of the driving factors behind everything we do and experience

in life. During the last decade, the computing community has been showing an ever

increasing interest for such a psychological construct, especially when it comes

to efforts aimed at making machines socially intelligent, i.e. capable of interacting with

people in the same way as people do. This talk will show the work being done in this

area at the School of Computing Science. After an introduction to the concept of

personality and its main applications, the presentation will illustrate experiments

on speech based automatic perception and recognition. Furthermore, the talk will

outline the main issues and challenges still open in the domain.  

Fast and Reliable Online Learning to Rank for Information Retrieval (06 May, 2013)

Speaker: Katja Hoffman

Online learning to rank for information retrieval (IR) holds promise for allowing the development of "self-learning search engines" that can automatically adjust to their users. With the large amount of e.g., click data that can be collected in web search settings, such techniques could enable highly scalable ranking optimization. However, feedback obtained from user interactions is noisy, and developing approaches that can learn from this feedback quickly and reliably is a major challenge.


In this talk I will present my recent work, which addresses the challenges posed by learning from natural user interactions. First, I will detail a new method, called Probabilistic Interleave, for inferring user preferences from users' clicks on search results. I show that this method allows unbiased and fine-grained ranker comparison using noisy click data, and that this is the first such method that allows the effective reuse of historical data (i.e., collected for previous comparisons) to infer information about new rankers. Second, I show that Probabilistic Interleave enables new online learning to rank approaches that can reuse historical interaction data to speed up learning by several orders of magnitude, especially under high levels of noise in user feedback. I conclude with an outlook on research directions in online learning to rank for IR, that are opened up by our results.

Entity Linking for Semantic Search (29 April, 2013)

Speaker: Edgar Meij

Semantic annotations have recently received renewed interest with the explosive increase in the amount of textual data being produced, the advent of advanced NLP techniques, and the maturing of the web of data. Such annotations hold the promise for improving information retrieval algorithms and applications by providing means to automatically understand the meaning of a piece of text. Indeed, when we look at the level of understanding that is involved in modern-day search engines (on the web or otherwise), we come to the obvious conclusion that there is still a lot of room for improvement. Although some recent advances are pushing the boundaries already, information items are still retrieved and ordered mainly using their textual representation, with little or no knowledge of what they actually mean. In this talk I will present my recent and ongoing work, which addresses the challenges associated with leveraging semantic annotations for intelligent information access. I will introduce a recently proposed method for entity linking and show how it can be applied to several tasks related to semantic search on collections of different types, genres, and origins. 

Query Classification for a Digital Library (18 March, 2013)

Speaker: Deirdre Lungley

The motivation for our query classification is the insight it gives the digital content provider into what his users are searching for and hence how his collection could be extended. This talk details two query classification methodologies we have implemented as part of the GALATEAS project ( one log-based, the other using wikified queries to learn a Labelled LDA model. An analysis of their respective classification errors indicates the method best suited to particular category groups. 

Reusing Historical Interaction Data for Faster Online Learning to Rank for IR (12 March, 2013)

Speaker: Anne Schuth


Online learning to rank for information retrieval (IR) holds promise for allowing the development of ³self-learning² search engines that can automatically adjust to their users. With the large amount of e.g., click data that can be collected in web search settings, such techniques could enable highly scalable ranking optimization. However, feedback obtained from user interactions is noisy, and developing approaches that can learn from this feedback quickly and reliably is a major challenge.


In this paper we investigate whether and how previously collected (historical) interaction data can be used to speed up learning in online learning to rank for IR. We devise the first two methods that can utilize historical data (1) to make feedback available during learning more reliable and (2) to preselect candidate ranking functions to be evaluated in interactions with users of the retrieval system. We evaluate both approaches on 9 learning to rank data sets and find that historical data can speed up learning, leading to substantially and significantly higher online performance. In particular, our preselection method proves highly effective at compensating for noise in user feedback. Our results show that historical data can be used to make online learning to rank for IR much more effective than previously possible, especially when feedback is noisy.

Scientific Lenses over Linked Data: Identity Management in the Open PHACTS project (11 March, 2013)

Speaker: Alasdair Gray,

Scientific Lenses over Linked Data: Identity Management in the Open PHACTS project

Alasdair Gray, University of Manchester


The discovery of new medicines requires pharmacologists to interact with a number of information sources ranging from tabular data to scientific papers, and other specialized formats. The Open PHACTS project, a collaboration of research institutions and major pharmaceutical companies, has developed a linked data platform for integrating multiple pharmacology datasets that form the basis for several drug discovery applications. The functionality offered by the platform has been drawn from a collection of prioritised drug discovery business questions created as part of the Open PHACTS project. Key features of the linked data platform are:

1) Domain specific API making drug discovery linked data available for a diverse range of applications without requiring the application developers to become knowledgeable of semantic web standards such as SPARQL;

2) Just-in-time identity resolution and alignment across datasets enabling a variety of entry points to the data and ultimately to support different integrated views of the data;

3) Centrally cached copies of public datasets to support interactive response times for user-facing applications.


Within complex scientific domains such as pharmacology, operational equivalence between two concepts is often context-, user- and task-specific. Existing linked data integration procedures and equivalence services do not take the context and task of the user into account. We enable users of the Open PHACTS platform to control the notion of operational equivalence by applying scientific lenses over linked data. The scientific lenses vary the links that are activated between the datasets which affects the data returned to the user



Alasdair is a researcher in the MyGrid team at the University of Manchester. He is currently working on the Open PHACTS project which is building an Open Pharmacological Space to integrate drug discovery data. Alasdair gained his PhD from Heriot-Watt University, Edinburgh, and then worked as a post-doctoral researcher in the Information Retrieval Group at the University of Glasgow. He has spent the last 10 years working on novel knowledge management projects investigating issues of relating data sets.

Modelling Time & Demographics in Search Logs (01 March, 2013)

Speaker: Milad Shokouhi

Knowing users' context offers a great potential for personalizing web search results or related services such as query suggestion and query completion. Contextual features cover a wide range of signals; query time, user’s location,  search history and demographics can all  be regarded as contextual features that can be used for search personalization.

In this talk, we’ll focus on two main questions:

1)      How can we use the existing contextual features, in particular time, for improving search results (Shokouhi & Radinsky, SIGIR’12).

2)      How can we infer missing contextual features, in particular user-demographics, based on search history (Bi et al., WWW2013).


Our results confirm that (1) contextual features matter and (2) that many of them can be inferred from search history.

Time-Biased Gain (21 February, 2013)

Speaker: Charlie Clark
Time-biased gain provides a unifying framework for information retrieval evaluation

Time-biased gain provides a unifying framework for information retrieval evaluation, generalizing many traditional effectiveness measures while accommodating aspects of user behavior not captured by these measures. By using time as a basis for calibration against actual user data, time-biased gain can reflect aspects of the search process that directly impact user experience, including document length, near-duplicate documents, and summaries. Unlike traditional measures, which must be arbitrarily normalized for averaging purposes, time-biased gain is reported in meaningful units, such as the total number of relevant documents seen by the user. In work reported at SIGIR 2012, we proposed and validated a closed-form equation for estimating time-biased gain, explored its properties, and compared it to standard approaches. In work reported at CIKM 2012, we used stochastic simulation to numerically approximate time-biased gain, an approach that provides greater flexibility, allowing us to accommodate different types of user behavior and increases the realism of the effectiveness measure. In work reported at HCIR 2012, we extended our stochastic simulation to model the variation between users. In this talk, I will provide an overview of time-biased gain, and outline our ongoing and future work, including extensions to evaluate query suggestion, diversity, and whole-page relevance. This is joint work with Mark Smucker.

[IR] Searching the Temporal Web: Challenges and Current Approaches (04 February, 2013)

Speaker: Nattiya Kanhabua

In this talk, we will give a survey of current approaches to searching the

temporal web. In such a web collection, the contents are created and/or

edited over time, and examples are web archives, news archives, blogs,

micro-blogs, personal emails and enterprise documents. Unfortunately,

traditional IR approaches based on term-matching only can give

unsatisfactory results when searching the temporal web. The reason for this

is multifold:  1) the collection is strongly time-dependent, i.e., with

multiple versions of documents, 2) the contents of documents are about

events happened at particular time periods, 3) the meanings of semantic

annotations can change over time, and 4) a query representing an information

need can be time-sensitive, so-called a temporal query.


Several major challenges in searching the temporal web will be discussed,

namely, 1) How to understand temporal search intent represented by

time-sensitive queries? 2) How to handle the temporal dynamics of queries

and documents? and 3) How to explicitly model temporal information in

retrieval and ranking models? To this end, we will present current

approaches to the addressed problems as well as outline the directions for

future research.

Probabilistic rule-based argumentation for norm-governed learning agents (28 January, 2013)

Speaker: Sebastian Riedel

There is a vast and ever-increasing amount of unstructured textual data at our disposal. The ambiguity, variability and expressivity of language makes this data difficult to analyse, mine, search, visualise, and, ultimately, base decisions on. These challenges have motivated efforts to enable machine reading: computers that can read text and convert it into semantic representations, such as the Google Knowledge Graph for general facts, or pathway databases in the biomedical domain. This representations can then be harnessed by machines and humans alike. At the heart of machine reading is relation extraction: reading text to create a semantic network of entities and their relations, such as employeeOf(Person,Company), regulates(Protein,Protein) or causes(Event,Event). 

In this talk I will present a series of graphical models and matrix factorisation techniques that can learn to extract relations. I will start by contrasting a fully supervised approach with one that leverages pre-existing semantic knowledge (for example, in the Freebase database) to reduce annotation costs. I will then present ways to extract additional relations that are not yet part of the schema, and for which no pre-existing semantic knowledge is available. I will show that by doing so we cannot only extract richer knowledge, but also improve extraction quality of relations within the original schema. This helps to improve over previous state-of-the-art by more than 10% points mean average precision. 

Context data in lifelog retrieval (19 November, 2012)

Speaker: Liadh Kelly
Context data in lifelog retrieval

Advances in digital technologies for information capture combined with
massive increases in the capacity of digital storage media mean that it is
now possible to capture and store much of one's life experiences in a
personal lifelog. Information can be captured from a myriad of personal
information devices including desktop computers, mobile phones, digital
cameras, and various sensors, including GPS, Bluetooth, and biometric
devices. This talk centers on the investigation of the challenges of
retrieval in this emerging domain and on the examination of the utility of
several implicitly recorded and derived context types in meeting these
challenges. For these investigations unique rich multimodal personal
lifelog collections of 20 months duration are used. These collections
contain all items accessed on subjects' PCs and laptops (email, web pages,
word documents, etc), passively captured images depicting subjects' lives
using the SenseCam device (, and
mobile text messages sent and received. Items are annotated with several
rich sources of automatically derived context data types including
biometric data (galvanic skin response, heart rate, etc), geo-location
(captured using GPS data), people present (captured using Bluetooth data),
weather conditions, light status, and several context types related to the
dates and times of accesses to items.


From Search to Adaptive Search (12 November, 2012)

Speaker: Udo Kruschwitz
Generating good query modification suggestions or alternative queries to assist a searcher remains however a challenging issue

Modern search engines have been moving away from very simplistic interfaces that aimed at satisfying a user's need with a single-shot query. Interactive features such as query suggestions and faceted search are now integral parts of Web search engines. Generating good query modification suggestions or alternative queries to assist a searcher remains however a challenging issue. Query log analysis is one of the major strands of work in this direction. While much research has been performed on query logs collected on the Web as a whole, query log analysis to enhance search on smaller and more focused collections (such as intranets, digital libraries and local Web sites) has attracted less attention. The talk will look at a number of directions we have explored at the University of Essex in addressing this problem by automatically acquiring continuously updated domain models using query and click logs (as well as other sources).

Events Webapp