<<  Online Search Committee Informational Module for Faculty, Department Chairs, and Deans GOOD NEWS, BAD NEWS FOR CCUsing accouting information to make managerial desisions  >>
Joint Relevance and Freshness Learning From Clickthroughs for News
Joint Relevance and Freshness Learning From Clickthroughs for News
Relevance v.s. Freshness
Relevance v.s. Freshness
Freshness is Important for News Search
Freshness is Important for News Search
Freshness is Important for News Search
Freshness is Important for News Search
Understand Users Information Need
Understand Users Information Need
Understand Users Information Need
Understand Users Information Need
Assess Users Information Need
Assess Users Information Need
Manipulate Editors Annotation
Manipulate Editors Annotation
Users Judgment on Relevance and Freshness
Users Judgment on Relevance and Freshness
Joint Relevance and Freshness Learning
Joint Relevance and Freshness Learning
Joint Relevance and Freshness Learning
Joint Relevance and Freshness Learning
Joint Relevance and Freshness Learning
Joint Relevance and Freshness Learning
Joint Relevance and Freshness Learning
Joint Relevance and Freshness Learning
Temporal Features
Temporal Features
Temporal Features
Temporal Features
Experiment Results
Experiment Results
Experiment Results
Experiment Results
Analysis of JRFL
Analysis of JRFL
Analysis of JRFL
Analysis of JRFL
Analysis of JRFL
Analysis of JRFL
Analysis of JRFL
Analysis of JRFL
Analysis of JRFL
Analysis of JRFL
Analysis of JRFL
Analysis of JRFL
Analysis of JRFL
Analysis of JRFL
Quantitative Comparison
Quantitative Comparison
Quantitative Comparison
Quantitative Comparison
Quantitative Comparison
Quantitative Comparison
Qualitative Comparison
Qualitative Comparison
Conclusions
Conclusions
References
References
Thank you
Thank you

: Joint Relevance and Freshness Learning From Clickthroughs for News Search. : Hongning. : Joint Relevance and Freshness Learning From Clickthroughs for News Search.pptx. zip-: 3396 .

Joint Relevance and Freshness Learning From Clickthroughs for News Search

Joint Relevance and Freshness Learning From Clickthroughs for News Search.pptx
1 Joint Relevance and Freshness Learning From Clickthroughs for News

Joint Relevance and Freshness Learning From Clickthroughs for News

Search

Hongning Wang+, Anlei Dong*, Lihong Li*, Yi Chang*, Evgeniy Gabrilovich* +CS@UIUC *Yahoo! Labs

2 Relevance v.s. Freshness

Relevance v.s. Freshness

Relevance Topical relatedness Metric: tf*idf, BM25, Language Model Freshness Temporal closeness Metric: age, elapsed time Trade-off Serve for users information need

3 Freshness is Important for News Search

Freshness is Important for News Search

Apple Company @ Oct. 4, 2011

4 Freshness is Important for News Search

Freshness is Important for News Search

Apple Company @ Oct. 5, 2011

5 Understand Users Information Need

Understand Users Information Need

Users emphasis on relevance/freshness varies Breaking news queries Prefer latest news reports freshness driven E.g., apple company Newsworthy queries Prefer high coverage and authority news reports relevance driven E.g., bin laden death

6 Understand Users Information Need

Understand Users Information Need

Users emphasis on relevance/freshness varies

7 Assess Users Information Need

Assess Users Information Need

Unsupervised integration [Efron 2011, Li 2003] Limited on timestamps Editors judgment [Dong 2010, Dai 2011] Expensive for timely annotation Inadequate to recover end-users information need

8 Manipulate Editors Annotation

Manipulate Editors Annotation

Freshness-demoted relevance Rule-based hard demotion [Dong 2010] E.g., if the result is somewhat outdated, it should be demoted by one grade (e.g., from excellent to good)

9 Users Judgment on Relevance and Freshness

Users Judgment on Relevance and Freshness

Users browsing behavior

10 Joint Relevance and Freshness Learning

Joint Relevance and Freshness Learning

JRFL: (Relevance, Freshness) -> Click

Query => trade-off

URL => relevance/freshness

11 Joint Relevance and Freshness Learning

Joint Relevance and Freshness Learning

Model formalization

Latent

12 Joint Relevance and Freshness Learning

Joint Relevance and Freshness Learning

Linear instantiation Associative property Relevance/Freshness model learning Query model learning

13 Joint Relevance and Freshness Learning

Joint Relevance and Freshness Learning

Coordinate descent for JRFL Randomly initialize , and set Repeat until converge Update Relevance/Freshness models: Update Query model: Return the final model

Convex programming

14 Temporal Features

Temporal Features

URL freshness features Identify freshness from content analysis

15 Temporal Features

Temporal Features

Query freshness features Capture latent preference

16 Experiment Results

Experiment Results

Data sets Two months Yahoo! News Search sessions Normal bucket: top 10 positions Random bucket [Li 2011] Randomly shuffled top 4 positions Unbiased evaluation corpus Editors judgment: 1 days query log Preference pair selection [Joachims 2005] Click > Skip above Click > Skip next Ordered by Pearsons value

17 Experiment Results

Experiment Results

Data sets Statistics

18 Analysis of JRFL

Analysis of JRFL

Convergence Train/Test sets: 90k/60k preference pairs Varying initial query weight

(a) Object Function Value Update

19 Analysis of JRFL

Analysis of JRFL

Convergence Train/Test sets: 90k/60k preference pairs Varying initial query weight

(b) Pairwise Error Rate Update

20 Analysis of JRFL

Analysis of JRFL

Convergence Train/Test sets: 90k/60k preference pairs Varying initial query weight

(c) Query Weight Update

21 Analysis of JRFL

Analysis of JRFL

Feature weight learning

22 Analysis of JRFL

Analysis of JRFL

Relevance and Freshness Learning Baseline: GBRank trained on Dong et al.s relevance/freshness annotation set Testing corpus: editors one day annotation set

23 Analysis of JRFL

Analysis of JRFL

Query weight analysis

24 Analysis of JRFL

Analysis of JRFL

Query weight analysis Query length differs in relevance/freshness driven queries significantly

25 Quantitative Comparison

Quantitative Comparison

Ranking performance Random bucket clicks

26 Quantitative Comparison

Quantitative Comparison

Ranking performance Normal clicks

27 Quantitative Comparison

Quantitative Comparison

Ranking performance Editorial annotations

28 Qualitative Comparison

Qualitative Comparison

CTR distribution revisit

29 Conclusions

Conclusions

Joint Relevance and Freshness Learning Query-specific preference Learning from query logs Temporal features Future work Personalized retrieval Broad spectral of users information need E.g., trustworthiness, opinion

30 References

References

[Efron 2011] M. Efron and G. Golovchinsky. Estimation methods for ranking recent information. In SIGIR, pages 495504, 2011. [Li 2003] X. Li and W. Croft. Time-based language models. In CIKM, pages 469475, 2003. [Dong 2010] A. Dong, Y. Chang, Z. Zheng, G. Mishne, J. Bai, R. Zhang, K. Buchner, C. Liao, and F. Diaz. Towards recency ranking in web search. In WSDM, pages 1120, 2010. [Dai 2011] N. Dai, M. Shokouhi, and B. D. Davison. Learning to rank for freshness and relevance. In SIGIR, pages 95104, 2011. [Li 2011] L. Li, W. Chu, J. Langford, and X. Wang. Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms. In Proceedings of ACM WSDM '11, pages 297306, 2011. [Joachims 2005] T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In SIGIR, pages 154161, 2005.

31 Thank you

Thank you

Q&A

Joint Relevance and Freshness Learning From Clickthroughs for News Search
http://900igr.net/prezentacija/anglijskij-jazyk/joint-relevance-and-freshness-learning-from-clickthroughs-for-news-search-137925.html
c

29
900igr.net > > > Joint Relevance and Freshness Learning From Clickthroughs for News Search