Ranking skyline points with an IR-style weighting scheme

1. George Valkanas1
, Apostolos N. Papadopoulos2
, Dimitrios Gunopulos1
Skyline Ranking à la IR
1
University of Athens, Greece
2
Aristotle University of Thessaloniki, Greece
1st
ExploreDB Workshop
Athens, Greece
28th
March, 2014

2. Skyline Problem Introduction
• Dataset D = (p1, p2, …, pn) in d-dimensional space
• Preferences for each dimension: min, max
• p dominates q iff pi ≤ qi i = 1,..,d && j: pj < qj

3. Usefulness of Skyline
• Multi-Objective optimization
• Exploratory Search
• Improve Recommendations
• Data summarization technique
• Building block for defining competitiveness

4. Skyline Cardinality Explosion
O( (ln n)d-1
)
• Skyline becomes too large
to inspect manually

5. Solving the Cardinality Problem
• Select subset of size k
– Coverage-based
– Contour representation
– Diversification
• Ranking
– Top-k Dominating
– Subspace dominance

6. Skyline + IR: Intuition
• Dominated points are not equally important
• Scheme similar to TF-IDF

7. Skyline + IR: How ?
• 2 Factors
– DP (~ tf)
– IDP (~ idf)
• DP-IDP

8. Ranking the Skyline
• Baseline:
– sp
• Iterate over its dominated points, and SUM
Slow
Unnecessary computations
• Alternative?
Bound the score
• Lower
• Upper
Prune skyline points

9. A Simpler Representation
• More comprehensive for bounds

10. Bounding the Score
• Q1: What is the score for B ?

11. Bounding the Score
• Q1: What is the score for B ?
• A1: Depends on the assignment of the
remaining edges

12. Bounding the Score
• Q1: What is the score for B ?
• A1: Depends on the assignment of the
remaining edges
• Q2: What is the maximum score for B ?

13. Bounding the Score
• Q1: What is the score for B ?
• A1: Depends on the assignment of the
remaining edges
• Q2: What is the maximum score for B ?
• A2: Assign appropriately the remaining
edges

14. Bounding the Score
• Q1: What is the score for B ?
• A1: Depends on the assignment of the
remaining edges
• Q2: What is the maximum score for B ?
• A2: Assign appropriately the remaining
edges
• Q3: What is the appropriate way?

15. Bounding the Score
• Q1: What is the score for B ?
• A1: Depends on the assignment of the
remaining edges
• Q2: What is the maximum score for B ?
• A2: Assign appropriately the remaining
edges
• Q3: What is the appropriate way?
• A3:
– Same layer → Higher score (dp)
– Minimum overlap → Higher score (idp)
• No overlap → Loose bounds

16. The SkyIR Algorithm

17. The SkyIR Algorithm

18. The SkyIR Algorithm

19. The SkyIR Algorithm

20. The SkyIR Algorithm

21. The SkyIR Algorithm

22. The SkyIR Algorithm
• Priority can be:
– Round Robin (RRB)
– Pending points (PND)
– Upper Bound (UBS)

23. Experimental Setup
• Datasets
• Algorithms
– Baseline
– SkyIR
• Bounds: Loose (LS), Collaborative (CB)
• 3 Priority schemes: RRB, PND, UBS

24. Total Runtime – IND distr
k=5, d=3
CB-UBS is 4x faster than the Baseline

25. Total Runtime – ANT distr
• Interesting fact: ANT is easier than IND
(fewer layers to extract)

26. Total Runtime – Forest Cover

27. Memory Consumption
CB, k=5
PND is the best memory-wise

28. Conclusions
• IR-style ranking for skyline
– Formal framework
– Bounds for efficient computation
• SkyIR algorithm
– Experimental evaluation
• Future Work
– Speed up / Scale up
– Improve bounds (lower, upper)
– Approximation technique(s)

29. Thank you!
Questions?
Acknowledgements: Heraclitus II fellowship, THALIS –
GeomComp, THALIS – DISFER, ARISTEIA – MMD, FP7 INSIGHT