Links tell us about lexical and semantic Web content
Filippo Menczer
Abstract
The latest generation of Web search tools is beginning to exploit hypertext link information to improve rankingBrin98,Kleinberg98 and crawlingMenczer00,Ben-Shaul99etal,Chakrabarti99 algorithms. The hidden assumption behind such approaches, a correlation between the graph structure of the Web and its content, has not been tested explicitly despite increasing research on Web topologyLawrence98,Albert99,Adamic99,Butler00. Here I formalize and quantitatively validate two conjectures drawing connections from link information to lexical and semantic Web content. The clink-content conjecture states that a page is similar to the pages that link to it, i.e., one can infer the lexical content of a page by looking at the pages that link to it. I also show that lexical inferences based on link cues are quite heterogeneous across Web communities. The link-cluster conjecture states that pages about the same topic are clustered together, i.e., one can infer the meaning of a page by looking at its neighbours. These results explain the success of the newest search technologies and open the way for more dynamic and scalable methods to locate information in a topic or user driven way.
Create a lesson
Related papers
Tracing and Taming Hallucinated Terms via Subsequence-Association Filtering and Breadth-Aware Selection for LLM Query Expansion
Amit Kumar Jaiswal, Sagar Uprety, Prayag Tiwari
SPARQL-LLM in the TEXT2SPARQL 2026 Challenge
Panayiotis Smeros, Vincent Emonet, Ruijie Wang et al.
NEAR: Neighborhood-Aware Residuals for Memory Retrieval
Han Luo, Zhan Zhuang, Xu He et al.
PRIME: Pre-Retrieval Index Modeling Enhancement for Structured Document Retrieval-Real
Cheuk Hei Chung, Li Liu
Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence
Han Luo, Zhan Zhuang, Xu He et al.
Compact and Efficient Indexes for Learned Sparse Retrieval
Franco Maria Nardini, Luca Rizzo, Cosimo Rulli et al.