Using sensors in the web crawling process
Ilya Zemskov
Abstract
This paper offers a short description of an Internet information field monitoring system, which places a special module-sensor on the side of the Web-server to detect changes in information resources and subsequently reindexes only the resources signalized by the corresponding sensor. Concise results of simulation research and an implementation attempt of the given "sensors" concept are provided.
Create a lesson
Related papers
Reasoning Quality Matters: Combating Reasoning Collapse in LLM-based Embedding Learning
Zihan Gong, Xiaohan Ye, Jiangchao Yao et al.
Think Thrice Before Reranking: Multi-perspective Evidence and Reasoning Integration for Text Reranking
Lijun Liu, Zhengzong Chen, Wenyan Li et al.
The Missing Complement: State-Conditioned Minimal Sufficient Evidence for Coding Agents
Zhexi Feng, Ruiyi Zhang, Yongbo Yang et al.
Reproducing Transparent and Scrutable Recommendations: Exploring Open-Weight Models via Natural-Language User Profiles
Noah Mamié, Laurin van den Bergh
Dense Feature Representation over Sequence Modeling: A Solution to the KDD Cup 2026 UniRec Challenge
Yi Zhang, Weiliang Ji
Self-Evolving Search Index
Sangam Lee, Wonjae Lee, Sunghwan Kim et al.