Back to Results
First PageMeta Content
Computing / Information retrieval / Focused crawler / Invisible Web / Robots exclusion standard / Web search engine / Internet Archive / Distributed web crawling / Web harvesting / Information science / World Wide Web / Web crawlers


Design and Implementation of a High-Performance Distributed Web Crawler Vladislav Shkapenyuk
Add to Reading List

Document Date: 2001-08-13 18:57:45

Open Document

File Size: 322,01 KB

Share Result on Facebook

Company

Sun Microsystems / Google / Intel Corporation / AltaVista / Cisco / Compaq / /

/

Facility

Polytechnic University / /

IndustryTerm

request services / query processing / search index / particular server / web search technology / hidden web / optimized crawling systems / Web Crawler Vladislav Shkapenyuk Torsten Suel CIS Department Polytechnic University Brooklyn / particular web server / main campus router / Internet Archive crawler / system management / web servers / search tools / separate web / search engines / web search engines / large search engine / distributed web crawler / Internet Archive / increased search engine size / web graph / web server / larger search engine / software architecture / search engine / web crawler / web crawlers / unrelated applications / /

OperatingSystem

GNU / /

Organization

Polytechnic University / National Science Foundation / High-Performance Distributed Web Crawler Vladislav Shkapenyuk Torsten Suel Department of Computer / New York State Center for Advanced Technology / Stanford / /

Person

Vladislav Shkapenyuk Torsten Suel / Downloader Downloader Downloader / /

/

Position

manager / first author / manager process / Crawl Manager The crawl manager / second crawl manager / manager for compression and storage / storage manager / manager in batches / designer / single manager / crawl manager / administrator / /

ProgrammingLanguage

Python / HTML / C++ / /

ProvinceOrState

New York / /

Technology

main campus router / search engine / machine learning / HTML / operating systems / DNS / Java / web search technology / HTTP / web server / /

URL

http /

SocialTag