URL TRACKER
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
seminar class
Active In SP
**

Posts: 5,361
Joined: Feb 2011
#1
18-02-2011, 02:06 PM



.docx   urldocument.docx (Size: 484.48 KB / Downloads: 55)
INTRODUCTION
The “URL TRACKER” is a multithreaded windows application that down-loads and stores Web pages Uniform Resource Identifier (URI’s), for a Web search engine. Roughly, a crawler starts off by placing an initial set of URLs, so, in a queue, where all URLs to be retrieved are kept and prioritized. From this queue, the crawler gets a URL (in some order), downloads the page, extracts any URLs in the downloaded page, and puts the new URLs in the queue. This process is repeated until the crawler decides to stop. Collected pages are later used for other applications, such as a Web search engine or a Web cache.
As the size of the Web grows, it becomes more difficult to retrieve the whole or a significant portion of the Web using a single process. Therefore, many search engines often run multiple processes in parallel to perform the above task, so that download rate is maximized.
PROJECT OVERVIEW
“URL Tracker” aims to develop a user interface which brings the information about a particular given website. This is a multithreaded windows application that downloads and stores Uniform Resource Identifiers of typical website. This application has got its use as a backend processing component for a search engine. The results gathered by the Website Fetcher will be given to the indexer which indexes page data so that the search query gives the results faster. The proposed project and implimentation once implemented can connect to the websites and download data which once indexed can be given to the search engine.
PROJECT DESCRIPTION
The Url Tracker is a multithreaded windows application that downloads and stores Web pages Uniform Resource Identifier (URI’s), for a Web search engine. Roughly, a crawler starts off by placing an initial set of URLs, so, in a queue, where all URLs to be retrieved are kept and prioritized. From this queue, the crawler gets a URL (in some order), downloads the page, extracts any URLs in the downloaded page, and puts the new URLs in the queue. This process is repeated until the crawler decides to stop. Collected pages are later used for other applications, such as a Web search engine or a Web cache.
As the size of the Web grows, it becomes more difficult to retrieve the whole or a significant portion of the Web using a single process. Therefore, many search engines often run multiple processes in parallel to perform the above task, so that download rate is maximized. We refer to this type of fetcher as a parallel crawler. This type of applications is often used in search engines where there is a need of collecting all the URL’s based on a query and indexing them on priority.
MODULES CRAWLER VIEW
This is a primary module to get initiate of tracking the URIS from URL. Firstly an Uri is given as input, then our crawler view takes the Uri and finds the URIS, URIS founded are placed into threads, if the threads memory is full, then they remain URIs are queued. When URIS is fetched with data, then the completed thread is killed and queued URI’s are placed into the threads.
There are two types of Functionalities in the module
1. Threads view
2. Crawler view.
Threads View and Requests View
First our system establish connection with the system after that user gives one URL (Uniform resource Locator) give one URL as input .It start searching or fetching the information of that URL by starting threads process. In This process 10 threads will be running continuously to get all the URI’S information and stores them in a queue.
At the time of down loading each URI it puts in threads view after completion of down load process it jus transfers the completed URI into the request phase .so while fetching any URI corresponding to URL, any difficulties or any errors occurs it just listed in error view phase.
CONFIGURATOR MODULE
 Mime Types
In this will set all kinds of data we need to extract from the particular URI like weather we need storing data, Boolean data and images information or not
 Output Settings
In this we mention the output folder name where we need to store the content about the website fetched.
 Advanced Settings
These are the settings made by the user in order to restrict some kind of website like with domain name as .NET,.AC.IN like this.
MULTITHREADED DOWNLOADER

Here the multi threaded downloader is responsible for starting threads and obtaining the information about the website being fetched. So the multi threaded downloader starts threads and it pushes all URI’s one queue. Each and every thread is starts with one Uri in the queue. After completion it just jumps to the next URI’s in the queue. In this module one folder creates in the user desired path and the files created with the URI names having the static information.
SYSTEM CONFIGURATION
SOFTWARE SPECIFICATIONS

 Microsoft .net framework
 Microsoft C# .net language
 Microsoft Windows 2000
 Microsoft Visual Studio 2005
HARDWARE SPECIFICATIONS
 PROCESSOR : P4 or higher
 RAM : 512 MB
 HARD DISK : 1GB
Reply
fruit
Active In SP
**

Posts: 18
Joined: May 2011
#2
19-05-2011, 12:42 PM

can you send the coding part to my email - fruit_ooi10@yahoo.com. thank~
Reply

Important Note..!

If you are not satisfied with above reply ,..Please

ASK HERE

So that we will collect data for you and will made reply to the request....OR try below "QUICK REPLY" box to add a reply to this page

Quick Reply
Message
Type your reply to this message here.


Image Verification
Please enter the text contained within the image into the text box below it. This process is used to prevent automated spam bots.
Image Verification
(case insensitive)

Possibly Related Threads...
Thread Author Replies Views Last Post
  MEDI TRACKER FULL REPORT study tips 0 320 26-06-2013, 12:19 PM
Last Post: study tips
  WEB PORTAL FOR BUG TRACKER PPT study tips 0 357 20-06-2013, 04:40 PM
Last Post: study tips
  Effort Tracker System Electrical Fan 9 6,112 11-03-2013, 09:30 AM
Last Post: study tips
  DEVELOPMENT OF EFFORT TRACKER SYSTEM ABSTRACT project girl 0 350 28-01-2013, 09:33 AM
Last Post: project girl
  Training Tracker report project girl 0 497 02-11-2012, 05:55 PM
Last Post: project girl
  Appraisal Tracker system seminar flower 0 480 29-09-2012, 03:15 PM
Last Post: seminar flower
  Design and development of a Skill and Activity Tracker System (SATS) seminar flower 0 647 14-09-2012, 04:21 PM
Last Post: seminar flower
  User friendly ,feature-rich, practical Appraisal Tracker seminar flower 0 458 14-09-2012, 01:22 PM
Last Post: seminar flower
  design and development of a skill tracker activity system Temiremi 3 2,444 17-07-2012, 09:51 PM
Last Post: OfeYrc
  Project Analysis for “RemindMe” Anniversary&Birthday Tracker seminar flower 0 561 19-06-2012, 01:59 PM
Last Post: seminar flower