Auctions Crawler

Completed Posted Apr 13, 2008 Paid on delivery
Completed Paid on delivery

This project will develop a crawler to mine old auction data then store it on a website that runs a database. For the server the coder will need to work with MySQL and PHP.

For the client I am open to any programming language but if you have C# experience I can make available the source code to crawlers that are working now. This would help create the new routines because much of the code can be reused.

SEE THE FULL SPECS FOR THIS PROJECT AT BOTTOM UNDER "OTHER REQUIREMENTS"

## Deliverables

This project will create a crawler to mine auction data from coin dealers. The completed auctions are stored at these two links:

[login to view URL]

[login to view URL]

The crawler will go to an auction of my choice then parse and post the data to my research website. The database is located at [login to view URL] (username rasiel, password eric2)

Before submitting your bid I strongly encourage you to visit the website so you can get an understanding of how the auctions are stored. When this project is complete the data should look very similar to the sections already present and the new data should integrate with the rest of the database. Much of the code you would need to use to make the new crawler has already been created. I can provide the source code for my other crawlers to help you create the new client. The previous clients I'm using now were written in C#.

The crawler will work sort of like this:

1) Ask for beginning parameters like, for example, Dealer Name = NAC, Auction = 40, Start URL = [login to view URL]

2) Script then checks to see if there is a section already titled "NAC" and if not create it. It then checks to see if there is an auction section under NAC titled "Auction 40" and if not create it.

3) Start the crawl record by record using the link I entered ([login to view URL])

4) On each record the crawler looks for keywords according to the list I will provide. If it has a match then it stores that record under the category bound to that keyword, if not then it assigns it to the Uncertain category where I will later on move it manually.

5) For each record scraped the crawler will get the info to fill the following fields: Auction title & lot number (this is also the record title), the name of the dealer, the price realized in dollars (after conversion), the text description of the lot and the largest available image.

6) The crawler takes the price realized and converts it to dollars *on the date of the auction* NOT today's date. I have the source code for a routine that does this in C# which the coder is welcome to use or, of course, you can make your own. Two sites that store old conversion rates are [login to view URL] and [login to view URL]

7) The client then posts the completed record to the website and then crawls the next record

I have had very good experience so far with the coders on RAC and look forward to working closely with you on this project. I am available almost 24/7 by MSN chat and email.

C# Programming Database Administration Engineering Microsoft MySQL PHP Project Management Software Architecture Software Testing SQL Windows Desktop

Project ID: #3867174

About the project

4 proposals Remote project Active Apr 15, 2008

Awarded to:

sgarside

See private message.

$126.65 USD in 14 days
(34 Reviews)
4.8

4 freelancers are bidding on average $328 for this job

bocodevw

See private message.

$391 USD in 14 days
(75 Reviews)
6.0
xolutionsvw

See private message.

$425 USD in 14 days
(10 Reviews)
4.8
vw2177283vw

See private message.

$369.75 USD in 14 days
(7 Reviews)
3.4