I'm looking for an advanced .NET professional that can do real-time webpage scraping for display purposes only. For example, I'm currently use frame techniques to display another website's content on my various domain names - for example:
Please visit this URL: [url removed, login to view] (SEO Friendly keywords based Domain Name)
Now, if you view the page source for this website URL, you notice that it looks like this:
And that's it - all you'll see here is an affiliate link that automatically redirects the user to the actual website that in return is viewed through this frameset. Now, this is good for displaying a website on your own domain name, but it's not too good for SEO work. And, that's where webpage scraping comes in. Because here's what I'm looking for:
1. I need for you to write .NET code that will take an affiliate hyperlink (as shwon above) and go to the end location as needed - then your code should scrape that webpage from beginning to end and then display that original HTML code under any of my various domain names. That way, instead of seeing a "" entry, I would see all of the original HTML code as part of my domain name.
2. Before the original HTML code is displayed on a foreign domain name, I want the code to execute a few "search and replace" commands so as to automatically remove certain lines, such as the meta tags lines. And, once the meta tags lines are removed, then I want you to write code that will create new meta tag lines of code. Although I want to have the ability to tell your program what keywords and what meta tags to automatically insert into the HTML code.
3. Then I want your code to search for all relative hyperlinks and make them absolute in return. For example, "../images/[url removed, login to view]" should be automatically updated to "[url removed, login to view]" - although hyperlinks should still function with the unique domain name.
4. After that is done, then I want to be able to do any additional searches and replaces based on what domain name is being used.
5. Then I want your code to add Google Analytics code at the end of the scrapped webpage.
6. Then I want you to implement URL Rewriting so that the search engines don't see the "[url removed, login to view]" page and instead they see a "[url removed, login to view]" page.
By doing these things, I'll be able to setup my own domain name websites that automatically displays a redirected webpage and yet do all the necessary SEO work based on a given domain name. This way Google and all the other search engines should be able to crawl all these new domain names and see the "redirected webpage" as a unique website that has been SEO Optimized.
This should not be too hard to do for the right developer. I did this exact same type of work years ago using classic ASP. In addition, I'm open to any suggestions that you may have about any of this - as long as your solution is Search Engine Friendly.