4 replies to this topic

[AtoZ]

Hello.

I'm trying to make a program that opens Google search results, and makes a list of all the URLs for that search topic.

The problem is, the HTML source code doesn't contain the links, just Javascript, any ideas?

[yamman13]

I dont think google lets crawlers use its search function

[Roger]

Are you asking about how to do this in C#?

I have done this a while back in PHP via Google's AJAX API: Google AJAX Search API - Google Code When we implemented it, the API had daily limits and only returned 8 results at a time. We also have the same implementation in Yahoo Search API, which has a much higher daily limit.

The result can look something like this: Datasheet & Application Note Database, PDF, Circuits, Datasheets | Datasheets.net : tlp160

[AtoZ]

I'll look into the API, I could always have C# interact with my own php webserver.

Does anyone know of a search engine, that keeps the links not hidden with Javascript?

[Roger]

How much traffic do you anticipate? If you don't expect too many queries, you can just scrap Google Search results... this is, of course, not recommended for legit business sites.