5 replies to this topic

[kemosabe]

Hi,

I am trying to develop something for my office, a personal project.

A common task for my position is to gather information from user client profiles. I have to click on about 10 web links per client profile to gather this information.

Is there some kind of program I can learn to build that will search all these predetermined links and display the data at one time so I dont have to waste time loading each one manually?

I have been researching what program can do this and the only thing ive got so far is a web crawler but theres probably soemthing more efficent at this right?

Any input is much appreciated!

Thanks.

[debtboy]

I would use wget in a shell script,
but the curl people probably have a different idea.

[kemosabe]

Hi,

Thanks for the quick response. Will wget work if the site is encrypted? Also, how difficult is to use as I have no background in computers at all.

Thanks.

[Orjan]

or you could create a php script or a c/c++ program that uses the curl library to retrieve this information, curl is able to handle login etc...

[debtboy]

As I figured..., curl is soo muchhh betterrr :rolleyes:

a Lynx dump will also work, but wget requires the least amount of code.
de-crypting is on you no matter what you use.

[WingedPanther]

Another important question: is it always the same 10 sites, or no?