1 reply to this topic

[Phishhook]

I've been given a project to help two of my professors with a research article and am trying to write some code to automate it. Basically, I need to find various info on 50 different business schools faculty. This includes things like their concentration, name, highest degree, gender, etc. Most of this info is widely available on the professors profile page via their respective schools faculty directory. There are at least 50 professors a school and at 50 schools that is 2500 people I need to collect data on. This would obviously take an extreme amount of time if i were to do it manually.

Can someone please provide me with some guidance on this. I know how to write macro's to grab data from websites, but I only know how to do one page at a time. I would really appreciate any help with this, you would absolutely be a life saver!

[ReignInChaos]

I would suggest using Perl to scrub each school's page. You may have to investigate how each school sets up their pages and some schools may even have a single page dedicated to all this information.