Jump to content

A program that can get information from a web browser???

- - - - -

  • Please log in to reply
11 replies to this topic

#1
Turambar

Turambar

    Newbie

  • Members
  • PipPip
  • 10 posts
I have some questions about my new idea.

First of all I'm doing this to improve my programming skills. I'm a new computer eng. student.

I'm planning to write a program for a web based game called ogame. Most of you probably heard about the game an if you didn't here is the address ogame.org .

I want to write a program that lists the Debris Field that appear in the galaxy menu. the program is going to list the coordinates and the amount of crystal and metal.
So the program gets values from the browser game, clicks something to go to the next page, and gets value from the next page... In the end it lists the values...

What programing language do i need to use?
How do I get values from the browser?
How do I make the program click on something on the browser?
How do I list the output?

Thanks for your help...

PS: This is my kind of my first topic. Sorry if I did anything wrong... :huh:

#2
LuthfiHakim

LuthfiHakim

    Programming God

  • Members
  • PipPipPipPipPipPipPip
  • 763 posts
Hi Turambar, welcome aboard to CodeCall! Just curious, are you somehow related with Turin? There was a Turin Turambar character in Silmarillion.

For this project, you need to be able to manipulate a web browser. For precise manipulation, usually you have to embed the web browser into your own application.

I don't know your OS preference, so I assume you will be using the project under Windows.

Quote

What programing language do i need to use?
You can use Delphi, VB, VB.NET, C++, and C#. For these language, the most popular web browser to embed is Internet Explorer. In Delphi, there is a ready to use control class named TWebBrowser which is a wrapper of IE.

Quote

How do I get values from the browser?
By reading the html source of the page. You can analyze raw html source (parsing the text, and maybe using regex), or you can use the help of the web browser which usually already parsed them into something in the format of DOM.

Quote

How do I make the program click on something on the browser?
By instructing the embedded web browser to click on something.

On this subject, perhaps you will find these topics interesting:

Quote

How do I list the output?
By reading the values and add some creativity? :)

#3
Turambar

Turambar

    Newbie

  • Members
  • PipPip
  • 10 posts
Thanks you for you're reply LuthfiHakim.

I'm a big fan of J.R.R Tolkien and I have read both Silmarillion and The Children of Húrin.

Thats a lot of information for now. Finally I can start somewhere. I guess I'm going to have to learn Delphi or just try C++ since I'm very little familiar with it.

Any way thank you again. I probably will come up with some new questions later. :)

#4
Turambar

Turambar

    Newbie

  • Members
  • PipPip
  • 10 posts
Ok I just used operas "inspect element" option on the game. I found what I was looking for. Now I need to learn how to make the program written in C++ language read it. So can anyone help me with that here is a pic of the html code i found:

Posted Image

Do you think I just have to learn more to do that or is it an easy thing to get the values from this html code? Or is the html code not enough to write a program that gets this values ?

#5
Alexander

Alexander

    It's Science!

  • Moderators
  • 4,118 posts
  • Location:Vancouver, Eh! Cleverness: 200
HTML are just simply nested elements, you can use a DOM parser to go through each element and get the value you are looking for.

I have found a good looking one here:
htmlcxx - html and css APIs for C++

You may need to use a function such as strtok or whathave you to extract the numbers from the text after you get the value.
Be sure to read the updated FAQ! || Health is achieved through the same 10,000 steps.
If a suggested code/method fails, informing us is less important than telling us why or what errors occurred.

#6
Turambar

Turambar

    Newbie

  • Members
  • PipPip
  • 10 posts
That looks very good but I just could not use it with visual basic 6.0. I Have copied the folder under a folder called "include" tried the examples on the link and got 63 erors :S...

I also have visual studio 2008 if 6.0 is too old for this project.

Also is there a free compiler for C++. The visual basics are from the university I am at. 6.0 is too old and 2008 is too complicated.

#7
Alexander

Alexander

    It's Science!

  • Moderators
  • 4,118 posts
  • Location:Vancouver, Eh! Cleverness: 200
I could have sworn you said C++ somewhere, my mistake although the concept remains the same for each language, I believe VB6 can use Microsoft's XMLDOM object as in the documentation here:
A Beginner's Guide to the XML DOM

I am sure there are many others too, remember that HTML can be parsed like XML.

If you are wishing for a free C++ compiler, there are a few available, two notable ones are the MinGW compiler and GCC, MinGW being a Windows port for much of GCC, although it may be more complex than VB.
Be sure to read the updated FAQ! || Health is achieved through the same 10,000 steps.
If a suggested code/method fails, informing us is less important than telling us why or what errors occurred.

#8
LuthfiHakim

LuthfiHakim

    Programming God

  • Members
  • PipPipPipPipPipPipPip
  • 763 posts

Quote

I could have sworn you said C++ somewhere, my mistake ..
He did, lol! I think he haven't decided which language he wants to use.

Quote

6.0 is too old and 2008 is too complicated.

They are actually different beast. :) VB in 2008 is VB.NET which is only slightly similar with VB6.

Note that I recommended the use of embedded web browser since i believe your game site is having dynamic pages. Many of its pages is rendered by javascript. Therefore values in their raw html might be different with what actually shown. This is why it's important to read values from what have been parsed by the web browser instead of relying on the raw html. It's different in sites with "static" pages. In this case I believe the most efficient way is simply requesting raw html source for the pages using much simpler and lighter socket library of your choice.

#9
Turambar

Turambar

    Newbie

  • Members
  • PipPip
  • 10 posts
Thank you so much guys. Really no body would care this much. You are just grade.

Ok guys I'm really confused now. I decided to use C++ because I'm familiar with it. I didn't get the point why you are thinking that I'm not going to use it. Is it because of VB 6.0??

LuthfiHakim I really have not much time any more because my lessons have started. I won't have time to learn Delphi. If I'll have to learn it than I will have to delay this all.

So I'm going to use C++. As I mentioned I'm familiar with it. So I just need to know three things right know:

compiler - Can VB 6.0 compile my C++ code? If not which one do you recommend?
html - How do I get a library that can recognize HTML?
congruity - How do I use them together?


The first two questions are answered but I still don't know which compiler to use and if the htmlcxx can be used with it.

#10
Alexander

Alexander

    It's Science!

  • Moderators
  • 4,118 posts
  • Location:Vancouver, Eh! Cleverness: 200
As you are choosing C++ you can look up at a few of my prior posts, I had linked a C++ compiler and an HTML parsing library, as for combining them it should be as simple as including the header, there should be documentation that will show you how.
Be sure to read the updated FAQ! || Health is achieved through the same 10,000 steps.
If a suggested code/method fails, informing us is less important than telling us why or what errors occurred.

#11
Turambar

Turambar

    Newbie

  • Members
  • PipPip
  • 10 posts
#include <iostream>


using namespace std;


int main ()

{

	int galaxy, startSolarS, endSolarS;//inputs

	int planet; //output

	double kristal, metal;//outputs


	kristal = 0;// temporary

	metal = 0;//temporary

	planet = 1;//temporary


	cout<<"Enter the number of galaxy: ";

	cin>>galaxy;


	cout<<"Enter the first solar system: ";

	cin>> startSolarS;


	cout<<"Enter the last solar system: ";

	cin>> endSolarS;


	cout<< "Scaning "<< startSolarS <<" --> "<< endSolarS <<" in galaxy "<< galaxy <<"\n\n";

	cout<<"                  Metal       Cyristal\n";

	

	[COLOR="#ff0000"][B]/*     Missing Part I

	 *!!-------------------!!

	 *Go to the "galaxy"

	 *!!-------------------!!*/[/B][/COLOR]


	while(startSolarS <= endSolarS)

	{

		[COLOR="#ff0000"][B]/*    Missing Part II

		 *!!-------------------!!

		 *Go to the "startSolarS"

		 *Scan The html code for Metal and Cyristal(look at post #4 in this thread)

	         *Use the "kristal" and "metal" vareables save the results

		 *Find the number of planet and put it in to "planet"

		 *!!-------------------!!*/[/B][/COLOR]


		cout<<""<< galaxy <<" : "<< startSolarS <<" : "<< planet <<"  --->   "<< metal <<"              "<< kristal <<"\n";


		startSolarS = startSolarS + 1;

	}


	return 0;

}


OK I have started to write the program. As you can see there are two missing parts.
And I have to add some new features so that the user can't make any mistakes...

I'll keep going but I need you're commands, tips and advices... This is going to be my first serious c++ program. I'm very exited about it :w00t:.

By the way I'm using VB 6 for now. I'll change it if necessary. And I keep trying to learn more about DOM parsers.

I have one question about it.
How will the DOM parser know that it has to use the Internet Explorer or any other explorer that is already lunched and prepared for the scanning process which will be done by the program I'm writing?:huh:

#12
Alexander

Alexander

    It's Science!

  • Moderators
  • 4,118 posts
  • Location:Vancouver, Eh! Cleverness: 200
You will need to download the page's source code in the program, and then parse through that. I believe the Windows API should provide enough to do that.
Be sure to read the updated FAQ! || Health is achieved through the same 10,000 steps.
If a suggested code/method fails, informing us is less important than telling us why or what errors occurred.




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users