I have some questions about my new idea.
First of all I'm doing this to improve my programming skills. I'm a new computer eng. student.
I'm planning to write a program for a web based game called ogame. Most of you probably heard about the game an if you didn't here is the address ogame.org .
I want to write a program that lists the Debris Field that appear in the galaxy menu. the program is going to list the coordinates and the amount of crystal and metal.
So the program gets values from the browser game, clicks something to go to the next page, and gets value from the next page... In the end it lists the values...
What programing language do i need to use?
How do I get values from the browser?
How do I make the program click on something on the browser?
How do I list the output?
Thanks for your help...
PS: This is my kind of my first topic. Sorry if I did anything wrong... :huh:
A program that can get information from a web browser???
Started by Turambar, Feb 09 2011 03:13 PM
11 replies to this topic
#1
Posted 09 February 2011 - 03:13 PM
|
|
|
#2
Posted 10 February 2011 - 01:48 AM
Hi Turambar, welcome aboard to CodeCall! Just curious, are you somehow related with Turin? There was a Turin Turambar character in Silmarillion.
For this project, you need to be able to manipulate a web browser. For precise manipulation, usually you have to embed the web browser into your own application.
I don't know your OS preference, so I assume you will be using the project under Windows.
You can use Delphi, VB, VB.NET, C++, and C#. For these language, the most popular web browser to embed is Internet Explorer. In Delphi, there is a ready to use control class named TWebBrowser which is a wrapper of IE.
By reading the html source of the page. You can analyze raw html source (parsing the text, and maybe using regex), or you can use the help of the web browser which usually already parsed them into something in the format of DOM.
By instructing the embedded web browser to click on something.
On this subject, perhaps you will find these topics interesting:
By reading the values and add some creativity? :)
For this project, you need to be able to manipulate a web browser. For precise manipulation, usually you have to embed the web browser into your own application.
I don't know your OS preference, so I assume you will be using the project under Windows.
Quote
What programing language do i need to use?
Quote
How do I get values from the browser?
Quote
How do I make the program click on something on the browser?
On this subject, perhaps you will find these topics interesting:
- http://forum.codecal...-same-time.html
- http://forum.codecal...en-clicked.html
- http://forum.codecal...webbrowser.html
Quote
How do I list the output?
#3
Posted 10 February 2011 - 05:58 AM
Thanks you for you're reply LuthfiHakim.
I'm a big fan of J.R.R Tolkien and I have read both Silmarillion and The Children of Húrin.
Thats a lot of information for now. Finally I can start somewhere. I guess I'm going to have to learn Delphi or just try C++ since I'm very little familiar with it.
Any way thank you again. I probably will come up with some new questions later. :)
I'm a big fan of J.R.R Tolkien and I have read both Silmarillion and The Children of Húrin.
Thats a lot of information for now. Finally I can start somewhere. I guess I'm going to have to learn Delphi or just try C++ since I'm very little familiar with it.
Any way thank you again. I probably will come up with some new questions later. :)
#4
Posted 10 February 2011 - 12:59 PM
Ok I just used operas "inspect element" option on the game. I found what I was looking for. Now I need to learn how to make the program written in C++ language read it. So can anyone help me with that here is a pic of the html code i found:

Do you think I just have to learn more to do that or is it an easy thing to get the values from this html code? Or is the html code not enough to write a program that gets this values ?

Do you think I just have to learn more to do that or is it an easy thing to get the values from this html code? Or is the html code not enough to write a program that gets this values ?
#5
Posted 10 February 2011 - 10:25 PM
HTML are just simply nested elements, you can use a DOM parser to go through each element and get the value you are looking for.
I have found a good looking one here:
htmlcxx - html and css APIs for C++
You may need to use a function such as strtok or whathave you to extract the numbers from the text after you get the value.
I have found a good looking one here:
htmlcxx - html and css APIs for C++
You may need to use a function such as strtok or whathave you to extract the numbers from the text after you get the value.
Be sure to read the updated FAQ! || Health is achieved through the same 10,000 steps.
If a suggested code/method fails, informing us is less important than telling us why or what errors occurred.
If a suggested code/method fails, informing us is less important than telling us why or what errors occurred.
#6
Posted 11 February 2011 - 06:49 AM
That looks very good but I just could not use it with visual basic 6.0. I Have copied the folder under a folder called "include" tried the examples on the link and got 63 erors :S...
I also have visual studio 2008 if 6.0 is too old for this project.
Also is there a free compiler for C++. The visual basics are from the university I am at. 6.0 is too old and 2008 is too complicated.
I also have visual studio 2008 if 6.0 is too old for this project.
Also is there a free compiler for C++. The visual basics are from the university I am at. 6.0 is too old and 2008 is too complicated.
#7
Posted 11 February 2011 - 10:57 AM
I could have sworn you said C++ somewhere, my mistake although the concept remains the same for each language, I believe VB6 can use Microsoft's XMLDOM object as in the documentation here:
A Beginner's Guide to the XML DOM
I am sure there are many others too, remember that HTML can be parsed like XML.
If you are wishing for a free C++ compiler, there are a few available, two notable ones are the MinGW compiler and GCC, MinGW being a Windows port for much of GCC, although it may be more complex than VB.
A Beginner's Guide to the XML DOM
I am sure there are many others too, remember that HTML can be parsed like XML.
If you are wishing for a free C++ compiler, there are a few available, two notable ones are the MinGW compiler and GCC, MinGW being a Windows port for much of GCC, although it may be more complex than VB.
Be sure to read the updated FAQ! || Health is achieved through the same 10,000 steps.
If a suggested code/method fails, informing us is less important than telling us why or what errors occurred.
If a suggested code/method fails, informing us is less important than telling us why or what errors occurred.
#8
Posted 12 February 2011 - 08:17 AM
Quote
I could have sworn you said C++ somewhere, my mistake ..
Quote
6.0 is too old and 2008 is too complicated.
They are actually different beast. :) VB in 2008 is VB.NET which is only slightly similar with VB6.
Note that I recommended the use of embedded web browser since i believe your game site is having dynamic pages. Many of its pages is rendered by javascript. Therefore values in their raw html might be different with what actually shown. This is why it's important to read values from what have been parsed by the web browser instead of relying on the raw html. It's different in sites with "static" pages. In this case I believe the most efficient way is simply requesting raw html source for the pages using much simpler and lighter socket library of your choice.
#9
Posted 13 February 2011 - 03:00 AM
Thank you so much guys. Really no body would care this much. You are just grade.
Ok guys I'm really confused now. I decided to use C++ because I'm familiar with it. I didn't get the point why you are thinking that I'm not going to use it. Is it because of VB 6.0??
LuthfiHakim I really have not much time any more because my lessons have started. I won't have time to learn Delphi. If I'll have to learn it than I will have to delay this all.
So I'm going to use C++. As I mentioned I'm familiar with it. So I just need to know three things right know:
compiler - Can VB 6.0 compile my C++ code? If not which one do you recommend?
html - How do I get a library that can recognize HTML?
congruity - How do I use them together?
The first two questions are answered but I still don't know which compiler to use and if the htmlcxx can be used with it.
Ok guys I'm really confused now. I decided to use C++ because I'm familiar with it. I didn't get the point why you are thinking that I'm not going to use it. Is it because of VB 6.0??
LuthfiHakim I really have not much time any more because my lessons have started. I won't have time to learn Delphi. If I'll have to learn it than I will have to delay this all.
So I'm going to use C++. As I mentioned I'm familiar with it. So I just need to know three things right know:
compiler - Can VB 6.0 compile my C++ code? If not which one do you recommend?
html - How do I get a library that can recognize HTML?
congruity - How do I use them together?
The first two questions are answered but I still don't know which compiler to use and if the htmlcxx can be used with it.
#10
Posted 13 February 2011 - 03:09 AM
As you are choosing C++ you can look up at a few of my prior posts, I had linked a C++ compiler and an HTML parsing library, as for combining them it should be as simple as including the header, there should be documentation that will show you how.
Be sure to read the updated FAQ! || Health is achieved through the same 10,000 steps.
If a suggested code/method fails, informing us is less important than telling us why or what errors occurred.
If a suggested code/method fails, informing us is less important than telling us why or what errors occurred.
#11
Posted 14 February 2011 - 12:22 PM
#include <iostream>
using namespace std;
int main ()
{
int galaxy, startSolarS, endSolarS;//inputs
int planet; //output
double kristal, metal;//outputs
kristal = 0;// temporary
metal = 0;//temporary
planet = 1;//temporary
cout<<"Enter the number of galaxy: ";
cin>>galaxy;
cout<<"Enter the first solar system: ";
cin>> startSolarS;
cout<<"Enter the last solar system: ";
cin>> endSolarS;
cout<< "Scaning "<< startSolarS <<" --> "<< endSolarS <<" in galaxy "<< galaxy <<"\n\n";
cout<<" Metal Cyristal\n";
[COLOR="#ff0000"][B]/* Missing Part I
*!!-------------------!!
*Go to the "galaxy"
*!!-------------------!!*/[/B][/COLOR]
while(startSolarS <= endSolarS)
{
[COLOR="#ff0000"][B]/* Missing Part II
*!!-------------------!!
*Go to the "startSolarS"
*Scan The html code for Metal and Cyristal(look at post #4 in this thread)
*Use the "kristal" and "metal" vareables save the results
*Find the number of planet and put it in to "planet"
*!!-------------------!!*/[/B][/COLOR]
cout<<""<< galaxy <<" : "<< startSolarS <<" : "<< planet <<" ---> "<< metal <<" "<< kristal <<"\n";
startSolarS = startSolarS + 1;
}
return 0;
}
OK I have started to write the program. As you can see there are two missing parts.
And I have to add some new features so that the user can't make any mistakes...
I'll keep going but I need you're commands, tips and advices... This is going to be my first serious c++ program. I'm very exited about it :w00t:.
By the way I'm using VB 6 for now. I'll change it if necessary. And I keep trying to learn more about DOM parsers.
I have one question about it.
How will the DOM parser know that it has to use the Internet Explorer or any other explorer that is already lunched and prepared for the scanning process which will be done by the program I'm writing?:huh:
#12
Posted 14 February 2011 - 06:46 PM
You will need to download the page's source code in the program, and then parse through that. I believe the Windows API should provide enough to do that.
Be sure to read the updated FAQ! || Health is achieved through the same 10,000 steps.
If a suggested code/method fails, informing us is less important than telling us why or what errors occurred.
If a suggested code/method fails, informing us is less important than telling us why or what errors occurred.
1 user(s) are reading this topic
0 members, 1 guests, 0 anonymous users


Sign In
Create Account


Back to top









