Jump to content




Recent Status Updates

  • Photo
      30 Sep
    rhossis

    laptop hard disk seated beneath motherboard but with no access panel. 7 hours to replace :(

    Show comments (3)
  • Photo
      19 Sep
    Chall

    I love it when you go to write a help thread, then while writing, you reach an enlightenment, and figure it out yourself.

    Show comments (3)
View All Updates

Developed by Kemal Taskin
Photo
- - - - -

Robot class


  • Please log in to reply
12 replies to this topic

#1 Sinipull

Sinipull

    CC Addict

  • Senior Member
  • PipPipPipPipPip
  • 383 posts

Posted 03 March 2010 - 12:00 PM

This tutorial is about java Robot class. (java.awt.Robot)

I will quote Sun here:

"This class is used to generate native system input events for the purposes of test automation, self-running demos, and other applications where control of the mouse and keyboard is needed. The

primary purpose of Robot is to facilitate automated testing of Java platform implementations."


So, basically this gives the low-level input control over your mouse and keyboard to your java program. Note, that you are still able to control your mouse and keyboard when the robot is running (in case of correct coding).

IMPORTANT: The Robot class gives your program as much power, as you have over your computer. It can do EVERYTHING you can do with your mouse and keyboard. The program has even ability to write it's own code, while running and these kind of programs should be written very carefully. (Not because Skynet is probable to evolve, but it can wreck something on your computer).

Ok, let's see how to start using the Robot class.
First we need to import the Robot class:

import java.awt.Robot;

Then we will need to make a Robot object:

Robot r = new Robot();

Now we have access to our Robot methods. (The complete list of methods and javadoc can be found here)

MOUSE:

r.mouseMove(x, y);
which moves your cursor instantly to the specified location.
Note, that x = 0 is located in the left of your screen and y = 0 is located at the top of your screen.
The largest values, where your cursor can move, is your screen resolution - 1. On my 1680x1050, it is x = 1679 and y = 1049.
If you tell the cursor to move out of the screen area, it will just slam against the respective wall.

r.mousePress(buttons);
r.mouseRelease(buttons);
Presses and instantly releases your specified button.
IMPORTANT: Never forget to release the button after pressing. (Can mess up something on your computer as a result, if lot of that is done)

the mousePress() and mouseRelease() takes int as an argument, which should be selected from amongst the constants in the InputEvent(java.awt.event.InputEvent) class:

those are the constants:
InputEvent.BUTTON1_MASK  //left button
InputEvent.BUTTON2_MASK  //middle button
InputEvent.BUTTON3_MASK  //right button

(I'm not sure, how they depend on the mouse, platform, configuration, etc, so you should check them out yourself.)


r.mouseWheel(wheelAmt);
This scrolls your mousewheel, if you have one. The wheelAmt argument is an int and should be positive or negative int value, depending how much or which way do you want to scroll the mouse.

KEYBOARD:

r.keyPress(key);
r.keyRelease(key);

Presses the specified key and releases it after.
IMPORTANT: Never forget to release the key after pressing. I actually had to restart my computer to fix the damage, that was caused by this mistake. Restarting just Eclipse didn't help.

The "key" argument is again int and should be selected as a constant from KeyEvent(java.awt.event.KeyEvent) class. Now, there are too many constants to copy them all here, so i'll just show you how they look:

r.keyPress(KeyEvent.VK_A); // press A-key
r.keyRelease(KeyEvent.VK_A); // release A-key

If you use proper IDE, you are able to find them all easily.


OTHER:

Robot class can also make screenshots or get the pixel color on screen:

r.createScreenCapture(new Rectangle(0, 0, 100, 100));
Creating screenshot requires Rectangle (java.awt.Rectangle) object as an argument, as this is the specified area, from where the screenshot is taken.
The result is BufferedImage (java.awt.image.BufferedImage) which can then be handeled properly (analyzed, saved or whatever needs to be done).

r.getPixelColor(x, y);
Gets you the pixel color as Color(java.awt.Color) object.

r.delay(ms);
Sleeps the specified amount of milliseconds. Is very useful when writing a Robot program. Is pretty much interchanceable with Thread.sleep(ms).

USEFUL STUFF TO KNOW WHEN WRITING A ROBOT

Get current screen resolution:
Toolkit.getDefaultToolkit().getScreenSize();
returns a Dimenson(java.awt.Dimension)

Get current mouse position
MouseInfo.getPointerInfo().getLocation();
returns a Point(java.awt.Point)


---

Again, beware of writing programs that take over your mouse and keyboard without sleeping. The only possible way to stop it, is the restart of the computer, from the physical button.
It's easy to stop the program from task manager, if only your mouse is stuck somewhere, but if you keyboard is being used too, then you are pretty much forced to plug off your computer.
Therefore, loops are dangerous, if not used properly!

Posted Image


The Robot class may not work on some platforms, as Sun has written:

Note that some platforms require special privileges or extensions to access low-level input control. If the current platform configuration does not allow input control, an AWTException will be thrown when trying to construct Robot objects.


Have fun ;)
  • 0

#2 Cander

Cander

    CC Resident

  • Advanced Member
  • PipPipPipPip
  • 55 posts

Posted 04 March 2010 - 01:05 PM

Good tutorial! rep+. You could have linked this page though, the full documentation of the java.awt.Robot class which tell all existing methods etc.

http://java.sun.com/.../awt/Robot.html
  • 0

#3 Sinipull

Sinipull

    CC Addict

  • Senior Member
  • PipPipPipPipPip
  • 383 posts

Posted 04 March 2010 - 01:26 PM

Thanks. :)
But i did link to that page, check right above the Mouse chapter.
  • 0

#4 Cander

Cander

    CC Resident

  • Advanced Member
  • PipPipPipPip
  • 55 posts

Posted 05 March 2010 - 05:21 AM

Oh, sorry! missed that
  • 0

#5 Prog4rammer

Prog4rammer

    CC Newcomer

  • Just Joined
  • PipPip
  • 14 posts

Posted 05 March 2010 - 07:11 AM

Good and useful tutorial rep+ Thanks You :) ...
  • 0

#6 unscripted_1

unscripted_1

    CC Newcomer

  • Just Joined
  • PipPip
  • 11 posts

Posted 21 April 2010 - 09:42 PM

so would you use r.getPixelColor() to try to get Robot to react to things happening on the screen?

Like could you create a boolean function, such as hasSomethingHappened() which keeps checking the pixel color, & just get the program to keep sleeping till it returns true?

Is there any more efficient way to get Robot to interact with screen events?
  • 0

#7 Sinipull

Sinipull

    CC Addict

  • Senior Member
  • PipPipPipPipPip
  • 383 posts

Posted 22 April 2010 - 12:10 AM

I wouldn't use r.getPixelColor(). I would use r.createScreenCapture() and get the Colors from there, i'm pretty sure, it is more efficient way for large data access than getPixelColor().

You have to create your own artificial intelligence for the machine, no 'easy way' here. It's very hard to create something that could actually understand what's on the screen. Neural Networks would be one possibility.
  • 0

#8 wim DC

wim DC

    Roar

  • Expert Member
  • PipPipPipPipPipPipPipPip
  • 2,574 posts
  • Programming Language:Java, JavaScript, PL/SQL
  • Learning:Java, PHP

Posted 24 April 2010 - 04:08 AM

Again, beware of writing programs that take over your mouse and keyboard without sleeping. The only possible way to stop it, is the restart of the computer, from the physical button.
It's easy to stop the program from task manager, if only your mouse is stuck somewhere, but if you keyboard is being used too, then you are pretty much forced to plug off your computer.
Therefore, loops are dangerous, if not used properly!


This has happened to me several times :( i let the robot control the mouse and click with it. But i got stuck in a loop several times and because it was constantly clicking i couldn't kill the process with the keyboard because task manager would lose focus. So now i always set up a timer which stops the program after a set time (mostly about 10-20 seconds) ;)
  • 0

#9 GMVResources

GMVResources

    CC Resident

  • Just Joined
  • PipPipPipPip
  • 71 posts

Posted 13 June 2010 - 06:10 PM

Nice job Sinipull!
  • 0

#10 linda

linda

    CC Lurker

  • Just Joined
  • Pip
  • 1 posts

Posted 17 December 2010 - 05:21 AM

@Sinipull, I am trying to build up an automation program to do some specific jobs.
Considering the capabilities of robot class, it works fine but i am stuck at two things:

1. Is there anyway to know if a process is finished in another application before pressing a key for example.
Like, if i run e.g a windows media player, how would i know it is loaded in OS and ready to be used, before
i click further using robot.xxx

2. Using robot, we can read the mouse and write back both mouse/keyboard actions. But is there anyway to
read the keyboard as well?
Like, if i automate documets and want my robot class to wait for some specific input from user, for (say) a
particular field, and rest it has to fill automatically, how can it read the key input?


Can someone please answer this?
  • 0

#11 Cander

Cander

    CC Resident

  • Advanced Member
  • PipPipPipPip
  • 55 posts

Posted 20 December 2010 - 02:34 AM

@Sinipull, I am trying to build up an automation program to do some specific jobs.
Considering the capabilities of robot class, it works fine but i am stuck at two things:

1. Is there anyway to know if a process is finished in another application before pressing a key for example.
Like, if i run e.g a windows media player, how would i know it is loaded in OS and ready to be used, before
i click further using robot.xxx

2. Using robot, we can read the mouse and write back both mouse/keyboard actions. But is there anyway to
read the keyboard as well?
Like, if i automate documets and want my robot class to wait for some specific input from user, for (say) a
particular field, and rest it has to fill automatically, how can it read the key input?


Can someone please answer this?


1. Nope. You can use Thread.sleep() while a started program by robot is starting up to wait for it to load, but there is no is it guarantee the started program is ready to use after sleeped time. Like if your computer has a slow day.
  • 0

#12 Xdawn90

Xdawn90

    CC Resident

  • Advanced Member
  • PipPipPipPip
  • 53 posts
  • Learning:C, Java, C++, PHP, JavaScript, Visual Basic .NET

Posted 20 December 2010 - 08:07 AM

This is something new for me. Nice tutorial.
  • 0