Digital Edition

Frank's Java Code Stack #2 Raising Automated Events
Frank's Java Code Stack #2 Raising Automated Events

(October 18, 2002) - Many Java developers have asked me why is it not possible, or rather, most cumbersome to write a Java based Trojan. Since Java by itself is constrained by many useful glitches like byte code verification, type checking, and other painfully time-consuming formalities, Java programmers might shun JNI calls for raising platform specific events including key logging, screen capture, and so on. (Hey, give them a break!). Now forget about Trojans. How can you code Java software, which can act as a Computer Based Tutorial (CBT), wherein the functionality of the software is explained interactively through a set of dynamic events-with automated mouse moves, key presses, screen grabbing et all? Simple. The Robot class in the AWT package lets you append events to the platform dependent native Event queue. Let us create a small code, which outlines basic Event automation.


1.        /*---A Simple class which performs platform
2.        dependent automated tasks-----*/
3.        import java.awt.*;
4.        import java.awt.event.*;
5.        import java.awt.image.*;
6.        import*;
7.        import com.sun.image.codec.jpeg.*;
9.        public class RobotJen{
11.        /*---A simple method which grabs the screen upon
12.        getting the Width, Height and Screen Resolution
13.        ---*/
14.        public static void screenGrab
15.        (int x1, int y1, int x2, int y2)
16.        {
17.        try{
18.        System.out.println("Trying to grab screen...");
20.        /*---java.awt.Robot class helps you in
21.        Automating most of the platform dependent tasks
22.        ---*/
23.        Robot afj=new Robot();
24.        BufferedImage shot = afj.createScreenCapture
25.        (new Rectangle(x1, y1, x2,y2));
27.        /*--Write the captured screen pixel data
28.        as a JPEG File Stream
29.        ---*/
30.        OutputStream out = new BufferedOutputStream
31.        (new FileOutputStream("shot.jpg"));
32.        JPEGImageEncoder enc =
33.        JPEGCodec.createJPEGEncoder(out);
34.        enc.encode(shot);
35.        out.close();
37.        }catch(Exception e){
38.        System.out.println("Error grabbing Screen!");}
39.        }
41.        /*--- This method demonstrates how keys can be
42.        automatically pressed by adding Events in the
43.        platform's native queue
44.        ---*/
45.        public static void changeAppFocus()
46.        {
47.        try{
48.        Robot afj=new Robot();
49.        System.out.println
50.        ("Trying to change App focus...");
51.        afj.delay(1000);
53.        /*-- Be sure to release the key after
54.        pressing it since keytype is logically
55.        keyPress+keyRelease. Else your system will
56.        act strangely
57.        ---*/
58.        /*--Trying to change App focus
59.        OS Dependent!!
60.        ---*/
61.        afj.keyPress(KeyEvent.VK_ALT);
62.        afj.keyPress(KeyEvent.VK_TAB);
63.        afj.keyRelease(KeyEvent.VK_TAB);
64.        afj.keyRelease(KeyEvent.VK_ALT);
66.        }catch(Exception e){
67.        System.out.println("Error changing App focus!");}
68.        }
70.        /*--- This method demonstrates how mouse pointer
71.        motion can be controlled through code. Note that
72.        unlike other AWT Events, mouseMove will move the
73.        mouse instantly and does not raise an event
74.        ---*/
75.        public static void moveMouse()
76.        {
77.        try{
78.        Robot afj=new Robot();
79.        System.out.println
80.        ("Trying to generate Mouse Events...");
82.        /*---Move mouse pointer to a random
83.        location on the screen, click the right
84.        button and left button once
85.        ---*/
86.        afj.mouseMove(400,400);
87.        afj.mousePress(InputEvent.BUTTON3_MASK);
88.        afj.mouseRelease(InputEvent.BUTTON3_MASK);
89.        afj.delay(1000);
90.        afj.mousePress(InputEvent.BUTTON1_MASK);
91.        afj.mouseRelease(InputEvent.BUTTON1_MASK);
93.        }catch(Exception e){
94.        System.out.println("Error raising events!");}
95.        }
97.        public static void main(String ar[]){
98.        screenGrab(0,0,1024,768);
99.        changeAppFocus();
100.        moveMouse();
101.        }
102.        }

Note: The above code works only from JDK1.3 onwards. The code works fine with Linux 2.4.1 and all versions of Windows. However if you get AWTException while creating Robot Objects, check if you have privileges to write events to the native queue. This might happen if your X server does not support valid extensions like XTEST 2.2.

Assignment: Create a simple CBT, which uses mouse press and key press Events to illustrate the usability of a software.

In order to post a comment you need to be registered and logged in.

Register | Sign-in

Reader Feedback: Page 1 of 1

Yes you can use this to automate existing apllications. What you need to understand is that we are forcing in events in the Platform Native queue. So you can automate any application including non-java by raising Key and Mouse Events.
Hint: If you want to automatically press a button in an existing application, just write an external patch which knows the screen co-ordinate of the button, and at a specified time move the mouse over that button (X,Y) and and perform a left click. You cannot compare JUnit with this humble Robot class.;-)

I am curious if this Robot class can be used to "attach" to a running Java app and "control" it in such a way that I can write test scenarios using this class to automate existing applications written in Java. This would allow developers to create full blown automation testing that can then be ran from an ant nightly build script and report results, so that the next day engineers could see the automation test results and know what to fix if anything broke.

Would this be possible? Or does it have to be done inline with the actual application? What is the difference, other than GUI control, of the Robot class as opposed to using JUnit to test.

Subscribe to the World's Most Powerful Newsletters


Intel is an American multinational corporation and technology company headquartered in Santa Clara, ...
Blockchain has shifted from hype to reality across many industries including Financial Services, Sup...
Concerns about security, downtime and latency, budgets, and general unfamiliarity with cloud technol...
In very short order, the term "Blockchain" has lost an incredible amount of meaning. With too many j...
Data center, on-premise, public-cloud, private-cloud, multi-cloud, hybrid-cloud, IoT, AI, edge, SaaS...
Cloud Storage 2.0 has brought many innovations, including the availability of cloud storage services...
For enterprises to maintain business competitiveness in the digital economy, IT modernization is req...
Cloud-Native thinking and Serverless Computing are now the norm in financial services, manufacturing...
Most modern computer languages embed a lot of metadata in their application. We show how this goldmi...
Moving to Azure is the path to digital transformation, but not every journey is effective. Organizat...
Public clouds dominate IT conversations but the next phase of cloud evolutions are "multi" hybrid cl...
On-premise or off, you have powerful tools available to maximize the value of your infrastructure an...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with exp...
Every organization is facing their own Digital Transformation as they attempt to stay ahead of the c...
Data center, on-premise, public-cloud, private-cloud, multi-cloud, hybrid-cloud, IoT, AI, edge, SaaS...
DevOps has long focused on reinventing the SDLC (e.g. with CI/CD, ARA, pipeline automation etc.), wh...
Now is the time for a truly global DX event, to bring together the leading minds from the technology...
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web...
Atmosera delivers modern cloud services that maximize the advantages of cloud-based infrastructures....
In today's always-on world, customer expectations have changed. Competitive differentiation is deliv...