Jump to content


Check out our Community Blogs

Register and join over 40,000 other developers!


Recent Status Updates

View All Updates

Photo
- - - - -

Controlling your pc with your Android smartphone.

socket server android controller

  • Please log in to reply
3 replies to this topic

#1 farrell2k

farrell2k

    CC Addict

  • Advanced Member
  • PipPipPipPipPip
  • 169 posts

Posted 07 March 2015 - 08:36 PM

Java is an amazing platform for software development.    There is probably a library for just about anything you want to do.   The idea for this application started out with my desire to play old video games with my smartphone as a game pad, as I don't have one for the pc, but like most simple projects the excitement died down after a day or so, so I am just going to show the principle behind how it works.    You're going to need to make sure that both the server and client you create here are on the same local network.   i.e. wifi.   

 
The way it works is pretty simple.  You have a Server Socket running on the pc , performing actions based on requests from a client via Sockets.  These actions can be to press keys on the keyboard, move the mouse, press mouse buttons, etc.   It's all done through the java Robot class.   The client will send a String to the server representing an action such as "press the A key", then the server will call the keyPress() method of the Robot class.   
 
The server class is pretty straightforward.  I set up a server socket, clients connect, then set up an inputstream from the client and a reader to read Strings.
 
/*
 * License: Public Domain.
 */
package smartphone.controller.server;

import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.KeyEvent;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.concurrent.atomic.AtomicBoolean;

/**
 *
 * @author Tom Farrell The Server class runs in the background responding to
 * messages from clients. It utilizes the AWT Robot class to simulate key
 * presses in response to messages from the client.
 */
public class ControllerServer {

    private int port;
    private Robot robot;
    private ServerSocket serverSocket;
    private Socket socket;
    private AtomicBoolean running;

    public ControllerServer(int port) throws AWTException {
        this.port = port;
        robot = new Robot();
        running = new AtomicBoolean();
    }

    /*Start the server and begin processing input. 
     The principle here is pretty simple.  Open an input stream from a client to
     strings from it.   The strings inform the server what actions to take.  See
     the handleAction method for details.
     */
    public void startServer() {
        Thread serverThread = new Thread(new Runnable() {
            public void run() {
                System.out.println("running");

                running.set(true);

                System.out.println(running.get());

                try {
                    serverSocket = new ServerSocket(port);

                    while (running.get()) {
                        System.out.println("Waiting for client.");

                        //When this method returns, the client is connected.
                        socket = serverSocket.accept();

                        //Get an input stream from the client.
                        InputStream input = socket.getInputStream();

                        //Set up a reader to read strings from the client.
                        BufferedReader reader = new BufferedReader(new InputStreamReader(input));

                        System.out.println("Client connected.");

                        while (running.get()) {
                            //Grab the action from the client.  May be null.
                            String action = reader.readLine();

                            //Only do something if a valid action was sent.
                            if (action != null) {
                                String[] commands = action.split(" ");

                                handleAction(commands[0], commands[1]);
                            }
                        }
                    }
                } catch (IOException e) {
                    e.printStackTrace();
                } finally {
                    try {
                        serverSocket.close();

                        if (socket != null) {
                            socket.close();
                        }
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }

            }
        });

        serverThread.start();
    }

    /**
     * This method handles key actions from the client.
     *
     * @param key - The key on which to perform the action.
     * @param state - The state of the action, press or release.
     */
    private void handleAction(String key, String state) {
        System.out.println(key + " " + state);

        switch (key) {
            case "escape":
                if (state.equals("press")) {
                    robot.keyPress(KeyEvent.VK_ESCAPE);
                } else {
                    robot.keyRelease(KeyEvent.VK_ESCAPE);
                }
                break;
            case "enter":
                if (state.equals("press")) {
                    robot.keyPress(KeyEvent.VK_ENTER);
                } else {
                    robot.keyRelease(KeyEvent.VK_ENTER);
                }
                break;
        }
    }

    /*
     Stops the server.  The interesting thing about java is that there is really
     no good way to stop a thread, so if the ServerSocket is blocking while 
     waiting for a connection, if will not respond to join() or interrupt(), so
     a good way to stop is is to close the ServerSocket, which will throw an 
     exception and end it.
     */
    public void stopServer() {
        running.set(false);

        try {
            serverSocket.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

The Main class to instantiate and start the server.

/*
 * License: Public Domain.
 */
package smartphone.controller.server;

import java.awt.AWTException;
import java.net.SocketException;
import java.net.UnknownHostException;
import smartphone.controller.util.ServerUtil;

/**
 *
 * @author Tom Farrell
 */
public class Main {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        ControllerServer server = null;

        try {

            /*Get the local ip of this computer so the client knows where to connect.
             You need to set this in the client manually.*/

            try {
                System.out.println(ServerUtil.getIpAddress());
            } catch (SocketException | UnknownHostException ex) {
                ex.printStackTrace();
            }

            //Create and start the server.
            server = new ControllerServer(5000);
            server.startServer();

        } catch (AWTException ex) {
            ex.printStackTrace();
        }
    }
}

The ServerUtil class.  You can ignore the getKeyEvent() method.  I originally planned this to be a controller for NES games, but lost interest in the project and decided to go another way with the transmission of key events to the server.
/*
 * License: Public Domain.
 */
package smartphone.controller.util;

import java.awt.event.KeyEvent;
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.net.SocketException;
import java.net.UnknownHostException;
import java.util.Enumeration;

/**
 *
 * @author Tom Farrell
 * Generic utility class to assist the Server in certain goals, such as getting 
 * the users IP address, translating key events between Android and JavaSE, etc.
 */
public class ServerUtil {
    /*Simple utility method to determine the private IP of a system on a network.
     Most routers are configured to assign 192.168 addresses, so for simplicity's 
     sake, I am skipping the 10. and 172. ranges.
     */

    public static String getIpAddress() throws SocketException, UnknownHostException {
        String ipAddress = null;

        try {
            Enumeration<NetworkInterface> interfaces = NetworkInterface.getNetworkInterfaces();
            while (interfaces.hasMoreElements()) {
                Enumeration<InetAddress> addresses = interfaces.nextElement().getInetAddresses();

                while (addresses.hasMoreElements()) {
                    InetAddress inetAddress = addresses.nextElement();

                    String[] ips = inetAddress.getHostAddress().split(" ");

                    for (String s : ips) {
                        if (s.startsWith("192.168")) {
                            ipAddress = s;
                        }
                    }
                }
            }
        } catch (SocketException e) {
            e.printStackTrace();
        }

        return ipAddress;
    }

    /*
     This method returns the Java SE KeyEvent fo a specific key.  e.g. The A key
     will return the int that represents KeyEvent.VK_A, etc.   I can think of no
     other way to do this with a lot of keys, other than using a switch statement
     or creating a map that maps android keyevent values to java keyevent values,
     as they are completely different.  I am only working with a few buttons, so
     I am OK to use a simple switch statement.     
     */
    public static int getKeyEvent(String key) {
        int keyEvent = 0;
        
        /*
        The client will send Strings and the switch statement will process them.
        */
        switch (key) {
            case "up":
                keyEvent = KeyEvent.VK_UP;
                break;
            case "down":
                keyEvent = KeyEvent.VK_DOWN;
                break;
            case "left":
                keyEvent = KeyEvent.VK_LEFT;
                break;
            case "right":
                keyEvent = KeyEvent.VK_RIGHT;
                break;
            case "select":
                keyEvent = KeyEvent.VK_SHIFT;
                break;
            case "start":
                keyEvent = KeyEvent.VK_ENTER;
                break;
            case "b":
                keyEvent = KeyEvent.VK_B;
                break;
            case "a":
                keyEvent = KeyEvent.VK_A;
        }

        return keyEvent;
    }
}
And now the Android client.
 
activity_main.xml
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:padding="50dip"
    android:orientation="vertical"
    android:gravity="center_horizontal"
    tools:context=".MainActivity">

    <Button
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Connect"
        android:id="@+id/connect"
        />

    <Button
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Escape"
        android:id="@+id/escape"
       />

    <Button
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Enter"
        android:id="@+id/enter"
         />
</LinearLayout>

AndroidManifest.xml

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.tj.controllerclient" >

    <uses-permission android:name="android.permission.INTERNET"/>

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:theme="@style/AppTheme" >
        <activity
            android:name=".MainActivity"
            android:label="@string/app_name"
            android:screenOrientation="portrait"
            android:configChanges="orientation">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

</manifest>

And finally the Android client MainActivity.

public class MainActivity extends ActionBarActivity {
    private Socket socket;
    private Writer writer;
    private Button escape, enter, connect;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);


        connect = (Button) findViewById(R.id.connect);

        //Connection to the server must not be done on the main thread.
        connect.setOnClickListener(new View.OnClickListener() {
            public void onClick(View view) {
                new ConnectTask().execute();
            }
        });

        escape = (Button) findViewById(R.id.escape);
        escape.setOnClickListener(new View.OnClickListener() {
            public void onClick(View view) {
                if (socket != null && socket.isConnected()) {
                    try {
                        sendAction("escape", "press");
                        sendAction("escape", "release");
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
            }
        });

        enter = (Button) findViewById(R.id.enter);
        enter.setOnClickListener(new View.OnClickListener() {
            public void onClick(View view) {
                if (socket != null && socket.isConnected()) {
                    try {
                        sendAction("enter", "press");
                        sendAction("enter", "release");
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
            }
        });
    }

    /*The IP address f my setup.  Yours will be different.  If you don't know how to get the ip
   address of the machine running your server, the Server Application has a class titled ServerUtil
   that has a method that will get it for you.
   */
    private void connect() throws IOException {
        String ip = "192.168.1.20";

        //Connect to the server.
        socket = new Socket(ip, 5000);

        writer = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
    }

    //Send action to the server in this format ("key" "state") e.g. (escape press).
    private void sendAction(String key, String state) throws IOException {
        ((BufferedWriter) writer).write(key + " " +  state + "\n");
        ((BufferedWriter) writer).flush();
    }

    //AsyncTask to connect to the server.
    private class ConnectTask extends AsyncTask<Void, Void, Void> {
        @Override
        protected Void doInBackground(Void... params) {
            try {
                connect();
            } catch (IOException e) {
                e.printStackTrace();
            }

            return null;
        }
    };

    public void onPause() {
        super.onPause();
        if (socket != null) {
            try {
                socket.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
}

Start the server, load the client on your android device, connect, then send some commands!

 

There are a lot of possibilities with this one.  You could use your Android device as a touchpad, emulating mouse movements and mouse presses.   You could even set up listeners to type on your pc with the soft keyboard on your Android phone, or you could even continue the original intent of this project and build yourself a functioning game controller for games.

   

Edited by farrell2k, 07 March 2015 - 08:39 PM.

  • 0

Averageloser.com - I used to be a programmer like you, then I took a -> in the knee. 


#2 BlackRabbit

BlackRabbit

    CodeCall Legend

  • Expert Member
  • PipPipPipPipPipPipPipPip
  • 3871 posts
  • Location:Argentina
  • Programming Language:C, C++, C#, PHP, JavaScript, Transact-SQL, Bash, Others
  • Learning:Java, Others

Posted 08 March 2015 - 02:55 PM

Excellent!

Mind security because anyone who steals your phone can also take control of your desktop, beware of that.

 

What practical use are you giving to it ?


  • 0

#3 farrell2k

farrell2k

    CC Addict

  • Advanced Member
  • PipPipPipPipPip
  • 169 posts

Posted 08 March 2015 - 11:05 PM

Could be used for a lot of things.  Originally, I planned on writing a client that would allow me to use my Android device as a 6 button controller for Super Nintendo and MAME emulation.   At this point, it is just for education.  I have no interest in it any longer.


  • 0

Averageloser.com - I used to be a programmer like you, then I took a -> in the knee. 


#4 BlackRabbit

BlackRabbit

    CodeCall Legend

  • Expert Member
  • PipPipPipPipPipPipPipPip
  • 3871 posts
  • Location:Argentina
  • Programming Language:C, C++, C#, PHP, JavaScript, Transact-SQL, Bash, Others
  • Learning:Java, Others

Posted 14 March 2015 - 12:26 AM

About making a controller for a game, it sounds interesting, because you'll have to work hard on the timing, and the response speed. I think working on it could teach you a lot.

 

On the other hand, I thought you had a practical use in mind, home security specifically. For example your home PC sends a warning, let's say motion detection somewhere, and you have a menu of actions to do, like sounding an alarm, or playing some "movement in the house" audio, etc.


  • 0