Jump to content

Photo

Sending MouseClicks & KeyStrokes to Multiple Applications at Once?


  • Please log in to reply
9 replies to this topic

#1 Vitium Posted 07 October 2013 - 01:43 AM

Vitium

    Soldier

  • Members
  • Pip
  • 5 posts
        private void Form1_Load(object sender, EventArgs e)
        {
            Process[] p = Process.GetProcessesByName("APPLICATION");
            foreach (Process proc in p)
            { nsComboBox1.Items.Add(proc.Id); }
        }

        private void nsButton1_Click(object sender, EventArgs e)
        {
            nsComboBox1.Items.Clear();
            Process[] p = Process.GetProcessesByName("APPLICATION");
            foreach (Process proc in p) 
            { nsComboBox1.Items.Add(proc.Id); }
        }

        private void nsButton2_Click(object sender, EventArgs e)
        {

            int PID = Int32.Parse(nsComboBox1.SelectedItem.ToString());

            using (var memory = new MemorySharp(Process.GetProcessById(PID)))
            {
                memory.Windows.MainWindow.Mouse.MoveTo(189, 636);
                memory.Windows.MainWindow.Mouse.DoubleClickLeft();
            }

        }

So I'm trying to send MouseClicks & KeyStrokes to a specific application, I want to be able to have 2-3 of that application running, and 2-3 of the application I'm trying to make.

Then I'd choose which PID I want each one to "attach" to and hit a Start Button which would then start sending these MouseClicks and KeyStrokes to that specific application and at a certain point start over/loop, while also still allowing the same to happen with the other 2-3.

 

The issue is, I can get the PID, and I can send the Clicks and Keys, however it requires the window to be in focus which prevents me from being able to send it to multiple applications, or use my computer for anything else while it runs.

 

Any and all help is greatly appreciated, I'm really new to this type of thing and looking to learn.


  • Back to top
  • Report

#2 ZenLulz Posted 07 October 2013 - 07:58 PM

ZenLulz

    Lead Developer

  • Administrators
  • 67 posts
  • LocationSwitzerland

Hi Vitium,

 

MemorySharp uses Windows messages to send keystrokes and by this fact, it doesn't require that the target window is the foreground application.

 

However, in its current implementation, the mouse simulation uses the function SendInput of the API Win32, which sends mouse events to the foreground application.

I decided to use this function because several applications/games ignore the Windows messages when used to simulate mouse events.

 

Nevertheless, your application maybe supports receiving Windows messages as mouse simulation. In this instance, it allows you to send mouse clicks to any background windows.

To send mouse click to a window , manually posts messages to it like this.

 

using (var memory = new MemorySharp(ApplicationFinder.FromProcessName("myProcess").First()))
{
    memory.Windows.MainWindow.SendMessage(WindowsMessages.LButtonDown, UIntPtr.Zero, new IntPtr(MakeLParam(x, y)));
    memory.Windows.MainWindow.SendMessage(WindowsMessages.LButtonUp, UIntPtr.Zero, new IntPtr(MakeLParam(x, y)));
}

 

where x and y are the position to click and the function MakeLParam is

 

public static int MakeLParam(int LoWord, int HiWord)
{
    return ((HiWord << 16) | (LoWord & 0xffff));
}

 

This code presses and releases the left button of the mouse. You can more events by inspecting the WindowsMessages enumeration or directly on MSDN here.

Be aware you need to send the mouse events to the correct window. The tool Spy++ (embedded with Visual Studio) can help you to identify the correct one.

 

I will probably write a nice wrapper to send mouse events through Windows messages in a next release.

 

Cheers,

ZenLulz


Edited by ZenLulz, 08 October 2013 - 07:05 AM.
Use of SendMessage

ZenLulz

  • Back to top
  • Report

#3 Vitium Posted 08 October 2013 - 02:07 AM

Vitium

    Soldier

  • Members
  • Pip
  • 5 posts

So I've been trying what you posted above and I end up with the following errors.

 

Error	1	The best overloaded method match for 'Binarysharp.MemoryManagement.Windows.RemoteWindow.PostMessage(Binarysharp.MemoryManagement.Native.WindowsMessages, System.UIntPtr, System.UIntPtr)' has some invalid arguments	C:\Users\VG-1\Desktop\Vice\Vice\Vice\Form1.cs	61	17	Vice
Error	2	Argument 3: cannot convert from 'System.IntPtr' to 'System.UIntPtr'	C:\Users\VG-1\Desktop\Vice\Vice\Vice\Form1.cs	61	98	Vice
Error	3	The best overloaded method match for 'Binarysharp.MemoryManagement.Windows.RemoteWindow.PostMessage(Binarysharp.MemoryManagement.Native.WindowsMessages, System.UIntPtr, System.UIntPtr)' has some invalid arguments	C:\Users\VG-1\Desktop\Vice\Vice\Vice\Form1.cs	62	17	Vice
Error	4	Argument 3: cannot convert from 'System.IntPtr' to 'System.UIntPtr'	C:\Users\VG-1\Desktop\Vice\Vice\Vice\Form1.cs	62	96	Vice


Edited by Vitium, 08 October 2013 - 09:45 AM.

  • Back to top
  • Report

#4 ZenLulz Posted 08 October 2013 - 07:04 AM

ZenLulz

    Lead Developer

  • Administrators
  • 67 posts
  • LocationSwitzerland

Yeah right, I edited my code (use of SendMessage instead of PostMessage).


ZenLulz

  • Back to top
  • Report

#5 Vitium Posted 08 October 2013 - 08:43 AM

Vitium

    Soldier

  • Members
  • Pip
  • 5 posts

Alright so I tried that out and it doesn't send a click at all. 

 

This

        private void nsButton2_Click(object sender, EventArgs e)
        {
            if (nsComboBox1.SelectedItem != null)
            {
                int PID = Int32.Parse(nsComboBox1.SelectedItem.ToString());
                using (var memory = new MemorySharp(ApplicationFinder.FromProcessId(PID)))
                {
                    memory.Windows.MainWindow.Mouse.MoveTo(189, 636);
                    memory.Windows.MainWindow.Mouse.DoubleClickLeft();
                }
            }
        }

Sends the Clicks provided the window is in focus, it doesn't move my mouse or anything, and it will click other windows that are in that location if they're on top of the application I'm trying to send it to.

 

I've ran Spy++ so I know I'm sending it to the right application, and if I have the application in focus and move it anywhere the clicks are in the right location of the application.

 

I also just ran Spy++ again to watch the Messages when I manually click the application the following shows up.


<000001> 000201C4 S WM_MOUSEACTIVATE hwndTopLevel:000201C4 nHittest:HTCLIENT uMsg:WM_LBUTTONDOWN
<000002> 000201C4 R WM_MOUSEACTIVATE fuActivate:MA_ACTIVATE
<000003> 000201C4 P WM_LBUTTONDOWN fwKeys:MK_LBUTTON xPos:185 yPos:604
<000004> 000201C4 P WM_LBUTTONUP fwKeys:0000 xPos:185 yPos:604

Not really sure if that helps with what the application can accept but figured I'd post it since I'm going to go google about it anyway.

 

Also something else that I should add, is another application that is built specifically for this application and can send things to multiples instances of it while they're minimized has an option "Force Foreground", its a general setting that is required to get the Clicks/Keystrokes to send to the Application, I have no idea how it is done but I think that might be part of the issue.

 

EDIT: Sending a friend what I have who has the Application I mentioned above to see if running them together (With the "Force Foreground" gets it to work to try and see if that's the issue I'm having.

 

EDIT2: Had him give it a try and FindApplication ends up showing an error, seems that this other Application interferes with that somehow, so basically learned nothing new from that. I might need to figure how to make it think its in foreground I might not.


Edited by Vitium, 08 October 2013 - 09:35 AM.

  • Back to top
  • Report

#6 ZenLulz Posted 08 October 2013 - 06:15 PM

ZenLulz

    Lead Developer

  • Administrators
  • 67 posts
  • LocationSwitzerland

The behavior "Force Foreground" you are describing is called Topmost Windows and can be applied using the function SetWindowPos (pinvoke info here).

Alternately, you can try to mimic the messages sent as you see them in Spy++ by playing with the functions PostMessage/SendMessage.

 

Note you can activate any windows using MemorySharp using the function Activate, like this.

 

memory.Windows.MainWindow.Activate();

 

Cheers


ZenLulz

  • Back to top
  • Report

#7 Vitium Posted 08 October 2013 - 07:28 PM

Vitium

    Soldier

  • Members
  • Pip
  • 5 posts

So I've tried a ton of different things and in some situations I get the MouseClick to work (however the Application still needs to be foreground) and not move my mouse. However having friends try it out has it completely move the mouse and such. 

 

So all in all I can't seem to get anything sent to the program while it's running in the background.

What would cause this? is it something that some Applications just prevent without a workaround?

 

Here is what I have maybe I'm just doing something completely wrong, this is also the code I've had the most success with (MouseClicks actually going through and such, although not in the background/not in focus.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.Diagnostics;
using Binarysharp;
using Binarysharp.MemoryManagement;
using Binarysharp.MemoryManagement.Assembly;
using Binarysharp.MemoryManagement.Helpers;
using Binarysharp.MemoryManagement.Native;
using Binarysharp.MemoryManagement.Windows;


namespace VTest
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        private void Form1_Load(object sender, EventArgs e)
        {
            Process[] p = Process.GetProcessesByName("APPLICATION");
            foreach (Process proc in p)
            { nsComboBox1.Items.Add(proc.Id); }
            
        }

        // Refresh
        private void nsButton1_Click(object sender, EventArgs e)
        {
            nsComboBox1.Items.Clear();
            Process[] p = Process.GetProcessesByName("APPLICATION");
            foreach (Process proc in p)
            { nsComboBox1.Items.Add(proc.Id); }
        }

        // Start
        private void nsButton2_Click(object sender, EventArgs e)
        {
            if (nsComboBox1.SelectedItem != null)
            {
                int PID = Int32.Parse(nsComboBox1.SelectedItem.ToString());
                using (var memory = new MemorySharp(ApplicationFinder.FromProcessId(PID)))
                {
                    memory.Windows.MainWindow.Mouse.MoveTo(189, 636);
                    memory.Windows.MainWindow.Mouse.ClickLeft();
                }
            }
        }

    }
}

Edited by Vitium, 08 October 2013 - 07:30 PM.

  • Back to top
  • Report

#8 ZenLulz Posted 08 October 2013 - 08:26 PM

ZenLulz

    Lead Developer

  • Administrators
  • 67 posts
  • LocationSwitzerland

You code is right and requires the application to be in foreground.

 

After, each program can implement its own way to query the cursor position. For example, the game World of Warcraft doesn't read the position of the mouse from the Windows Messages but uses the function GetCursorPos from the API Win32. The Windows messages are only used to simulate the click of the mouse (like in my first message). In this case, you have to hook the function GetCursorPos.

 

You have to perform some reverse engineering on your application to know how it exactly works... or ask to your friends to know how they do. :P


ZenLulz

  • Back to top
  • Report

#9 Vitium Posted 09 October 2013 - 01:46 AM

Vitium

    Soldier

  • Members
  • Pip
  • 5 posts

If it's not too much trouble could you recommend some reading on Reverse Engineering?

 

The software a friend uses is purchased, so they aren't to knowledgeable on it.

 

EDIT: Been reading about Reverse Engineering all night and looking into IDA Pro w/ Hex-Rays, I can find various dwords for reading things from the Application. I'm under the assumption I want the same thing but for the MouseClicks, or do I want to find the dword for whatever I'm specifically trying to click, I'd guess with the MouseClicks it would allow me to click anywhere within the application, and Keys would let me send keys. 

 

All that being said, how do I take the dwords and use them with MemorySharp? 

var address = IntPtr.Zero; var sharp = new MemorySharp(Process.GetCurrentProcess());

// Read an array of 3 integers 
int[] integers = sharp.Read(address, 3);
// Write a string
sharp.WriteString(address, "I love managed languages.");

I tried playing around with that example, but wasn't entirely sure that was the right example for what I wanted, and I couldn't compile because 

sharp.Read(address, 3)

Would give an error back.

 

Thanks again for the help!


Edited by Vitium, 09 October 2013 - 06:50 AM.

  • Back to top
  • Report

#10 ZenLulz Posted 09 October 2013 - 09:11 PM

ZenLulz

    Lead Developer

  • Administrators
  • 67 posts
  • LocationSwitzerland

I recommend you to read these two books, they are really interesting: IDA Pro Book Unofficial Disassembler  and Reversing: Secrets of Reverse Engineering.

 

IDA Pro is a very powerful tool but requires a good knowledge in reversing engineering to be used correctly. DWORD is just a data type, it is equal to 4 bytes (unsigned), as expressed here. You can find thousand of DWORD within an application and they won't especially help you to perform a click somewhere in your application.

 

I suggest reading some basis articles on how Windows messages work and how an application can get the position of the cursor, instead blindly disassemble your process and try to look for something you ignore.

 

You can read DWORD using MemorySharp like this.

 

sharp.Read<uint>(address)

 

This reads a value of 4 bytes (unsigned) from the main module base address.

 

EDIT: I fixed the example that threw the error. I forgot the function was generic. ^_^

The correct example is:

var address = IntPtr.Zero; var sharp = new MemorySharp(Process.GetCurrentProcess());

// Read an array of 3 integers
int[] integers = sharp.Read<int>(address, 3);
// Write a string
sharp.WriteString(address, "I love managed languages.");


Edited by ZenLulz, 09 October 2013 - 09:17 PM.

ZenLulz

  • Back to top
  • Report




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users