introduction

this article describes the implementation of a windows mobile desktop remote controller. with this application, you will be able to remotely control your windows mobile device by using the mouse and keyboard.

background

the code in this article builds on my previous article: a remote windows mobile screen grabber. instead of blocking rapi calls, this application implements a streamed rapi server that allows the desktop application to have a permanent connection to the device. also, in this code, i have dropped the gapi and directdraw screen grabbing techniques, and used a simpler gdi based screen grabbing technique. to improve communications performance, the code uses the zlib compression library on both ends.

desktop code

please refer to the ceremoteclient directory on the distribution zip file for the desktop project.

the bulk of the desktop code is on the ceremoteclientview.h file. this is in fact a wtl 8.0 frame child window that implements all the desktop client features.

device connection is performed via the public connect() and disconnect() methods. these are called by the menu and toolbar handlers in the frame window class implementation (mainfrm.h). when the desktop successfully connects to the device, a 200 millisecond timer is created to poll the device for the compressed screen bitmap.

the device screen is retrieved by the private getscreen() method. it first kills the timer, and sends a message to the device server requesting the current screen:

killtimer(screen_timer);
hr = write(rcm_getscreen);

the device server returns a message containing the same message code, the compressed size of the screen buffer, its expanded size, and the compressed byte stream. after reading the three first dwords, the code makes sure there is enough room on both the compressed and expanded buffers, and then reads the compressed byte stream:

// read the compressed buffer
hr = m_pstream->read(m_pzipbuf, cbzipbuf, &ulread);

if all is well, the compressed buffer is decompressed and the resulting dib is queried for the bitmap dimensions. if the dimensions are different than the last time, then it is very likely that the device screen was rotated, so the whole window is invalidated to erase any garbage:

zr = uncompress(m_pscrbuf, &ndestlen, m_pzipbuf, cbzipbuf);
if(zr == z_ok)
{
    dibinfo* pdibinfo = (dibinfo*)m_pscrbuf;
    byte*    pbmpdata = (byte*)(m_pscrbuf + sizeof(dibinfo));
    bool     berase   = false;

    if(m_xdevscr != pdibinfo->bmiheader.biwidth || 
       m_ydevscr != pdibinfo->bmiheader.biheight)
    {
        m_xdevscr = pdibinfo->bmiheader.biwidth;
        m_ydevscr = pdibinfo->bmiheader.biheight;

        setscrollsize(m_xdevscr, m_ydevscr);

        berase = true;
    }

    m_dib.setbitmap((bitmapinfo*)pdibinfo, pbmpdata);

    invalidaterect(null, berase);
    updatewindow();
}

after forcing the window to update, the timer is restarted so we can get the next screen.

sending input

sending keyboard and mouse input to the device is pretty simple: handle the corresponding window messages, convert their data content to input structures, and send them to the server for processing. here's the wm_keydown handler:

lresult onkeydown(tchar vk, uint crepeat, uint flags)
{
    hresult hr;
    input   input;

    if(!m_bconnected)
        return 0;

    input.type           = input_keyboard;
    input.ki.wvk         = mapkey(vk);
    input.ki.wscan       = 0;
    input.ki.dwflags     = 0;
    input.ki.time        = 0;
    input.ki.dwextrainfo = 0;

    hr = write(rcm_setinput);
    if(succeeded(hr))
    {
        hr = write(&input, sizeof(input));
        if(succeeded(hr))
            getscreen();
    }

    return 0;
}

the mapkey() function performs basic key mappings between the desktop and the device keyboards. use the f1 key for the left function button, and f2 for the right. the f3 and f4 keys naturally map to the phone keys.

sending mouse actions is similar:

lresult onlbuttondown(uint flags, cpoint pt)
{
    hresult    hr;
    input    input;

    if(!m_bconnected)
        return 0;

    m_bleftbtn           = true;
    input.type           = input_mouse;
    input.mi.dwflags     = mouseeventf_absolute | 
                           mouseeventf_leftdown;
    input.mi.dx          = pt.x * 65536 / m_xdevscr;
    input.mi.dy          = pt.y * 65536 / m_ydevscr;
    input.mi.mousedata   = 0;
    input.mi.time        = 0;
    input.mi.dwextrainfo = 0;

    hr = write(rcm_setinput);
    if(succeeded(hr))
    {
        hr = write(&input, sizeof(input));
        if(succeeded(hr))
            hr = getscreen();
    }

    return 0;
}

note how the mouse screen coordinates are normalized for the device screen. this is a requirement of the sendinput api used on the device server.

now that i mentioned it, let's take a closer look at the device server code.

device code

please refer to the ceremsrv directory on the distribution zip file for the device project.

the bulk of the device code is implemented in the cremotecontrol class. all messages sent by the desktop client are processed and dispatched on the executive loop implemented in the run method.

the device screen is captured by the sendscreen method which has a very similar structure to its desktop counterpart. note how the device screen is captured so easily:

hdc = getwindowdc(null);

after getting the hdc of the device screen, you can very easily copy it into a bitmap and serialize it to the desktop. there's no need for fancy gapi or directdraw techniques like i used before.

after getting the device screen copied into a dib, the whole thing is compressed and sent back to the desktop client:

memcpy(m_pscrbuf + i, m_dib.getbitmapinfo(), sizeof(dibinfo));
i += sizeof(dibinfo);

memcpy(m_pscrbuf + i, m_dib.getdibits(), m_dib.getimagesize());
i += m_dib.getimagesize();

ulong len = m_cbzipbuf;
int   zr  = compress(m_pzipbuf, &len, m_pscrbuf, cbnew);

if(zr != z_ok)
    len = 0;

hr = m_pstream->write(&dwmsg,    sizeof(dword), &ulwritten);
hr = m_pstream->write(&len,      sizeof(ulong), &ulwritten);
hr = m_pstream->write(&cbnew,    sizeof(dword), &ulwritten);
hr = m_pstream->write(m_pzipbuf, len,           &ulwritten);

handling input from the desktop is even simpler:

hresult cremotecontrol::getinput()
{
    input   input;
    hresult hr;
    dword   dwread;

    hr = m_pstream->read(&input, sizeof(input), &dwread);
    if(failed(hr))
        return hr;

    if(dwread != sizeof(input))
        return e_fail;

    sendinput(1, &input, sizeof(input));

    return s_ok;
}

a very simple implementation indeed.

points of interest

there are two interesting things you may like to know: how i simulated the double-click mouse event, and why this code will not work out of the box on wm5 and wm6 devices.

double-clicks had to be simulated by sending four messages to the device. this happens because the desktop window manager will merge the four mouse events (down - up - down - up) into a single message - wm_lbuttondblclk. if you look at my code, you will see how this was undone...

on wm5 and wm6 devices, you may have to enable rapi connectivity in order for the device server dll to respond to the client. i once wrote a simple device tool to help you with this nasty chore. you can get it here. copy the exe to the device and execute it.

history

• 2008-03-28 - corrected flickering.
• 2008-03-17 - first release.