Leap Motion in Python 3

Purpose of the article

There is a motion tracker called Leap Motion. Using this API for Python, I tried to get the distance between the thumb and index finger in real time. There are many articles that show how to use Leap Motion in Python, but it didn't go smoothly in my environment. So I will leave the method as a memo for myself.

environment

• Windows 10 64bit
• Python 3.7.5

First in Python 2.x

Leap Motion's Python API only supports Python 2.x. I'll run it on Python 3 series later, but first I'll run it on Python 2.7. Create a Python 2.7 environment using Anaconda.

Preparing the Leap Motion SDK

Then download the Desktop SDK (https://developer.leapmotion.com/setup/desktop) from the Leap Motion Developer Site (https://developer.leapmotion.com/setup). There are V2 and V4 options, but choose V2.

When you unzip the DL zip, there is an installer inside, so let's execute it. (← Is it unnecessary if you don't need Visualizer?) afterwards, C:\Program Files\Leap Motion Core Services\Visualizer.exe If you execute, you can check the operation of Leap Motion, so let's try it.

If nothing is displayed in Visualizer, refer to this article and execute dpinst64.exe to install the driver.

Along with the installer, there is a folder called LeapSDK. Furthermore, it seems that Leap Motion can be handled from Python by ʻimport lib / Leap.py in it. However, for example, if you move to the same directory as Leap.py and ʻimport, you will get ʻImport Error: No module named Leap Python. With reference to [this article](https://blog.mezquita.jp/archives/4043), with Leap.py Copy the contents of LeapSDK / lib / x64 and \ _ \ _ init \ _ \ _.py (empty file) into one folder (named leapmotion here). Go to the directory where leap motion is located and do from leap motion import Leap` from Python to use the Leap Motion API.

Using the Leap Motion API

Articles around here and [Official Documentation](https://developer-archive.leapmotion.com/documentation/v2/python/ Use devguide / Leap_Overview.html) and dir () to output the distance between your thumb and index finger.

If you do the following, it seems that you can get the hand information at that moment as frame.

controller = Leap.Controller()
frame = controller.frame()

By the way, if you do controller.frame (1), it seems that you can get the information one frame before.

The finger information of frame is stored in frame.fingers. The information of each finger seems to be retrieved as follows.

frame.fingers[0]    # <-Thumb
frame.fingers[1]    # <-Index (index finger
frame.fingers[2]    # <-Middle (middle finger
frame.fingers[3]    # <-Ring (ring finger
frame.fingers[4]    # <-Pinky (pinky)

To output the distance between the tip of your thumb and index finger, do the following.

pos_thumb = frame.fingers[0].tip_position
pos_index = frame.fingers[1].tip_position
print(pos_thumb.distance_to(pos_index))

In summary, it looks like this.

from time import sleep

from packages.leapmotion import Leap

controller = Leap.Controller()

THUMB = 0
INDEX = 1
MIDDLE = 2
RING = 3
PINKY = 4

while not controller.is_connected:
    print('Connecting...')
    sleep(.5)
    pass
print('...Connected!')

while True:
    sleep(.01)
    frame = controller.frame()

    pos_thumb = frame.fingers[THUMB].tip_position
    pos_index = frame.fingers[INDEX].tip_position

    print(pos_thumb.distance_to(pos_index))

Runs on Python 3

This article I will do it based on. The difference from the setting in the article is that you are using 64-bit Windows and Python Ver. Is 3.7.

Swig Have them generate a program for Python using Swig.

Installation

Install Swig on Windows by referring to this site. The latest version at the time of writing the article was 4.0.1, but it seems to cause an error if it is 4.0.0 or later (Reference). I don't get an error (probably) with 3.0.12, but I use 2.0.9 as mentioned in the article. Let's go as far as adding the path to swig.exe to the environment variable PATH.

Reworking the script

Create an appropriate folder and copy the following files.

• LeapSDK/include/Leap.h
• LeapSDK/include/LeapMath.h
• LeapSDK/include/Leap.i
• LeapSDK/lib/x64/Leap.lib

I can run Swig in this folder, but Leap.i doesn't seem to be good enough and I get an error. Refer to this article and fix it manually. Open Leap.i in a text editor. The part enclosed in % pythoncode {%}} is the code for Python, but there is an error in the indentation. Fix the parts that do not need indentation, such as before def, so that there is no indentation, and the other indentations are blank. For the time being, leave Repaired Leap.i.

At the command prompt, cd to the location of that folder and execute the following command.

swig -c++ -python -o LeapPython.cpp -interface LeapPython Leap.i

If you get an error message, review Leap.i's indentation again based on it. If successful, Leap.py, LeapPython.cpp and LeapPython.h will be created in the same folder.

Visual Studio Using Visual Studio, compile the LeapPython.cpp generated earlier so that it can be run from Python.

Installation

For Python 3.7, you need to prepare the 2017 version instead of the latest 2019 version (Reference )), So I will download it. https://docs.microsoft.com/ja-jp/visualstudio/releasenotes/vs2017-relnotes Select Download Enterprise 2017 and DL & launch the installer. I didn't know what to do, so I checked Desktop Development with C ++ from the Workload tab and installed it.

Create an empty project

From the menu bar, select [File] → [New] → [Project]. Select [Visual C ++] → [Empty Project] from the menu on the left. Decide on a name and location, [OK]. Copy all the contents of the folder where you moved Swig to the project folder (LeapPython.cpp alone?). It's the same directory as the file that has .filters.

Various settings

View → Solution Explorer. Right-click the project name in Solution Explorer and select [Add] → [Existing Item]. [Add] all the files you copied from the Swig-completed folder.

Next, right-click the project name again and select [Properties]. If you don't see Properties, use the icon at the top to change from folder view to solution view, as shown in the screenshot below. Click Configuration Manager and change the calibration to Release and the platform to x64. Select the [General] menu and set [Target Name] to LeapPython and [Configuration Type] to Dynamic Library (.dll). Next, open the [C / C ++]-> [General] menu and specify the Python include folder in the additional include directory. For example, in my Anaconda (Miniconda) environment, C: \ Users \ (user name) \ Miniconda3 \ envs \ (environment name) \ include It's like that. Then jump to the Linker-> Input menu and add the full path to Leap.lib and python37.lib in the additional dependent files. python37.lib should be in the following location. C: \ Users \ (user name) \ Miniconda3 \ envs \ (environment name) \ libs \ python37.lib The setting is complete. Let's close the property.

Build

Compile. Right-click on the project name → [Build] If you get an error message, make sure that python33.lib and Leap.lib are for 64bit. When the build is finished, LeadPython.dll will be created in the displayed directory. Rename it to .pyd with the extension. For the time being, put Generated LeadPython.pyd.

Run on Python 3.7

Open the folder that created \ _ \ _ init \ _ \ _. Py by digging into the contents of the Leap SDK that was run on Python 2.7. Of these, replace Leap.py and LeapPython.pyd with those generated by Swig or Visual Studio.

Finally, you also need to manually fix Leap.py. Let's open it with a text editor. There are extra %s here and there, so please delete them. It is faster to replace 2 half-width spaces +% and erase them. Reworked Leap.py is here.

Now you can get the Leap Motion data in Python 3.7 as you did in Python 2.7.

Combine with PsychoPy (bonus)

Write a script like this ↓

leapmotion.py

from time import sleep

from .leapsdk import Leap


class LeapMotion:
    def __init__(self):
        self.controller = Leap.Controller()
        self.fingers = {'thumb': 0,
                        'index': 1,
                        'middle': 2,
                        'ring': 3,
                        'pinky': 4}

        while not self.controller.is_connected:
            sleep(.5)

    def calc_dist(self, finger_from: str, finger_to: str):
        frame = self.controller.frame()
        pos_from = frame.fingers[self.fingers[finger_from]].tip_position
        pos_to = frame.fingers[self.fingers[finger_to]].tip_position
        return pos_from.distance_to(pos_to)

Move it like this ↓

from time import sleep

from packages.device.leapmotion import LeapMotion

from psychopy import visual

leapmotion = LeapMotion()
window = visual.Window(
    colorSpace='rgb255',
    fullscr=False,
    size=(900, 900),
    allowGUI=True,
    wintype='pygame',
    units='pix')
circle = visual.Circle(window, size=1, autoDraw=True)

while True:
    circle.size = leapmotion.calc_dist('thumb', 'index')
    window.flip()
    sleep(.01)
    print(leapmotion.calc_dist('thumb', 'index'))

It looks like this! ↓

MinGW instead of Visual Studio (failed)

I tried compiling with g ++ installed with MinGW, but it didn't work. Keep a record until it gets stuck.

Installation

Install MinGW-w64 based on this article. It's OK if you add it to your PATH. By the way, in my case, I added the following. C:\Program Files\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\bin

Then, at the command prompt, cd to the SWIG completed folder and

g++ -I C:\Users\issakuss\Miniconda3\envs\study09t\include LeapPython.cpp Leap.lib C:\Users\issakuss\Miniconda3\envs\study09t\libs\python37.lib -shared -o LeapPython.pyd 

I thought I could do the same thing I did in Visual Studio, but I got stuck with a lot of errors. If I could do it here, it would have been easier because I wouldn't have to install Visual Studio.