Welcome, guest | Sign In | My Account | Store | Cart

This demo requires you to be using a Linux machine and to have libfreenect installed with the python wrapper. This demo also requires you to have opencv, numpy, and pygame. They can all be installed using sudo apt-get install {PROGRAM NAME HERE}. Here are instructions on installing libfreenect: http://openkinect.org/wiki/Getting_Started. Okay, so what this demo does is finds the minimum point on the depth image and uses the index of this minimum point to create a four button joystick that codes for 'A', 'B', 'X', 'Y'. It also uses the minimum point and the RGB image to put a circle on the minimum point on the screen. So basically a dot will follow your hand on the RGB image. If the minimum value is larger than 600 (arbitrary units), the color of the dot will change from red to purple and the dot will remain stationary. Also, if the Kinect is not properly opened the first time, unplug it and plug it back in and run in the terminal sudo freenect-glview. After try running the program again. One more thing, this code requires super user privileges so run it through the terminal. Here is how to do this for linux n00bs.

  1. Change your directory the the directory the code is in (use cd {PATH})
  2. Run the code using sudo python {FILENAME}
  3. Don't forget to add the the .py at the end
Python, 132 lines
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
#!/usr/bin/env python
"""
Kinect Demo using minimum values from the depth image. 
@Author   = Alex Wallar
@Date     = 17 March, 2012
@Version  = 1.1
@Filename = KinectJoystickMin.py
"""
from freenect import sync_get_depth as get_depth, sync_get_video as get_video, init, close_device, open_device, set_led
import cv  
import numpy as np
import pygame
from math import *
from numpy import mean

def doloop():
    
    #Series of commands to do pointer operations on the kinect (motor, led, accelerometer)
    ctx = init() #Initiates device
    mdev = open_device(ctx, 0) #Opens the device for commands
    set_led(mdev, 1) #Sets LED to green
    close_device(mdev #Closes device. Device must be closed immediately after usage
    
    #Mean filter caches
    yList = [0,0,0,0,0,0]
    xList = [0,0,0,0,0,0]
    
    #Sets color tuples
    RED = (255,0,0)
    BLUE = (0,0,255)
    TEAL = (0,200,100)
    BLACK = (0,0,0)
    
    #Sets the size of the screen
    xSize = 640
    ySize = 480
    
    done = False #Main while loop bool counter
    pygame.init() #Initiates pygame
    screen = pygame.display.set_mode((xSize, ySize), pygame.RESIZABLE) #Creates the pygame window
    screen.fill(BLACK) #Fills the window black
    
    #Initiates the xTempPos and yTempPos values so that the point will remain stationary
    #if the minimum value is larger than 600
    xTempPos = xSize/2
    yTempPos = ySize/2
    
    global depth, rgb #Makes the depth and rgb variables global
    
    while not done:
        screen.fill(BLACK) #Makes the pygame window black after each iteration
        
        # Get a fresh frame
        (depth,_) = get_depth()
        (rgb, _) = get_video()
        
        minVal = np.min(depth) #This is the minimum value from the depth image
        minPos = np.argmin(depth) #This is the raw index of the minimum value above
        xPos = np.mod(minPos, xSize) #This is the x component of the raw index
        yPos = minPos//xSize #This is the y component of the raw index
        
        #This is the mean filter process
        """
        A mean filter works by collecting values in a cache list and taking the mean of them
        to determine the final value. It works in this case to decrease the amount of
        volatility the minimum position experiences to get a smoother display with a more
        consistent value. My computer works smoothly with a 5 bit cache where as a faster
        computer may need a larger cache and a slower computer may need a smaller cache
        """
        xList.append(xPos)
        del xList[0]
        xPos = int(mean(xList))
        yList.append(yPos)
        del yList[0]
        yPos = int(mean(yList))
        
        """
        This if statement says that if the minimum value is below 600 to store the minimum 
        positions in xTempPos and yTempPos and to make the dot color red. Also if the minimum
        value is larger than 600, xPos and yPos become the last stored minimum and maximum
        positions. It also changes the color to purple
        """
        if minVal < 600:
            xTempPos = xPos
            yTempPos = yPos
            COLOR = cv.RGB(255,0,0)
        else:
            xPos = xTempPos
            yPos = yTempPos
            COLOR = cv.RGB(100,0,100)
            
        cv.Circle(rgb, (xPos, yPos), 2, COLOR, 40) #draws a circle of a certain color at minimum position
    
        cv.ShowImage('Image',rgb) #Shows the image
        cv.WaitKey(5) #Keyboard interupt
        
        """
        The if statement below sets up the virtual joystick by basically breaking the pygame
        window into four parts. A dot representing the minimum position is drawn on the window
        and the corresponding button based on the position is "pressed". The quarter of the
        window in which the button "pressed" corresponds to turns teal after being "pressed"
        
        Top Right   : A
        Bottom Right: B
        Bottom Left : Y
        Top Right   : X
        """        
        if xPos <= xSize/2 and yPos <= ySize/2:
            command = 'A'
            rect1 = pygame.Rect((xSize/2,0),(xSize/2,ySize/2))
            pygame.draw.rect(screen,TEAL,rect1)
        elif xPos <= xSize/2 and yPos > ySize/2:
            command = 'B'
            rect1 = pygame.Rect((xSize/2,ySize/2),(xSize/2,ySize/2))
            pygame.draw.rect(screen,TEAL,rect1)
        elif xPos > xSize/2 and yPos <= ySize/2:
            command = 'X'
            rect1 = pygame.Rect((0,0),(xSize/2,ySize/2))
            pygame.draw.rect(screen,TEAL,rect1)
        else: 
            command = 'Y'
            rect1 = pygame.Rect((0,ySize/2),(xSize/2,ySize/2))
            pygame.draw.rect(screen,TEAL,rect1)
        pygame.draw.line(screen, BLUE, (xSize/2, ySize/2), (xSize - xPos,yPos)) #Draws a line from the middle to the minimum position
        pygame.draw.circle(screen, RED, (xSize - xPos,yPos), 10) #Draws the circle on pygame window
        pygame.display.flip() #Displays the processed pygame window
        print command, minVal #Prints the "pressed" button and the minimum value
        for e in pygame.event.get(): #Itertates through current events
            if e.type is pygame.QUIT: #If the close button is pressed, the while loop ends
                done = True
        
doloop()

This is just a 1.0 version and if anybody wants to email me about any problems, please do not hesitate. My email is wallarelvo@hotmail.com. I am also sort of a beginner and tried to make this code for beginners. There is no blob analysis or machine learning techniques for a reason. This code is meant for a beginner to learn the basics of computer vision processes using the Kinect with the OpenKinect SDK for Python. Also, please no rude comments about the code. If there is a problem please email.