FollowMeBot

Team Members: Richie Choy, Daniel Hamilton, Rahul Shetty

Description

The FollowMeBot follows a person based on a color selected by the user. The device hosts a webcam on a servo to find the object and orient the robot. The color is chosen through the user interface with push buttons. Currently, the MATLAB code supports red and green detection. The purpose of FollowMeBot is to be able to follow the user in order to be helpful for daily tasks.

Here are the main components of the FollowMeBot:

• mbed LPC1768
• iRobot Create
• Atom PC
• Logitech Webcam
• Sharp IR sensor
• Servo
• 6V battery supply

MBED: The mbed is responsible for controlling the iRobot's movements. We leveraged the iRobot libraries to build the "following" algorithm. The robot polls the area to find the selected color until it finds the user. The mbed receives push button inputs for the colors on the breadboard on the surface. The green and red LEDs near the buttons correspond with the selected color.

Image Detection: The image processing is done with MATLAB running on the Atom PC. The MATLAB image processing toolbox is used to read the input of the webcam and produce X,Y centroid data that is sent to the mbed. We used serial interfacing to communicate between mbed and the Atom PC. The mbed uses the X,Y data to re-orient the servo to center the object and move the robot accordingly.

Power: The mbed runs off the USB cable from the Atom. This provides power to the IR sensor and LEDs. The Atom PC runs off the iRobot battery. A stand alone 6V power supply powers the servo.

Images

Connections

Matlab Compiler Runtime (MCR) Instructions

1. Download the MCR Installer from: http://www.mathworks.com/products/compiler/mcr/

2. Follow the instructions of the installer and install in the PC you want it on

3. In the Matlab Application, run the "deploytool" instruction in the Matlab command window.

4. Then choose the folder where you want the standalone application to be stored.

5. Follow the instructions by picking the main file and all its dependent files

6. Then click build and wait!

These are the steps to make a standalone Matlab application. The MCR is free to use and was helpful to us in this project because the Atom PC we were using did not have Matlab installed. We installed MCR on the Atom PC and then was able to run Matlab applications on the Atom.

Code

Matlab Code

mbed = getSerialPort();


delete(timerfind); 
 
try
   % For windows
   vid = videoinput('winvideo', 1);
catch
   errordlg('No webcam available');
   
end

% Set the properties of the video object
set(vid, 'FramesPerTrigger', Inf);
set(vid, 'ReturnedColorspace', 'rgb')

t = timer;
t.TimerFcn = {@getCentroid,vid, mbed}
t.Period = 0.1;
t.TasksToExecute = Inf;                                                                                                               
t.ExecutionMode = 'fixedRate';

start(vid);
start(t);

stop(t);
delete(t);
stop(vid);
delete(vid);
fclose(mbed);   %close connection to prevent COM port being locked open
delete(mbed);
clear mbed; 

function [mbed] = getSerialPort()

info = instrhwinfo('serial');
serials = size(info.SerialPorts);
%if(serials(1) >= 2)
 %  comPort = info.SerialPorts{2};
%else %just hardcode a comport
 %   comPort = 'COM22';
%end

%comPort = 'COM14';
comPort = 'COM32';

try
    delete(instrfindall); % first delete all serials
    
    mbed = serial(comPort, ...
       'BaudRate', 9600, ...
       'Parity', 'none', ...
       'DataBits', 8, ...
       'StopBits', 1, ...
       'Timeout', 10);       %change depending on mbed configuration

   fopen(mbed);
catch
	%disp('Could Not Find MBED COM Port');
    fclose(mbed);   %close connection to prevent COM port being lokced open
    delete(mbed);
    clear mbed;
    ed = errordlg({'Serial connection not established.', 'Please check connections and try again'}); 
    uiwait(ed);
end

end

function [] = sendSerialData(mbed, x, y, boxArea)
    
    buff = strcat(int2str(x), ';', int2str(y), ';', int2str(boxArea), ';');    
    try
        fprintf(mbed, buff);    %get values into vector
    catch
    end
    %values = fscanf(mbed);    %get values into vector
    %disp('read somethin')
    %disp(values)
 
    %fclose(mbed);   %close connection (this should never be reached when using while(1), but included for completeness)
    
end 

function [x, y, boxArea] = getCentroid(obj, event,vid, mbed)

    % Get the snapshot of the current frame
    data = getsnapshot(vid);
    %size(data)
    
    persistent values;
    
    if exist('values')==0
        values = 0
    end
    
    if mbed.BytesAvailable > 0
        values = fscanf(mbed);
        %disp(values);
    end

     color = str2double(values);
    %disp(color);
    
    % Now to track red objects in real time
    % we have to subtract the red component 
    % from the grayscale image to extract the red components in the image.
    if color == 0
        diff_im = imsubtract(data(:,:,2), rgb2gray(data));
        disp('green');
    else
        diff_im = imsubtract(data(:,:,1), rgb2gray(data));
        disp('red');
    end
        
    %Use a median filter to filter out noise
    diff_im = medfilt2(diff_im, [3 3]);
    % Convert the resulting grayscale image into a binary image.
    diff_im = im2bw(diff_im,0.18);
    
    % Remove all those pixels less than 300px
    diff_im = bwareaopen(diff_im,300);
    
    % Label all the connected components in the image.
    bw = bwlabel(diff_im, 8);
    
    % Here we do the image blob analysis.
    % We get a set of properties for each labeled region.
%     stats = regionprops(bw, 'BoundingBox', 'Centroid');
     stats = regionprops(bw, 'Area', 'Centroid');

%     imshow(data)   
%     hold on
    
    largest = 0;
    myIndex = 0;

    for object = 1:length(stats)
        tempArea = stats(object).Area;

        if(tempArea > largest)
            largest = tempArea;
            myIndex = object;
        end
        
    end
    
    if largest > 0
        bc = stats(myIndex).Centroid;
        x = bc(1);
        y = bc(2);
        boxArea = stats(myIndex).Area;
    end
    
    if exist('x')==0
        x = 0;
        %disp(x)
    end
    if exist('y')==0
        y = 0;
        %disp(y)
    end
    if exist('boxArea')==0
        boxArea = 0;
        %disp(y)
    end
%     hold off

    sendSerialData(mbed, x, y, boxArea);
    
    if imaqmem('FrameMemoryLimit') > 5000000
        flushdata(vid); 
    end
    
    
end

mbed Code

Video

Future Development

Instead of user holding color paper, the image processing can be improved to work on solid color T-shirts. This way the robot can follow with the user having his back turned.