The webcam module uses a webcam and a stored image to detect if multiple users are in the room. The presence of multiple users implies that the user is busy and should not be disturbed.
Operating System : Windows.
Language: Java
Hardware: Webcam
Software: Java Media Framework, Java Advanced Imaging(JAI) must be setup on the host computer.
*.dll files from the library directory of JAI must be put in jre/bin/
The classpath of JMF and JAI must be setup correctly
MJSIP sipstack, Publish Class
Install the software and driver needed by the Webcam.
Setup Java Media Framework and Java Advanced Imaging.
Image of the room and occupant under normal circumstances.
Setup: Stored image of the room with the single user.
Configuration File: A configuration file is used by the module to setup the presence server information it needs and the time duration parameter which defines the amount of time after which the program will run again.
The project Final ABD contains the following directories
1) log: log files are kept which show the publish messages and the responses received to them.
2)config: configuration files such as webcam.cfg which lets users setup username, hostname and dest-port information, duration after which the program should be run again.
3) lib: contains all the jar files needed the project such as sip.jar (sip stack), and jar files needed for Java Advanced Image Processing.
To run the capture program to capture the fixed picture:
java -classpath lib\sip.jar;lib\jai_codec.jar;lib\jai_core.jar;
lib\mlibwrapper_jai.jar;capture.jar;webcam.jar
BusyDiscovery.webcamcapture.CaptureImage
To run the webcam program:
java -classpath lib\sip.jar;lib\jai_codec.jar;
lib\jai_core.jar;lib\mlibwrapper_jai.jar;capture.jar;webcam.jar
BusyDiscovery.webcamcapture.Webcam
The configuration file webcam.cfg is read using the Parser Class which is part of the MJSIP library. The format of the configuration file is as follows:
# Automated Busy Discovery Webcam configuration
file
# ___________________________________________
#
# ********** sip ***********
local_port=6060
username=Rekha
hostname=cs.columbia.edu
min_duration=30
Location=123,Claremont,Ave,Room234,NY
Busy_status=Meeting
The local_port parameter contains the information about the port that the sip stack is listening on for incoming responses. The username host name parameters are used in the creation of the PUBLISH message such as for the SIP Request. The min_duration parameter specifies how often the program will run. The location parameter is set equal to a user's office room address, and finally the Busy_status parameter defines the activity sent in the PUBLISH message when there are multiple people in the room.
The primary objective of the webcam module is to process an image of the room to detect if more than one person is present. Multiple approaches in image processing were considered such as Motion Vectors and Object recognition. In the motion vectors approach, it is difficult to differentiate between two users who are sitting very close together, especially if they have similar physical characteristics.
On the other hand Object Recognition method is difficult to implement successfully, especially when the objects are human beings. For example, a user may make a small change to their physical appearance which may lead to incorrect results from the system.
The image processing method considered in the project, was the method of calculating the difference between the two images. In case there is little difference between the images, the difference image is a mostly black image. The program then calculates the mean over the difference image; if there were lots of differences between the snapshot and the fixed image , the mean would be above a certain experimentally derived threshold value. The threshold value method was found to be more flexible since the use of a range gives the program more leeway in regard to detected differences. This method of comparing images does not follow any fixed algorithm, it was found to be the most effective method for implementing this system.
Pre-requisites: A stored image which displays the usual setup and arrangement of the user and the room. This picture is taken using the CaptureImage program, which captures an image of the required size(300 x 240). The program then converts the image into a grey-scaled image.
The program takes a snapshot of the room using the CaptureImage program. Snapshots are taken periodically as defined in the configuration file. The image taken by the webcam is then converted into a gray-scaled image, since they given more precise results for comparison purposes.
The snapshot and permanent pictures are compared and the difference in backgrounds form a basis for deciding if there is more than one person in the room. The image processing is done by calculating the difference vectors of the two pictures. The mean of the resultant picture which is mostly black incase there are very few differences between the images. This mean is then compared against an experimentally derived threshold value. The threshold value is set so that the program does not give incorrect results in cases where the user moves.
In cases where the differences between two pictures is great, the mean of the difference image is higher than the threshold, in this case it is assumed that a meeting is in progress. Mean values have been found experimentally for cases where there is more than one person in the webcam range and when the room is empty. It is assumed that the webcam is placed in a office room which has artificial lighting available, so the webcam module would have enough light to function correctly. The threshold value is adjusted so that it does not trigger incorrectly incase there is a slight shift in light.
The program then calls the publishPIDF method of the PUBLISH class which is described in under the PUBLISH module. Incase there is a very big difference between the current settings and the stored image, the publish message states that a meeting is in progress. The PIDF skeleton structure is fixed with each module filling in information such as the location, activity of the user, username and whether they are busy or not, which is set in the <basic> xml element as closed or opened.
Sample Publish Message Sent by the program to the Presence Server
PUBLISH sip:Rekha@cs.columbia.edu SIP/2.0
Via: SIP/2.0/UDP honamsun.cs.columbia.edu:5060;rport;branch=z9hG4bK42589
Max-Forwards: 70
To: <sip:Rekha@cs.columbia.edu>
From: <sip:Rekha@cs.columbia.edu>;tag=z9hG4bK97843157
Call-ID: 652825108423@honamsun.cs.columbia.edu
CSeq: 1 PUBLISH
Contact: <sip:Rekha@cs.columbia.edu:6060>
Expires: 3600
User-Agent: mjsip stack 1.6
Event: presence
Content-Length: 797
Content-Type: application/cpim-pidf+xml
<?xml version="1.0" encoding="UTF-8"?>
<presence xmlns="urn:ietf:params:xml:ns:cpim-pidf"
xmlns:local="urn:automatedBusy:pidf-status-type"
xmlns:dm="urn:ietf:params:xml:ns:pidf:person"
xmlns:es="urn:ietf:params:xml:ns:pidf:status:rpid-status"
xmlns:et="urn:ietf:params:xml:ns:pidf:rpid-tuple"
xmlns:ts="urn:ietf:params:xml:ns:pidf:timed-status"
entity="pres:Rekha@cs.columbia.edu">
<tuple id="ee2924f3">
<status>
<basic>closed</basic>
<dm:person>
<et:class>Rekha</et:class>
<dm:status>
<es:activities>
<es:activity>Meeting</es:activity>
</es:activities>
</dm:status>
</dm:person>
<local:location>123,Claremont,Ave,Room234,NY</local:location>
</status>
</tuple>
</presence>
The webcam modules was tested by:
Running the application in different areas with different lighting and it was found that there was some variation in the results.
The application was tested with the user at different distances. The person has to be at a constant distance from the webcam, to get correct results.
A image with a user in the far background was compared with the image.
Test the program by running it in different locations.
The program is dependent on the distance between the user and the camera; incase the distance between the user and the camera for the standard picture is different from the picture taken for image comparison, the program may give incorrect results.
The program is also dependent on changes in lighting.
Learning image processing fundamentals.
Learning to work with Java Media Framework and Java Advanced Imaging.