Using Multiple threads for image processing with semaphores

Have you ever tried using OpenCV where a single frame of image is processed for different processes like template matching, blob analysis etc? Have you ever faced a problem where it takes a few seconds to update a new frame? If you are working on a problem that has any of the two above scenarios, the solution would be to use multiple threads in your code.

Why multiple threads?

Multiple threads allow you to process your image as individual processes. For e.g : Image capture and Image processing can be executed under different cores of your processor. This accelerates the performance of your system.

Problems related to multiple threads:

The main problem behind using multiple threads is that there may be variables that might be shared between threads. In the case of splitting the image capture and image processing, the captured frames are shared between two threads. This problem is overcome by using mutex locks or semaphores.

The concept of mutex locks and semaphores has been explained in simple English in this link. I created a header file using the information available from the link.

Asynchronous threads:

I faced another issue where my threads were two asynchronous processes. My image capture rate was 30 frames/sec but my image processing rate was 5 frames/sec. I had to use a circular buffer to save the remaining 25 images. The circular buffer examples have been very clearly explained in this link. I used the boost circular buffer libraries to store my captured frames.

I created a buffer size of 200 frames ( Just in case…. 😉 .

“Window freeze”

Even the simplest problem’s solution wouldn’t start working right away. I had issues related to the highgui windows stalling when I executed my code. I could capture the frames only for a few seconds and the windows would stall from updating the frames.

Earlier, the process flow was as shown in the figure below:

Stack overflow came to my rescue. I posted my question here.

So the person who answered my question asked me to do it like this:

The problem got resolved! Since the documentation was very scarce for multi-threaded opencv processing, I thought I must share my experience. I am also sharing the sample file and the additional header file required for trying this simple solution. You may download the files from here. Code header file

WordPress doesn’t allow .cpp files and .h files. I believe that my pdf files could be used a reference to write your own!

Advertisements

4 Responses to Using Multiple threads for image processing with semaphores

  1. David says:

    Hi, this is a great post.
    I’m working in a school’s project and I’ve downloaded code and the header file, but those are the same.
    Could you please link these files? I have to do something similar to your work and this is very difficult for me.
    thanks in advance.
    David

  2. Sam Rad says:

    Hi yamanoorsai,

    This is great. Thanks for sharing your experience. I’m trying to implement similar technique for an Android application which processes the camera feed in real time.

    Since it’s processing 5 frames per second and receiving 30 frames per second, we fall 25 frames behind in each second. Which means the buffer gets bigger and bigger and eventually out of memory. What do you think?

    Best regards,
    Sam

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: