imshow() and waitkey() from within a thread --> not updating the display

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

imshow() and waitkey() from within a thread --> not updating the display

nomis_ch
Hello !
General informations:
I am working on a framework to record and process multiple stream from
different sensors. (based on CRNT)I am actually implementing the
webcam/kinect part.
I am using OpenCV to retrieve and process images.Retrieving and saving
images is fine, but I have problem displaying the images in a window.
The framework si based on different modules (threads) each handling some
part of the information and sending it to a different module.
For webcam I coded the following modules: WebcamReader (to read images
from a camera), WebcamWriter (to save images as an avi file) and
WebcamVisualizer to display the video on the screen.
I use opencv 2.3.1, on macosx 10.6.8 with MacPorts.
Problem:
When I try to display the images, the first received frame is displayed
in the window but then the window is not updated anymore (but I still
receive new frames which are not displayed) !From what I read, the
problem is because the imshow and waitkey functions are called from
within a secondary thread, but I have not seen any solution.
Tried solutions
I tried to increase the time for waitkey(), to create the namedWindow at
different places to call sleep() to ensure the thread let enough tiem to
display the image.
Note that I made a small test with the  same code with a simple
application (within the main and not a thread), and the image gets
updated without problem.
Is there anyway I can force the update of the window through a signal or
anything ?
Code:

/*

  TO COMPLETE

  Module to display an image/video received through a connection stream.

  */

#include "WebcamVisualizer.h"

using namespace std;

WebcamVisualizer::WebcamVisualizer( ) : StreamTask(1,0)

{

//Constructor

cv::namedWindow("WebcamWindow", CV_WINDOW_AUTOSIZE);




}

WebcamVisualizer::WebcamVisualizer( const WebcamVisualizer& r) :
StreamTask(r)

{

log("Copy constructor");

}




WebcamVisualizer::~WebcamVisualizer()

{

log("WebcamVisualizer: released (destructor)");

//cv::destroyAllWindows();



}




void WebcamVisualizer::paramsChanged(){

log("WebcamVisualizer:  params changed");

}




void WebcamVisualizer::run()

{

//INIT

printf("Webcam: Starting visualizer thread\n");





DataPacket *p;

InPort *inPort = inPorts[0];



//init variables for time statistics

struct timeval start, end, actual;

long mtime, seconds, useconds;






//Get an image to initialize parameters

p = inPort->receive();

if(p==NULL){

printf("WebCamVisualizer: Failed to initialize resolution with received
image");

}else{

//do something with p

Value *val = p->dataVector[0];

cv::Mat mat = (dynamic_cast<CvMatValue*>(val))->getCvMat();



}



int nFrame = 0;



while (running) {

//Read inPort

if(p == NULL){

if(inPort->notEmpty()){

p=inPort->receive();

}

}else{

gettimeofday(&start, NULL);

//Read header



//Read cv::Mat

Value *val = p->dataVector[0];




//Display image

cv::imshow("WebcamWindow",
(dynamic_cast<CvMatValue*>(val))->getCvMat());



cv::waitKey(20);



gettimeofday(&end, NULL);



if(debug){

seconds = end.tv_sec - start.tv_sec;

useconds= end.tv_usec - start.tv_usec;

mtime += ((seconds)*1000 + useconds/1000.0)+0.5;

}

delete p;

p=NULL;

nFrame++;

}



}

if(debug){

//printf("WebCamWriter: received #%d frames, displayed with %dx%d
resolution  \n", nFrame, resolutionW, resolutionH);

printf("WebCamWriter: Average time taken pro frame: %.2f
milliseconds\n", (mtime/((double)nFrame)));

fflush(stdout);

}

log("Running Visualizer finished succesfully");

}


Reply | Threaded
Open this post in threaded view
|

Re: imshow() and waitkey() from within a thread --> not updating the display

nomis_ch
No one has a hint about that problem ?

Reply | Threaded
Open this post in threaded view
|

Re: imshow() and waitkey() from within a thread --> not updating the display

nomis_ch
In reply to this post by nomis_ch
No one as a hint about this problem ?

Reply | Threaded
Open this post in threaded view
|

Re: imshow() and waitkey() from within a thread --> not updating the display

Shervin Emami
In reply to this post by nomis_ch
Many GUI systems only let you display things on the screen and get user
input on the main thread. So try just using imshow & waitkey from the main
thread (the one that your program starts on).

Cheers,
Shervin Emami.
Mobile Computer Vision Engineer, NVIDIA.
http://www.shervinemami.info/openCV.html


On Mon, Oct 24, 2011 at 7:40 PM, nomis_ch <[hidden email]> wrote:

> **
>
>
> No one has a hint about that problem ?
>
>  
>
Reply | Threaded
Open this post in threaded view
|

Re: imshow() and waitkey() from within a thread --> not updating the display

nomis_ch
Hello !

Thanks for the hint.
Sadly it would be too problematic to put those calls in the main thread. It would involve removing some of the modularity of the framework; something I want to avoid as much as possible.

What I do now is that I use the CImg library to display the images. A solution that is not very clean as I have to convert the cv::Mat to CImg before each display.

No solution to have imshow() and waitkey() working from secondary threads ? Anything like Callbacks methods or signals ?

Cheers


--- In [hidden email], Shervin Emami <shervin.emami@...> wrote:

>
> Many GUI systems only let you display things on the screen and get user
> input on the main thread. So try just using imshow & waitkey from the main
> thread (the one that your program starts on).
>
> Cheers,
> Shervin Emami.
> Mobile Computer Vision Engineer, NVIDIA.
> http://www.shervinemami.info/openCV.html
>
>
> On Mon, Oct 24, 2011 at 7:40 PM, nomis_ch <simon.ruffieux@...> wrote:
>
> > **
> >
> >
> > No one has a hint about that problem ?
> >
> >  
> >
>


Reply | Threaded
Open this post in threaded view
|

Re: imshow() and waitkey() from within a thread --> not updating the display

vincent_ravier
Hi,
I've experienced a quite similar problem, but while working on a single thread. It's referenced in this post http://tech.groups.yahoo.com/group/OpenCV/message/82672 as a code example, but actually it's for a bigger app purpose.
The only I've found was in an article, mentionning it was not really a proper way to do this (and I agree ^^). I force window refresh with those 2 lines:

HWND hWnd = (HWND)cvGetWindowHandle(OPENCV_WINDOW_NAME);
::SendMessage(hWnd, WM_PAINT, 0, 0);

It doesn't seem OpenCV's highgui module has been developped to manage real time image display in an effective way, but just to display images for debug purpose, which is actually done very simply but putting 2 lines wherever in your code, and it's fine.
But just write a program which, for example, increase or decrease some Canny edge value and displays the result when you press a specific key. Now keep that key pressed, you will see the result when releasing it, but no refresh while you keep it pressed.
However, strange thing is it works fine while grabbing images from a webcam.

Well, I can't really give you any solution for this problem, but somehow I'm relieved not to be the only one to have experienced this.

My 2 cents.


Reply | Threaded
Open this post in threaded view
|

Re: imshow() and waitkey() from within a thread --> not updating the display

Gus K Lott v3.0
In reply to this post by nomis_ch
Move the NamedWindow call into the run method.  I've had success in the
past with creating the window in the thread in which you update it using
the imshow and waitkey calls.

Good Luck

On Thu, Oct 27, 2011 at 4:23 AM, nomis_ch <[hidden email]> wrote:

> **
>
>
> Hello !
>
> Thanks for the hint.
> Sadly it would be too problematic to put those calls in the main thread.
> It would involve removing some of the modularity of the framework;
> something I want to avoid as much as possible.
>
> What I do now is that I use the CImg library to display the images. A
> solution that is not very clean as I have to convert the cv::Mat to CImg
> before each display.
>
> No solution to have imshow() and waitkey() working from secondary threads
> ? Anything like Callbacks methods or signals ?
>
> Cheers
>
>
> --- In [hidden email], Shervin Emami <shervin.emami@...> wrote:
> >
> > Many GUI systems only let you display things on the screen and get user
> > input on the main thread. So try just using imshow & waitkey from the
> main
> > thread (the one that your program starts on).
> >
> > Cheers,
> > Shervin Emami.
> > Mobile Computer Vision Engineer, NVIDIA.
> > http://www.shervinemami.info/openCV.html
> >
> >
> > On Mon, Oct 24, 2011 at 7:40 PM, nomis_ch <simon.ruffieux@...> wrote:
> >
> > > **
>
> > >
> > >
> > > No one has a hint about that problem ?
> > >
> > >
> > >
> >
>
>  
>



--
Gus K Lott III, PhD, PMP
Reply | Threaded
Open this post in threaded view
|

Re: imshow() and waitkey() from within a thread --> not updating the display

Sanjiv K. Bhatia
This was a very helpful post.  Wonder if this should be included in the documentation.  This will save a lot of time.

Is there an FAQ for OpenCV?  If not, is it time to start one?

Sanjiv

--- In [hidden email], "Gus K Lott v3.0" <guslott@...> wrote:

>
> Move the NamedWindow call into the run method.  I've had success in the
> past with creating the window in the thread in which you update it using
> the imshow and waitkey calls.
>
> Good Luck
>
> On Thu, Oct 27, 2011 at 4:23 AM, nomis_ch <simon.ruffieux@...> wrote:
>
> > **
> >
> >
> > Hello !
> >
> > Thanks for the hint.
> > Sadly it would be too problematic to put those calls in the main thread.
> > It would involve removing some of the modularity of the framework;
> > something I want to avoid as much as possible.
> >
> > What I do now is that I use the CImg library to display the images. A
> > solution that is not very clean as I have to convert the cv::Mat to CImg
> > before each display.
> >
> > No solution to have imshow() and waitkey() working from secondary threads
> > ? Anything like Callbacks methods or signals ?
> >
> > Cheers
> >
> >
> > --- In [hidden email], Shervin Emami <shervin.emami@> wrote:
> > >
> > > Many GUI systems only let you display things on the screen and get user
> > > input on the main thread. So try just using imshow & waitkey from the
> > main
> > > thread (the one that your program starts on).
> > >
> > > Cheers,
> > > Shervin Emami.
> > > Mobile Computer Vision Engineer, NVIDIA.
> > > http://www.shervinemami.info/openCV.html
> > >
> > >
> > > On Mon, Oct 24, 2011 at 7:40 PM, nomis_ch <simon.ruffieux@> wrote:
> > >
> > > > **
> >
> > > >
> > > >
> > > > No one has a hint about that problem ?
> > > >
> > > >
> > > >
> > >
> >
> >  
> >
>
>
>
> --
> Gus K Lott III, PhD, PMP
>


Reply | Threaded
Open this post in threaded view
|

Re: imshow() and waitkey() from within a thread --> not updating the display

Shervin Emami
There is an official wiki (but old) at "
http://opencv.willowgarage.com/wiki/FullOpenCVWiki" and an old FAQ at "
http://opencv.willowgarage.com/wiki/faq". I agree that there should be a
better place for people to keep OpenCV knowledge. Some people have been
trying to start a "StackOverflow" type website but it needs more people to
join before it can start, so please do: "
http://area51.stackexchange.com/proposals/11036/computer-vision?referrer=HYTKvJuWQIHZ4DM45o_rrw2
"

Cheers,
Shervin Emami.
Mobile Computer Vision Engineer, NVIDIA.
http://www.shervinemami.info/openCV.html


On Fri, Nov 18, 2011 at 6:46 AM, skbnerd <[hidden email]> wrote:

> **
>
>
> This was a very helpful post. Wonder if this should be included in the
> documentation. This will save a lot of time.
>
> Is there an FAQ for OpenCV? If not, is it time to start one?
>
> Sanjiv
>
> --- In [hidden email], "Gus K Lott v3.0" <guslott@...> wrote:
> >
> > Move the NamedWindow call into the run method. I've had success in the
> > past with creating the window in the thread in which you update it using
> > the imshow and waitkey calls.
> >
> > Good Luck
> >
> > On Thu, Oct 27, 2011 at 4:23 AM, nomis_ch <simon.ruffieux@...> wrote:
> >
> > > **
> > >
> > >
> > > Hello !
> > >
> > > Thanks for the hint.
> > > Sadly it would be too problematic to put those calls in the main
> thread.
> > > It would involve removing some of the modularity of the framework;
> > > something I want to avoid as much as possible.
> > >
> > > What I do now is that I use the CImg library to display the images. A
> > > solution that is not very clean as I have to convert the cv::Mat to
> CImg
> > > before each display.
> > >
> > > No solution to have imshow() and waitkey() working from secondary
> threads
> > > ? Anything like Callbacks methods or signals ?
> > >
> > > Cheers
> > >
> > >
> > > --- In [hidden email], Shervin Emami <shervin.emami@> wrote:
> > > >
> > > > Many GUI systems only let you display things on the screen and get
> user
> > > > input on the main thread. So try just using imshow & waitkey from the
> > > main
> > > > thread (the one that your program starts on).
> > > >
> > > > Cheers,
> > > > Shervin Emami.
> > > > Mobile Computer Vision Engineer, NVIDIA.
> > > > http://www.shervinemami.info/openCV.html
> > > >
> > > >
> > > > On Mon, Oct 24, 2011 at 7:40 PM, nomis_ch <simon.ruffieux@> wrote:
> > > >
> > > > > **
> > >
> > > > >
> > > > >
> > > > > No one has a hint about that problem ?
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
> > --
> > Gus K Lott III, PhD, PMP
> >
>
>  
>
Reply | Threaded
Open this post in threaded view
|

Re: imshow() and waitkey() from within a thread --> not updating the display

peb aryan
Well,

the other solution might be not using highgui for multithreaded
applications (copy to native GUI bitmap). I'm using C# (opencvsharp),
WinForms, and threading. it can show highgui window (cvShowImage), but i
must call the underlying GUI to ProcessMessages. the drawback is it can
only show the first video. if i change the video file parameter (thus,
creating another thread), it won't show highgui window again.

HTH

On Mon, Nov 21, 2011 at 5:03 AM, Shervin Emami <[hidden email]>wrote:

> **
>
>
> There is an official wiki (but old) at "
> http://opencv.willowgarage.com/wiki/FullOpenCVWiki" and an old FAQ at "
> http://opencv.willowgarage.com/wiki/faq". I agree that there should be a
> better place for people to keep OpenCV knowledge. Some people have been
> trying to start a "StackOverflow" type website but it needs more people to
> join before it can start, so please do: "
> http://area51.stackexchange.com/proposals/11036/computer-vision?referrer=HYTKvJuWQIHZ4DM45o_rrw2
> "
>
>
> Cheers,
> Shervin Emami.
> Mobile Computer Vision Engineer, NVIDIA.
> http://www.shervinemami.info/openCV.html
>
>
> On Fri, Nov 18, 2011 at 6:46 AM, skbnerd <[hidden email]> wrote:
>
>> **
>>
>>
>> This was a very helpful post. Wonder if this should be included in the
>> documentation. This will save a lot of time.
>>
>> Is there an FAQ for OpenCV? If not, is it time to start one?
>>
>> Sanjiv
>>
>> --- In [hidden email], "Gus K Lott v3.0" <guslott@...> wrote:
>> >
>> > Move the NamedWindow call into the run method. I've had success in the
>> > past with creating the window in the thread in which you update it using
>> > the imshow and waitkey calls.
>> >
>> > Good Luck
>> >
>> > On Thu, Oct 27, 2011 at 4:23 AM, nomis_ch <simon.ruffieux@...> wrote:
>> >
>> > > **
>> > >
>> > >
>> > > Hello !
>> > >
>> > > Thanks for the hint.
>> > > Sadly it would be too problematic to put those calls in the main
>> thread.
>> > > It would involve removing some of the modularity of the framework;
>> > > something I want to avoid as much as possible.
>> > >
>> > > What I do now is that I use the CImg library to display the images. A
>> > > solution that is not very clean as I have to convert the cv::Mat to
>> CImg
>> > > before each display.
>> > >
>> > > No solution to have imshow() and waitkey() working from secondary
>> threads
>> > > ? Anything like Callbacks methods or signals ?
>> > >
>> > > Cheers
>> > >
>> > >
>> > > --- In [hidden email], Shervin Emami <shervin.emami@> wrote:
>> > > >
>> > > > Many GUI systems only let you display things on the screen and get
>> user
>> > > > input on the main thread. So try just using imshow & waitkey from
>> the
>> > > main
>> > > > thread (the one that your program starts on).
>> > > >
>> > > > Cheers,
>> > > > Shervin Emami.
>> > > > Mobile Computer Vision Engineer, NVIDIA.
>> > > > http://www.shervinemami.info/openCV.html
>> > > >
>> > > >
>> > > > On Mon, Oct 24, 2011 at 7:40 PM, nomis_ch <simon.ruffieux@> wrote:
>> > > >
>> > > > > **
>> > >
>> > > > >
>> > > > >
>> > > > > No one has a hint about that problem ?
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> >
>> >
>> >
>> > --
>> > Gus K Lott III, PhD, PMP
>> >
>>
>>
>  
>