Updating Pixels with Optical Flow

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Updating Pixels with Optical Flow

lalithrafernando
Hello all,

I'm trying to create a system that uses optical flow to see where
objects in a video travel.  Then it would color the pixel that moved
the same color as the pixel that moved from the previous frame.  The
previous frame would have been colored as specified by an image.  It
would ideally be somewhat similar to the system they used to make A
Scanner Darkly.

So it would spread the coloring specified by the image throughout the
video.

I'm having a problem with the coloring part.  I use the following to
calculate the optical flow (in case it matters):

        cvCalcOpticalFlowPyrLK( frame1_1C, frame2_1C, pyramid1,
                pyramid2, frame1Features, frame2Features,
                numFeatures, opticalFlowWindow, 5,
                opticalFlowFoundFeature, opticalFlowFeatureError,
                opticalFlowTerminationCriteria, 0 );

Then I iterate through the found features, if they exist, like so:

        CvPoint p,q;
        p.x = (int) frame1Features[i].x;
        p.y = (int) frame1Features[i].y;
        q.x = (int) frame2Features[i].x;
        q.y = (int) frame2Features[i].y;

        uchar* tempFillPixPtr = &((uchar*)(imgFill->imageData +
imgFill->widthStep*p.y))[p.x*3];
        uchar* tempFramePixPtr = &((uchar*)(frame2->imageData +
frame2->widthStep*q.y))[q.x*3];

        /*****CRASHES AFTER THIS POINT*************
        (after multiple times through this part****/
        tempFramePixPtr[0] = 100;//tempFillPixPtr[0];
        tempFramePixPtr[1] = 100;//tempFillPixPtr[1];
        tempFramePixPtr[2] = 100;//tempFillPixPtr[2];

Unfortunately, after writing 122 frames successfully, it crashes
nearly halfway (135 of 300) through the features of the next frame, at
the specified point above.

I have no idea what's wrong... Since it passed this point many times
before, maybe it's the video?  Unfortunately, this is the only video
openCV has accepted and I can't seem to generate any other video's it
likes. I've tried wmv, avi, mov with multiple codecs (using Sony Vegas
6).  The input video (an avi) came with some sample code I adapted.

Anyone have any idea what I could be doing wrong?

Thanks a lot!
Lalithra

Reply | Threaded
Open this post in threaded view
|

Re: Updating Pixels with Optical Flow

lordfinesse6
--- In [hidden email], "lalithrafernando" <singha321@...> wrote:

>
> Hello all,
>
> I'm trying to create a system that uses optical flow to see where
> objects in a video travel.  Then it would color the pixel that moved
> the same color as the pixel that moved from the previous frame.  The
> previous frame would have been colored as specified by an image.  It
> would ideally be somewhat similar to the system they used to make A
> Scanner Darkly.
>
> So it would spread the coloring specified by the image throughout the
> video.
>
> I'm having a problem with the coloring part.  I use the following to
> calculate the optical flow (in case it matters):
>
> cvCalcOpticalFlowPyrLK( frame1_1C, frame2_1C, pyramid1,
> pyramid2, frame1Features, frame2Features,
> numFeatures, opticalFlowWindow, 5,
> opticalFlowFoundFeature, opticalFlowFeatureError,
> opticalFlowTerminationCriteria, 0 );
>
> Then I iterate through the found features, if they exist, like so:
>
> CvPoint p,q;
> p.x = (int) frame1Features[i].x;
> p.y = (int) frame1Features[i].y;
> q.x = (int) frame2Features[i].x;
> q.y = (int) frame2Features[i].y;
>
> uchar* tempFillPixPtr = &((uchar*)(imgFill->imageData +
> imgFill->widthStep*p.y))[p.x*3];
> uchar* tempFramePixPtr = &((uchar*)(frame2->imageData +
> frame2->widthStep*q.y))[q.x*3];
>
>         /*****CRASHES AFTER THIS POINT*************
>         (after multiple times through this part****/
> tempFramePixPtr[0] = 100;//tempFillPixPtr[0];
> tempFramePixPtr[1] = 100;//tempFillPixPtr[1];
> tempFramePixPtr[2] = 100;//tempFillPixPtr[2];
>
> Unfortunately, after writing 122 frames successfully, it crashes
> nearly halfway (135 of 300) through the features of the next frame, at
> the specified point above.
>
> I have no idea what's wrong... Since it passed this point many times
> before, maybe it's the video?  Unfortunately, this is the only video
> openCV has accepted and I can't seem to generate any other video's it
> likes. I've tried wmv, avi, mov with multiple codecs (using Sony Vegas
> 6).  The input video (an avi) came with some sample code I adapted.
>
> Anyone have any idea what I could be doing wrong?
>
> Thanks a lot!
> Lalithra
>

I don't understand (and I don't like) this part of the code:
uchar* tempFillPixPtr = &((uchar*)(imgFill->imageData +
imgFill->widthStep*p.y))[p.x*3];
uchar* tempFramePixPtr = &((uchar*)(frame2->imageData +
frame2->widthStep*q.y))[q.x*3];
Two advices:
1)Use the function cvGet2D and cvSet2D to access pixels (if you are
using color images), or the macro CV_IMAGE_ELEM(if you are using
gray-level images);
2)Step into the code with a debugger

Giulio