2D to 3D projection, with given plane

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

2D to 3D projection, with given plane

susanne_stutz
Hey guys,

I'm trying to get 3d coordinates of 2d image points using one camera, whereas the real world object points all lie on the same plane (the floor).

To obtain a set of object points I used a chessboard lying on the floor and findChessboardCorners().
Till now I have calibrated the camera using calibrateCamera(), so I have the intrinsic camera parameters, distortion coefficients, rotation vectors and translation vectors.
With these results I built the camera matrix (containing the intrinsic parameters) and the joint rotation-translation matrix. The first idea was to multiply these matrices and invert the result to get the inverse projection matrix (2d->3d).
The problem is that this matrix of course is not invertible (it is a 3x4 matrix).

How can I now use the fact that my object points all lie on the same plane (namely the plane of the chessboard), to build a transformation matrix which gives me the 3d coordinates of a given 2d image point?

Thanks in advance,
Susanne

Reply | Threaded
Open this post in threaded view
|

Re: 2D to 3D projection, with given plane

pavan shinde
You can calculate inverse of 3x4 non-square matrix by SVD method. 
cvInvert(const CvArr* src, CvArr* dst, int method=CV_SVD); in OpenCV version >2.0
 
Best regards,
Pavan Shinde


________________________________
 From: susanne_stutz <[hidden email]>
To: [hidden email]
Sent: Friday, 10 February 2012 5:38 PM
Subject: [OpenCV] 2D to 3D projection, with given plane
 

 
Hey guys,

I'm trying to get 3d coordinates of 2d image points using one camera, whereas the real world object points all lie on the same plane (the floor).

To obtain a set of object points I used a chessboard lying on the floor and findChessboardCorners().
Till now I have calibrated the camera using calibrateCamera(), so I have the intrinsic camera parameters, distortion coefficients, rotation vectors and translation vectors.
With these results I built the camera matrix (containing the intrinsic parameters) and the joint rotation-translation matrix. The first idea was to multiply these matrices and invert the result to get the inverse projection matrix (2d->3d).
The problem is that this matrix of course is not invertible (it is a 3x4 matrix).

How can I now use the fact that my object points all lie on the same plane (namely the plane of the chessboard), to build a transformation matrix which gives me the 3d coordinates of a given 2d image point?

Thanks in advance,
Susanne


 
Reply | Threaded
Open this post in threaded view
|

Re: 2D to 3D projection, with given plane

susanne_stutz
Ok thanks, now I have an inverse projection matrix (4x3 matrix).

Is it correct to just multiply this matrix with the vector of the 2D point (u, v, 1) in order to obtain the 3D world point (X, Y, Z, 1)?

The remaining question is:
At which point of calculation do I consider the information that the 3D world points all lie on the plane of the chessboard?
Do I have to determine the camera's position relative to the chessboard?

Has anybody done the calculation of 3D world points, lying on a known plane, given 2D image points before and wants to tell about his or her experience?

Thanks in advance,
Susanne

--- In [hidden email], Pawan Shinde <discoverpvn@...> wrote:

>
> You can calculate inverse of 3x4 non-square matrix by SVD method. 
> cvInvert(const CvArr* src, CvArr* dst, int method=CV_SVD); in OpenCV version >2.0
>  
> Best regards,
> Pavan Shinde
>
>
> ________________________________
>  From: susanne_stutz <stutz1977@...>
> To: [hidden email]
> Sent: Friday, 10 February 2012 5:38 PM
> Subject: [OpenCV] 2D to 3D projection, with given plane
>  
>
>  
> Hey guys,
>
> I'm trying to get 3d coordinates of 2d image points using one camera, whereas the real world object points all lie on the same plane (the floor).
>
> To obtain a set of object points I used a chessboard lying on the floor and findChessboardCorners().
> Till now I have calibrated the camera using calibrateCamera(), so I have the intrinsic camera parameters, distortion coefficients, rotation vectors and translation vectors.
> With these results I built the camera matrix (containing the intrinsic parameters) and the joint rotation-translation matrix. The first idea was to multiply these matrices and invert the result to get the inverse projection matrix (2d->3d).
> The problem is that this matrix of course is not invertible (it is a 3x4 matrix).
>
> How can I now use the fact that my object points all lie on the same plane (namely the plane of the chessboard), to build a transformation matrix which gives me the 3d coordinates of a given 2d image point?
>
> Thanks in advance,
> Susanne
>


Reply | Threaded
Open this post in threaded view
|

Re: 2D to 3D projection, with given plane

susanne_stutz
In reply to this post by pavan shinde
Ok thanks, now I have the inverse projection matrix (4x3 matrix).

Is it correct to just multiply this matrix with the vector of the 2D point (u, v, 1) in order to obtain the 3D world point (X, Y, Z, 1)?

The remaining question is:
At which point of calculation do I consider the information that the 3D world points all lie on the plane of the chessboard?
Do I have to determine the camera's position relative to the chessboard?

Has anybody done the calculation of 3D world points, lying on a known plane, given 2D image points before and wants to tell about his or her experience?

Thanks in advance,
Susanne



--- In [hidden email], Pawan Shinde <discoverpvn@...> wrote:

>
> You can calculate inverse of 3x4 non-square matrix by SVD method. 
> cvInvert(const CvArr* src, CvArr* dst, int method=CV_SVD); in OpenCV version >2.0
>  
> Best regards,
> Pavan Shinde
>
>
> ________________________________
>  From: susanne_stutz <stutz1977@...>
> To: [hidden email]
> Sent: Friday, 10 February 2012 5:38 PM
> Subject: [OpenCV] 2D to 3D projection, with given plane
>  
>
>  
> Hey guys,
>
> I'm trying to get 3d coordinates of 2d image points using one camera, whereas the real world object points all lie on the same plane (the floor).
>
> To obtain a set of object points I used a chessboard lying on the floor and findChessboardCorners().
> Till now I have calibrated the camera using calibrateCamera(), so I have the intrinsic camera parameters, distortion coefficients, rotation vectors and translation vectors.
> With these results I built the camera matrix (containing the intrinsic parameters) and the joint rotation-translation matrix. The first idea was to multiply these matrices and invert the result to get the inverse projection matrix (2d->3d).
> The problem is that this matrix of course is not invertible (it is a 3x4 matrix).
>
> How can I now use the fact that my object points all lie on the same plane (namely the plane of the chessboard), to build a transformation matrix which gives me the 3d coordinates of a given 2d image point?
>
> Thanks in advance,
> Susanne
>


Reply | Threaded
Open this post in threaded view
|

Re: 2D to 3D projection, with given plane

avanindra singh
In reply to this post by susanne_stutz
Hi,

I don't think you would need even calibration information to compute 3d co-ordinates of the image points on the plane.

You just need the 3d-2d homography ( which is actually used internally in cvCalibrateCamera. )

I beleive , in calibration process you would have set all the z co-ordinates to some constant( e.g. 0 ) and passed 2d and 3d points in that calibrate camera method.

For your problem you don't need to compute camera matrix. Just take (X , Y ) of ( X ,Y , Z ) co-ordinates of 3d points and compute homography against the image points.

Let's say it come out to be H.

so 3D co-ordinate for any point p( x , y ) would be ( H * p( x , y ) , Z ).

Hope this helps.

avanindra singh.

--- In [hidden email], "susanne_stutz" <stutz1977@...> wrote:

>
> Hey guys,
>
> I'm trying to get 3d coordinates of 2d image points using one camera, whereas the real world object points all lie on the same plane (the floor).
>
> To obtain a set of object points I used a chessboard lying on the floor and findChessboardCorners().
> Till now I have calibrated the camera using calibrateCamera(), so I have the intrinsic camera parameters, distortion coefficients, rotation vectors and translation vectors.
> With these results I built the camera matrix (containing the intrinsic parameters) and the joint rotation-translation matrix. The first idea was to multiply these matrices and invert the result to get the inverse projection matrix (2d->3d).
> The problem is that this matrix of course is not invertible (it is a 3x4 matrix).
>
> How can I now use the fact that my object points all lie on the same plane (namely the plane of the chessboard), to build a transformation matrix which gives me the 3d coordinates of a given 2d image point?
>
> Thanks in advance,
> Susanne
>