I’m setting up some cameras now in Gazebo and I haven’t found any documentation on what exactly does the camera sensor position mean. Should I place the sensor in the middle of the CMOS/CCD chip of the modeled real camera (which would be the image plane, also called focal plane for objects in infinity)? Or in the focal point in front of the image plane? Or somewhere in the lens?
I tend to put it in the focal plane, but I’m not sure whether that’s right.
Does anyone know?
Hmmmm my guess for practical purposes is wherever it is, it just has to be so that when you multiply the camera intrinsic matrix, you get the right projection, and when you multiply by the extrinsic matrix, you get to where the camera is. Those are the things that the camera pose would be used for, right?
The conventions in ROS are documented in sensor_msgs/CameraInfo.
So if you get the right center (cx, cy) and focal lengths fx and fy from calibration, then that’s all that matters?
Without knowing the correct answer to your question, I’d guess if you have the calibrated matrix from the real camera – though it’ll be different for each camera anyway – that could be used for sanity check of obvious errors. Not sure how much effort it’d be.
I think I figured it out (theoretically, not looking into code).
It seems that the parameter of interest here is called “Entrance pupil position”. It is a parameter lens makers might publish (e.g. Basler usually does). And according to wikipedia, if you subtract this distance from the out-most vertex of the last pupil (lens), you get exactly the point which is the center of the lens’ perspective projection, and e.g. the point around which you should rotate the camera when shooting panoramic images. I think this should satisfy Gazebo needs (and as there is no focus simulation, it’s not a problem that this formula holds only when focused to infinity).
If your lens doesn’t have this parameter published, I came with an approximation. Measure the radius of the out-most pupil (lens) on your lens, and divide it by tan(diagonal fov). This should give you coordinates of a vertex from which, if you look through a fov-wide cone, the cone will intersect exactly the circumference of the out-most pupil.
A slightly related issue (even if in our case we had more doubts about the orientation) is Missing documentation for what concerns camera sensors frame · Issue #425 · osrf/sdformat · GitHub .