文档结构  
翻译进度:82%     翻译赏金:0 元 (?)    ¥ 我要打赏
参与翻译: toypipi (5), ExDevilLee (4), CY2 (1)

照相机已经存在了很久很久。然而,在二十世纪末,随着便宜的针孔相机的引入,他们在我们的日常生活中屡见不鲜。不幸的是,这便宜的价格(意味着):明显的失真。幸运的是,这些都可以通过常量和校准然后重绘来修正他们。此外,随着校准你也会测定出照相机的自然单位(像素)与真实世界单位(例如:毫米)的关系。

原理

OpenCV 对于失真会考虑到径向因子和切向因子。对于径向因子的使用有一个如下的公式:

第 1 段(可获 2 积分)

x_{corrected} = x( 1 + k_1 r^2 + k_2 r^4 + k_3 r^6) \\ y_{corrected} = y( 1 + k_1 r^2 + k_2 r^4 + k_3 r^6)

因此对于在输入图像中的(x,y)坐标处的旧像素点,其在校正的输出图像上的位置将是(x_{corrected} y_{corrected})。径向变形的存在表现为“barrel”或“fish-eye”效应的形式。

由于图像拍摄镜头不完全平行于成像平面,因此发生切线失真。它可以通过以下公式校正:

x_{corrected} = x + [ 2p_1xy + p_2(r^2+2x^2)] \\ y_{corrected} = y + [ p_1(r^2+ 2y^2)+ 2p_2xy]

我们有五个失真参数,在OpenCV中表现为具有5列的一行矩阵:

Distortion_{coefficients}=(k_1 \hspace{10pt} k_2 \hspace{10pt} p_1 \hspace{10pt} p_2 \hspace{10pt} k_3)

对于单位转换,我们使用以下公式:

\left [ \begin{matrix} x \\ y \\ w \end{matrix} \right ] = \left [ \begin{matrix} f_x & 0 & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{matrix} \right ] \left [ \begin{matrix} X \\ Y \\ Z \end{matrix} \right ]

这里, w的存在通过使用单对应坐标系 (和w=Z)来解释。未知参数是 f_xf_y (相机焦距) ,并且(c_x, c_y) 是以像素坐标表示的光学中心。 如果对于两个轴,使用具有给定a纵横比(通常为1)的公共焦距 ,则在f_y=f_x*a 和上面的公式中,将具有单个焦距 f。 包含这四个参数的矩阵被称为相机矩阵。不管使用的相机分辨率是多少,失真系数都是相同的,它们应当与来自校准分辨率的当前分辨率一起缩放。

第 2 段(可获 2 积分)

确定这两个矩阵的过程称为校准。 通过基本几何方程计算这些参数。 所使用的方程式取决于所选择的校准对象。 目前OpenCV支持三种类型的校准对象:

  • 古典黑白棋盘
  • 对称的圆形图案
  • 不对称的圆形图案

基本上,你需要用你的相机拍摄这些模式的快照,并让OpenCV找到它们。 每个找到的模式产生一个新的方程。 为了求解方程,你需要至少预先设定数量的模式快照以形成合适的方程系统。 这个数字对于棋盘模式较高,对于圆形模式较少。 例如,在理论上棋盘模式需要至少两个快照。 然而,在实践中,我们在输入图像中存在大量的噪声,因此为了获得好的结果,您可能需要至少10张好的在不同位置的输入模式快照。

第 3 段(可获 2 积分)

目标

示例应用程序将:

  • 确定失真矩阵
  • 确定相机矩阵
  • 从相机,视频和图像文件列表中获取输入
  • 从XML/YAML 文件读取配置
  • 将结果保存到XML/YAML 文件中
  • 计算再投影误差

源代码

您还可以在OpenCV源库的samples/cpp/tutorial_code/calib3d/camera_calibration/文件夹中找到源代码,或从此处下载。该程序有一个参数:其配置文件的名称。如果没有给出,那么它将尝试打开一个名为“default.xml”的配置文件。这里有一个XML格式的示例配置文件。在配置文件中,您可以选择使用摄像头,视频文件或图像列表作为输入。如果选择最后一个,您将需要创建一个配置文件,枚举要使用的兔像。这里有一个例子。要记住的重要部分是,需要使用绝对路径或相对于应用程序工作目录的相对路径来指定图像。你可以在上面提到的samples目录中找到所有内容。

第 4 段(可获 2 积分)

The application starts up with reading the settings from the configuration file. Although, this is an important part of it, it has nothing to do with the subject of this tutorial: camera calibration. Therefore, I’ve chosen not to post the code for that part here. Technical background on how to do this you can find in the File Input and Output using XML and YAML files tutorial.

Explanation

  1. Read the settings.

    Settings s;
    const string inputSettingsFile = argc > 1 ? argv[1] : "default.xml";
    FileStorage fs(inputSettingsFile, FileStorage::READ); // Read the settings
    if (!fs.isOpened())
    {
          cout << "Could not open the configuration file: \"" << inputSettingsFile << "\"" << endl;
          return -1;
    }
    fs["Settings"] >> s;
    fs.release();                                         // close Settings file
    
    if (!s.goodInput)
    {
          cout << "Invalid input detected. Application stopping. " << endl;
          return -1;
    }
    

    For this I’ve used simple OpenCV class input operation. After reading the file I’ve an additional post-processing function that checks validity of the input. Only if all inputs are good then goodInput variable will be true.

  2. Get next input, if it fails or we have enough of them - calibrate. After this we have a big loop where we do the following operations: get the next image from the image list, camera or video file. If this fails or we have enough images then we run the calibration process. In case of image we step out of the loop and otherwise the remaining frames will be undistorted (if the option is set) via changing from DETECTION mode to the CALIBRATED one.

    for(int i = 0;;++i)
    {
      Mat view;
      bool blinkOutput = false;
    
      view = s.nextImage();
    
      //-----  If no more image, or got enough, then stop calibration and show result -------------
      if( mode == CAPTURING && imagePoints.size() >= (unsigned)s.nrFrames )
      {
            if( runCalibrationAndSave(s, imageSize,  cameraMatrix, distCoeffs, imagePoints))
                  mode = CALIBRATED;
            else
                  mode = DETECTION;
      }
      if(view.empty())          // If no more images then run calibration, save and stop loop.
      {
                if( imagePoints.size() > 0 )
                      runCalibrationAndSave(s, imageSize,  cameraMatrix, distCoeffs, imagePoints);
                break;
      imageSize = view.size();  // Format input image.
      if( s.flipVertical )    flip( view, view, 0 );
      }
    

    For some cameras we may need to flip the input image. Here we do this too.

  3. Find the pattern in the current input. The formation of the equations I mentioned above aims to finding major patterns in the input: in case of the chessboard this are corners of the squares and for the circles, well, the circles themselves. The position of these will form the result which will be written into the pointBuf vector.

    vector<Point2f> pointBuf;
    
    bool found;
    switch( s.calibrationPattern ) // Find feature points on the input format
    {
    case Settings::CHESSBOARD:
      found = findChessboardCorners( view, s.boardSize, pointBuf,
      CV_CALIB_CB_ADAPTIVE_THRESH | CV_CALIB_CB_FAST_CHECK | CV_CALIB_CB_NORMALIZE_IMAGE);
      break;
    case Settings::CIRCLES_GRID:
      found = findCirclesGrid( view, s.boardSize, pointBuf );
      break;
    case Settings::ASYMMETRIC_CIRCLES_GRID:
      found = findCirclesGrid( view, s.boardSize, pointBuf, CALIB_CB_ASYMMETRIC_GRID );
      break;
    }
    

    Depending on the type of the input pattern you use either the findChessboardCorners or the findCirclesGrid function. For both of them you pass the current image and the size of the board and you’ll get the positions of the patterns. Furthermore, they return a boolean variable which states if the pattern was found in the input (we only need to take into account those images where this is true!).

    Then again in case of cameras we only take camera images when an input delay time is passed. This is done in order to allow user moving the chessboard around and getting different images. Similar images result in similar equations, and similar equations at the calibration step will form an ill-posed problem, so the calibration will fail. For square images the positions of the corners are only approximate. We may improve this by calling the cornerSubPix function. It will produce better calibration result. After this we add a valid inputs result to the imagePoints vector to collect all of the equations into a single container. Finally, for visualization feedback purposes we will draw the found points on the input image using findChessboardCorners function.

    if ( found)                // If done with success,
      {
          // improve the found corners' coordinate accuracy for chessboard
            if( s.calibrationPattern == Settings::CHESSBOARD)
            {
                Mat viewGray;
                cvtColor(view, viewGray, CV_BGR2GRAY);
                cornerSubPix( viewGray, pointBuf, Size(11,11),
                  Size(-1,-1), TermCriteria( CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, 30, 0.1 ));
            }
    
            if( mode == CAPTURING &&  // For camera only take new samples after delay time
                (!s.inputCapture.isOpened() || clock() - prevTimestamp > s.delay*1e-3*CLOCKS_PER_SEC) )
            {
                imagePoints.push_back(pointBuf);
                prevTimestamp = clock();
                blinkOutput = s.inputCapture.isOpened();
            }
    
            // Draw the corners.
            drawChessboardCorners( view, s.boardSize, Mat(pointBuf), found );
      }
    
  4. Show state and result to the user, plus command line control of the application. This part shows text output on the image.

    //----------------------------- Output Text ------------------------------------------------
    string msg = (mode == CAPTURING) ? "100/100" :
              mode == CALIBRATED ? "Calibrated" : "Press 'g' to start";
    int baseLine = 0;
    Size textSize = getTextSize(msg, 1, 1, 1, &baseLine);
    Point textOrigin(view.cols - 2*textSize.width - 10, view.rows - 2*baseLine - 10);
    
    if( mode == CAPTURING )
    {
      if(s.showUndistorsed)
        msg = format( "%d/%d Undist", (int)imagePoints.size(), s.nrFrames );
      else
        msg = format( "%d/%d", (int)imagePoints.size(), s.nrFrames );
    }
    
    putText( view, msg, textOrigin, 1, 1, mode == CALIBRATED ?  GREEN : RED);
    
    if( blinkOutput )
       bitwise_not(view, view);
    

    If we ran calibration and got camera’s matrix with the distortion coefficients we may want to correct the image using undistort function:

    //------------------------- Video capture  output  undistorted ------------------------------
    if( mode == CALIBRATED && s.showUndistorsed )
    {
      Mat temp = view.clone();
      undistort(temp, view, cameraMatrix, distCoeffs);
    }
    //------------------------------ Show image and check for input commands -------------------
    imshow("Image View", view);
    

    Then we wait for an input key and if this is u we toggle the distortion removal, if it is g we start again the detection process, and finally for the ESC key we quit the application:

    char key =  waitKey(s.inputCapture.isOpened() ? 50 : s.delay);
    if( key  == ESC_KEY )
          break;
    
    if( key == 'u' && mode == CALIBRATED )
       s.showUndistorsed = !s.showUndistorsed;
    
    if( s.inputCapture.isOpened() && key == 'g' )
    {
      mode = CAPTURING;
      imagePoints.clear();
    }
    
  5. Show the distortion removal for the images too. When you work with an image list it is not possible to remove the distortion inside the loop. Therefore, you must do this after the loop. Taking advantage of this now I’ll expand the undistort function, which is in fact first calls initUndistortRectifyMap to find transformation matrices and then performs transformation using remap function. Because, after successful calibration map calculation needs to be done only once, by using this expanded form you may speed up your application:

    if( s.inputType == Settings::IMAGE_LIST && s.showUndistorsed )
    {
      Mat view, rview, map1, map2;
      initUndistortRectifyMap(cameraMatrix, distCoeffs, Mat(),
          getOptimalNewCameraMatrix(cameraMatrix, distCoeffs, imageSize, 1, imageSize, 0),
          imageSize, CV_16SC2, map1, map2);
    
      for(int i = 0; i < (int)s.imageList.size(); i++ )
      {
          view = imread(s.imageList[i], 1);
          if(view.empty())
              continue;
          remap(view, rview, map1, map2, INTER_LINEAR);
          imshow("Image View", rview);
          char c = waitKey();
          if( c  == ESC_KEY || c == 'q' || c == 'Q' )
              break;
      }
    }
    
第 5 段(可获 2 积分)

校准并保存

因为每个摄像机只需要进行一次校准,所以在成功校准后保存它是很有意义的。 这种方式使得你可以稍后加载这些值到你的程序。 因此,我们首先进行校准,如果成功,我们将结果保存为OpenCV样式的XML或YAML文件,具体取决于您在配置文件中给出的扩展名。

因此,在第一个函数中,我们只是拆分了这两个过程。 因为我们要保存许多校准变量,我们将在这里创建这些变量,并将它们传递到校准和保存功能。 再次,我不会显示保存部分,因为它与校准不太一样。 请浏览源文件,以了解如何做和做什么:

第 6 段(可获 2 积分)
bool runCalibrationAndSave(Settings& s, Size imageSize, Mat&  cameraMatrix, Mat& distCoeffs,vector<vector<Point2f> > imagePoints )
{
 vector<Mat> rvecs, tvecs;
 vector<float> reprojErrs;
 double totalAvgErr = 0;

 bool ok = runCalibration(s,imageSize, cameraMatrix, distCoeffs, imagePoints, rvecs, tvecs,
                          reprojErrs, totalAvgErr);
 cout << (ok ? "Calibration succeeded" : "Calibration failed")
     << ". avg re projection error = "  << totalAvgErr ;

 if( ok )   //  当且仅当该校验成功时保存 
     saveCameraParams( s, imageSize, cameraMatrix, distCoeffs, rvecs ,tvecs, reprojErrs,
                         imagePoints, totalAvgErr);
 return ok;
}
第 7 段(可获 2 积分)

We do the calibration with the help of the calibrateCamera function. It has the following parameters:

  • The object points. This is a vector of Point3f vector that for each input image describes how should the pattern look. If we have a planar pattern (like a chessboard) then we can simply set all Z coordinates to zero. This is a collection of the points where these important points are present. Because, we use a single pattern for all the input images we can calculate this just once and multiply it for all the other input views. We calculate the corner points with the calcBoardCornerPositions function as:

    void calcBoardCornerPositions(Size boardSize, float squareSize, vector<Point3f>& corners,
                      Settings::Pattern patternType /*= Settings::CHESSBOARD*/)
    {
    corners.clear();
    
    switch(patternType)
    {
    case Settings::CHESSBOARD:
    case Settings::CIRCLES_GRID:
      for( int i = 0; i < boardSize.height; ++i )
        for( int j = 0; j < boardSize.width; ++j )
            corners.push_back(Point3f(float( j*squareSize ), float( i*squareSize ), 0));
      break;
    
    case Settings::ASYMMETRIC_CIRCLES_GRID:
      for( int i = 0; i < boardSize.height; i++ )
         for( int j = 0; j < boardSize.width; j++ )
            corners.push_back(Point3f(float((2*j + i % 2)*squareSize), float(i*squareSize), 0));
      break;
    }
    }
    

    And then multiply it as:

    vector<vector<Point3f> > objectPoints(1);
    calcBoardCornerPositions(s.boardSize, s.squareSize, objectPoints[0], s.calibrationPattern);
    objectPoints.resize(imagePoints.size(),objectPoints[0]);
    
  • The image points. This is a vector of Point2f vector which for each input image contains coordinates of the important points (corners for chessboard and centers of the circles for the circle pattern). We have already collected this from findChessboardCorners or findCirclesGrid function. We just need to pass it on.

  • The size of the image acquired from the camera, video file or the images.

  • The camera matrix. If we used the fixed aspect ratio option we need to set the f_x to zero:

    cameraMatrix = Mat::eye(3, 3, CV_64F);
    if( s.flag & CV_CALIB_FIX_ASPECT_RATIO )
         cameraMatrix.at<double>(0,0) = 1.0;
    
  • The distortion coefficient matrix. Initialize with zero.

    distCoeffs = Mat::zeros(8, 1, CV_64F);
    
  • For all the views the function will calculate rotation and translation vectors which transform the object points (given in the model coordinate space) to the image points (given in the world coordinate space). The 7-th and 8-th parameters are the output vector of matrices containing in the i-th position the rotation and translation vector for the i-th object point to the i-th image point.

  • The final argument is the flag. You need to specify here options like fix the aspect ratio for the focal length, assume zero tangential distortion or to fix the principal point.

第 8 段(可获 2 积分)
double rms = calibrateCamera(objectPoints, imagePoints, imageSize, cameraMatrix,
                            distCoeffs, rvecs, tvecs, s.flag|CV_CALIB_FIX_K4|CV_CALIB_FIX_K5);
  • 该函数返回平均再投影误差。 这个数字对于发现参数的精度给出了良好的估计。 它应该尽可能接近于零。 给定内部,失真,旋转和平移矩阵,我们可以通过使用  projectPoints 首先将对象点变换到图像点来计算一个视图的误差。 然后我们计算变换所得和 corner/circle 搜索算法之间的绝对范数。 为了找到平均误差,我们计算对所有校准图像计算的误差的算术平均值。

    double computeReprojectionErrors( const vector<vector<Point3f> >& objectPoints,
                              const vector<vector<Point2f> >& imagePoints,
                              const vector<Mat>& rvecs, const vector<Mat>& tvecs,
                              const Mat& cameraMatrix , const Mat& distCoeffs,
                              vector<float>& perViewErrors)
    {
    vector<Point2f> imagePoints2;
    int i, totalPoints = 0;
    double totalErr = 0, err;
    perViewErrors.resize(objectPoints.size());
    
    for( i = 0; i < (int)objectPoints.size(); ++i )
    {
      projectPoints( Mat(objectPoints[i]), rvecs[i], tvecs[i], cameraMatrix,  // project
                                           distCoeffs, imagePoints2);
      err = norm(Mat(imagePoints[i]), Mat(imagePoints2), CV_L2);              // difference
    
      int n = (int)objectPoints[i].size();
      perViewErrors[i] = (float) std::sqrt(err*err/n);                        // save for this view
      totalErr        += err*err;                                             // sum it up
      totalPoints     += n;
    }
    
    return std::sqrt(totalErr/totalPoints);              // calculate the arithmetical mean
    }
    
第 9 段(可获 2 积分)

结果

让我们来输入一个尺寸为 9 X 6的 棋盘图案 。我用 AXIS IP 相机照了一些板子的快照并保存在了 VID5 目录下。我把他们放到我工作目录下的 images/CameraCalibration 文件夹中,并且创建了一个如下的 VID5.XML 文件,用来描述使用的图像:

<?xml version="1.0"?>
<opencv_storage>
<images>
images/CameraCalibration/VID5/xx1.jpg
images/CameraCalibration/VID5/xx2.jpg
images/CameraCalibration/VID5/xx3.jpg
images/CameraCalibration/VID5/xx4.jpg
images/CameraCalibration/VID5/xx5.jpg
images/CameraCalibration/VID5/xx6.jpg
images/CameraCalibration/VID5/xx7.jpg
images/CameraCalibration/VID5/xx8.jpg
</images>
</opencv_storage>
第 10 段(可获 2 积分)

然后把 images/CameraCalibration/VID5/VID5.XML 作为输入放到配置文件中。 当程序运行时,它会发现如下的棋盘图案:

A found chessboard

在应用了消除失真操作后,我们可以得到下图:

Distortion removal for File List

这也等同于对 这些非对称的圆形图案 设置宽为4,高为11的输入。这一次我用的是实时摄像机并指定它的 ID (“1”) 作为输入。在这里,可以看到它检测到的图案:

Asymmetrical circle detection

这两种情况,在指定的 XML/YAML (格式的)输出文件中,你将会找到照相机矩阵和失真系数矩阵:

<Camera_Matrix type_id="opencv-matrix">
<rows>3</rows>
<cols>3</cols>
<dt>d</dt>
<data>
 6.5746697944293521e+002 0. 3.1950000000000000e+002 0.
 6.5746697944293521e+002 2.3950000000000000e+002 0. 0. 1.</data></Camera_Matrix>
<Distortion_Coefficients type_id="opencv-matrix">
<rows>5</rows>
<cols>1</cols>
<dt>d</dt>
<data>
 -4.1802327176423804e-001 5.0715244063187526e-001 0. 0.
 -5.7843597214487474e-001</data></Distortion_Coefficients>

把这些值作为常量添加到你的程序中, 调用 initUndistortRectifyMapremap 函数来消除失真,接下来就享受便宜且低质量相机的无失真输入的乐趣吧。

你也可以在 这里 查看这个实例的运行情况。

第 11 段(可获 2 积分)

文章评论