CVTracked is the pure virtual base class for tracking features in video. More...

#include <CVTracked.h>

Inheritance diagram for CVTracked:

Public Member Functions
	CVTracked ()

virtual	~CVTracked ()=default

virtual bool	track (CVMat imageGray, CVMat imageBgr, CVCalibration *calib)=0

void	drawDetection (bool draw)

bool	isVisible ()

bool	drawDetection ()

CVMatx44f	objectViewMat ()

Static Public Member Functions
static cv::Matx44f	createGLMatrix (const CVMat &tVec, const CVMat &rVec)
	Create an OpenGL 4x4 matrix from an OpenCV translation & rotation vector. More...

static void	createRvecTvec (const CVMatx44f &glMat, CVMat &tVec, CVMat &rVec)
	Creates the OpenCV rvec & tvec vectors from an column major OpenGL 4x4 matrix. More...

static CVMatx44f	calcObjectMatrix (const CVMatx44f &cameraObjectMat, const CVMatx44f &objectViewMat)

static CVVec3f	averageVector (vector< CVVec3f > vectors, vector< float > weights)

static SLQuat4f	averageQuaternion (vector< SLQuat4f > quaternions, vector< float > weights)

static void	resetTimes ()
	Resets all static variables. More...

Static Public Attributes
static AvgFloat	trackingTimesMS
	Averaged time for video tracking in ms. More...

static AvgFloat	detectTimesMS
	Averaged time for video feature detection & description in ms. More...

static AvgFloat	detect1TimesMS
	Averaged time for video feature detection subpart 1 in ms. More...

static AvgFloat	detect2TimesMS
	Averaged time for video feature detection subpart 2 in ms. More...

static AvgFloat	matchTimesMS
	Averaged time for video feature matching in ms. More...

static AvgFloat	optFlowTimesMS
	Averaged time for video feature optical flow tracking in ms. More...

static AvgFloat	poseTimesMS
	Averaged time for video feature pose estimation in ms. More...

Protected Attributes
bool	_isVisible
	Flag if marker is visible. More...

bool	_drawDetection
	Flag if detection should be drawn into image. More...

CVMatx44f	_objectViewMat
	view transformation matrix More...

HighResTimer	_timer
	High resolution timer. More...

Detailed Description

CVTracked is the pure virtual base class for tracking features in video.

The static vector trackers can hold multiple of CVTracked that are tracked in scenes that require a live video image from the device camera. A tracker is bound to a scene node. If the node is the camera node the tracker calculates the relative position of the camera to the tracker. This is the standard augmented reality case. If the camera is a normal scene node, the tracker calculates the object matrix relative to the scene camera. See also the derived classes CVTrackedAruco, CVTrackedChessboard, CVTrackedFaces and CVTrackedFeature for example implementations. The update of the tracking per frame is implemented in onUpdateTracking in AppDemoTracking.cpp and called once per frame within the main render loop.

Definition at line 49 of file CVTracked.h.

Constructor & Destructor Documentation

◆ CVTracked()

CVTracked::CVTracked ( )

inlineexplicit

Definition at line 52 of file CVTracked.h.

52 : _isVisible(false), _drawDetection(true) {}

CVTracked::_isVisible

bool _isVisible

Flag if marker is visible.

Definition: CVTracked.h:91

CVTracked::_drawDetection

bool _drawDetection

Flag if detection should be drawn into image.

Definition: CVTracked.h:92

◆ ~CVTracked()

virtual CVTracked::~CVTracked ( )

virtualdefault

Member Function Documentation

◆ averageQuaternion()

SLQuat4f CVTracked::averageQuaternion	(	vector< SLQuat4f >	quaternions,
		vector< float >	weights
	)

static

Definition at line 153 of file CVTracked.cpp.

 {
     // Based on: https://math.stackexchange.com/questions/61146/averaging-quaternions
  
     if (quaternions.size() == 1)
         return quaternions[0];
  
     SLQuat4f total(0.0f, 0.0f, 0.0f, 0.0f);
  
     for (int i = 0; i < quaternions.size(); i++)
     {
         SLQuat4f quaternion = quaternions[i];
         float    weight     = weights[i];
  
         if (i > 0 && quaternion.dot(quaternions[0]) < 0.0)
             weight = -weight;
  
         total.set(total.x() + weight * quaternion.x(),
                   total.y() + weight * quaternion.y(),
                   total.z() + weight * quaternion.z(),
                   total.w() + weight * quaternion.w());
     }
  
     return total.normalized();
 }

◆ averageVector()

CVVec3f CVTracked::averageVector	(	vector< CVVec3f >	vectors,
		vector< float >	weights
	)

static

Definition at line 134 of file CVTracked.cpp.

 {
     if (vectors.size() == 1)
         return vectors[0];
  
     CVVec3f total;
     float   totalWeights = 0.0f;
  
     for (int i = 0; i < vectors.size(); i++)
     {
         float weight = weights[i];
         total += vectors[i] * weight;
         totalWeights += weight;
     }
  
     return total / totalWeights;
 }

◆ calcObjectMatrix()

CVMatx44f CVTracked::calcObjectMatrix	(	const CVMatx44f &	cameraObjectMat,
		const CVMatx44f &	objectViewMat
	)

static

Calculates the object matrix from the cameraObject and the object view matrix.
Nomenclature: T = homogenous transformation matrix
^aT_b = homogenous transformation matrix with subscript b and superscript a
Subscrips and superscripts: w = world; o = object; c = camera
^cT_o = Transformation of object with respect to camera coordinate system. It describes the position of an object in the camera coordinate system. We get this transformation from OpenCVs solvePNP function.
^wT_c = (^cT_w)^-1 = Transformation of camera with respect to world coord.-system. Inversion exchanges sub- and superscript.
The inverse of the camera to world matrix is the view matrix or camera matrix.
We can combine two or more homogenous transformations to a new one if the inner sub- and superscript fit together. The resulting transformation inherits the superscript from the left and the subscript from the right transformation. The following transformation is what we want to do:
^wT_o = ^wT_c * ^cT_o = Transformation of object with respect to world coordinate system (object matrix)

Definition at line 125 of file CVTracked.cpp.

 {   
     // new object matrix = camera object matrix * object-view matrix
     return cameraObjectMat * objectViewMat;
 }

◆ createGLMatrix()

CVMatx44f CVTracked::createGLMatrix	(	const CVMat &	tVec,
		const CVMat &	rVec
	)

static

Create an OpenGL 4x4 matrix from an OpenCV translation & rotation vector.

Definition at line 46 of file CVTracked.cpp.

 {
     // 1) convert the passed rotation vector to a rotation matrix
     CVMat rMat;
     Rodrigues(rVec, rMat);
  
     // 2) Create an OpenGL 4x4 column major matrix from the rotation matrix and 
     // translation vector from openCV as described in this post:
     // www.morethantechnical.com/2015/02/17/
     // augmented-reality-on-libqglviewer-and-opencv-opengl-tips-wcode
       
     // The y- and z- axis have to be inverted:
     /*
     tVec = |  t0,  t1,  t2 |
                                         |  r00   r01   r02   t0 |
            | r00, r10, r20 |            | -r10  -r11  -r12  -t1 |
     rMat = | r01, r11, r21 |    glMat = | -r20  -r21  -r22  -t2 |
            | r02, r12, r22 |            |    0     0     0    1 |
     */
  
     CVMatx44f glM((float) rMat.at<double>(0, 0), (float) rMat.at<double>(0, 1), (float) rMat.at<double>(0, 2), (float) tVec.at<double>(0, 0),
                   (float)-rMat.at<double>(1, 0), (float)-rMat.at<double>(1, 1), (float)-rMat.at<double>(1, 2), (float)-tVec.at<double>(1, 0),
                   (float)-rMat.at<double>(2, 0), (float)-rMat.at<double>(2, 1), (float)-rMat.at<double>(2, 2), (float)-tVec.at<double>(2, 0),
                 0.0f,                          0.0f,                          0.0f,                          1.0f);
     return glM;
 }

◆ createRvecTvec()

void CVTracked::createRvecTvec	(	const CVMatx44f &	glMat,
		CVMat &	tVec,
		CVMat &	rVec
	)

static

Creates the OpenCV rvec & tvec vectors from an column major OpenGL 4x4 matrix.

Definition at line 74 of file CVTracked.cpp.

 {
     // The y- and z- axis have to be inverted:
     /*
     tVec = |  t0,  t1,  t2 |
                                         |  r00   r01   r02   t0 |
            | r00, r10, r20 |            | -r10  -r11  -r12  -t1 |
     rMat = | r01, r11, r21 |    glMat = | -r20  -r21  -r22  -t2 |
            | r02, r12, r22 |            |    0     0     0    1 |
     */
     
     // 1) Create cv rotation matrix from OpenGL rotation matrix
     CVMatx33f rMat(glMat.val[0], -glMat.val[1], -glMat.val[2],
                    glMat.val[4], -glMat.val[5], -glMat.val[6],
                    glMat.val[8], -glMat.val[9], -glMat.val[10]);
     
     // 2) Convert rotation matrix to Rodrigues rotation vector
     Rodrigues(rMat, rVec);
     
     // 3) Create tvec vector from translation components
     tVec.at<double>(0, 0) =  glMat.val[3];
     tVec.at<double>(1, 0) = -glMat.val[7];
     tVec.at<double>(2, 0) = -glMat.val[11];
 }

◆ drawDetection() [1/2]

bool CVTracked::drawDetection ( )

inline

Definition at line 64 of file CVTracked.h.

64 { return _drawDetection; }

◆ drawDetection() [2/2]

void CVTracked::drawDetection ( bool draw )

inline

Definition at line 60 of file CVTracked.h.

60 { _drawDetection = draw; }

◆ isVisible()

bool CVTracked::isVisible ( )

inline

Definition at line 63 of file CVTracked.h.

63 { return _isVisible; }

◆ objectViewMat()

CVMatx44f CVTracked::objectViewMat ( )

inline

Definition at line 65 of file CVTracked.h.

65 { return _objectViewMat; }

CVTracked::_objectViewMat

CVMatx44f _objectViewMat

view transformation matrix

Definition: CVTracked.h:93

◆ resetTimes()

void CVTracked::resetTimes ( )

static

Resets all static variables.

Definition at line 31 of file CVTracked.cpp.

 {
     // Reset all timing variables
     CVTracked::trackingTimesMS.init(60, 0.0f);
     CVTracked::detectTimesMS.init(60, 0.0f);
     CVTracked::detect1TimesMS.init(60, 0.0f);
     CVTracked::detect2TimesMS.init(60, 0.0f);
     CVTracked::matchTimesMS.init(60, 0.0f);
     CVTracked::optFlowTimesMS.init(60, 0.0f);
     CVTracked::poseTimesMS.init(60, 0.0f);
 }

◆ track()

virtual bool CVTracked::track	(	CVMat	imageGray,
		CVMat	imageBgr,
		CVCalibration *	calib
	)

static

Averaged time for video feature pose estimation in ms.

Definition at line 88 of file CVTracked.h.

◆ trackingTimesMS

AvgFloat CVTracked::trackingTimesMS

static

Averaged time for video tracking in ms.

Definition at line 82 of file CVTracked.h.

The documentation for this class was generated from the following files:

Public Member Functions

Static Public Member Functions

Static Public Attributes

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ CVTracked()

◆ ~CVTracked()

Member Function Documentation

◆ averageQuaternion()

◆ averageVector()

◆ calcObjectMatrix()

◆ createGLMatrix()

◆ createRvecTvec()

◆ drawDetection() [1/2]

◆ drawDetection() [2/2]

◆ isVisible()

◆ objectViewMat()

◆ resetTimes()

◆ track()

Member Data Documentation

◆ _drawDetection

◆ _isVisible

◆ _objectViewMat

◆ _timer

◆ detect1TimesMS

◆ detect2TimesMS

◆ detectTimesMS

◆ matchTimesMS

◆ optFlowTimesMS

◆ poseTimesMS

◆ trackingTimesMS