SENSCvCamera Class Reference

#include <SENSCvCamera.h>

Public Types
enum	ConfigReturnCode { SUCCESS = 0 , WARN_TARGET_SIZE_NOT_FOUND , ERROR_CAMERA_INVALID , ERROR_CAMERA_IS_RUNNING , ERROR_FACING_NOT_AVAILABLE , ERROR_UNKNOWN }
	error code definition, will be returned by configure in case of failure. See method advice() for explanation. More...

Public Member Functions
	SENSCvCamera (SENSCamera *camera)
ConfigReturnCode	configure (SENSCameraFacing facing, int targetWidth, int targetHeight, int manipWidth, int manipHeight, bool mirrorH, bool mirrorV, bool convertManipToGray)
bool	started ()
bool	start ()
void	stop ()
SENSFramePtr	latestFrame ()
bool	supportsFacing (SENSCameraFacing facing)
SENSCamera *&	cameraRef ()
bool	isConfigured ()
const SENSCvCameraConfig *	config ()
const SENSCalibration *const	calibration () const
	returns SENSCalibration if it was started (maybe a guessed one from a fovV guess). Else returns nullptr. The calibration is used for computer vision applications. So, if a manipulated image is requested (see imgManipSize in SENSCamera::start(...), SENSFrame::imgManip and SENSCameraConfig) this calibration is adjusted to fit to this image, else to the original sized image (see SENSFrame::imgBGR) More...
const SENSCalibration *const	calibrationManip () const
void	guessAndSetCalibration (float fovDegFallbackGuess)
cv::Mat	scaledCameraMat ()
void	setCalibration (const SENSCalibration &calibration, bool buildUndistortionMaps)
	Set calibration and adapt it to current image size. Camera has to be configured, before this function is called. More...
void	clearCalibration ()
	clear calibration set from outsilde with setCalibration. Only then an automatic guess will be possible. More...

Static Public Member Functions
static std::string	advice (const ConfigReturnCode returnCode)
	returns advice to an error code More...

Private Member Functions
SENSFramePtr	processNewFrame (const SENSTimePt &timePt, cv::Mat bgrImg, cv::Mat intrinsics, bool intrinsicsChanged)

Private Attributes
SENSCamera *	_camera
std::unique_ptr< SENSCvCameraConfig >	_config
std::unique_ptr< SENSCalibration >	_calibration
	The calibration is used for computer vision applications. This calibration is adjusted to fit to the original sized image (see SENSFrame::imgBGR and SENSCameraConfig::targetWidth, targetHeight) More...
std::unique_ptr< SENSCalibration >	_calibrationManip
std::unique_ptr< SENSCalibration >	_calibrationOverwrite

Detailed Description

Camera wrapper for computer vision applications that adds convenience functions and that makes post processing steps.

Definition at line 64 of file SENSCvCamera.h.

Member Enumeration Documentation

ConfigReturnCode

enum SENSCvCamera::ConfigReturnCode

error code definition, will be returned by configure in case of failure. See method advice() for explanation.

Enumerator
SUCCESS
WARN_TARGET_SIZE_NOT_FOUND
ERROR_CAMERA_INVALID
ERROR_CAMERA_IS_RUNNING
ERROR_FACING_NOT_AVAILABLE
ERROR_UNKNOWN

Definition at line 68 of file SENSCvCamera.h.

    {
        SUCCESS = 0,
        WARN_TARGET_SIZE_NOT_FOUND,
        ERROR_CAMERA_INVALID,
        ERROR_CAMERA_IS_RUNNING,
        ERROR_FACING_NOT_AVAILABLE,
        ERROR_UNKNOWN
    };

Constructor & Destructor Documentation

SENSCvCamera()

SENSCvCamera::SENSCvCamera

(

SENSCamera *

camera

)

Definition at line 81 of file SENSCvCamera.cpp.

  : _camera(camera)
{
    assert(camera);
    //retrieve capture properties
    if (_camera)
        _camera->captureProperties();
}

Member Function Documentation

advice()

static std::string SENSCvCamera::advice ( const ConfigReturnCode  returnCode )

inlinestatic

returns advice to an error code

Definition at line 79 of file SENSCvCamera.h.

    {
        switch (returnCode)
        {
            case SUCCESS: return "Camera was configured successfully";
            case WARN_TARGET_SIZE_NOT_FOUND: return "Only a smaller resolution was found (you can still start the camera)";
            case ERROR_CAMERA_INVALID: return "Camera is null";
            case ERROR_CAMERA_IS_RUNNING: return "Camera is running, you have to stop it first";
            case ERROR_FACING_NOT_AVAILABLE: return "Camera facing is not available as configured";
            case ERROR_UNKNOWN: return "Unknown error";
            default: return "Unknown return code";
        }
    }

calibration()

const SENSCalibration* const SENSCvCamera::calibration ( ) const

inline

returns SENSCalibration if it was started (maybe a guessed one from a fovV guess). Else returns nullptr. The calibration is used for computer vision applications. So, if a manipulated image is requested (see imgManipSize in SENSCamera::start(...), SENSFrame::imgManip and SENSCameraConfig) this calibration is adjusted to fit to this image, else to the original sized image (see SENSFrame::imgBGR)

Definition at line 120 of file SENSCvCamera.h.

{ return _calibration.get(); }

calibrationManip()

const SENSCalibration* const SENSCvCamera::calibrationManip ( ) const

inline

Definition at line 121 of file SENSCvCamera.h.

{ return _calibrationManip.get(); }

cameraRef()

SENSCamera*& SENSCvCamera::cameraRef ( )

inline

Definition at line 115 of file SENSCvCamera.h.

{ return _camera; }

clearCalibration()

void SENSCvCamera::clearCalibration ( )

inline

clear calibration set from outsilde with setCalibration. Only then an automatic guess will be possible.

Definition at line 132 of file SENSCvCamera.h.

{ _calibrationOverwrite.reset(); }

config()

const SENSCvCameraConfig* SENSCvCamera::config ( )

inline

Definition at line 117 of file SENSCvCamera.h.

{ return _config.get(); }

configure()

SENSCvCamera::ConfigReturnCode SENSCvCamera::configure	(	SENSCameraFacing	facing,
		int	targetWidth,
		int	targetHeight,
		int	manipWidth,
		int	manipHeight,
		bool	mirrorH,
		bool	mirrorV,
		bool	convertManipToGray
	)

configure camera: the function checks with the wrapped camera if it could successfully configure the camera.

configure camera: the function checks with the wrapped camera if it could successfully configure the camera. If not, it will return false. This may have dirrerent reasons:

• the wrapped camera is not valid
• it has to extrapolate the maximum found stream config size to get targetWidth, targetHeight. Transfer -1 for manipWidth and manipHeight to disable generation of manipulated image

Definition at line 106 of file SENSCvCamera.cpp.

{
    //this can happen, because it is possible to retrieve a camera pointer reference for camera simulation
    if (!_camera)
        return ERROR_CAMERA_INVALID;
 
    bool cameraWasStarted = false;
    if (_camera->started())
    {
        _camera->stop();
        cameraWasStarted = true;
    }
 
    const SENSCaptureProps& props = _camera->captureProperties();
    //check if facing is available
    if (!props.supportsCameraFacing(facing))
        return ERROR_FACING_NOT_AVAILABLE;
 
    ConfigReturnCode returnCode = SUCCESS;
 
    //search for best matching stream config
    int   trackingImgW = 640;
    float searchWdivH;
    if (manipWidth > 0 && manipHeight > 0 &&
        ((float)targetWidth / (float)targetHeight) >
          ((float)manipWidth / (float)manipHeight))
        searchWdivH = (float)manipWidth / (float)manipHeight;
    else
        searchWdivH = (float)targetWidth / (float)targetHeight;
 
    //approximately what resolution we search for visualiation image
    int aproxHighImgW = targetWidth;
 
    std::pair<const SENSCameraDeviceProps*, const SENSCameraStreamConfig*> bestConfig =
      props.findBestMatchingConfig(facing, 65.f, aproxHighImgW, (int)((float)aproxHighImgW / searchWdivH));
 
    //warn if extrapolation needed (image will not be extrapolated)
    if (!bestConfig.first && !bestConfig.second)
    {
        //we reduce this size
        aproxHighImgW = 640;
        bestConfig    = props.findBestMatchingConfig(facing, 65.f, aproxHighImgW, (int)((float)aproxHighImgW / searchWdivH));
        if (!bestConfig.first && !bestConfig.second)
            return ERROR_UNKNOWN;
        else
            returnCode = WARN_TARGET_SIZE_NOT_FOUND;
    }
 
    _config = std::make_unique<SENSCvCameraConfig>(bestConfig.first,
                                                   bestConfig.second,
                                                   targetWidth,
                                                   targetHeight,
                                                   manipWidth,
                                                   manipHeight,
                                                   mirrorH,
                                                   mirrorV,
                                                   convertManipToGray);
 
    //guess calibrations if no calibration is set from outside
    if (!_calibrationOverwrite)
        guessAndSetCalibration(65.f);
 
    if (cameraWasStarted)
        _camera->start(bestConfig.first->deviceId(), *bestConfig.second);
 
    return returnCode;
}

guessAndSetCalibration()

void SENSCvCamera::guessAndSetCalibration

(

float

fovDegFallbackGuess

)

Definition at line 54 of file SENSCvCamera.cpp.

{
    //We make a calibration with full resolution and adjust it to the manipulated image size later if neccessary:
    //For the initial setup we have to use streamconfig values because that is where the fovV fits too
    float horizFOVDev = fovDegFallbackGuess;
    if (_config->streamConfig->focalLengthPix > 0)
        horizFOVDev = SENS::calcFOVDegFromFocalLengthPix(_config->streamConfig->focalLengthPix, _config->streamConfig->widthPix);
 
    //init calibration with streaconfig image size, because that is where the fovV fits too
    _calibration = std::make_unique<SENSCalibration>(cv::Size(_config->streamConfig->widthPix, _config->streamConfig->heightPix),
                                                     horizFOVDev,
                                                     false,
                                                     false,
                                                     SENSCameraType::BACKFACING, //todo
                                                     Utils::ComputerInfos().get());
    //now we adapt the calibration to the target size
    if (_config->targetWidth != _config->streamConfig->widthPix || _config->targetHeight != _config->streamConfig->heightPix)
        _calibration->adaptForNewResolution({_config->targetWidth, _config->targetHeight}, false);
 
    //update second calibration
    if (_config->manipWidth > 0 && _config->manipHeight > 0)
    {
        _calibrationManip = std::make_unique<SENSCalibration>(*_calibration);
        _calibrationManip->adaptForNewResolution(cv::Size(_config->manipWidth, _config->manipHeight), false);
    }
}

isConfigured()

bool SENSCvCamera::isConfigured ( )

inline

Definition at line 116 of file SENSCvCamera.h.

{ return _config != nullptr; }

latestFrame()

SENSFramePtr SENSCvCamera::latestFrame

(

)

Definition at line 210 of file SENSCvCamera.cpp.

{
    SENSFramePtr latestFrame;
 
    SENSFrameBasePtr frameBase = _camera->latestFrame();
    if (frameBase)
    {
        //process
        latestFrame = processNewFrame(frameBase->timePt, frameBase->imgBGR, frameBase->intrinsics, !frameBase->intrinsics.empty());
    }
 
    //update calibration if necessary
    //we wont update it, if overwrite calibration is valid (set from outside)
    //todo: if frame size changed (or config) then we also have to adjust the overwrite calibration?
    if (latestFrame && !_calibrationOverwrite && !latestFrame->intrinsics.empty())
    {
        HighResTimer t;
        //todo: mutex for calibration?
        _calibration = std::make_unique<SENSCalibration>(latestFrame->intrinsics,
                                                         cv::Size(_config->streamConfig->widthPix, _config->streamConfig->heightPix),
                                                         _calibration->isMirroredH(),
                                                         _calibration->isMirroredV(),
                                                         _calibration->camType(),
                                                         _calibration->computerInfos());
        //now we adapt the calibration to the target size
        if (_config->targetWidth != _config->streamConfig->widthPix || _config->targetHeight != _config->streamConfig->heightPix)
            _calibration->adaptForNewResolution({_config->targetWidth, _config->targetHeight}, false);
 
        //update second calibration
        if (_config->manipWidth > 0 && _config->manipHeight > 0)
        {
            _calibrationManip = std::make_unique<SENSCalibration>(*_calibration);
            _calibrationManip->adaptForNewResolution(cv::Size(_config->manipWidth, _config->manipHeight), false);
        }
 
        //SENS_DEBUG("calib update duration %f", t.elapsedTimeInMilliSec());
    }
 
    return latestFrame;
}

processNewFrame()

SENSFramePtr SENSCvCamera::processNewFrame ( const SENSTimePt &  timePt, cv::Mat  bgrImg, cv::Mat  intrinsics, bool  intrinsicsChanged  )

private

Definition at line 7 of file SENSCvCamera.cpp.

{
    //todo: accessing config readonly should be no problem  here, as the config only changes when camera is stopped
    cv::Size inputSize = bgrImg.size();
 
    // Crop Video image to required aspect ratio
    int cropW = 0, cropH = 0;
    SENS::cropImage(bgrImg, (float)_config->targetWidth / (float)_config->targetHeight, cropW, cropH);
 
    // Mirroring
    SENS::mirrorImage(bgrImg, _config->mirrorH, _config->mirrorV);
 
    cv::Mat manipImg;
    float   scale = 1.0f;
    //problem: eingangsbild 16:9 -> targetImg 4:3 -> crop left and right -> manipImg 16:9 -> weiterer crop oben und unten -> FALSCH
    if (_config->manipWidth > 0 && _config->manipHeight > 0)
    {
        manipImg  = bgrImg;
        int cropW = 0, cropH = 0;
        SENS::cropImage(manipImg, (float)_config->manipWidth / (float)_config->manipHeight, cropW, cropH);
        scale = (float)_config->manipWidth / (float)manipImg.size().width;
        cv::resize(manipImg, manipImg, cv::Size(), scale, scale);
    }
    else if (_config->convertManipToGray)
    {
        manipImg = bgrImg;
    }
 
    // Create grayscale
    if (_config->convertManipToGray)
    {
        cv::cvtColor(manipImg, manipImg, cv::COLOR_BGR2GRAY);
    }
 
    SENSFramePtr sensFrame = std::make_unique<SENSFrame>(timePt,
                                                         bgrImg.clone(),
                                                         manipImg,
                                                         _config->mirrorH,
                                                         _config->mirrorV,
                                                         1 / scale,
                                                         intrinsics.clone(),
                                                         manipImg.cols,
                                                         manipImg.rows);
 
    return sensFrame;
}

scaledCameraMat()

cv::Mat SENSCvCamera::scaledCameraMat

(

)

Definition at line 251 of file SENSCvCamera.cpp.

{
    if (!_calibrationManip)
        return cv::Mat();
    else
        return _calibrationManip->cameraMat();
}

setCalibration()

void SENSCvCamera::setCalibration	(	const SENSCalibration &	calibration,
		bool	buildUndistortionMaps
	)

Set calibration and adapt it to current image size. Camera has to be configured, before this function is called.

Definition at line 260 of file SENSCvCamera.cpp.

{
    _calibrationOverwrite = std::make_unique<SENSCalibration>(calibration);
 
    //update first calibration
    _calibration = std::make_unique<SENSCalibration>(*_calibrationOverwrite);
    _calibration->adaptForNewResolution(cv::Size(_config->targetWidth, _config->targetHeight), buildUndistortionMaps);
 
    //update second calibration
    if (_config->manipWidth > 0 && _config->manipHeight > 0)
    {
        _calibrationManip = std::make_unique<SENSCalibration>(*_calibrationOverwrite);
        _calibrationManip->adaptForNewResolution(cv::Size(_config->manipWidth, _config->manipHeight), buildUndistortionMaps);
    }
}

start()

bool SENSCvCamera::start

(

)

Definition at line 181 of file SENSCvCamera.cpp.

{
    if (!_camera)
        return false;
 
    if (!_config)
        return false;
 
    _camera->start(_config->deviceProps->deviceId(),
                   *_config->streamConfig,
                   true);
 
    return true;
}

started()

bool SENSCvCamera::started

(

)

Definition at line 196 of file SENSCvCamera.cpp.

{
    if (_camera)
        return _camera->started();
    else
        return false;
}

stop()

void SENSCvCamera::stop

(

)

Definition at line 204 of file SENSCvCamera.cpp.

{
    if (_camera)
        _camera->stop();
}

supportsFacing()

bool SENSCvCamera::supportsFacing

(

SENSCameraFacing

facing

)

Definition at line 90 of file SENSCvCamera.cpp.

{
    if (!_camera)
        return false;
 
    const SENSCaptureProps& props = _camera->captureProperties();
    return props.supportsCameraFacing(facing);
}

Member Data Documentation

_calibration

std::unique_ptr<SENSCalibration> SENSCvCamera::_calibration

private

The calibration is used for computer vision applications. This calibration is adjusted to fit to the original sized image (see SENSFrame::imgBGR and SENSCameraConfig::targetWidth, targetHeight)

Definition at line 141 of file SENSCvCamera.h.

_calibrationManip

std::unique_ptr<SENSCalibration> SENSCvCamera::_calibrationManip

private

Definition at line 143 of file SENSCvCamera.h.

_calibrationOverwrite

std::unique_ptr<SENSCalibration> SENSCvCamera::_calibrationOverwrite

private

Definition at line 146 of file SENSCvCamera.h.

_camera

SENSCamera* SENSCvCamera::_camera

private

Definition at line 137 of file SENSCvCamera.h.

_config

std::unique_ptr<SENSCvCameraConfig> SENSCvCamera::_config

private

Definition at line 138 of file SENSCvCamera.h.

---

The documentation for this class was generated from the following files:

• SENSCvCamera.h
• SENSCvCamera.cpp