Autonomy Software C++ 24.5.1
Welcome to the Autonomy Software repository of the Mars Rover Design Team (MRDT) at Missouri University of Science and Technology (Missouri S&T)! API reference contains the source code and other resources for the development of the autonomy software for our Mars rover. The Autonomy Software project aims to compete in the University Rover Challenge (URC) by demonstrating advanced autonomous capabilities and robust navigation algorithms.
Loading...
Searching...
No Matches
Functions
torchtag Namespace Reference

Namespace containing functions related to torch tag detections operations on images using PyTorch. More...

Functions

std::vector< tagdetectutils::ArucoTag > Detect (const cv::Mat &cvFrame, yolomodel::pytorch::PyTorchInterpreter &trPyTorchDetector, const float fMinObjectConfidence=0.40f, const float fNMSThreshold=0.60f)
 Detect ArUco tags in the provided image using a YOLO DNN model.
 
void DrawDetections (cv::Mat &cvDetectionsFrame, const std::vector< tagdetectutils::ArucoTag > &vDetectedTags)
 Given a vector of tagdetectutils::ArucoTag structs draw each tag corner and confidence onto the given image.
 

Detailed Description

Namespace containing functions related to torch tag detections operations on images using PyTorch.

Author
clayjay3 (clayt.nosp@m.onra.nosp@m.ycowe.nosp@m.n@gm.nosp@m.ail.c.nosp@m.om)
Date
2025-02-13

Function Documentation

◆ Detect()

std::vector< tagdetectutils::ArucoTag > torchtag::Detect ( const cv::Mat &  cvFrame,
yolomodel::pytorch::PyTorchInterpreter &  trPyTorchDetector,
const float  fMinObjectConfidence = 0.40f,
const float  fNMSThreshold = 0.60f 
)
inline

Detect ArUco tags in the provided image using a YOLO DNN model.

Parameters
cvFrame- The RGB camera frame to run detection on.
trPyTorchDetector- The PyTorch model interpreter to run inference on.
fMinObjectConfidence- The minimum confidence required for an object to be considered a valid detection.
fNMSThreshold- The threshold for Non-Maximum Suppression, controlling overlap between bounding box predictions.
Returns
std::vector<tagdetectutils::ArucoTag> - The resultant vector containing the detected tags in the frame.
Author
clayjay3 (clayt.nosp@m.onra.nosp@m.ycowe.nosp@m.n@gm.nosp@m.ail.c.nosp@m.om)
Date
2025-02-13
49 {
50 // Check if the input frame is in RGB format.
51 if (cvFrame.channels() != 3)
52 {
53 // Submit logger message.
54 LOG_ERROR(logging::g_qSharedLogger, "Detect() requires a RGB image.");
55 return {};
56 }
57
58 // Declare instance variables.
59 std::vector<tagdetectutils::ArucoTag> vDetectedTags;
60
61 // Check if the PyTorch interpreter hardware is opened and the model is loaded.
62 if (trPyTorchDetector.IsReadyForInference())
63 {
64 // Run inference on YOLO model with current image.
65 std::vector<yolomodel::Detection> vOutputTensorTags = trPyTorchDetector.Inference(cvFrame, fMinObjectConfidence, fNMSThreshold);
66
67 // Repackage detections into tensorflow tags.
68 for (const yolomodel::Detection& stTagDetection : vOutputTensorTags)
69 {
70 // Create and initialize new TensorflowTag.
71 tagdetectutils::ArucoTag stDetectedTag;
72 stDetectedTag.dConfidence = stTagDetection.fConfidence;
73 stDetectedTag.pBoundingBox = std::make_shared<cv::Rect2d>(stTagDetection.cvBoundingBox);
74 stDetectedTag.nID = stTagDetection.nClassID;
75 stDetectedTag.szClassName = stTagDetection.szClassName;
76 stDetectedTag.eDetectionMethod = tagdetectutils::TagDetectionMethod::eTorch;
77 stDetectedTag.cvImageResolution = cvFrame.size();
78
79 // Add the newly detected tag to the vector.
80 vDetectedTags.emplace_back(stDetectedTag);
81 }
82 }
83 else
84 {
85 // Submit logger message.
86 LOG_WARNING(logging::g_qSharedLogger,
87 "TorchDetect: Unable to detect tags using YOLO torch detection because hardware is not opened or model is not initialized.");
88 }
89
90 // Return the detected tags.
91 return vDetectedTags;
92 }
cv::Mat::size
MatSize size
cv::Mat::channels
int channels() const
yolomodel::pytorch::PyTorchInterpreter::IsReadyForInference
bool IsReadyForInference() const
Check if the model is ready for inference.
Definition YOLOModel.hpp:946
yolomodel::pytorch::PyTorchInterpreter::Inference
std::vector< Detection > Inference(const cv::Mat &cvInputFrame, const float fMinObjectConfidence=0.85, const float fNMSThreshold=0.6)
Given an input image forward the image through the YOLO model to run inference on the PyTorch model,...
Definition YOLOModel.hpp:859
tagdetectutils::ArucoTag
Represents a single ArUco tag. Combines attributes from TorchTag, TensorflowTag, and the original Aru...
Definition TagDetectionUtilty.hpp:59
yolomodel::Detection
This struct is used to.
Definition YOLOModel.hpp:45
Here is the call graph for this function:
Here is the caller graph for this function:

◆ DrawDetections()

void torchtag::DrawDetections ( cv::Mat &  cvDetectionsFrame,
const std::vector< tagdetectutils::ArucoTag > &  vDetectedTags 
)
inline

Given a vector of tagdetectutils::ArucoTag structs draw each tag corner and confidence onto the given image.

Parameters
cvDetectionsFrame- The frame to draw overlay onto.
vDetectedTags- The vector of tagdetectutils::ArucoTag structs used to draw tag corners and confidences onto image.
Author
clayjay3 (clayt.nosp@m.onra.nosp@m.ycowe.nosp@m.n@gm.nosp@m.ail.c.nosp@m.om)
Date
2025-02-13
104 {
105 // Check if the given frame is a 1 or 3 channel image. (not BGRA)
106 if (!cvDetectionsFrame.empty() && (cvDetectionsFrame.channels() == 1 || cvDetectionsFrame.channels() == 3))
107 {
108 // Loop through each detection.
109 for (const tagdetectutils::ArucoTag& stTag : vDetectedTags)
110 {
111 // Check if the tag detection type is Torch.
112 if (stTag.eDetectionMethod == tagdetectutils::TagDetectionMethod::eTorch)
113 {
114 // Draw bounding box onto image.
115 cv::rectangle(cvDetectionsFrame, *stTag.pBoundingBox, cv::Scalar(255, 255, 255), 2);
116 std::string szText = stTag.szClassName + " " + std::to_string(static_cast<int>(stTag.dConfidence * 100)) + "%";
117 cv::Size cvTextSize = cv::getTextSize(szText, cv::FONT_HERSHEY_SIMPLEX, 0.75, 1, nullptr);
118 // Draw classID background box onto image.
119 cv::rectangle(cvDetectionsFrame,
120 stTag.pBoundingBox->tl() + cv::Point2d(0, stTag.pBoundingBox->height),
121 stTag.pBoundingBox->tl() + cv::Point2d(cvTextSize.width, stTag.pBoundingBox->height + cvTextSize.height),
122 cv::Scalar(255, 255, 255),
123 cv::FILLED);
124 // Draw class text onto image.
125 cv::putText(cvDetectionsFrame,
126 szText,
127 stTag.pBoundingBox->tl() + cv::Point2d(0, stTag.pBoundingBox->height + cvTextSize.height),
128 cv::FONT_HERSHEY_SIMPLEX,
129 0.5,
130 cv::Scalar(0, 0, 0));
131 }
132 }
133 }
134 else
135 {
136 // Submit logger message.
137 LOG_ERROR(logging::g_qSharedLogger,
138 "TorchDetect: Unable to draw markers on image because it is empty or because it has {} channels. (Should be 1 or 3)",
139 cvDetectionsFrame.channels());
140 }
141 }
cv::Mat::empty
bool empty() const
Point_< double >
Scalar_< double >
cv::Size_
cv::Size_::height
_Tp height
cv::Size_::width
_Tp width
cv::rectangle
void rectangle(InputOutputArray img, Point pt1, Point pt2, const Scalar &color, int thickness=1, int lineType=LINE_8, int shift=0)
cv::getTextSize
Size getTextSize(const String &text, int fontFace, double fontScale, int thickness, int *baseLine)
cv::putText
void putText(InputOutputArray img, const String &text, Point org, int fontFace, double fontScale, Scalar color, int thickness=1, int lineType=LINE_8, bool bottomLeftOrigin=false)
cv::FONT_HERSHEY_SIMPLEX
FONT_HERSHEY_SIMPLEX
cv::FILLED
FILLED
Here is the call graph for this function:
Here is the caller graph for this function: