Qt6 multimedia开发一个摄像头录像机

Qt 6 附加模块multimedia可用于多媒体的开发，今天使用它可以快速开发一个摄像头录像机。毕业季用作本科毕业设计软件应该可以的。

支持的功能

无边框窗口，并且支持拖拽，调整窗口大小
切换摄像头
配置摄像头原格式、分辨率、帧率、画面质量、
抓图拍摄
录像，支持声音

不支持硬件加速，纯CPU编解码，当前我没在qt文档中找到硬件加速的接口。
可使用ffmpeg实现GPU加速，将来再写文章。

效果展示

在这里插入图片描述

源码地址

https://gitee.com/noevilme/QtDemo/tree/master/CameraRecorder

源码分析

1. 外观设计

这里有人分享的一些无边框窗口的示例，可以参考修改
https://gitee.com/feiyangqingyun/QWidgetDemo/tree/master/ui/uidemo08
他这里使用的黑色，可以参考这个仓库下其他示例，更换一个外观风格。

皮肤资源

复制资源文件lightblue.css和lightblue目录到CameraRecorder\core_qss\qss
资源文件中添加刚才的两个资源。
在这里插入图片描述

修改外观

修改main.cpp中资源文件名

    //加载样式表
    QFile file(":/qss/lightblue.css");
    if (file.open(QFile::ReadOnly)) {
        QString qss = QLatin1String(file.readAll());
        QString paletteColor = qss.mid(20, 7);
        qApp->setPalette(QPalette(QColor(paletteColor)));
        qApp->setStyleSheet(qss);
        file.close();
    }

2. 无边框窗口的放大缩小拖拽

效果如下
在这里插入图片描述

由于setWindowFlags(Qt::FramelessWindowHint);// 设置窗口为无边框窗口，所以窗口是不支持边框放大与缩小拖拽的，因为已经没有边框了。
要支持拖拽，需要自己模拟实现，窗口边框附近变更鼠标符号，然后就计算坐标变化，再去变更窗口大小。

frmmain.h

private:
    //边距+可移动+可拉伸
    int padding;
    bool moveEnable;
    bool resizeEnable;

    //无边框窗体
    QWidget *widget;

    //鼠标是否按下+按下坐标+按下时窗体区域
    bool mousePressed;
    QPoint mousePoint;
    QRect mouseRect;

    //鼠标是否按下某个区域+按下区域的大小
    //依次为 左侧+右侧+上侧+下侧+左上侧+右上侧+左下侧+右下侧
    QList<bool> pressedArea;
    QList<QRect> pressedRect;

    void initResizeMembers();

frmmain.cpp 中需要去初始化这些成员变量，installEventFilter很重要，然后再去eventFilter响应鼠标事件，实现拉伸。

void frmMain::initResizeMembers() {
    //设置鼠标追踪为真，不然只会在鼠标按下时才会触发鼠标移动事件
    this->setMouseTracking(true);
    //设置悬停为真,必须设置这个,不然当父窗体里边还有子窗体全部遮挡了识别不到MouseMove,需要识别HoverMove
    this->setAttribute(Qt::WA_Hover, true);
    // setWindowFlags(Qt::FramelessWindowHint);// 设置窗口为无边框窗口
    // 只有安装了事件过滤器，才会进入到eventFilter，很重要！！
    installEventFilter(this);

    padding = 8;
    moveEnable = true;
    resizeEnable = true;
    widget = this;

    mousePressed = false;
    mousePoint = QPoint(0, 0);
    mouseRect = QRect(0, 0, 0, 0);

    for (int i = 0; i < 8; ++i) {
        pressedArea << false;
        pressedRect << QRect(0, 0, 0, 0);
    }
}


bool frmMain::eventFilter(QObject *watched, QEvent *event) {
    if (event->type() == QEvent::MouseButtonDblClick) {
        if (watched == ui->widgetTitle) {
            on_btnMenu_Max_clicked();
            return true;
        }
    }

    // qDebug() << "watched " << watched << ", event " << event->type();

    if (widget && watched == widget) {
        int type = event->type();
        if (type == QEvent::WindowStateChange) {
            //解决mac系统上无边框最小化失效的bug
#ifdef Q_OS_MACOS
            if (widget->windowState() & Qt::WindowMinimized) {
                isMin = true;
            } else {
                if (isMin) {
                    //设置无边框属性
                    widget->setWindowFlags(flags | Qt::FramelessWindowHint);
                    widget->setVisible(true);
                    isMin = false;
                }
            }
#endif
        } else if (type == QEvent::Resize) {
            //重新计算八个描点的区域,描点区域的作用还有就是计算鼠标坐标是否在某一个区域内
            int width = widget->width();
            int height = widget->height();

            //左侧描点区域
            pressedRect[0] = QRect(0, padding, padding, height - padding * 2);
            //右侧描点区域
            pressedRect[1] =
                QRect(width - padding, padding, padding, height - padding * 2);
            //上侧描点区域
            pressedRect[2] = QRect(padding, 0, width - padding * 2, padding);
            //下侧描点区域
            pressedRect[3] =
                QRect(padding, height - padding, width - padding * 2, padding);

            //左上角描点区域
            pressedRect[4] = QRect(0, 0, padding, padding);
            //右上角描点区域
            pressedRect[5] = QRect(width - padding, 0, padding, padding);
            //左下角描点区域
            pressedRect[6] = QRect(0, height - padding, padding, padding);
            //右下角描点区域
            pressedRect[7] =
                QRect(width - padding, height - padding, padding, padding);
        } else if (type == QEvent::HoverMove) {
            //设置对应鼠标形状,这个必须放在这里而不是下面,因为可以在鼠标没有按下的时候识别
            QHoverEvent *hoverEvent = (QHoverEvent *)event;
            QPoint point = hoverEvent->pos();
            if (resizeEnable) {
                if (pressedRect.at(0).contains(point)) {
                    widget->setCursor(Qt::SizeHorCursor);
                } else if (pressedRect.at(1).contains(point)) {
                    widget->setCursor(Qt::SizeHorCursor);
                } else if (pressedRect.at(2).contains(point)) {
                    widget->setCursor(Qt::SizeVerCursor);
                } else if (pressedRect.at(3).contains(point)) {
                    widget->setCursor(Qt::SizeVerCursor);
                } else if (pressedRect.at(4).contains(point)) {
                    widget->setCursor(Qt::SizeFDiagCursor);
                } else if (pressedRect.at(5).contains(point)) {
                    widget->setCursor(Qt::SizeBDiagCursor);
                } else if (pressedRect.at(6).contains(point)) {
                    widget->setCursor(Qt::SizeBDiagCursor);
                } else if (pressedRect.at(7).contains(point)) {
                    widget->setCursor(Qt::SizeFDiagCursor);
                } else {
                    widget->setCursor(Qt::ArrowCursor);
                }
            }

            //根据当前鼠标位置,计算XY轴移动了多少
            int offsetX = point.x() - mousePoint.x();
            int offsetY = point.y() - mousePoint.y();

            //根据按下处的位置判断是否是移动控件还是拉伸控件
            if (moveEnable && mousePressed) {
                widget->move(widget->x() + offsetX, widget->y() + offsetY);
            }

            if (resizeEnable) {
                int rectX = mouseRect.x();
                int rectY = mouseRect.y();
                int rectW = mouseRect.width();
                int rectH = mouseRect.height();

                if (pressedArea.at(0)) {
                    int resizeW = widget->width() - offsetX;
                    if (widget->minimumWidth() <= resizeW) {
                        widget->setGeometry(widget->x() + offsetX, rectY,
                                            resizeW, rectH);
                    }
                } else if (pressedArea.at(1)) {
                    widget->setGeometry(rectX, rectY, rectW + offsetX, rectH);
                } else if (pressedArea.at(2)) {
                    int resizeH = widget->height() - offsetY;
                    if (widget->minimumHeight() <= resizeH) {
                        widget->setGeometry(rectX, widget->y() + offsetY, rectW,
                                            resizeH);
                    }
                } else if (pressedArea.at(3)) {
                    widget->setGeometry(rectX, rectY, rectW, rectH + offsetY);
                } else if (pressedArea.at(4)) {
                    int resizeW = widget->width() - offsetX;
                    int resizeH = widget->height() - offsetY;
                    if (widget->minimumWidth() <= resizeW) {
                        widget->setGeometry(widget->x() + offsetX, widget->y(),
                                            resizeW, resizeH);
                    }
                    if (widget->minimumHeight() <= resizeH) {
                        widget->setGeometry(widget->x(), widget->y() + offsetY,
                                            resizeW, resizeH);
                    }
                } else if (pressedArea.at(5)) {
                    int resizeW = rectW + offsetX;
                    int resizeH = widget->height() - offsetY;
                    if (widget->minimumHeight() <= resizeH) {
                        widget->setGeometry(widget->x(), widget->y() + offsetY,
                                            resizeW, resizeH);
                    }
                } else if (pressedArea.at(6)) {
                    int resizeW = widget->width() - offsetX;
                    int resizeH = rectH + offsetY;
                    if (widget->minimumWidth() <= resizeW) {
                        widget->setGeometry(widget->x() + offsetX, widget->y(),
                                            resizeW, resizeH);
                    }
                    if (widget->minimumHeight() <= resizeH) {
                        widget->setGeometry(widget->x(), widget->y(), resizeW,
                                            resizeH);
                    }
                } else if (pressedArea.at(7)) {
                    int resizeW = rectW + offsetX;
                    int resizeH = rectH + offsetY;
                    widget->setGeometry(widget->x(), widget->y(), resizeW,
                                        resizeH);
                }
            }
        } else if (type == QEvent::MouseButtonPress) {
            //记住鼠标按下的坐标+窗体区域
            QMouseEvent *mouseEvent = (QMouseEvent *)event;
            mousePoint = mouseEvent->pos();
            mouseRect = widget->geometry();

            //判断按下的手柄的区域位置
            if (pressedRect.at(0).contains(mousePoint)) {
                pressedArea[0] = true;
            } else if (pressedRect.at(1).contains(mousePoint)) {
                pressedArea[1] = true;
            } else if (pressedRect.at(2).contains(mousePoint)) {
                pressedArea[2] = true;
            } else if (pressedRect.at(3).contains(mousePoint)) {
                pressedArea[3] = true;
            } else if (pressedRect.at(4).contains(mousePoint)) {
                pressedArea[4] = true;
            } else if (pressedRect.at(5).contains(mousePoint)) {
                pressedArea[5] = true;
            } else if (pressedRect.at(6).contains(mousePoint)) {
                pressedArea[6] = true;
            } else if (pressedRect.at(7).contains(mousePoint)) {
                pressedArea[7] = true;
            } else {
                mousePressed = true;
            }
        } else if (type == QEvent::MouseMove) {
            //改成用HoverMove识别
        } else if (type == QEvent::MouseButtonRelease) {
            //恢复所有
            widget->setCursor(Qt::ArrowCursor);
            mousePressed = false;
            for (int i = 0; i < 8; ++i) {
                pressedArea[i] = false;
            }
        }
    }

    return QWidget::eventFilter(watched, event);
}

3. 控件设计

摄像头列表，使用QTreeWidget
配置与操作，很多个小控件，比较常用的
视频预览，使用拖放一个QWidget即可，后面需要提升到QVideoWidget

在qmake配置文件form.pri中需要添加。CameraRecorder.pro添加也行。

QT += multimedia multimediawidgets

提升显示控件QWidget到QVideoWidget
在这里插入图片描述

4. 加载摄像头列表

在这里插入图片描述

使用 QMediaDevices::videoInputs()获取摄像头信息QCameraDevice列表，
QMediaDevices::audioInputs() 获取音频设备QAudioDevice列表
QCameraDevice和QAudioDevice都可以用QVariant::fromValue()直接保存到控件的Qt::UserRole中，打开设备的时候直接可用。

void frmMain::loadDevices() {
    ui->treeWidgetCamera->setHeaderHidden(true);
    QTreeWidgetItem *computer = new QTreeWidgetItem(ui->treeWidgetCamera);
    computer->setIcon(0, QIcon(":/image/computer.png"));
    computer->setText(0, "此电脑");
    computer->setExpanded(true);
    ui->treeWidgetCamera->addTopLevelItem(computer);

    auto cameras = QMediaDevices::videoInputs();
    for (const QCameraDevice &camera : cameras) {
        QTreeWidgetItem *item = new QTreeWidgetItem(computer);
        item->setIcon(0, QIcon(":/image/camera.png"));
        item->setText(0, camera.description());
        item->setData(0, Qt::UserRole, QVariant::fromValue(camera));

        qDebug() << camera.description() << ", id: " << camera.id();
        auto videoFormats = camera.videoFormats();
        for (auto &format : videoFormats) {
            qDebug() << " - resolution " << format.resolution()
                     << ", frame rate [" << format.minFrameRate() << ", "
                     << format.maxFrameRate() << "], format "
                     << format.pixelFormat();
        }
    }

    //    ui->comboBoxAudioDevice->addItem(tr("Default"), QVariant(QString()));
    for (auto device : QMediaDevices::audioInputs()) {
        auto name = device.description();
        ui->comboBoxAudioDevice->addItem(name, QVariant::fromValue(device));
    }

    ui->comboBoxQuality->addItem("很低", int(QImageCapture::VeryLowQuality));
    ui->comboBoxQuality->addItem("低", int(QImageCapture::LowQuality));
    ui->comboBoxQuality->addItem("正常", int(QImageCapture::NormalQuality));
    ui->comboBoxQuality->addItem("高", int(QImageCapture::HighQuality));
    ui->comboBoxQuality->addItem("很高", int(QImageCapture::VeryHighQuality));
    ui->comboBoxQuality->setCurrentIndex(2);
}

QCameraDevice::videoFormats()可以获取视频设备的分辨率、FPS、图像格式。
每种格式下支持的分辨率和FPS也是不一样的。YUYV数据量很大所以在高分辨率下FPS很低。
如果要高分辨率、最大FPS必须使用JPEG，这样USB带宽才够。

 - resolution  QSize(1920, 1080) , frame rate [ 30 ,  30 ], format  Format_NV12
 - resolution  QSize(1920, 1080) , frame rate [ 30 ,  30 ], format  Format_Jpeg
 - resolution  QSize(1920, 1080) , frame rate [ 25 ,  25 ], format  Format_NV12
 - resolution  QSize(1920, 1080) , frame rate [ 25 ,  25 ], format  Format_Jpeg
 - resolution  QSize(1280, 960) , frame rate [ 30 ,  30 ], format  Format_NV12
 - resolution  QSize(1280, 960) , frame rate [ 30 ,  30 ], format  Format_Jpeg
 - resolution  QSize(1280, 960) , frame rate [ 25 ,  25 ], format  Format_NV12
 - resolution  QSize(1280, 960) , frame rate [ 25 ,  25 ], format  Format_Jpeg
 - resolution  QSize(1280, 720) , frame rate [ 30 ,  30 ], format  Format_NV12
 - resolution  QSize(1280, 720) , frame rate [ 30 ,  30 ], format  Format_Jpeg
 - resolution  QSize(1280, 720) , frame rate [ 25 ,  25 ], format  Format_NV12
 - resolution  QSize(1280, 720) , frame rate [ 25 ,  25 ], format  Format_Jpeg
 - resolution  QSize(640, 480) , frame rate [ 30 ,  30 ], format  Format_NV12
 - resolution  QSize(640, 480) , frame rate [ 30 ,  30 ], format  Format_Jpeg
 - resolution  QSize(640, 480) , frame rate [ 25 ,  25 ], format  Format_NV12
 - resolution  QSize(640, 480) , frame rate [ 25 ,  25 ], format  Format_Jpeg
 - resolution  QSize(1920, 1080) , frame rate [ 5 ,  5 ], format  Format_YUYV
 - resolution  QSize(1280, 960) , frame rate [ 5 ,  5 ], format  Format_YUYV
 - resolution  QSize(1280, 720) , frame rate [ 10 ,  10 ], format  Format_YUYV
 - resolution  QSize(640, 480) , frame rate [ 30 ,  30 ], format  Format_YUYV
 - resolution  QSize(1920, 1080) , frame rate [ 5 ,  5 ], format  Format_NV12
 - resolution  QSize(1280, 960) , frame rate [ 10 ,  10 ], format  Format_NV12
 - resolution  QSize(1280, 720) , frame rate [ 15 ,  15 ], format  Format_NV12
 - resolution  QSize(640, 480) , frame rate [ 30 ,  30 ], format  Format_NV12

5. 打开摄像头及预览

双击树形控件的时候打开该节点的摄像头

void frmMain::on_treeWidgetCamera_itemDoubleClicked(QTreeWidgetItem *item,
                                                   int column) {

   // https://blog.csdn.net/u011442415/article/details/129370856

   auto cameraDevice = item->data(0, Qt::UserRole).value<QCameraDevice>();
   qDebug() << "激活摄像头" << cameraDevice.description() << ", id "
            << cameraDevice.id();

   // 加载摄像头支持的参数到控件
   loadCameraProperties(cameraDevice);
   setCamera(cameraDevice);

   auto videoFormats = cameraDevice.videoFormats();
   for (auto &format : videoFormats) {
       qDebug() << "resolution " << format.resolution() << ", frame rate ["
                << format.minFrameRate() << ", " << format.maxFrameRate()
                << "], format " << format.pixelFormat();
   }

   // https://doc.qt.io/qt-6/qcameraformat.html
   startCamera();
}

由于每个摄像头支持的格式、分辨率、FPS都不一样，所以得把这个信息保留下来，留到后面切换格式及分辨率的时候用。

void frmMain::loadCameraProperties(const QCameraDevice &camera) {
    ui->comboBoxPixFormat->clear();
    cameraFormats = camera.videoFormats();

    QSet<QVideoFrameFormat::PixelFormat> pixFormats;
    for (auto &format : cameraFormats) {
        if (!pixFormats.contains(format.pixelFormat())) {
            ui->comboBoxPixFormat->addItem(
                QVideoFrameFormat::pixelFormatToString(format.pixelFormat()),
                format.pixelFormat());
            pixFormats.insert(format.pixelFormat());
        }
    }

    ui->comboBoxPixFormat->setCurrentIndex(0);
}

然后就是用QCamera建立设备,并设置到QMediaCaptureSession中。
建立QMediaRecorder，QImageCapture，图片截图可以设置成PNG，其他的可以自己改。

captureSession.setVideoOutput(ui->widgetVideo) 将输出显示设置到刚才提升的那个QVideoWidget控件。

void frmMain::setCamera(const QCameraDevice &cameraDevice) {
    curCamera.reset(new QCamera(cameraDevice));
    captureSession.setCamera(curCamera.data());

    if (!mediaRecorder) {
        mediaRecorder.reset(new QMediaRecorder);
        captureSession.setRecorder(mediaRecorder.data());
    }

    if (!imgCapture) {
        imgCapture.reset(new QImageCapture);
        imgCapture->setFileFormat(QImageCapture::PNG);

        captureSession.setImageCapture(imgCapture.get());
    }

    captureSession.setVideoOutput(ui->widgetVideo);
}

最后就可以通过curCamera->start() 打开摄像头了。

6. 截图

设置图像质量，imgCapture->captureToFile()就完成截图了，默认是在“图片”目录，可以使用绝对路径替换。

void frmMain::on_toolButtonCapture_clicked() {
    // https://doc.qt.io/qt-6.5/qimagecapture.html
    if (!curCamera) {
        QUIHelper::showMessageBoxError("没有打开摄像头");
        return;
    }

    auto quality =
        ui->comboBoxQuality->currentData().value<QImageCapture::Quality>();
    imgCapture->setQuality(quality);

    QDateTime currentDateTime = QDateTime::currentDateTime();
    QString fileName = currentDateTime.toString("yyyy-MM-dd_hhmmss");
    imgCapture->captureToFile(fileName);

    qDebug() << "已截图" << fileName;
}

7. 录音录像

QMediaFormat可以设置音频和视频编码格式，音频一般aac，视频H264即可。（H265需要硬件支持，一般不行就会回退到H264）。
Quality, OutputLocation, EncodingMode设置完毕之后record()即可开启录像了。保存格式是mp4，默认在“视频”目录。

void frmMain::on_toolButtonRecord_clicked() {
    if (!curCamera) {
        QUIHelper::showMessageBoxError("没有打开摄像头");
        return;
    }
    if (!recording) {
        recording = true;

        if (ui->checkBoxAudio->isChecked()) {
            auto audioDevice =
                ui->comboBoxAudioDevice->currentData().value<QAudioDevice>();
            audioInput.reset(new QAudioInput(audioDevice));
            captureSession.setAudioInput(audioInput.get());
        } else {
            captureSession.setAudioInput(nullptr);
        }

        QMediaFormat format;
        format.setAudioCodec(QMediaFormat::AudioCodec::Unspecified); // aac
        format.setVideoCodec(QMediaFormat::VideoCodec::Unspecified); // h264
        mediaRecorder->setMediaFormat(format);

        // 值一样，控件共用
        auto quality =
            ui->comboBoxQuality->currentData().value<QMediaRecorder::Quality>();
        mediaRecorder->setQuality(quality);
        QDateTime currentDateTime = QDateTime::currentDateTime();
        QString fileName = currentDateTime.toString("yyyy-MM-dd_hhmmss");
        mediaRecorder->setOutputLocation(QUrl::fromLocalFile(fileName));
        mediaRecorder->setEncodingMode(QMediaRecorder::ConstantQualityEncoding);
        mediaRecorder->record();
        ui->toolButtonRecord->setText("停止");
    } else {
        recording = false;

        mediaRecorder->stop();
        ui->toolButtonRecord->setText("录像");
    }
}