Feature/aff on vff#515
Conversation
|
Pull request must be merged with a description containing the required fields, Summary: If there is no jira releated to this change, please put 'Jira: NO-JIRA'. Description can be changed by editing the top comment on your pull request and making a new commit. |
There was a problem hiding this comment.
Pull request overview
This PR introduces end-to-end “first frame received” notifications (including audio first-frame support) by wiring GStreamer first-frame signals (and an audio-sink probe fallback) through gstplayer task scheduling, server forwarding, IPC transport, and client callbacks, with accompanying unit/component test coverage.
Changes:
- Add first-frame detection during element setup (video signal + audio signal / audio-sink probe fallback) and schedule first-frame tasks.
- Propagate first-frame events through server internals, IPC (
FirstFrameReceivedEvent), and client callbacks (notifyFirstFrameReceived). - Add/extend unit and component tests to validate first-frame notifications and non-regressions.
Reviewed changes
Copilot reviewed 63 out of 63 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unittests/media/server/mocks/gstplayer/GstGenericPlayerPrivateMock.h | Extends gstplayer private mock API for first-frame/probe lifecycle hooks. |
| tests/unittests/media/server/mocks/gstplayer/GstGenericPlayerClientMock.h | Adds client mock notification for first-frame received. |
| tests/unittests/media/server/mocks/gstplayer/GenericPlayerTaskFactoryMock.h | Adds task-factory mock for FirstFrameReceived task creation. |
| tests/unittests/media/server/main/mediaPipeline/CallbackTest.cpp | Adds server callback forwarding tests for first-frame received. |
| tests/unittests/media/server/ipc/mediaPipelineModuleService/MediaPipelineModuleServiceTestsFixture.h | Adds fixture helpers for first-frame IPC event testing. |
| tests/unittests/media/server/ipc/mediaPipelineModuleService/MediaPipelineModuleServiceTestsFixture.cpp | Adds matcher/helpers and send path for FirstFrameReceivedEvent. |
| tests/unittests/media/server/ipc/mediaPipelineModuleService/MediaPipelineModuleServiceTests.cpp | Adds module-service test for first-frame event emission. |
| tests/unittests/media/server/gstplayer/genericPlayer/tasksTests/SetupElementTest.cpp | Adds unit tests for first-video/audio-frame signal hookup. |
| tests/unittests/media/server/gstplayer/genericPlayer/tasksTests/GenericPlayerTaskFactoryTest.cpp | Adds unit test coverage for createFirstFrameReceived. |
| tests/unittests/media/server/gstplayer/genericPlayer/tasksTests/FirstFrameReceivedTest.cpp | New unit tests for FirstFrameReceived task execution behavior. |
| tests/unittests/media/server/gstplayer/genericPlayer/GstGenericPlayerPrivateTest.cpp | Adds tests for scheduling first video/audio frame tasks. |
| tests/unittests/media/server/gstplayer/genericPlayer/common/GenericTasksTestsContext.h | Extends shared test context with first-frame callbacks. |
| tests/unittests/media/server/gstplayer/genericPlayer/common/GenericTasksTestsBase.h | Adds helper methods for first-frame signal/probe testing flows. |
| tests/unittests/media/server/gstplayer/genericPlayer/common/GenericTasksTestsBase.cpp | Implements first-frame signal hookup expectations + trigger helpers. |
| tests/unittests/media/server/gstplayer/CMakeLists.txt | Adds FirstFrameReceived unit test to build. |
| tests/unittests/media/client/mocks/ipc/MediaPipelineIpcClientMock.h | Adds IPC client mock callback for first-frame received. |
| tests/unittests/media/client/main/mediaPipeline/CallbackTest.cpp | Adds client callback forwarding test for first-frame received. |
| tests/unittests/media/client/ipc/mediaPipelineIpc/CallbackTest.cpp | Adds IPC callback test handling FirstFrameReceivedEvent. |
| tests/unittests/media/client/ipc/mediaPipelineIpc/base/MediaPipelineIpcTestBase.h | Adds event tag/callback slot for FirstFrameReceivedEvent. |
| tests/unittests/media/client/ipc/mediaPipelineIpc/base/MediaPipelineIpcTestBase.cpp | Adds subscribe/unsubscribe expectations for FirstFrameReceivedEvent. |
| tests/componenttests/server/tests/mediaPipeline/UnderflowTest.cpp | Updates expectations impacted by additional signal-list/free calls. |
| tests/componenttests/server/tests/mediaPipeline/FirstFrameNotificationTest.cpp | New server component tests for first video/audio frame IPC emission. |
| tests/componenttests/server/tests/CMakeLists.txt | Adds new server component test to build. |
| tests/componenttests/server/stubs/ClientStub.cpp | Subscribes client stub to FirstFrameReceivedEvent. |
| tests/componenttests/client/tests/mse/FirstFrameNotificationTest.cpp | New client component test validating first-frame callbacks for A/V. |
| tests/componenttests/client/tests/base/MediaPipelineTestMethods.h | Adds helper methods for first-frame event expectations/sends. |
| tests/componenttests/client/tests/base/MediaPipelineTestMethods.cpp | Implements first-frame expectation + event send helpers. |
| tests/componenttests/client/stubs/MediaPipelineModuleStub.h | Adds stub method to send FirstFrameReceivedEvent. |
| tests/componenttests/client/stubs/MediaPipelineModuleStub.cpp | Implements stub event construction and send for FirstFrameReceivedEvent. |
| tests/componenttests/client/CMakeLists.txt | Adds new client component test to build. |
| tests/common/publicClientMocks/MediaPipelineClientMock.h | Adds first-frame callback to public client mock. |
| proto/mediapipelinemodule.proto | Adds FirstFrameReceivedEvent protobuf message definition. |
| openspec/changes/audio-first-frame/tasks.md | New openspec task checklist for audio first-frame change. |
| openspec/changes/audio-first-frame/specs/audio-first-frame/spec.md | New openspec requirements spec for audio first-frame. |
| openspec/changes/audio-first-frame/proposal.md | New openspec proposal describing rationale and scope. |
| openspec/changes/audio-first-frame/design.md | New openspec design doc for detection/probe/guard decisions. |
| openspec/changes/audio-first-frame/.openspec.yaml | New openspec metadata for the change. |
| media/server/main/source/MediaPipelineServerInternal.cpp | Adds server internal forwarding for first-frame received by source type. |
| media/server/main/include/MediaPipelineServerInternal.h | Declares notifyFirstFrameReceived on server internal interface. |
| media/server/ipc/source/MediaPipelineClient.cpp | Sends FirstFrameReceivedEvent over IPC. |
| media/server/ipc/include/MediaPipelineClient.h | Declares notifyFirstFrameReceived on IPC client interface. |
| media/server/gstplayer/source/Utils.cpp | Adds first-frame signal lookup utility (video + audio signal names). |
| media/server/gstplayer/source/tasks/generic/Stop.cpp | Clears audio first-frame guard/probe state on stop. |
| media/server/gstplayer/source/tasks/generic/SetupElement.cpp | Connects first-frame callbacks and installs audio sink-pad probe fallback. |
| media/server/gstplayer/source/tasks/generic/GenericPlayerTaskFactory.cpp | Adds task creation for FirstFrameReceived. |
| media/server/gstplayer/source/tasks/generic/FirstFrameReceived.cpp | New task that notifies client of first-frame received per source type. |
| media/server/gstplayer/source/tasks/generic/AttachSource.cpp | Resets audio first-frame guard on attach/reattach. |
| media/server/gstplayer/source/GstGenericPlayer.cpp | Adds scheduling APIs and audio probe lifecycle management + guard state. |
| media/server/gstplayer/interface/IGstGenericPlayerClient.h | Adds gstplayer client callback notifyFirstFrameReceived(MediaSourceType). |
| media/server/gstplayer/include/Utils.h | Declares getFirstFrameSignalName utility. |
| media/server/gstplayer/include/tasks/IGenericPlayerTaskFactory.h | Declares createFirstFrameReceived factory method. |
| media/server/gstplayer/include/tasks/generic/GenericPlayerTaskFactory.h | Declares override for createFirstFrameReceived. |
| media/server/gstplayer/include/tasks/generic/FirstFrameReceived.h | New header for FirstFrameReceived task. |
| media/server/gstplayer/include/IGstGenericPlayerPrivate.h | Extends private interface for first-frame scheduling + probe management. |
| media/server/gstplayer/include/GstGenericPlayer.h | Adds new IGstGenericPlayerPrivate method implementations. |
| media/server/gstplayer/include/GenericPlayerContext.h | Adds audio first-frame guard + probe state to context. |
| media/server/gstplayer/CMakeLists.txt | Builds new FirstFrameReceived task source. |
| media/public/include/IMediaPipelineClient.h | Adds public client callback notifyFirstFrameReceived(sourceId). |
| media/client/main/source/MediaPipeline.cpp | Forwards first-frame received callback to application client. |
| media/client/main/include/MediaPipeline.h | Declares notifyFirstFrameReceived on MediaPipeline IPC client implementation. |
| media/client/ipc/source/MediaPipelineIpc.cpp | Subscribes to FirstFrameReceivedEvent and forwards to IPC client. |
| media/client/ipc/interface/IMediaPipelineIpcClient.h | Adds IPC client callback notifyFirstFrameReceived(sourceId). |
| media/client/ipc/include/MediaPipelineIpc.h | Declares onFirstFrameReceived handler. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| #### Scenario: Existing protocol remains unchanged | ||
| - **WHEN** audio first-frame support is added | ||
| - **THEN** the system continues to use the existing `FirstFrameReceivedEvent` transport shape | ||
| - **THEN** no new public callback name or protobuf message is required No newline at end of file |
| * Initalise the control state to running for this test application. | ||
| * Initalise a audio video media session playing. |
|
tests/unittests/media/server/gstplayer/genericPlayer/tasksTests/FirstFrameReceivedTest.cpp:33:103: error: syntax error [syntaxError] |
| <<<<<<< HEAD | ||
|
|
||
| TEST_F(FirstFrameReceivedTest, shouldNotifyFirstAudioFrameReceived) | ||
| { | ||
| firebolt::rialto::server::tasks::generic::FirstFrameReceived task{m_context, m_gstPlayer, &m_gstPlayerClient, | ||
| kAudioSourceType}; | ||
|
|
||
| EXPECT_CALL(m_gstPlayerClient, notifyFirstFrameReceived(kAudioSourceType)); | ||
|
|
||
| task.execute(); | ||
| } | ||
| ======= | ||
| >>>>>>> master |
| void GenericTasksTestsBase::shouldSetVideoUnderflowCallback() | ||
| { | ||
| ASSERT_TRUE(testContext->m_videoUnderflowCallback); | ||
| EXPECT_CALL(testContext->m_gstPlayer, scheduleVideoUnderflow()); | ||
| } | ||
|
|
||
| void GenericTasksTestsBase::shouldSetFirstVideoFrameCallback() | ||
| { | ||
| ASSERT_TRUE(testContext->m_firstVideoFrameCallback); | ||
| EXPECT_CALL(testContext->m_gstPlayer, scheduleFirstVideoFrameReceived()); | ||
| } | ||
|
|
||
| void GenericTasksTestsBase::shouldSetupBaseParse() |
) Summary: Fix reattaching audio source at dynamic audio codec change. Type: Fix Test Plan: UT/CT, Fullstack Jira: RDKEMW-17771 --------- Co-authored-by: smudri85 <smudri85@gmail.com> Co-authored-by: Marcin Wojciechowski <marcin.wojciechowski@sky.uk> Co-authored-by: Sasa Mudri <Sasa_Mudri@comcast.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
tests/unittests/media/server/gstplayer/genericPlayer/tasksTests/FirstFrameReceivedTest.cpp:33:103: error: syntax error [syntaxError] |
| EXPECT_CALL(*testContext->m_gstWrapper, gstObjectUnref(_)); | ||
| } | ||
|
|
||
| void GenericTasksTestsBase::expectSetupVideoDecoderElementWithFirstVideoFrameCallback() |
| void GenericTasksTestsBase::shouldSetVideoUnderflowCallback() | ||
| { | ||
| ASSERT_TRUE(testContext->m_videoUnderflowCallback); | ||
| EXPECT_CALL(testContext->m_gstPlayer, scheduleVideoUnderflow()); | ||
| } | ||
|
|
||
| void GenericTasksTestsBase::shouldSetFirstVideoFrameCallback() | ||
| { | ||
| ASSERT_TRUE(testContext->m_firstVideoFrameCallback); | ||
| EXPECT_CALL(testContext->m_gstPlayer, scheduleFirstVideoFrameReceived()); | ||
| } | ||
|
|
||
| void GenericTasksTestsBase::shouldSetupBaseParse() |
| MediaSourceType type = sourceIter->first; | ||
|
|
||
| m_gstPlayer->removeSource(type); | ||
| m_needMediaDataTimers.erase(type); | ||
| m_noAvailableSamplesCounter.erase(type); |
| else if (isAudioSink(*m_gstWrapper, m_element)) | ||
| { | ||
| GstPad *sinkPad = m_gstWrapper->gstElementGetStaticPad(m_element, "sink"); | ||
| if (sinkPad) | ||
| { | ||
| gulong probeId = | ||
| m_gstWrapper->gstPadAddProbe(sinkPad, GST_PAD_PROBE_TYPE_BUFFER, firstAudioFrameProbeCallback, | ||
| &m_player, nullptr); | ||
|
|
||
| if (probeId != 0) | ||
| { | ||
| RIALTO_SERVER_LOG_INFO("Installed first audio frame fallback probe on sink"); | ||
| m_player.setAudioFirstFrameFallbackProbe(sinkPad, probeId); | ||
| } | ||
| else | ||
| { | ||
| m_gstWrapper->gstObjectUnref(sinkPad); | ||
| } | ||
| } |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| m_context.audioSourceRemoved = true; | ||
| if (m_gstPlayerClient) | ||
| { | ||
| m_gstPlayerClient->invalidateActiveRequests(m_type); | ||
| } | ||
| auto sourceElem = m_context.streamInfo.find(m_type); | ||
| if (sourceElem == m_context.streamInfo.end()) | ||
| { | ||
| RIALTO_SERVER_LOG_WARN("Failed to remove source - streamInfo not found"); | ||
| return; | ||
| } |
| RemoveSource::RemoveSource(GenericPlayerContext &context, IGstGenericPlayerPrivate &player, | ||
| IGstGenericPlayerClient *client, | ||
| const std::shared_ptr<firebolt::rialto::wrappers::IGstWrapper> &gstWrapper, | ||
| const MediaSourceType &type) | ||
| : m_context{context}, m_player{player}, m_gstPlayerClient{client}, m_gstWrapper{gstWrapper}, m_type{type} | ||
| { |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| void GenericTasksTestsBase::shouldSetVideoUnderflowCallback() | ||
| { | ||
| ASSERT_TRUE(testContext->m_videoUnderflowCallback); | ||
| EXPECT_CALL(testContext->m_gstPlayer, scheduleVideoUnderflow()); | ||
| } |
| EXPECT_CALL(*m_gstWrapperMock, gstObjectRef(m_audioDecoder)).WillOnce(Return(m_audioDecoder)); | ||
|
|
||
| EXPECT_CALL(*m_gstWrapperMock, gstElementFactoryListIsType(m_elementFactory, GST_ELEMENT_FACTORY_TYPE_DECODER)) | ||
| .WillOnce(Return(TRUE)); | ||
| EXPECT_CALL(*m_gstWrapperMock, | ||
| gstElementFactoryListIsType(m_elementFactory, GST_ELEMENT_FACTORY_TYPE_MEDIA_VIDEO)) | ||
| .WillOnce(Return(FALSE)); | ||
| EXPECT_CALL(*m_gstWrapperMock, | ||
| gstElementFactoryListIsType(m_elementFactory, | ||
| GST_ELEMENT_FACTORY_TYPE_DECODER | | ||
| GST_ELEMENT_FACTORY_TYPE_MEDIA_AUDIO)) | ||
| .WillOnce(Return(TRUE)) | ||
| .RetiresOnSaturation(); | ||
| EXPECT_CALL(*m_gstWrapperMock, | ||
| gstElementFactoryListIsType(m_elementFactory, | ||
| GST_ELEMENT_FACTORY_TYPE_SINK | GST_ELEMENT_FACTORY_TYPE_MEDIA_VIDEO)) | ||
| .WillOnce(Return(FALSE)) | ||
| .RetiresOnSaturation(); | ||
|
|
| void GstGenericPlayer::setAudioFirstFrameFallbackProbe(GstPad *pad, gulong id) | ||
| { | ||
| clearAudioFirstFrameFallbackProbe(); | ||
|
|
||
| m_context.audioFirstFrameProbePad = pad; | ||
| m_context.audioFirstFrameProbeId = id; | ||
| } |
| GstPad *sinkPad = m_gstWrapper->gstElementGetStaticPad(m_element, "sink"); | ||
| if (sinkPad) | ||
| { | ||
| gulong probeId = | ||
| m_gstWrapper->gstPadAddProbe(sinkPad, GST_PAD_PROBE_TYPE_BUFFER, firstAudioFrameProbeCallback, | ||
| &m_player, nullptr); |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 24 out of 24 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
tests/unittests/media/server/gstplayer/genericPlayer/common/GenericTasksTestsBase.cpp:1120
GenericTasksTestsBase.hdeclaresshouldSetFirstVideoFrameCallback()andshouldSetFirstAudioFrameCallback(), andSetupElementTestcalls them, but neither is defined inGenericTasksTestsBase.cppanymore (the video one was removed in this hunk). This will break the unit test build/link. Reintroduce the video helper and add the new audio helper so the new test can assert scheduling.
void GenericTasksTestsBase::shouldSetupBaseParse()
{
EXPECT_CALL(*testContext->m_gstWrapper, gstBaseParseSetPtsInterpolation(_, FALSE));
expectSetupBaseParseElement();
}
| void GstGenericPlayer::scheduleFirstAudioFrameReceived() | ||
| { | ||
| if (m_context.firstAudioFrameReceived || !m_workerThread) | ||
| { | ||
| return; | ||
| } | ||
|
|
||
| m_context.firstAudioFrameReceived = true; | ||
| m_workerThread->enqueueTask(m_taskFactory->createFirstFrameReceived(m_context, *this, MediaSourceType::AUDIO)); | ||
| } |
| void GstGenericPlayer::clearAudioFirstFrameFallbackProbe() | ||
| { | ||
| if (m_context.audioFirstFrameProbePad && m_context.audioFirstFrameProbeId != 0) | ||
| { | ||
| m_gstWrapper->gstPadRemoveProbe(m_context.audioFirstFrameProbePad, m_context.audioFirstFrameProbeId); | ||
| } | ||
|
|
||
| clearAudioFirstFrameFallbackProbeState(); | ||
| } | ||
|
|
||
| void GstGenericPlayer::clearAudioFirstFrameFallbackProbeState() | ||
| { | ||
| if (m_context.audioFirstFrameProbePad) | ||
| { | ||
| m_gstWrapper->gstObjectUnref(m_context.audioFirstFrameProbePad); | ||
| m_context.audioFirstFrameProbePad = nullptr; | ||
| } | ||
|
|
||
| m_context.audioFirstFrameProbeId = 0; | ||
| } |
| /** | ||
| * @brief True when first audio frame has already been scheduled for the current audio source lifecycle. | ||
| */ | ||
| std::atomic_bool firstAudioFrameReceived{false}; | ||
|
|
||
| /** | ||
| * @brief Fallback probe id for first audio frame detection on sink pad. | ||
| */ | ||
| gulong audioFirstFrameProbeId{0}; | ||
|
|
||
| /** | ||
| * @brief Fallback sink pad that owns audio first frame probe. | ||
| */ | ||
| GstPad *audioFirstFrameProbePad{nullptr}; |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| void GenericTasksTestsBase::shouldSetVideoUnderflowCallback() | ||
| { | ||
| ASSERT_TRUE(testContext->m_videoUnderflowCallback); | ||
| EXPECT_CALL(testContext->m_gstPlayer, scheduleVideoUnderflow()); | ||
| } |
| RIALTO_SERVER_LOG_DEBUG("Executing Stop"); | ||
| m_context.firstAudioFrameReceived = false; | ||
| m_player.clearAudioFirstFrameFallbackProbe(); |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| void GenericTasksTestsBase::shouldSetVideoUnderflowCallback() | ||
| { | ||
| ASSERT_TRUE(testContext->m_videoUnderflowCallback); | ||
| EXPECT_CALL(testContext->m_gstPlayer, scheduleVideoUnderflow()); | ||
| } |
| EXPECT_CALL(*m_gstWrapperMock, gstElementFactoryListIsType(m_elementFactory, GST_ELEMENT_FACTORY_TYPE_DECODER)) | ||
| .WillOnce(Return(TRUE)); | ||
| EXPECT_CALL(*m_gstWrapperMock, | ||
| gstElementFactoryListIsType(m_elementFactory, GST_ELEMENT_FACTORY_TYPE_MEDIA_VIDEO)) | ||
| .WillOnce(Return(FALSE)); |
| GCallback m_audioUnderflowCallback; | ||
| GCallback m_videoUnderflowCallback; | ||
| GCallback m_firstVideoFrameCallback; | ||
| GCallback m_firstAudioFrameCallback; | ||
| GCallback m_childAddedCallback; | ||
| GCallback m_childRemovedCallback; |
No description provided.