Share via

Test Cases for the DirectShow Playback and Latency Tests

  • Article

Other versions of this page are also available for the following:

Windows Mobile Not SupportedWindows Embedded CE Supported

8/27/2008

This section shows the test cases as provided in the XML file, the tests in the test binary, and the verifiers in the test binary.

Test Cases in the DirectShow Playback and Latency Tests

The following table shows the test cases for the DirectShow Playback and Latency Tests.

Test case Description

0

Build verification test (BVT): Graph building - adding a source filter - Verify that source filter gets added, for this particular media

For the media clip specified, it will verify if we have a source filter that will match.

1

BVT: Graph building - preferential filter loading - Verify that the correct graph gets built, forcing certain filters to be present

Currently configured for a WMV clip. Will force loading of the WM Source filter and then tries to create the entire filter graph.

Can force loading of other filter by specifying the colon separated names in the <FilterList></FilterList> tag

2

BVT: Graph building - Render pin to a complete graph - Verify that we can render a graph from the source filter onwards

Add the source filter for the URL. Try to connect the unconnected pins of the source filter.

3

BVT: Graph building - intelligent connect source to sink - Verify that we can connect the Source Filter to the renderer

Currently configured for a WMV clip.

Will force loading of the source filter and the renderer filter specified. They are specified as colon separated names in the <FilterList></FilterList>i tag. First filter it encounters, it considers the source filter, second filter it considers as the renderer.

4

BVT: Graph building - build graph for supported media - Verify that RenderFile will build the correct graph

Calls RenderFile on the URL specified

Verifies that the following filters are in the graph: WM Source filter, Object dispatcher, WMV decoder, Video Renderer. This can be updated by changing the <CorrectGraph></CorrectGraph> tag

5

BVT: Graph building - build graph for supported media and query interfaces - Verify that the correct interfaces exist in the graph

Calls RenderFile on the URL specified. Queries the created graph for the following interfaces: IMediaControl, IMediaEvent, IMediaSeeking

100

Playback duration test: Graph playback - manual playback test - playback a clip, manually verify and pass/fail the test

Calls RenderFile on the URL specified. Plays the media clip and waits for manual verification from the tester as to whether the clip played as expected. This test expects tester interaction.

By default will play the clip from beginning to end but can specify the % positions to play from as a comma separated list in the <PositionList></PositionList> tag. Test assumes first position is start position, second position specified is stop position.

101

Playback duration test: Graph playback - manual playback test - copy a clip to hard disk first and then playback clip, manually verify and pass/fail test

Copies the media clip specified in the URL down to device first. Can specify location to copy to in the <DownloadTo></DownloadTo> tag. Currently it is set as \Hard Disk\.

Calls RenderFile on the URL specified. Plays the media clip and waits for manual verification from the tester as to whether the clip played as expected. This test expects tester interaction.

By default will play the clip from beginning to end but can specify the % positions to play from as a comma separated list in the <PositionList></PositionList> tag. Test assumes first position is start position, second position specified is stop position.

102

Playback duration test: Graph playback - end-end playback test - verify duration, verify it played till end

Calls RenderFile on the URL specified. Plays the media clip.

By default will play the clip from beginning to end but can specify the % positions to play from as a comma separated list in the <PositionList></PositionList> tag. Test assumes first position is start position, second position specified is stop position.

200

Startup latency test: Graph playback - end-end playback test - verify startup latency

Calls RenderFile on the URL specified. Plays the media clip. Verifies that the time from asking the clip to play to the time the first sample comes out of the decoder is under a specified threshold.

By default will play the clip from beginning to end but can specify the % positions to play from as a comma separated list in the <PositionList></PositionList> tag. Test assumes first position is start position, second position specified is stop position.

201

Startup latency test: Graph playback - end-end playback test - verify startup latency, higher bitrate content

Calls RenderFile on the URL specified. Plays the media clip. Verifies that the time from asking the clip to play to the time the first sample comes out of the decoder is under a specified threshold.

By default will play the clip from beginning to end but can specify the % positions to play from as a comma separated list in the <PositionList></PositionList> tag. Test assumes first position is start position, second position specified is stop position.

300

State change/latency test: Graph playback - state change test - verify state change (Play to Pause) and measure latency

Calls RenderFile on the URL specified. Plays the media clip. Verifies how long it takes to change graph states, from play to pause & back.

Can specify the state changes needed by updating comma separated list in the <StateChangeSequence></StateChangeSequence> tag.

  • First item is the initial state to start from, currently Stopped
  • Type = what states to switch between. Valid options: PlayPause, PlayStop, PauseStop, PlayPauseStop, RandomSequence, currently PlayPause
  • HowMany= how many times to switch the states and measure latency, currently 50
  • TimeBetween= time in ms to sleep between each state change, currently 50 ms

301

State change/latency test: Graph playback - state change test - verify state change (Random state changes) and measure latency

Calls RenderFile on the URL specified. Plays the media clip. Verifies how long it takes to change graph states. It will randomly switch between play, pause and stop states.

Can specify the state changes needed by updating comma separated list in the <StateChangeSequence></StateChangeSequence> tag.

  • First item is the initial state to start from, currently Stopped
  • Type = what states to switch between. Valid options: PlayPause, PlayStop, PauseStop, PlayPauseStop, RandomSequence, currently RandomeSequence
  • HowMany= how many times to switch the states and measure latency, currently 50
  • TimeBetween= time in ms to sleep between each state change, currently 500 ms

302

State change/latency test: Graph playback - state change test - verify state (Play to Stop) change and measure latency

Calls RenderFile on the URL specified. Plays the media clip. Verifies how long it takes to change graph states. It will switch between play and stop states.

Can specify the state changes needed by updating comma separated list in the <StateChangeSequence></StateChangeSequence> tag.

  • First item is the initial state to start from, currently Stopped
  • Type = what states to switch between. Valid options: PlayPause, PlayStop, PauseStop, PlayPauseStop, RandomSequence, currently PlayStop
  • HowMany= how many times to switch the states and measure latency, currently 50
  • TimeBetween= time in ms to sleep between each state change, currently 500 ms

303

State change/latency test: Graph playback - run, pause, run - verify latency of first sample to come out of the decoder, after calling Run

Calls RenderFile on the URL specified. Plays the media clip. Pauses the clip and then plays the again. Measures how long it takes to get the first sample out of the video decoder after the call to run/play.

Tests in the Test Binary

The following table shows the individual tests found in the test binary.

Test Description

EmptyGraphQueryInterfaceTest

Graph build

  1. Queries an empty filter graph for common interfaces.
  2. Fails the test if some of the common interfaces aren't returned. The essential interfaces are considered to be IMediaControl, IMediaEvent, IMediaSeeking.

AddSourceFilterTest

Graph build

  1. Adds a source filter for media that is supported.
  2. Fails the test if unsuccessful.

AddUnsupportedSourceFilterTest

Graph build

  1. Adds a source filter for media that is unsupported.
  2. Fails the test if successful.

BuildGraphTest

Graph build

  1. Starts verification
  2. Builds a filter graph using RenderFile for specified media.
  3. Gets verification results
  4. Fails the test if unsuccessful or if verification fails

Verifiers:

  • Handles BuildGraphLatency within the test
  • Any appropriate verifier

BuildGraphMultipleTest

Graph build

  1. Repeatedly builds a filter graph using RenderFile for specified media. The number of repetitions is 10 by default.
  2. Fails the test if any of the RenderFile calls is unsuccessful.

BuildGraphQueryInterfaceTest

Graph build

  1. Builds a filter graph using RenderFile for specified media.
  2. Fails the test if unsuccessful.
  3. Queries the graph for essential interfaces.
  4. Fails the test if any of the essential interfaces aren't returned - the essential interfaces are considered to be IMediaControl, IMediaEvent, IMediaSeeking.

BuildGraphUnsupportedMediaTest

Graph build

  1. Builds a filter graph using RenderFile for media that is unsupported.
  2. Fails the test if successful.

BuildMultipleGraphTest

Graph build

  1. Builds multiple filter graphs for the list of media specified in the config file for this test.
  2. Fails the test if building the graph fails for any of the media specified.

RenderPinTest

Graph build

  1. Adds a source filter for the media that is supported and renders all the unconnected pins of the source filter.
  2. The test fails if adding the source filter does not succeed or rendering the unconnected pins does not succeed.

ConnectSourceFilterToRendererTest

Graph build

  1. Retrieves the source and renderer filters specified in config file.
  2. Adds the source and renderer filters.
  3. Loads the source filter with the specified media url.
  4. Connects the zeroth pins of the source and renderer filter using intelligent connect.
  5. Test fails if adding the source and renderer filter fail or if setting the media fails or if connecting the source and renderer fails.

BuildGraphPreLoadFilterTest

Graph build

  1. Retrieve the list of filters to be preloaded specified in the config file
  2. Load the filters into an empty graph.
  3. RenderFile the specified media.
  4. Verify that the pre-loaded filters have at-least one input pin (if any) and output pin (if any) connected.

ManualPlaybackTest

Graph playback

  1. Builds graph for the media specified.
  2. Sets start and stop positions (if any)
  3. Starts playback and waits for completion with a timeout of twice the expected playback duration.
  4. Asks the tester if playback was successful with no or few glitches?
  5. Returns pass/fail depending on response.

PlaybackTest

Graph playback

  1. Builds graph for the media specified.
  2. Enables specified verifiers.
  3. VerifyPlaybackDuration is handled intrinsically.
  4. Sets start and stop positions (if any)
  5. Starts verification.
  6. Changes graph state to Running
  7. Waits for completion with a timeout of twice the duration
  8. Gets verification results
  9. Fails the test if wait for completion fails or any of the verifiers fail

Verifiers:

Do not mix VerifyPlaybackDuration and StartupLatency. The test internally uses StartupLatency to determine playback duration.

PlaybackDurationTest

Graph playback

  1. Builds graph for the media specified.
  2. Enables VerifyPlaybackDuration verifier
  3. Sets start and stop positions (if any)
  4. Starts verification
  5. Changes graph state to Running
  6. Waits for completion with a timeout of twice the duration
  7. Query the verifier for the first sample time and the EOS time.
  8. Calculate the EC completion latency time and the actual playback duration from first sample to EOS and compare against a default threshold.

MultiplePlaybackTest

Graph playback

  1. For each of the media specified, instantiate a test graph object and set the media.
  2. Build graphs for each of the media
  3. Get event handles from each of the test graphs
  4. Change each of the test graphs to state Running
  5. Wait for completion on all the handles with infinite timeout
  6. Test succeeds the wait succeeds.

StateChangeTest

State change

  1. Builds graph for the media specified.
  2. VerifyStateChangeLatency is handled intrinsically.
  3. Change states according to the sequence and count specified.
  4. Fails the test the state change latency was over the threshold or if any of the state-changes fail

Verifiers:

Only VerifyStateChangeLatency

Run_Pause_Run_Test

State change

  1. Builds graph for the media specified.
  2. Enables specified verifiers.
  3. Starts verification.
  4. Sets the graph in Running state.
  5. Waits
  6. Sets the graph in Paused state
  7. Waits
  8. Start verification
  9. Set the graph in running state
  10. Get verification results
  11. Fails the test if any of the state changes fail or any of the verifiers fail

Verifiers:

Any appropriate verifier

Verifiers in the Test Binary

Most of the verifiers work by inserting a tap filter in the existing DirectShow filter graph. When the verifier is created, it decides where it wants a tap filter inserted and the test graph will add it for the verifier (for example, between the video decoder and the video renderer). The verifier then registers callbacks into the tap filter.

The tap filter does not do anything to the samples and messages passing through it. It acts as a passthrough and just passes it as is to its output pin. However, every time it gets a sample or a message, it signals the registered callback into the verifiers. Thus the verifiers know of what is happening in the graph, including what sample is currently coming out of the filter and can verify as needed.

The following table shows the verifiers included in the test binary.

Verifier Supported verifications

DecoderOutputLatencyVerifier

DecodedVideoLatencyPauseToFirstSample

Measures the latency from the next Pause to the first sample received after the pause at the output of the video decoder.

DecodedVideoLatencyRunToFirstSample

Measures the latency from the next Run to the first sample received after the Run at the output of the video decoder.

GraphBuildVerifier

CorrectGraph

This verifier checks if the specified filters are in the filter-graph.

PlaybackDurationVerifier

VerifyPlaybackDuration

This measures the time from the start of the verifier to the first sample time and then to the time that EOS is received. The caller is responsible for taking the measurements and verifying.

SampleDeliveryVerifier

VerifySampleDelivered

This verifies that a sample is delivered at a specified location in the graph after a specified event.

StartupLatencyVerifier

StartupLatency

This measures the latency of the first sample to be received after the verifier is started.

See Also

Other Resources

DirectShow Playback and Latency Tests