3. Implementation Details

Source Package

The complete MCN Streaming source package consists of source files, configure and make scripts. Source files are located in the "src" sub-directory, which in turn contains two sub-directories named as receiver and sender, respectively. MCN Streaming sender and receiver implementations are located in these two directories.

GStreamer

MCN Streaming is built upon GStreamer, which is an open source multimedia framework. GStreamer is a powerful and versatile framework for creating streaming media applications. Many of the virtues of the GStreamer framework come from its modularity: GStreamer can seamlessly incorporate new plug-in modules for a specific purpose.
The framework is based on plug-ins that will provide the various codec and other functionality. The plug-ins can be linked and arranged in a pipeline, which defines the flow of the data. GStreamer plug-ins could be classified into following categories:

protocol handling.
sources: for audio and video (involves protocol plug-ins).
formats: parsers, formaters, muxers, demuxers, metadata, subtitles.
codecs: coders and decoders.
filters: converters, mixers, effects.
sinks: for audio and video (involves protocol plug-ins).

Above figure shows an example GStreamer pipeline for MCN Streaming sender. Here the video source is a video4linux2 device. The read-in frames are encoded by an x264 encoder, followed by a RTP H.264 payloader. The sending and receiving of RTP and RTCP packets are managed by a GStreamer RTP bin. In this example, RTP packets are sent on port 5000, sender RTCP packets are sent on port 5001. The receiver RTCP packets are received on port 5005.
Note that an identity element is inserted between udpsrc and rtpbin. It is used for inspecting the received RTCP packets and invoking a call-back RTCP handling function.

A Receiver Pipeline Example

Above figure shows an example GStreamer pipeline for MCN Streaming receiver. RTP and RTCP packets from sender are received on port 5000 and 5001, respectively. The sender RTCP packets are sent on port 5005. Received RTP packets are decoded and displayed by an xvimage sink.

Receiver RTCP Feedback to Video Encoder

The receiver RTCP packet feedback to video encoder is implemented via GStreamer upstream event, which is generated by an element somewhere downstream in the pipeline.
In MCN Streaming sender, once a RTCP packet is inspected by the identity element, a call-back function will generate a custom upstream event, and this event will be sent to the source pad of the video encoder element, as shown in following code snippet. Here we create an upstream event called "rtcp_event".

This "rtcp_event" will then be consumed by the video encoder. Since it is a custom event, if we are using an existing video encoder element (e.g., x264enc), we need to modify the element accordingly to handle the custom event. Following code snippet is an illustrative example, based on GStreamer x264enc plug-in.

Write a New GStreamer Plugin

To implement the adaptive video encoding algorithm, it might be necessary to write your own GStreamer video encoder plug-in. There are currently two ways to develop a new plug-in for GStreamer: You can write the entire plug-in by hand, or you can copy an existing plug-in template and write the plug-in code you need. The second method is by far the simpler of the two. To use the second method, the first step is to check out a copy of the gst-template git module to get an important tool and the source code template for a basic GStreamer plug-in. For more details, please refer to GStreamer Plug-in Writers Guide.