Training Session on GStreamer Plugin Development

1. Gstreamer Plugin Development
Session VII

2. Element
• Core of Gstreamer
• Object derived from GstElement
• Source elements provides data to stream
• Filter elements acts on a data in stream
• Sink elements consumes data of stream

3. Plugin
• Loadable block of code
• Usually shared object or a dynamically linked
library
• Contains implementation of many elements or
just a single one
• Very basic functions reside in the core library
• All other are implemented as plugins
• Are only loaded when their provided elements
are requested

4. Pads
• Negotiate links and data flows between elements
• Is an port of an element – Output or Input Port
• Specific data handling capabilities
• Can restrict the data that flows through it.
• Links are allowed between two pads when data
types of two pads compatible
• Is similar to plug or jack of a physical device

5. Pads
• Source Pads
– Data flows out of one element through one or
more source pads
• Sink Pads
– Accepts incoming data flow through one or more
sink pads

6. GstMiniObject
• Stream of data are chopped up into chunks
that are passed from a source pad on one
element to a sink pad on another element
• Structure used to hold these chunks of data
• Exact type indicating what type of data
• GstMiniObject Types
– Events ( Control )
– Buffers ( Control )

7. GstBuffer
• Contains any sort of data that two linked pads
know how to handle
• Contains some sort of audio or video data that
flows from one element to another.
• Metadata describing the buffer’s contents
– Pointers to one or more GstMemory Objects
– Ref Counted objects the encapsulate a region of
memory
– Timestamp indicating the preferred display timestamp
of the content in the buffer.

8. Buffer Allocation
• Able to store chunks of memory of several
different types
• Most generic type of buffer contains memory
allocated by malloc
– Very convenient
– Not always fast, since data often needs to be
copied

9. Specialized Buffers
• Many specialized elements create buffers that
point to special memory.
– filesrc element maps a file into address space of
application ( using mmap() )
– Creates buffers that point into that address range
– Created by filesrc act like generic buffers, except
that are read only
• Buffer freeing code automatically freed by the
correct method of freeing

10. GstBufferPool
• Element might get specialized buffers is to request them
from a downstream peer through a GstBufferPool
• Downstream able to provide these objects, upstream can
use them to allocate buffers.
• Accelerated methods for copying data to hardware or
direct access to hardware
• These elements able to create a GstBufferPool for their
upstream peers.
• Example Ximagesink
– Create a buffer that contain Ximages
– Upstream peer copies the data into the buffer, it is copying
directly into the Ximage
– Enables Ximagesink draws the image directly without copy

11. GstEvent
• Information on the state of the stream flowing
between two linked pads
• Will only be sent if element explicitly support
them, else core will try to handle
automatically
• Events used to indicate ( for example )
– A media type
– End of media stream
– Cache should be flushed

12. Events
• Subtype indicating the type of contained
event
• Other contents of event depend on the
specific event type

13. Media Types & Properties
• Uses type system to ensure that the data passed
between the elements in recognized format.
• Type system is important for ensuring that the
parameters required to fully specify the format
match up correctly when linking pads between
elements.
• Each link is made between elements has a
specified type and optionally a set of properties.
• Capabilities Negotiation.

14. Basic Types
S.No Media Type Description Property Description
1 audio/* All audio types rate Sample Rate
2 channels integer Greater than 0 No of channels
3 audio/x-raw raw integer
audio
Format String
4 audio/mpeg MPEG Audio
encoding
mpegversion Integer
5 framed boolean 0 or 1 True => Frame
available,
false => no frame
6 layer Integer 1,2 or 3 Compression
layer
7 bitrate integer Greater than 0
8 audio/x-vorbis Vorbis audio
data

15. Building a plugin
• Example file element
– Single input and single output pad
– Simply pass Media Data and Event data from its
sink pad to its source pad without modification.
• Including properties
• Including signal handling
• Examples/pwg/examplefilter

16. Plugin Template
• To check out gst template,
– git clone git://anongit.freedesktop.org/gstreamer/gst-
template.git
– cd gst-template/gst-plugin/src
– ../tools/make_element MyFilter
– Creates two files
• gstmyfilter.c
• gstmyfilter.h
– autogen.sh
– make && sudo make install

17. Examing Code
• Basic code structure
• Element metadata
• GstStaticPad
• Constructor
• Plugin_init
• Specifying the pads

18. Plugin Header
#include <gst/gst.h>
/* Definition of structure storing data for this element. */
typedef struct _GstMyFilter {
GstElement element;
GstPad *sinkpad, *srcpad;
gboolean silent;
} GstMyFilter;
/* Standard definition defining a class for this element. */
typedef struct _GstMyFilterClass {
GstElementClass parent_class;
} GstMyFilterClass;
/* Standard macros for defining types for this element. */
#define GST_TYPE_MY_FILTER (gst_my_filter_get_type())
#define GST_MY_FILTER(obj) (G_TYPE_CHECK_INSTANCE_CAST((obj),GST_TYPE_MY_FILTER,GstMyFilter))
#define GST_MY_FILTER_CLASS(klass) (G_TYPE_CHECK_CLASS_CAST((klass),GST_TYPE_MY_FILTER,GstMyFilterClass))
#define GST_IS_MY_FILTER(obj) (G_TYPE_CHECK_INSTANCE_TYPE((obj),GST_TYPE_MY_FILTER))
#define GST_IS_MY_FILTER_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE((klass),GST_TYPE_MY_FILTER))
/* Standard function returning type information. */
GType gst_my_filter_get_type (void);

19. Setup Object
#include "filter.h"
G_DEFINE_TYPE (GstMyFilter, gst_my_filter,
GST_TYPE_ELEMENT);

20. Element Meta data
gst_element_class_set_static_metadata (klass,
"An example plugin",
"Example/FirstExample",
"Shows the basic structure of a plugin",
"your name <your.name@your.isp>");
static void gst_my_filter_class_init (GstMyFilterClass * klass)
{
GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
[..]
gst_element_class_set_static_metadata (element_klass,
"An example plugin",
"Example/FirstExample",
"Shows the basic structure of a plugin",
"your name <your.name@your.isp>");
}

21. Pad Template
static GstStaticPadTemplate sink_factory =
GST_STATIC_PAD_TEMPLATE (
"sink",
GST_PAD_SINK,
GST_PAD_ALWAYS,
GST_STATIC_CAPS ("ANY")
);

22. Plugin init
static gbooleanplugin_init (GstPlugin *plugin)
{
return gst_element_register (plugin, "my_filter", GST_RANK_NONE, GST_TYPE_MY_FILTER);
}
GST_PLUGIN_DEFINE (
GST_VERSION_MAJOR,
GST_VERSION_MINOR,
my_filter,
"My filter plugin",
plugin_init,
VERSION,
"LGPL",
"GStreamer",
"http://gstreamer.net/"
)

23. Specifying Pads
static voidgst_my_filter_init (GstMyFilter *filter)
{
/* pad through which data comes in to the element */
filter->sinkpad = gst_pad_new_from_static_template (&sink_template,
"sink");
/* pads are configured here with gst_pad_set_*_function () */
gst_element_add_pad (GST_ELEMENT (filter), filter->sinkpad);
/* pad through which data goes out of the element */
filter->srcpad = gst_pad_new_from_static_template (&src_template,
"src");
/* pads are configured here with gst_pad_set_*_function () */
gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad);
/* properties initial value */
filter->silent = FALSE;
}

24. Chain Function
• The function in which all data processing takes
place
• In simple filter, mostly linear functions for
each incoming buffers, one buffer will go out.

25. Simple Chain Implementation
static GstFlowReturn gst_my_filter_chain (GstPad *pad, GstObject *parent, GstBuffer *buf);
static void gst_my_filter_init (GstMyFilter * filter)
{
/* configure chain function on the pad before adding the pad to the element */
gst_pad_set_chain_function (filter->sinkpad, gst_my_filter_chain);
}
static GstFlowReturn gst_my_filter_chain (GstPad *pad, GstObject *parent, GstBuffer *buf)
{
GstMyFilter *filter = GST_MY_FILTER (parent);
if (!filter->silent)
g_print ("Have data of size %" G_GSIZE_FORMAT" bytes!n",
gst_buffer_get_size (buf));
return gst_pad_push (filter->srcpad, buf);
}