java.net.RTP
Waqar Ali
Columbia University
New York, NY 10027
USA
aliw@usa.net
Akhil Nigam
Columbia University
New York, NY 10027
USA
anigam@lehman.com
Abstract
The Real-time Transport Protocol (RTP [1]) has become a widely implemented internet-standard protocol for transport of real-time data. It provides a mechanism for the participants to give feedback about the service using the RTP Control Protocol (RTCP [1]). This document presents the platform independent RTP implementation in Java. This implementation, which is available as a java.net.RTP package, can be easily incorporated into an application, which then has access to all of the transport level features RTP and RTCP provide. This package also provides out of band initialization classes, which implement a late-initialization protocol. These classes could be used to implement a late initialization server which is capable of synchronizing late-joining clients.
Introduction
This report describes the Java implementation of RTP protocol. It covers the following topics.
1) Related Work
2) Background
3) Architecture
3.1) Object Model
3.2) Event Model
3.3) Late Initialization Client/Server
4) Program Documentation
5) Java RTP API documentation
6) References
7) Downloads
RTP Library:
C++ library which provides application independent implementation of the RTP protocol. This library is available for DEC OSF/1 v3.2 and Solaris 2.3 platforms only.
Java Media Framework:
The Java Media Framework API (JMF) specifies a simple, unified architecture to synchronize and display time-based data, such as audio and video, within Java applications and applets. The JMF 1.0 API was developed by Sun Microsystems, Inc., Silicon Graphics Inc., and Intel Corporation.[2]
Elemedia H.323 stack
The elemedia PX3230S H.323 Protocol Stack software module is a software implementation of the ITU-T H.323 standard. The RTP/RTCP module supports header validity checks, statistic logging, sender reports, reception reports, and a system time-to-media time utility. [3]
Webcanal
WebCanal contains a set of networking packages and applications mainly for information distribution using IP Multicast. WebCanal networking packages include a Java implementation of RTP (real time transport protocol), LRMP, SAP/SDP (session announcement protocol) and more. [3]
Background
The project was motivated by the need to have a platform independent, multicast-capable RTP implementation, which could be used in collaborative applications like the Whiteboard. The whiteboard application has special initialization requirements where all the instances of the collaborating whiteboards have to display the identical content, i.e. they need to have a common state. This becomes a problem for late joiners which then have to be synchronized and brought to a common state like the other instances, hence there must exist either a dedicated server or such, which is capable of serving such clients and is responsible to provide late joiners with historic data. A generic functionality to build such a client and a server needs to be available for such an application. The package was designed and developed to be modular with a generic late-initialization client/server classes implementing a late-initialization protocol.
Architecture
This section describes the architecture of the software. When appropriate, the design decisions and alternatives are discussed.
Overall architecture of the software was motivated by the need to have a modular, simple and platform-independent RTP implementation which could be easily integrated into any application requiring RTP as its transport protocol. By choosing Java as the development platform, the goal of having platform-independent implementation was immediately met. The object oriented features of Java and the language's simplicity made it possible to make the software simple and modular.
Session:
The top most class the user of the package interacts with is Session.
This class encapsulates the RTP and RTCP related setup, startup and
shutdown procedures. Session also serves as the interface using which
object's states and their interactions are controlled.
Source objects. Source class encapsulates the source-related attributes, which are updated by the sender and receiver modules. The participants include active senders as well as listeners. Source table is implemented as a Java hash-table, which essentially is an associative array with search functionality built into it. The key in the hash-table is the SSRC of the source and the value is the Source object itself. Java Map and Vector classes (in java.util) were considered as implementation alternatives. Both of these are container classes just like the hash-table but due to the reasons discussed below, it was agreed upon to use the hash-table.
Map (java.util.Map) is only available in JDK 1.2 (which at the time of writing this document is in beta release). Selecting this as a source container would have limited the usability of the package to 1.2 or later versions of JDK. Hash-table, on the other hand, is available in all JDK releases as early as the version 1.02.
Vector (java.util.Vector) was not chosen because it does not provide the key-value associations and would have required manual search to find the desired source object. The hash-table, on the other hand, has the search functionality and association built-in making it trivial to retrieve the desired source object given its SSRC as the key.
An implementation aspect of the source table needs to be pointed out;
it is that the sample implementation suggested rfc-1889 distinguishes
between the senders and members by maintaining two separate tables. In
this package, the source table provided by the session holds the members
and there is no separate table for senders. A source attribute
ActiveSender is used to distinguish the senders from listeners in the
member table. Functions in Session are provided to get the senders or
the members.
From the user's perspective, the Session interacts with the network
and is responsible for sending and receiving RTP and RTCP packets. What
constitutes the network interaction can be classified into two distinct
processes:
(i) Synchronous processes:
Sending RTP packets
(i) Asynchronous processes:
Receiving RTCP packets
Receiving RTP packets
Sending RTCP packets
The synchronous interaction with the network is
straightforward, this is driven by the application (the user of the
package) and is invoked by calling the Session.SendRTPPacket() function.
The asynchronous interaction, on the other hand, requires the RTP and RTCP receivers to run on separate threads and wait for the packet arrival. Following the reception of a packet, several tasks are performed among which is posting of the packet arrival event. Using events, the application has the option to ignore a particular event family or subscribe to it. The event model and the object interaction that takes place when asynchronous events occur are described in the Event Model section of this document.
Figure 1: Higher level functional modules
RTP Packet Sending and Receiving:
Figure 1 depicts the object model where Session uses the
RTPThreadHandler object to send and receive RTP packets.
RTPThreadHandler class is inherited from Thread and the function which
receives the RTP packets, runs in an infinite loop. This loop is
started directly by calling the or using Session.Start() method which
starts all the receivers. RTPThreadHandler class also doubles as a
sender by providing the SendPacket() function.
RTCP Packet Sending and Receiving
RTCP receiver RTCPReceiverThread is similar to RTPThreadHandler
described above as it receives RTCP packets in an endless loop, but it
is dissimilar as it does not provide any send functionality.
For sending RTCP packets, a separate threaded class RTCPSenderThread
is provided. The functionality of sending and receiving RTCP packets is
not folded into one class to keep the design modular and simple. RTCP
sender and receiver both have complicated algorithms and to keep the
changes in the send RTCP module from effecting the receiver module (or
vice-versa), these functional modules are implemented as separate
encapsulations. This was decided upon to keep the overall design in
accordance with the design guideline of isolating the package modules
from the changes made in other modules, while allowing for
extensibility.
RTCPThreadHandler class is the parent class, which serves as an
intstantiator of the sender and receiver thread classes. This class has
the sole purpose of policing the RTCP sender and receiver threads.
Although sending of the RTCP packets is an asynchronous process, the
sender thread does not run in an endless loop like the receiver -- the
Session has some control
over it. For instance, after the RTCP sender thread is started, it does
not send any RTCP packets unless the application (user of the Session) sends a RTP packet.
Likewise, the RTCP sender stops sending packets after application
requests a shutdown by calling the Session.Stop() function.
According to the RTP protocol specifications, it is mandatory for the
RTCP sender thread to be silent following the transmission of the BYE
packet. Although the Session user has some control
over the RTCPSenderThread's
packet sending behavior, it has no explicit control over when the RTCP
packet will be sent, this is due to the randomness introduced while
calculating the RTCP sending interval.
Sending RTCP packets has to follow certain rules and guidelines as dictated by the rfc[1]. The flowchart shown in Figure 2 summarizes the algorithms used in the implementation of the sender.
Figure 2: RTCP Send flowchart. The thread loop is entered at the top-left and exited at the right.
This flow depicted in the flow-diagram above implements the BYE back-off algorithm (rfc-1889, section 7.3.7). Furthermore, SSRC of sources are timed out following the algorithm described in Option B of rfc-1889 (section 7.3.6).
RTCP reception involves parsing the incoming RTCP packets, which can be of type described in RTCPConstants class. After parsing the packet, source table is updated if required and an event is generated for the registered listener.
RTP and RTCP reception update the Session and source level statistics that are necessary to sustain an RTP session with minimal interaction from the application. This makes it easy for applications, which do not necessary need to concern themselves with the protocol intricacies, to use this software package and with only a few lines of code, have a fully functional RTP session. For instance, notice the following sample code
// Construct a new
Session object
rtpSession = new
Session ( "234.5.6.7", // MulticastGroupIPAddress
8000, // MulticastGroupPort
8001, // RTCPGroupPort
8051, // RTPSendFromPort
8052, // RTCPSendFromPort
10000 // bandwidth
);
// Set the session parameters
rtpSession.setPayloadType ( 5 );
rtpSession.setCName ( "RTPUser" );
rtpSession.setEMail ( "user@pluto.com" );
// Start the session
rtpSession.Start();
// Send a test packet.
rtpSession.SendPacket ( String ( "TestString" ).getBytes() );
// Stop the session
rtpSession.Stop();
However, for the applications requiring finer control over the RTP session, it is necessary to expose some of the session internals. This allows for applications, requiring distinct level of control, to use the same software package. Session exposes some of the internal functionality, such as, thread handling, source-table modification methods etc. for flexibility and extendibility.
It was mentioned earlier that when a RTP or RTCP packet arrives, session level state and statistics are updated, furthermore, it is necessary to forward the RTP packets over to the user of the session (application or any other class requiring the received RTP packets). To avoid using hard-coded callback functions, an event model was designed which is discussed in the next section.
The event model in this package is based on the one implemented by the Abstract Windowing Toolkit (AWT) 1.1. In the AWT event-model, asynchronous events, such as button-click etc. are handled by registered listeners. A listener is a class which implements an interface specified by the event-model. For instance, there exists in AWT, the mouseListener interface. Any class interested in mouse events, must implement this interface. Additionally, the interested class instance must register itself with the event source by calling the method addMouseListener and passing its own instance reference. This step of registration will allow the event dispatcher to call the appropriate event handling functions in the listener when an event is posted. The dispatcher is guaranteed that the listener, which declares that it implements the interface, indeed does implement all the methods. This guarantee is in effect by the compiler, which will produce compile-time errors until an implementation, even if empty, is provided by the interface implementor.
Following the same model, two interfaces, RTCP_actionListener are made available. It is not required by the application classes to implement any of these interfaces. The only requirement is that once an application class is declared that it implements the interface, it must provide handlers for each and every method. The objects passed to the event handler are classes which encapsulate the RTP and RTCP packets. Following object model diagram will help clarify how these packet classes are interrelated.
RTP Packet:
Attributes of the RTP Packet are listed in the following model diagram.
When the Session.postAction() is called. This method is provided with the newly constructed RTPPacket object. An alternative is to post the byte data stream instead, but from the user's point of view, it is easier to work with objects and attributes than to parse bits and bytes out of a byte datagram. To the user, an incoming RTP packet is nothing more than the object instance of the RTPPacket class. Note that the user is not required to do anything with any of the attributes, most often, it will only need the data attribute, which contains the payload. But an application like RTP monitor needs to have access to the RTP packet header and this encapsulation targets such users.
RTCPSenderReportPacket,
RTCPReceiverReportPacket:
RTCP receiver and sender report packets inherit from the RTCPPacket class, which contains zero or one Report Block. It is possible that there are more than 31 members in an RTP session, and the specifications allow for a RTCP packet to have at most 31 report blocks, hence, the feedback about this source might be in the next RTCP packet.
When a RTCP sender report or receiver report packet is encountered in the compound packet, a corresponding packet object is instantiated and handleRTCPEvent() method and hands the packet object to it.
SDES Packets:
Following object diagram displays the SDES and BYE packets. Similar to the RTCP sender/receiver report reception, when SDES and BYE packets are received, respective packet objects are created and posted to Session.
Sequence Diagram:
Following sequence diagram shows the interaction between the Session, the session instantiator which implements the RTP_actionListener interface, the RTP receiver and the RTPPacket object.
Sequence 1 and 1.5: The registration takes place. Here instance of the class which is implementing the reference in order to post the RTP events to this listener.
Sequence 2: The asynchronous event, i.e. the RTP packet reception.
Sequence 3: The RTP receiver, after performing the house-keeping tasks, such as updating the source table, upgrading the sender of this RTP packet to active sender etc., instantiates a RTPPacket object and populates its fields. It then posts the packet object with the session in sequence 3.5.
Sequence 4: Session determines if any RTP_actionListener implementing object is registered. In this case, there is a registered object.
Sequence 5: The packet object is handled by the RTP_actionListener.
Following rules and guidelines apply while working with the events model and actionListener interfaces:
- It is recommended that a class implement the RTP_actionListener interface.
class MyClass implements RTP_actionListener { …}
- Session maintains registration and in the current implementation, there can only be one listener per interface. Registration is performed using the methods Session.addRTCP_actionListener().
- Only one registered listener will get the event notifications. For instance, there are two classes, A and B, that implement the RTP_actionListener interface, and if instance of A followed by B registers itself by calling the Session.addRTP_actionListener() function, then the last registration will overwrite the one preceding it. It is necessary to understand that there can only be one instance registered for RTP_actionListener. Similarly, for RTCP events, there should be no more than one class implementing the RTCP_actionListener interface and registering itself.
- If a class implements both interfaces, it must provide implementation for methods in both the interfaces.
Following code examples may help clarify the event model.
// Example 1: One class that implements the RTP_actionListener interface and registers itself
class MyClass implements RTP_actionListener {
MyClass {
Session rtpSession = new Session (…);
rtpSession.addRTP_actionListener ( this );
…
}
public void handleRTPEvent( RTPPacket rtppkt) {
// RTP event handler..
}
} // end
// Example 2: One class that implements both RTP_actionListener and RTCP_actionListener
class MyClass implements RTP_actionListener, RTCP_actionListener {
MyClass {
Session rtpSession = new Session ( … );
rtpSession.addRTP_actionListener ( this );
…
}
public void handleRTPEvent(RTPPacket pkt) {
// RTP event handler
..
}
public void handleRTCPEvent(RTCPReceiverReportPacket RRpkt) {
// receiver report packet event handler
..
}
public void handleRTCPEvent(RTCPSenderReportPacket SRpkt) {
// sender report packet event handler
..
}
public void handleRTCPEvent(RTCPSDESPacket sdespkt) {
// Sdes packet event handler
..
}
public void handleRTCPEvent(RTCPBYEPacket byepkt) {
// bye packet event handler
..
}
One or more than one class can implement one or both the interfaces but only one class should register.
Late Initialization Client/Server
Consider a whiteboard implementation which uses this package as its transport. Assume that a conference using these whiteboard instances has been active for some time. Now a new instance of the whiteboard is brought up which starts to receive packets originated after its instantiation. The contents of the late-joiner whiteboard are not in sync with other instances that have been active for a while. In order to synchronize the contents of this with others, there must exist a server which is capable of replaying all the historic data. The Late Initialization Client/server tries to provide such functionality.
The repository of the historic data is maintained by the session as a Vector named Session.InitializationData. The applications is responsible for keeping this vector up-to-date. The Vector is assumed to contain objects which could be converted to String (proprietary classes must override the .toString() method). Contents of this vector are given to the late-clients, which update their Session.LateInitializationData vector. Hence, after a successful initialization, all the collaborating participants will have identical data in the LateInitializationData vector.
A sample usage of this scheme would consist of a dedicated server, which is always alive and listens to the RTP packets on the multicast groups. This server stores the data it receives (in String format) in the LateInitializationData vector. The clients, upon instantiation, will connect to this server and populate their LateInitializationData vector, thereby allowing all the clients to share a common state irrespective of their startup times.
High Level Overview
Late Initialization Server
This encapsulates the server side functionality of the late initialization procedure. In the application it is implemented as separate thread, which monitors the activity on a specified port, waiting for client connections. All clients initially connect to this port when they want to initiate a new session with the server. As soon as the server gets a client request on this initial port, it assigns the client a random port number (which is available) to connect to, so that the initial port is freed for other client requests.. The server spawns a new thread connecting to the randomly generated port number and creates a Server Socket on that port . The client and the server go through a series of "handshaking" signals which maintain the state of the server and the client. The server transfers all the data to the client and sends a bye to the client which closes the connection on that particular port and the port is freed up.
Late Initialization Client
This encapsulates the client side functionality of the late initialization procedure. It starts the client thread which connects to the server on a specified port to initialize a new session with the server. The server assigns the client a newly generated port number (which is available) to connect to, so that the initial port is freed for other client requests. The client closes that connection and spawns a new thread connecting to the port number assigned by the server and the client and the server go through a series of "handshaking" signals which maintain the state of the server and the client. The server transfers all the data to the client and sends a bye to the client which closes the connection on that particular port and the port is freed up.
The application used to test the sender and receiver functionality is available to download from here .
1 Henning. Schulzrinne, et. al., RFC-1889, RTP, a Transport Protocol for Realtime Applications
2.JavaTM Media Framework FAQ
3.Henning. Schulzrinne, RTP Libraries and Tools.