ActiveStreaming: NetServ Module for Live Streaming

Amandeep Singh
Columbia University
New York
as3947@columbia.edu

Aditya Muthyala
Columbia University
New York
am3551@columbia.edu

Abstract

This document describes NetServ ActiveStreaming module for live streaming of media with simple cache management implementation in NetServ routers.

Introduction

NetServ is a signal driven modular network router. It breaks up the functions provided by Internet services and make these functionalities available as modular building blocks for network services. The project was conceived to address the limitations of services in the current Internet. It presents a clean state Internet architecture based on the concepts of service virtualization [1].

ActiveCDN is a video on demand (VoD) module developed on top of NetServ. For supporting live streaming we have implemented ActiveStreaming module which is installed on a NetServ router on getting a signal from the content server. Content server sends a signal to install the module when it receives a request to serve live streaming content. The NetServ router caches the content based upon the popularity of the content. Subsequent requests to content server results in redirection to the NetServ node.

The report is organized as follows:

Introduction
Architecture
Results
Future work
References

Architecture

The module is written in Java, it is divided into two main parts:

Content Server
NetServ Streaming Module

Content Server

Content server contains an embedded VLC streaming server, and a servlet listening for live streaming requests. Embedded VLC streaming server is supported only on systems which support libVLC library, we have tested it on Ubuntu Linux. On Mac and Windows we can use VLC player to stream media over the network as well. With no NetServ node the requests are directly handled by streaming server.

Components

Servlet Container - Embedded Jetty server which is listening for requests at port 8088 & /stream/ context.
NetServ Setup/Probe - Thread used to check any NetServ nodes along the path of the client request, if any NetServ nodes are found we install ActiveStreaming module on it.
Embedded Streaming Server - Starts a embedded VLC streaming server
Content Singleton - Stores NetServ nodes corresponding to each client.

Fig.1 - NetServ Content Server Architecture

NetServ ActiveStreaming Module

NetServ ActiveStreaming Module is an OSGI bundle, which gets installed when content server sends a signal. When the client requests for a live stream, the request is redirected to the NetServ node, which serves the cached stream to the clients. We have implemented a simple threshold (number of active connections) based cache management scheme.

Components

Servlet Container - Embedded Jetty servlet which is listening for requests at port 8888 & /stream-cdn/ context.
Active Stream Map - Singleton Map which keeps track of multiple live streams and their corresponding video cache object.
Video Cache - Each live stream is represented by this class object. It stores the meta data related to the live stream.
URL Reader - This method is used to directly access the live stream from the streaming server.
Local File Reader - This method is used to read the live stream from local file system cache.
File Writer - Thread that writes to local file storage once the threshold value is reached.

Fig.2 - NetServ Streaming Module Architecture

NetServ serves the client request directly from the streaming server until the threshold number of clients are reached. Once the threshold is crossed, a writer thread is spawned, which caches the stream in the file system.The client that triggers the caching and all the subsequent clients view the stream from the beginning and can switch to live stream by clicking a "Live Mode" hyperlink.

VLC Client Browser plug-in

We need a VLC browser plug-in installed on the client machine to render all the live streams. There are two viewing modes in the application, by default all the live stream requests start from the beginning if the local storage at NetServ node is activated. User can go to Live Mode by clicking "View Live" button.

Control flow

Control flow for 3 clients use cases, with following settings:

Content server is running at 10.0.1.1
Streaming server is running at 10.0.1.2
NetServ node is running at 10.0.1.3
Threshold value, T =1.

Fig.3 - Control Flow of ActiveStreaming

Client 1, requests for http://10.0.1.1/stream/?file=kung_fu_panda
Content Server, checks for any NetServ nodes along the path, if not found sends initiates NetServ setup. Replies with a direct connection to streaming server.
Client 1, VLC browser plugin, contacts streaming server http://10.0.1.2/stream/kung_fu_panda
Client 1, gets the live stream from streaming server
Client 2, requests for http://10.0.1.1/stream/?file=kung_fu_panda
Content Server, sends a redirect to NetServ node, http://10.0.1.3/stream-cdn/?url="http://10.0.1.2/stream/kung_fu_panda" with parameter url for live stream set.
Client 2 contacts NetServ node which contacts streaming server and sends the stream directly to Client 2. Also, as T=1, NetServ node starts saving the file locally as well.
Client 2, gets the live stream.
Client 3, requests for http://10.0.1.1/stream/?file=kung_fu_panda
Content Server, sends a redirect to NetServ node, http://10.0.1.3/stream-cdn/?url="http://10.0.1.2/stream/kung_fu_panda" with parameter url for live stream set.
Client 3 contacts NetServ node - As T is now 2, NetServ node uses its local cache to serve all the subsequent request.
Client 3, gets the live stream starting from the point where NetServ node starts caching.
Client 3, clicks "Live View"
Content Server redirect the request to NetServ node with parameter mode="live" set.
Client 3 contacts NetServ node with mode="live", NetServ node shifts the position of local cache to point to live stream position.
Client 3 gets live stream from NetServ node.

Algorithm psuedocode

ContentServer():

  map = NetServNodeMap

  file = getFileRequest()

  client = getClient()

  node = checkNodes(client)

  if (node is not present):

    start a new thread:

      node = initialize netserv setup

      if(node):

        map.add(node)

    return directURL(file)

  else:

    return sendRedirect(node)

NetServNode():

  T = Threshold value

  map = cacheVideo map

  url = recieveRequest()

  if (url NOT in map):

    cv = new VideoCache(url)

    map.add(cv)

  cv.activeConnections += 1

  if (cv.activeConnections < T):

    return URLReader(url)

  else if (cv.activeConnections == T):

    start writer thread and start saving to local
cache.

    return URLReader(url)

  else if (cv.activeConnections > T):

    return FileReader(url)

Results

Testing is done with 1 physical machine running Mac OS X 10.6 and 2 Ubuntu 10.10 virtual machines running on Virtual Box . The physical machine is used as the streaming server & content server. One of the virtual machines is used as a netserv node and the other virtual machine is used as client.

Fig.4 - NetServ ActiveStreaming Screenshot

Future work

New technologies that can be tested on top of NetServ architecture.

Better cache management at NetServ nodes.
HTTP Live streaming can be tested.
Multiple bit-rates support in live streaming.
WebSockets can be an ideal solution for streaming application.

References

[1] Srinivasan S, Lee J W, Liu E, Kester M, Schulzrinne H, Hilt V, Seetharaman S, Khan A, "NetServ: Dynamically Deploying In-network Services", ACM ReArch '09 (CoNEXT workshop), December 2009.

[2] Meeyoung Cha, Haewoon Kwak, Pablo Rodriguez, Yong-Yeol Ahn, Sue Moon, "I Tube, You Tube, Everybody Tubes: Analyzing the World’s Largest User Generated Content Video System"

[3] Phillipa Gill, Martin Arlittz, Zongpeng Li, Anirban Mahanti, "YouTube Traffic Characterization: A View From the Edge"

[4] VLC Java Bindings - http://code.google.com/p/vlcj/

[5] VLC Http Streaming - http://www.videolan.org/doc/streaming-howto/en/ch02.html