Building a topology generator, a fault generator and
a distributed platform for running simulations using
Opnet as a simulation tool.

Panagiotis Sebos
Columbia University
New York 10027 USA
psebos@comet.columbia.edu


Abstract

The main aim of this project is to build a distributes system for running simulations through Opnet simulation tool. This can be done by entering certain parameters to the program such as nodes numbers, redudancy factors etc to produce files that can be used as an input for Opnet, without interfering with the GUI of the program. Also it is provided a tool that can be named as a simulation manager and can handle the simulations and run them in remotely located machines. This is quite useful when we need to run large simulations with the same main characteristics but different specific ones. Also all the results are indexed inside a main database giving to the user the flexibility to access them through a well defined language (SQL)

Introduction

Opnet is a software tool for simulating networks. It provides the user with a specific graphical user interface so that he can build a specific network, specify the characteristics of each node, as well for the whole network and afterwards run a simulation and "probe" specific statistics from the network, draw specific graphs.

The main problem using opnet is that if someone would like to run a simulation with specific characteristics, he has always to go and change them through the GUI of the program. There is no way of doing it automatically through a batch way. So for example if a someone would like to run a simulation for 20 nodes (routers) of a network with a variation of the number and the location of the clients (senders and receivers of the traffic) all his has to go with the traditional GUI and change the number and the location of the clients and rerun the simulation.

Also another problem is when someone would like add fault inside the network (fault means for a link to bring the link down and after some time to recover it) he has to go inside each node and add code inside each model, for the specific fault. In this way the user all he is doing is adding fault (making a apecific link of the model to go down for a certain time period). We are going to discuss the exact procedure later.

So the main aspect of the project is giving the user the tools to generate automatically a randomly generated network and afterwards inject inside in it fault without interfering at all with the GUI of Opnet. This can be done by supplying certain parameters which are: nodes number, redudancy factor 1, reudancy factor 2, senders fraction, receivers fraction, link to go down, duration of the damage. We are going to discuss about each one of this later.

After building those files we "compile" and "link" them inside Opnet environment and afterwards we are ready to run the simulation based on the files we have produced.

But in many cases we need to run the same simulation with somehow different characteristics (e.g. bringing for the same network topology different links down and watch the performance and specific results of the simulation). So we need something like a simulation manager that can handle and send the configuration files and the simulation commands (through network) to a client that has opnet installed, run the simulation there and get back the output of the simulation. This manager should have also a simple graphical user interface for specifying the main characteristics for the topology generator and the fault injection program so that he can setup the simulation and distributed it to the opnet clients. Of course for the manager we should use not a proprietary user interface but something that is widely acceptable. And one of the best solutions is WWW and java. So anyone, anywhere can access the manager through a web client setup some characteristics of some simulations and launch them. After the end of the simulations he can retieve back the results from the manager.

There is one more issue for this. What is going to happen to the results that the client is retreiving? So, need a database inside the network so that we can insert and keep all the statistics that we have retreived from a specific simulation. After viewing the results we insert them inside the database and we can have them for future reference.

So the whole project consists from the following parts:


Related Work

Speaking of network simulation tools, one of the most known freely available Network Simulator is the UCB/LBNL/VINT Network Simulator - ns (version 2) which is a discrete event simulator targeted at networking research. Ns provides substantial support for simulation of TCP, routing, and multicast protocols.

The main thing is that ns is not a polished and finished product, but is the result of an on-going effort of research and development. In particular, significant bugs in the software are still being discovered and corrected. So it is primarily used for research issues and not for commercial.


Topology generators
The most widely known algorithm for creating graphs is an extension in Waxman's algorithm. The nodes in the network are placed at random points in a two dimentional grid. The most widely know implementations for those topology generators are two: GT-ITM topology generator and Tiers topology generator. The main problem abot those two is that the output of those two is ASCII and cannot be inserted directly to OPNET. Also our main aim is to produce a network which consists from nodes (routers), links (between the routers), senders/receivers (on ethernets that exist on the ethernet interfaces of the routers). Those topology generators the only thing they provide us with is a graph which consists of specific nodes and nothing more. So we need to implement a way of extending those graphs to our need and the most important thing to write the topology in a format that OPNET can understand (create EMA code) but all these things automatically.


What we have used for the topology generator is a modified version of tiers algorithm. (I am going to discuss about it later)


Fault injection
Inside the ns someone can inject fault but he has to do it by hand. Also inside OPNET the user should write some code again. There is no tool currently available for automating this procedure.


Simulation Manager/Client, Database
Inside Opnet someone can use many machines to run a single simulation. This system can send multiple simulation to different machines and track them through a graphically interface and gather back the results. Also there is no connection with a database.


Background
 

Opnet

OPNET is a comprehensive software environment for modeling, simulating, and analyzing the performance of communications networks, computer systems and applications, and distributed systems. Is provides the user with a well builted and detailed graphical user interface giving him the advantage to create difficult network topologies with different node characteristics and apply various simulations on it

OPNET is also used as a decision support tool to provide insight into the performance and behavior of existing or proposed networks, systems, and processes. Specific tools are included with each OPNET license that assist users through the following phases of the modeling and simulation cycle:

For model Building and Configuration we can use the Network Editor (define or change network topology models), the Node Editor (define or change data flow - systems architecture - models), the Process Editor (define or change control flow - behavioral logic - models)

For running simulations we can use the Simulation Tool (define and run simulation studies using models constructed with the OPNET Editors Interactive Debugging Tool - interact with running simulations )

For analyzing results we can use the Analysis Tool (display and compare statistical results) and the Animation Viewer (watch dynamic behavior of models during simulation runs)

What is the main power of OPNET is the EMA code. (EMA stands for External Model Access)

External model access is an OPNET term that is defined as follows: the technique of accessing a model external to the opnet program (i.e., without using the services provided by the opnet graphical editors). In this context, the definition of accessing a model includes creating it, modifying it, and extracting data from it. External model access is supported via a library of C callable functions, which serves as a programmatic specification and query language. This library is named the External Model Access (Ema) package, and it can be viewed as an Application Program Interface (API) for creating and extracting data from OPNET model files. External model access is provided to adequately support a number of special requirements:

  • Large models, involving many objects and interconnections, can be tedious to enter using the graphical editors, and it is easy to make errors of omission when specifying attribute values. If large models have a regular structure (such as a ring, grid, tree, etc.), they are especially conducive to algorithmic specification using a programmatic interface.

  • Many applications of opnet involve model construction based on sets of parameters associated with real world equipment configurations. Often, the parameter sets are conveniently stored in an on-line database. In these situations, it is useful to have an automated model construction process, especially if the number of models to be built is large or openended. Automated model construction is possible with a programmatic specification language.


To support the convenient embedding of Ema statements in C programs, Ema statements are implemented as C function calls. Models in Ema are sets of objects, each of which has an object type. As with objects, there can be multiple models present at the same time in an Ema application. The maximum number of models present at any one time in an Ema application is limited by the amount of available virtual memory.

So by using this External Model Access model we have written the topology generation and the fault injection generator which actually by giving certain parameters to the program it generates the specific EMA code.

 

Topology Generator

Each routing domain in the Internet can be classified as either a stub domain or a transit domain.

A stub domain carries only traffic that originates or terminates in the domain. Transit domains do not have this restriction. The purpose of transit domains is to interconnect stub domains efficiently; without them, every pair of stub domains would need to be directly connected to each other. Stub domains generally correspond to campus networks or other collections of interconnected LANs, while transit domains are almost always wide- or metropolitan-area networks (WANs or MANs).

A transit domain consists of a set of backbone nodes. In a transit domain each backbone node may also connect to a number of stub domains, via gateway nodes in the stubs. Some backbone nodes also connect to other transit domains. Stub domains can be further classi ed as single- or multi homed. Multi-homed stub domains have connections to more than one transit domain, single-homed stubs connect to only one transit domain. Some stubs domains may have links to other stubs. Transit domains may themselves be organized in hierarchies, e.g. MANs connect mainly to stubs domains and WANs.

One of the most commonly used models for generating random networks algorithmically is due to Waxman. The nodes in the network are placed in random points in a two dimentional grid. Links are added to the network by considering all possible pairs of nodes and then deciding whether a link should exist according to a probability function involving how far apart the teo nodes are and how many links are expected to be in the whole network.

This algorithm actually has many drawbacks (it does not guarantee a connected network, the output graphs do not look like the real Internet graphs, etc) when used for generating typical internets. So over the past few years a better method has been devised for graphs that reflect the hierarchical domain of the Internet). This method constructs the graph piecewise, where the pieces correspond to domains at the different levels in the hierarchy. The connectivity within the domain (intranetwork connectivity) is dealt with separately from that betwween domains (internetwrk connectivity).

We are using actually an implementation of this algorithm called Tiers: In this model, the three levels of hierarchy, or tiers, are referred to as WAN, MAN and LAN levels, corresponding to the transit domains, stub domains and LANs of the basic method. The model does not currently support multiple WANs, thus T = 1. The parameter values for Tiers also diff er slightly from that of the basic method in that number of node" parameters are per higher level domain, not per node in the higher-level domain. The Tiers model produces connected sub-graphs by joining all the nodes in a single domain using a minimum spanning tree. The use of a minimum spanning tree is a crucial feature of Tiers and is particularly appropriate since it is sometimes used in reality as the basis for laying out large networks. When adding edges for intranetwork redundancy, edges are added to the closest nodes in the network, in increasing order of Euclidean distance. For internetwork redundancy, extra edges to the closest nodes in the next higher domain are added. This ensures some degree of local connectivity within geographic constraints. Nodes are permitted to occupy the same points in the grid, since the purpose of the grid is to generate costs using an approximation of nodes' phys ical locations, and this can represent multiply connected hosts.


When a node is a gateway node, the node is represented as two interconnected nodes, e.g. a LAN node and a MAN node. The metrics of the edge between the two types of nodes reflect the processing delay and bandwidth constraints within the host.This approach permits better estimates of network performance than using strict approach that every edge is a physical link in a network. If the redundancy parameters are small integers, the time complexity of Tiers is O(NH2). For comparison, the complexity of Waxman's original algorithm is also O(NH2), but with potentially longer generation times for some values of the probability function.

 

Fault Injection

Fault injection part is quite complicated inside OPNET. When running a certain simulation for each object there some interrupts that can be be generated from the program. Two of those interrupts are failure and recovery. So to inject fault inside a network model is to somehow schedule manually the occurence of those interrupts.

For this purpose I have built a special proccess that gets the input of a file with the following format:

# time                  user_id         event
  11.420000             1               failure
  12.670000             3               failure
  16.860001             5               failure
  19.000000             1               failure
  21.129999             3               failure
  38.529999             1               recovery
  51.020000             3               recovery
  72.349998             5               recovery
  81.870003             1               recovery
  92.940002             3               recovery

and schedule the interrupts when the simulation is running.

Here is how the proccess look like.

We start from the init state where we read the file with the event and we move on to the sche state where we schedule the next event. In the idle state we are waiting for a failre or recovery interrupt. When this occured in the intr state we bring the corresponding link up or down. (Unfortunately when using EMA code, the EMA code includes this file with the events. So the proccess is not a statically file but it changes)

Each link inside the EMA code is beig described with the user_id attribute. So when I create the topology, I insert inside the correct user_id attribute for each link.

This proccess is being placed inside the node model of one router as the proccess fault. The new model looks like this.
  Simulation Manager/Client, Database

For making possible the communication between the manager application and the opnet client we have used Java RMI:

Java Remote Method Invocation (RMI) allows you to write distributed objects using Java. RMI provides a simple and direct model for distributed computation with Java objects. These objects can be new Java objects, or can be simple Java wrappers around an existing API. Java embraces the "Write Once, Run Anywhere model. RMI extends the Java model to be run everywhere."

Because RMI is centered around Java, it brings the power of Java safety and portability to distributed computing. You can move behavior, such as agents and business logic, to the part of your network where it makes the most sense. When you expand your use of Java in your systems, RMI allows you to take all the advantages with you.

For the implementation we have used for the manager and the client Java RMI. The Client has an RMI interface so that the manager can invoke methods remotely. We avoid to insert to the manager an RMI interface also so that we can make the whole system portable for a user who is just having a laptop and cannot has difficult configuration



For the database we have used Mini SQL (version 2.0) from Hughes Technologies . This database is widely known and used among Internet and specially is being mostly used as a database behind Web Servers.

Architecture

The main architecture of the project consists of the following parts:

Topology Generator
Actually what I wrote is program that gets some parameters and gives an output of the .em.c file (EMA code file) which using m3_mkema utility can be compiled and linked inside OPNET. EMA language is quite difficult and complicated and the only manual that was available inside OPNET's documentation was a reference manual, so I had to do reverse engineering (creating ema code for topologies that have been produced with OPNET GUI and undestand what was going on inside the code).

The tiers algorithm that I am using produce a 3-plane graph: one for WAN one for MAN and one for LAN. For each plane it takes a redudancy factor (e.g. if for the MAN we have redudancy factor 2 it means that each node inside the MAN can reach every node inside the MAN through a number of ways that the redudancy factor gives us). Also it reflects the number of connections with the next level up in the hierarchy. (for MANs to WANs and for LANs to MANs).

The redudancy factor in the plane of LANs does not work correctly in the implementation of tiers (it assumes that LANs are mostly buses without being a mess graph). Also this redudancy factor of the LANs affects the topology of the MANs (how to connect to the next level of hierarchy).

So I am ruuning the tiers algorithm for 0LANs and nMANs and using two redudancy factors r1 for the LANs and r2 for the MANs. After experimenting I have notice that after producing the topology with tiers the graph was not fully interconnect. So what I did is inside the tiers implemantation code before dumping the object into the output device, I chech whether all the nodes connect to someplace. If not by a random generator (based on the time of the host) I connect the unconnected node someplace inside the graph (by using also the MANs redudancy factor).

The output is a file (e.g. test.em.c which is a EMA file)

The topology generator gets the input of:

For the router model I am using ethcoax2_slip4_gtwy_int model. For the senders I am using ethcoax_server_int and for receivers ethcoax_wkstn_int. This is highly and pretty strateforward configurable because I have to change just two lines in each output EMA code which are fixed each time. It can be even done with a sed filter afterwards:

                "model",                COMP_CONTENTS, "ethcoax_server_int",
                "icon name",            COMP_CONTENTS_SYMBOL, "server",

                "model",                COMP_CONTENTS, "ethcoax2_slip4_gtwy_int",
                "icon name",            COMP_CONTENTS_SYMBOL, "gateway_4ports",

                "model",                COMP_CONTENTS, "ethcoax_wkstn_int",
                "icon name",            COMP_CONTENTS_SYMBOL, "wkstn",

These lines have to be cahnged in each OPNETs output. Actually all these models have to be built just once and afterwards can be statically linked inside OPNET so that my topology generator can change the models.

The senders and receivers are connected to the ethernet cards of each router through a bus.

The IP adressing is being like: we enter a value like 147.102. The program goes directly and assign in the WAN links of the routers the netmask of 255.255.255.252 (break each C class in 64 networks) and each peer of the link get one IP address of these. The ethernets of earch router (that has senders and/or receivers) gets a whole C-Class starting from 147.103.1.0 (not 102).


Fault Injection
I have inmplemented two ways of injecting faults into the network. The first one is injecting faults automatically or manually.

To inject fault into the network all I have to do is create a proccess which is going to handle the fault injection part (read from someplace the fault times and produce the corresponding ema code for the proccess). This process can be inside each router and andle only the local router or it can be a centralized process). I prefer the model of a centralized proccess because if we have one OPNET proccess in each router we are haveing a scaling problem when having many nodes - OPNET needs alot of memory and we have to be careful when handling it).

This centralized proccess we have placed it inside the node model of one router (This model is slightly different than the other). The node model is statically changed, this means that we change it once and it going to be there.

In the automatic mode I give the input of: Simulation length, maximum failures and maximum time per failure and based again on a random generator it produces errors (bring a specific link of the network down and up again). These are written in a file. Afterwards by having the input of this file I produce the responsible EMA code for the proccess.

In the manual mode I give again the simlulation length and the input file we are getting the information from about the fault and the program produces again the corrent EMA code for the proccess.


Simulation Manager/Simulation Client/Database

The architecture of the whole scheme will be like:

We have some machines which run OPNET and also a daemon for listening from the Simulation Manager commands. The Simulation manager first authenticates the user by communicating with the DataBase. Then he see what machines are available for running the simulations (again seraches the database) and afterwards he specifies some parameters to the simulation he wants to run. These parameters are: nodes_number, redudancy factor1, redudancy factor2, receivers fraction, senders fraction, IP=, simulation length, maximum failures as described before.

The Simulation Manager sends the commands for creating the simulation files and the commands for starting the simulation to the cilents that are using opnet. The simulation manager is written in Java (the advantage of web browsers) and it communicates with the clients that are running OPNET through Java RMI. So the Simulation Manager invokes a method for starting the simulation and afterwards monitors the simulations in each client. After the end of the simulation the output is being written inside the database by the clients directly. The methods that I currently support are:

I am using the Mini SQL databse with the following schema:


The pswd matrix is used for authentication, the clients matrix is being used for the available OPNET clients and the storage is for storing the output from the simulations.

The Simulation manager and the OPNET clients talk to the database through a standard TCP connection. In order to do it with SSL we need a special commercial toolkit for java. But after installing it, it is pretty easy to add support for this.


Program Documentation

The installation demands various compilations and patching on various source code of many programs.

Follow this link for installation and operation.

Task List

I got a implementation of tiers algorithm. Although this one only supports certain graphs (that is routers). There was a problem in producing fully interconnected graphs. And I had I've modified the code so that it can give the correct output. Also it gives the correct number of senders and receivers attached in each router.

After taking the output of the tiers algorithm (in an ASCII format) I have written a program that converts it into EMA code that Opnet Can understand. Also I came up with a algorihm to design the output into the opnet workarea without being a mess.

There is also a program producing EMA code for the fault generation part.

There is a java applet for the simulation manager interface. And the implemantation of the whole scheme of communication (between the manager the Opnet client and the database).

I am currently working on extending the SSL support for the MSQL so that I can speak directly through the Java application to the database through SSL.


References

Modeling Internet Topology, Kenneth L. Galvert, Georgia Tech

Tiers: A better Model for Generating Test Networks (Matthew B. Doar)

Opnet Manuals

Hughees Technologies (Mini SQL)

JDK 1.1.5 Documentation

UCB/LBNL/VINT Network Simulator - ns (version 2)