|
|
A Fiber Optic Primer and Tutorial: Designing Networks for Optimum Performance
By Joe St Sauver (joe@oregon.uoregon.edu)
The Simplest Case | More Complicated Network Topologies | Bibliography |
Recent fiber optic projects in Eugene/Springfield and elsewhere in the state have generated much interest in fiber optic networking. While many projects have fiber in place, at least in some cases fiber projects tend to stall when it comes to actually sending traffic over that fiber--that is, you will hear a lot of people talking about what's involved in "lighting dark fiber."
In an article of this length we can't make you a fiber optic expert, nor do we mean to imply that any of the technologies discussed in this piece will be a perfect solution for any particular actual fiber project currently underway. Our goal is simply to give you some basic background knowledge to help you begin thinking about options related to the fiber projects you may encounter.
In the simplest of cases, a run of fiber optic cable can interconnect two points (such as the main university campus and a branch location sited away from the main campus), and essentially operates just as an extension of the local campus network.
Types of Fiber. There are two basic types of fiber that are commonly deployed: multimode and single mode. Multimode fiber has a large core relative to a wavelength of light (~1.3 micron)--typically 62.5 microns. The relatively large size of multimode fiber means that light injected at one end of the fiber strand ends up bouncing along through the fiber, caroming off the cladding (the "walls" of the fiber optic "pipe"), and traveling many different paths to the other end of the fiber strand.
Fig. 1.: The elements of a multi-fiber fiber optic cable
Some of those paths are relatively short and direct, while others are longer and include lots of reflections. Such differences in path lengths result in a phenomena called modal dispersion (pulse "spreading," or a "smearing" of the wave form that was originally injected into the fiber). Modal dispersion limits multimode fiber use to relatively short runs--typically no more than a few hundred meters at gigabit ethernet (1000Mbps) speeds, or a few kilometers at fast ethernet (100Mbps) speeds.
Given this relatively short permissible distance, why do sites bother using multimode fiber at all? First of all, multimode fiber was available before single mode fiber, so a lot of multimode fiber was installed where it was the only common option and gigabit speeds weren't on the horizon. Way back when, a couple of kilometers at fast ethernet speeds probably sounded like more than enough "reach."
Even today, multimode fiber continues to be installed in some circumstances because it is comparatively inexpensive and works fine over restricted distances (e.g., within a building), and because multimode transceivers (the devices that inject and receive light in fiber optic cables) tend to cost less than single mode transceivers.
Where distances are longer, however, single mode fiber is the clear choice. Single mode fiber has a relatively thin core (typically 8-9 microns), thereby virtually eliminating modal dispersion problems. In the case of single mode fiber, the primary factor that serves to limit fiber reach is "attenuation," or loss of optical signal strength as the light passes through the fiber.
Figuring Out How Far Single Mode Fiber Can Reach. Fiber's transparency to light varies from wavelength to wavelength, with some wavelengths losing power in fiber more rapidly than others. For most fiber, the "primary spectral window," (its "sweet spot," or wavelength exhibiting the least optical attenuation) is 1300 nanometers (nm). For long haul applications, a different type of fiber with a primary spectral window centered at 1550nm is commonly deployed instead.
Attenuation, or loss of optical power, is usually quoted in terms of decibels per kilometer (dB/Km). At 1300nm, single mode fiber has a typical attenuation of 0.3dB/Km and a worst-case attenuation of 0.5 dB/Km. Additional power will be lost at splice points (if any), and at the connectors at each end of the cable. Let's assume that splices typically use up 0.06dBm each, and connectors cause a loss of 0.5 to 1.0 dBm each.
Two additional factors affect how far we can go. In order to compute our available fiber loss budget, we also need to know our fiber optic transmitter's launch power and our fiber optic receiver's sensitivity.
For example, consider the Foundry Networks Gigabit LX optical transmitter: it has a minimum of -11.5dBm of power output, and the optical receiver has a minimum sensitivity of -20dBm. Subtracting the receiver sensitivity from the launch power, we see that we have a total of (-11.5)-(-20)=8.5dBm worth of power available. If we assume no splices and subtract a total of 2dBm worth of loss for connectors at either end, that implies we have 6.5dBm left to overcome attenuation in the fiber itself. At 0.3dB/Km, that implies we should (theoretically) be able to go 6.5/0.3=21.66Km, or a little over 20 kilometers.
What do we do if we need to go farther? We can (a) increase the power we put into the fiber, or (b) increase the sensitivity of the receiver on the other end, or (c) use fiber that has lower attenuation per kilometer.
To consider just one simple example of how we might go farther than we could with a standard gigabit LX interface, Foundry has a proprietary long-haul gigabit interface called "LHB," which has a power budget of 29dB minimum. At that, even assuming 3dB worth of loss associated with connectors and other miscellaneous causes, and assuming you can reduce attenuation to .2dB/Km of cable, you could theoretically go 26/0.2=130Km--a very long way! (In reality, due to random manufacturing variation and installation-related artifacts, you won't really know how far you'll be able to go in a particular situation until you actually test a particular fiber run with a particular transmitter and a particular receiver.)
Why does it matter how far we can drive fiber? Consider a hypothetical fiber run between Eugene and Salem, at a minimum distance of about 70 miles (or roughly 110Km). (Actual distances for as-deployed fiber tend to be longer than as-the-crow-flies distances, but let's assume that it is 110km between Eugene and Salem.) If we had fiber and equipment that could handle distances of 130Km, we could cover that entire span without needing access to the fiber at any intermediate point, a tremendous convenience (and a great cost savings, since we wouldn't require any intervening electronics, nor any facilities to house those electronics).
Fiber Strand Count Requirements. Now let's consider a different issue: fiber strand count. Just as it normally takes a pair of wires to deliver electricity, so, too, you'll normally need a pair of fibers to deliver network connectivity.
In many common network scenarios, however, you may find that you actually want more than two fibers, although in a pinch you can actually make do with only a single fiber. To understand why, you need to develop a pessimistic "network engineering mindset," recognizing that any fiber you deploy will exhibit a perverse affinity for backhoes, brushfires, hungry rodents, and sundry other natural disasters, all aimed at destroying your connectivity.
The normal solution to this problem is to deploy fiber redundantly along two physically separate routes, thereby insuring that if a backhoe cuts the pair of fibers you happen to be using, traffic will automatically reroute onto the backup ("protection") pair. Clearly, this is not a particularly elegant solution nor a particularly cheap one, but if the circuit is crucial, or if it's running through remote areas where quick repairs would be difficult, it's routine to spend whatever it takes to get the required backup circuit deployed, even though that may mean doubling (or more than doubling) the cost.
Fig. 2: Unprotected and protected point-to-point connections
But what if you had only one strand of fiber? Would that single strand of glass be completely worthless? No, by using a device called a "fiber singler" or "single fiber full duplex unit" you can actually send traffic in both directions over a single piece of glass. Traffic going in one direction is sent on one wavelength (e.g., 1310nm), while traffic going in the other direction is sent on a different wavelength (e.g., 1550nm). Examples of this type of product include Canoga Perkins' L65x and 6001 fiber singlers and NBase's Single Fiber Full duplex product family.
Required Bandwidth. On a point-to-point single mode fiber optic circuit running in LAN extension mode, obtaining 100Mbps (fast ethernet speeds) or 1000Mbps (gigabit ethernet speeds) is, for the most part, a simple matter of selecting appropriate interfaces to attach to each end.
When you don't have a tremendous number of fiber circuits to deploy, one expedient and economical (less than $1,000 per end) way of lashing a conventional 100baseT copper network connection to a fiber optic link is via a fiber converter, such as Allied Telesyn's AT-FS201/202 series, which take 10/100baseT (copper) connections in on one side and convert them to 100baseFX (fiber) connections on the other. These devices are available in configurations that will support distances ranging from a couple of kilometers all the way out to 100 kilometers.
At gigabit speeds, connections will normally be made directly between gigabit switches or gigabit router interfaces.
Adding More Sites to the Network Using SONET Rings. What happens if we need to add more sites to the network? Traditionally, once you moved beyond simple point-to-point fiber optic links, you'd install a SONET-based ring network topology. Think of a SONET "ring" as two fiber rings running in opposite directions. The two rings run over two physically separate fibers: one live, and one deployed as a backup (or protection) fiber. Each site thus has four fiber connections (in plus out on the live fiber, and in plus out on the protection fiber). If a catastrophic failure should occur, such as a fiber severed between two of the sites connected to a ring, the SONET ring will typically be able to recover and reroute traffic quite quickly (on the order of 50 msec). SONET rings are also very good at handling multiple categories of traffic, e.g., data traffic plus traditional voice traffic, or data plus traditional voice plus dedicated video traffic.
Fig. 3: SONET "rings"
Unfortunately, SONET-based rings also have a number of drawbacks. For instance, the SONET add/drop multiplexers ("MUXes" or "ADMs"), the devices that put traffic onto or take traffic off of a SONET ring, are quite expensive. If money is tight, the sheer cost of these SONET MUXes (tens of thousands to hundreds of thousands of dollars per location) can be a "show stopper" when it comes to lighting a shared fiber optic infrastructure.
Second, SONET rings traditionally allocate their available capacity on a fixed basis using a technology called "time division multiplexing" (TDM). TDM divides the available capacity of a SONET optical ring into fixed time slices. The time slices reserved for a particular partner's traffic are not available for use by anyone other than that partner. This means that often a ring's capacity is wasted, with some partners having capacity that's unused, while other partners may have capacity that's insufficient to accommodate their requirements. This is especially true in the case of "bursty" data flows.
Third , traditional SONET add/drop MUXes have a limited range of tributary speeds. (Tributaries are the connection made to the SONET ring, either direct connections made to a particular customer, or connections to a slower speed subsidiary SONET ring.)
That is to say, if your primary SONET ring is running at OC48 (2.5Gbps) speeds, a typical SONET add/drop MUX (such as the Fujitsu FLM 2400) connected to that ring may only directly support OC12 (622Mbps), OC3 (155Mbps), and DS3 (45Mbps) tributary speeds. To deploy slower speeds, such as T1 circuits (1.5Mbps), you'd typically run cascading tributary SONET rings at lower speeds. That is to say, in order to be able to carve off T1 circuits, you'd first build an OC3 SONET ring attached to the OC48 SONET ring, and then use a slower speed ADM (such as the Fujitsu FLM 150) to carve T1's out of that OC3 ring. Of course, the equipment required for that sort of hierarchical network structure (rings connected to rings) just adds to the cost of deploying a SONET-based system.
For all those reasons and for many others (including the need for special SONET test equipment, the need for SONET trained support personnel, etc.), if you can avoid SONET rings you can greatly simplify your network and escape some very substantial costs.
Adding More Partners By Building Upon Point-to-Point Circuit Technology. So what might we do if our goal was adding more partners while avoiding deployment of SONET based rings?
It doesn't take a tremendous amount of imagination to see that we might be able to extend our basic point-to-point circuits (running at fast ethernet or gigabit ethernet speeds) in a way that might work for multiple partners. For example, assume the university has a number of network partners distributed all around Eugene/Springfield, all of whom want to connect to the UO to exchange traffic at the UO-run Oregon Internet Exchange. How can that be accomplished?
Tree network topologies. One possible solution is to have each partner connect to its nearest adjacent partner using a point-to-point link, thereby forming a tree topology. While doing this would minimize the amount of fiber required, tree network topologies aren't very robust: partners located at the far end of "branches" are quite vulnerable to disruptions anywhere between them and the "trunk" of the tree. Some partners located at the end of branches might also worry that their traffic is being monitored or interfered with by intermediary sites through which it might pass, or be concerned that there wouldn't be sufficient capacity to meet their needs in shared upstream branches of the tree.
Fig. 4: Examples of tree network topology (newly-added partners are represented by lighter lines).
A more resilient topology would be a "star" or "hub and spoke" topology, where all remote sites connect directly to a single central hub site. As long as we have enough fiber between each of the remote partners and the central hub, and as long as redundancy isn't an issue, it is a comparatively simple matter to build this type of network out of point-to-point circuits. Each link could be a different speed than the other link(s)--for instance, some fast ethernet and some gigabit ethernet--and any problems with one of the links should not impact any of the others. On the other hand, loss of the central hub site could take all the other sites "off the air" because the center of the star is a highly undesirable "single point of failure."
One way to avoid the hub's role as a single point of failure would actually be to connect each remote partner site via two physically separate pairs of fiber, with one pair from each partner site going to the main central hub site, and another (backup) pair going to some second (physically separate) alternate hub site.
Following this approach, we obtain improved reliability, but it comes at a cost: the number of fiber links we require doubles, as does the number of fiber optic transmitters and receivers. Recovery in the event of a hub failure ("reconvergence") also tends to take somewhat longer than in the case of SONET-based solutions, running on the order of a few seconds, rather than fractions of a second (as is the case for SONET).
Fig. 6: An example of "star" topology with redundant hubs
We believe this type of star topology with redundant hubs, or still more sophisticated designs based on this fundamental underpinning, has a tremendous amount of potential when it comes to lighting dark fiber, particularly when you realize that it can be deployed for a fraction of the cost of a SONET-based solution.
3Com: Gigabit Ethernet Comes of Age
http://www.3com.com/technology/tech_net/white_papers/503003.html
Allied Telesyn Media Converters
http://www.alliedtelesyn.com/products/conv_feth.htm
Building New Generation MANs
http://www.extremenetworks.com/technology/whitepapers/MANWhite.pdf
"Canoga Perkins Wavelength Division Multiplexers"
http://www.canoga.com/wdm.htm
Dynamic Packet Transport Technology and Applications
http://www.cisco.com/warp/public/cc/cisco/mkt/servprod/opt/dpt/dpta_wp.htm
Fiber Driver Single Fiber
http://fiberdriver.nbase.com/single_fiber.html
Fiber Optic Cable Tutorial
http://www.arcelect.com/fibercable.htm
FNC - Technology Transmission Products FLM 2400 ADM
http://www.fnc.fujitsu.com/products/view_294.html
Foundry Networks Gigabit Info Center High Performance LAN Alternatives
http://www.foundrynet.com/wpvol1.html
Foundry Networks - Service & Support FAQs - Media
http://www.foundrynet.com/genFaqMedia.html
Gigabit Campus Network Design
http://www.cisco.com/warp/public/cc/sol/mkt/ent/cmps/gcnd_wp.htm
Gigabit Ethernet MANs: Where the real action is, 10/04/99
http://www.nwfusion.com/archive/1999b/1004petrosky.html
Gigabit Ethernet ventures into the land beyond the LAN, 05/10/99
http://www.nwfusion.com/archive/1999/65004_05-10-1999.html
Gigabit Networking Gigabit Ethernet Solutions
http://www.cisco.com/warp/public/cc/cisco/mkt/switch/gig/tech/gesol_wp.htm
Service Provider - Optical Internetworking
http://www.cisco.com/warp/public/779/servpro/solutions/optical/opt_rings.html
Technology White Papers - The Glass Story
http://www.occfiber.com/wpglass.html
Tolly Switch Central - Gigabit Ethernet goes Metropolitain
http://switchcentral.tolly.com/tolly_articles/articles/res_926260166_906.html
Transition Networks Fiber White Paper
http://www.transition.com/products/fiber_wp.html
Transporting Gigabit Ethernet and Fiber Channel over the MAN
http://www.csdmag.com/main/2000/02/0002feat2.htm
Web ProForum Tutorial: SONET
http://www.webproforum.com/sonet/
Wide Area and Long Haul Gigabit Ethernet
http://www.canet3.net/papers/wideandlonggigabit.html