DataTrafficAviatNetworks

Substation automation: How to calculate the bandwidth you need

Follow this practical step-by-step guide to assessing substation data capacity needs and designing a network around them.
Published: Tue 28 Nov 2017

Utility applications like teleprotection, SCADA, security and operational services share at least two things in common - they are mission critical but also produce growing volumes of data.

In an Engerati webinar on prioritising data traffic, networking solutions company Aviat Networks highlighted how the dual forces of new grid applications and the moving of time-division multiplexing networks to packed-based transport is pressuring utilities to scale up their data bandwidth.

During the presentation, Director of Product and Regional Marketing Ronil Prasad advises utilities to assess their mission-critical data applications before designing a network with sufficient capacity around them.

Prasad has now compiled a white paper that gives a detailed breakdown of capacity used per grid application. The study also looks at how to design a network “without sacrificing reliability or increasing cost”, he says.

Data consumption - the need to assess

The need to assess traffic loading and network topology derives from the fact that networks that were built to sustain 100-200Mb/s of TDM/SONET/SDH traffic are now facing bottleneck situations, says Aviat, with a newer network requiring a minimum of 300Mb/s, and upwards of 1Gb/s to handle traffic.

Prasad says the approach to designing a substation data network uses a typical ring configuration and some typical traffic loading assumptions in order to simplify the calculations and provide a baseline for calculating the network link capacities.

Factors such as the size of the network, network topology, failure scenarios, the volume of services and anticipated growth will affect these estimates, he advises.  

This analysis can and should be extended to other configurations and can easily be adjusted to fit either different network topologies, different traffic loading of individual services, or a combination of the two.

Grid applications - data consumption

Teleprotection - one of the most critical services for utilities and while each teleprotection circuit is inherently low in capacity (one DS0 or less), as networks migrate to packet based systems, the transport of these circuits over packet infrastructure can introduce issues with latency and delay.

Traffic Requirement per substation: ~300kb/s to 10Mb/s per substation

SCADA traffic represents the command and control network of the utility and like Teleprotection is inherently very low in capacity but high in importance.  

Traffic Requirement per substation: ~300kb/s per substation.

Traditional and Emergency Voice - Utilities still maintain private networks for their internal and mission-critical voice communications including traditional push-to-talk (PTT) mobile radio, private branch exchange (PBX) lines, and newer voice systems like VoIP and LTE. In terms of capacity, networks are designed for peak load since while the usage of voice is intermittent and not as frequent as in commercial LTE networks, the requirement can increase in cases of emergencies or if there is aggregation at the site.

Traffic Requirement per substation: ~1Mb/s to 3Mb/s per substation.

Operational services include data access applications like metering, fault reporting and event analysis, which are all individual low capacity services but once aggregated, can become a large consumer of available bandwidth.  

Traffic Requirement per substation: ~10Mb/s per substation.

Corporate Networking - with the availability of Ethernet pipes on a private network, applications and services that sustain the corporate environment will increase.The bandwidth of this type of traffic can vary depending on the type of applications and these services are often times the lowest in priority but are still very capacity hungry.

Traffic Requirement per substation: ~10Mb/s per substation.

Security might be the single largest driver of capacity in utility networks today and the most bandwidth intensive security application is surveillance video.  

Traffic Requirement per substation: ~25Mb/s per substation.

Phasor Measurement Unit (PMU) Services - Comparing the data resolution of PMU’s to SCADA, while SCADA typically collects 1 sample every 2-4 seconds, PMUs collect 10-60 samples per second and measure both magnitude and phase angles of a phasor signal (SCADA only measures magnitude).

Traffic Requirement per substation: ~3Mb/s to 11MB/s

Designing a data network to meet capacity needs

Once a utility has analysed its traffic requirements, the next step is to create a sample network and see how the individual services contribute to the total required network capacity.

AviatNetworksDataCapacityPlanning

The graphic shows the per site capacity required for each mission-critical grid application using the lower-end range of each of the requirements.

Site one on the diagram represents the control site where most services originate and terminate. Traffic loading for each site is calculated by aggregating traffic for all the applications running at that site.

Traffic loading for each microwave link is calculated by aggregating traffic flows from each site to the control site that traverses the link. Under normal conditions, traffic from each site will follow the shortest path to the control site, assuming a basic IP network transport scheme.

AviatNetworksDataTraffic

The green bars represent the density of traffic as you move closer to the control site and represent how different links can require a different capacity based on their proximity and route to the control site.

The proper dimensioning of each microwave link needs to consider additional traffic loading under failure scenarios.

In a ring topology, a failure on one of the two links feeding the control site (Link 1 or Link 10) represents the worst-case scenario for traffic loading as it shifts the traffic from all remote sites to the other link feeding the control site.   

AviatNetworksDataTrafficWhitePaper

The figure above shows traffic loading for each link after a failure of Link 10. In this case, Link 1 will need to handle traffic from all remote sites.

Worst case scenarios and scaling network size  

With ring systems, link and network capacities need to be based on worst case failure scenarios, so in this case, the loading impact of failover should be considered. It is also important to dimension each microwave link so that it operates without congestion, no more than 80% of its capacity, under the worst-case failure scenario.

Another factor to consider when dimensioning the microwave links is the anticipated growth in network size, by adding spur links or subrings for example, as well increasing traffic demands of the existing applications or new applications. Growth factors can be included in the initial design of the links by reducing the load factor under worse case failure from 80% to 60% or less. In summary, the equation for link capacity is as follows:

Link capacity = (Worst Case Load / 0.8) x growth factor

For a deeper analysis of calculating link capacity and growth, see the Aviat Networks white paper

In conclusion, Prasad says the above process can and should be extended to other configurations and can easily be adjusted to fit either different network topologies, different traffic loading of individual services, or a combination of the two. The methods and formulas that were used can still be applied to any type of network and adjusted accordingly.

Related Webinar