Data Rich, Information Poor

The smart grid offers utilities a wealth of information which, if handled effectively, can be used to enhance service delivery and improve energy efficiency. Engerati’s The Future of the Smart Grid, the ICT and Data Management Perspective report discusses the challenges that smart grid operators face.
Published: Thu 17 Jan 2013

The underlying challenge to smart grid operators is that they are about to find themselves in a situation familiar during the early adopter phase of almost all major new systems, that is “Data Rich and Information Poor” underscored by a number of issues that new operators in this area need to tackle and representing a step change in scale of task:

  • Data volumes (“big data”)
  • Real time processing
  • Multiple data types: dealing with complexity
  • Data integration (or the impact of the lack of integration): ownership and compatibility issues
  • Transformation issues: legacy systems
  • Data quality: structured and unstructured, new and old
  • Acquiring skills: aging/retiring workforce potentially leads to shortage of skills

The alternative view of the new data and data analytics market place           

An alternative and insightful view of the new data and data analytics market place is provided by the Versant “Magic Cube.” 

The three dimensions to the cube are: 

  • Data volume:  the volume of data that needs to be managed is a moving target. Exponential growth of datasets means that whatever is “big” today will be average tomorrow, making the future proofing of solutions both central and difficult to achieve
  • Data/ Model Complexity: data complexity embodied in the relationships as well as the relationships between data objects continues to grow. Quoting IBM, Versant highlight that the majority of CEOs see complexity as their organization’s greatest challenge
  • Concurrent Data: The amount of concurrent data transactions (the number of users and the level of concurrent access to the data base, and in particular, instances of conflict resolution) is growing exponentially. Latency is directly related to a system’s ability to handle this issue and governed by whatever is defined as the critical limit for concurrency

Versant further position existing systems within this cube , highlighting two facts:

Versant conclude that the future lies in Object-Oriented Database Management Systems (ODBMS/NoSQL systems)

Engerati Analysis

With the smart grid, there is significant interaction between data, communications and processing which makes its difficult to treat these areas as wholly separate. In order for utilities to see the benefits that smart grid has to offer, they will need to change the ways in which they handle data, analytics and processing.