Fionn Murtagh’s Blog

Themes: information economy, intellectual property, research

Archive for May 2012

Moore’s Law, the Rise of the Data Economy, and the Problem of “Dirty Data”

leave a comment »

In the piece on “Are Greenpeace attacking the younger generation and their ‘dirty data’?”, I found the discussion interesting as to whether data centres were “dirty” or not, in the sense of using energy that is high in CO2 or equivalent greenhouse gases.    In that British Computer Society blog post, the rates of growth are looked at in the data economy, the ability of computational technologies to respond, and then just how green this computational response is, in practice.   Let’s have a closer look at this. 

First I will touch on compute power and space.   Moore’s Law refers to the doubling of processor, storage, and related computing capability every two years.   Hence the compound annual growth rate, or CAGR, for doubling every two years, is 41.42% per annum.   

Hydropower is “the most widely used renewable resource for clean power generation across the globe”, and stable, flexible and inexpensive, as noted by ”Hydropower continues to account for major share of renewable power through 2020”.   A CAGR for hydropower in the period 2011-2020 is estimated as 3.6%. 

Moore’s law and hydropower indicate just well we can do to handle data growth. 

Citing from “IDC Releases First Worldwide Big Data Technology and Services Market Forecast, Shows Big Data as the Next Essential Capability and a Foundation for the Intelligent Economy”, that discusses an IDC (International Data Corp.) report on the Big Data Economy to 2015:

“IDC defines Big Data technologies as a new generation of technologies and architectures designed to extract value economically from very large volumes of a wide variety of data by enabling high-velocity capture, discovery, and/or analysis. Further, the study segments the Big Data market into server, storage, networking, software, and services segments.”

Compound annual growth rate (CAGR) is looked at in this IDC report, and for 2015 it is estimated at 44% CAGR for Big Data deployments, and up to 61.4% CAGR for storage.    Servers and software are less, being estimated respectively at 27.3% and 34.2%.  

So – my conclusion, echoing the Greenpeace and “dirty data” discussion above: the soaring rise of the data economy, and in particular storage, point to high growth of up to around a CAGR of 60%.  Technology as expressed by Moore’s law can only partially keep up with this pace – 41%.   Renewable power generation, as represented by the leading category of hydropower, is incapable of doing the job alone, in the green and global picture.   The growth of data outstrips greatly computing technologies and green technologies. 

So we have something of a conundrum here: just how do we handle our data economy, while avoiding or mitigating the data economy’s “dirty data” side?   Potential answers: nuclear, hydrogen, a battery miracle.   Or the Spirit of Ireland solution.