Data Maturity Literature Review: Appendices

These are the appendices to the Data Maturity in the Social Sector 2016 report.

Appendix 1 Key Sources on Data Maturity

Accenture, Analytics Maturity Assessment, Netherlands 2015

Anderson C.,”Creating a Data-Driven organisation: practical advice from the trenches“, 2015

Booz, Allan & Hamilton,  “The Field Guide to Data Science“, 2015 (2nd Edition).
(Lit. Review list & Data Science Maturity Model on page 35)

Davenport T., Assessing your analytical and big data capabilities, Wall St Journal, July 2014

Eckerson W., The Data Warehousing Institute, Business Intelligence Maturity Model

Fisher D., “Data Analytics Maturity Model“, 2014

Howard J., Review of the INFORMS analytical model, 2014

Howson C., TScore Overview for BI and Analytics, Gartner 2015

Marsh M., “Review of skills and leadership in the VCS sector” (section on data-informed social change), 2013

Mason H., video and e-mail comms,  2016.

McSweeney A., “Review of Data Management Maturity Models” 2013

Parenteau P., Sallam R., Howson C., Tapadinhas J., Schlegel C., Oestreich T., Magic Quadrant for Business Intelligence and Analytics Platforms, Feb 2016

Polynumeral Blog, the number one question CEOs ask about data 2016

Patil D.J., Mason H., “Data-driven – Creating a Data Culture“, 2015

Sedar J., Data Science maturity model blog, March 2016

Soares S., “The IBM data governance Unified Process“, Sept 2010

Yanosky R., Arroway P., The Analytics Landscape in Higher Education, Educause, Oct 2015

Appendix 2 People we interviewed

  • Jake Porway, Founder and Executive Director at DataKind, New York
  • Duncan Ross, Data and Analytics Director, TES Global (Founder/Chair DataKind UK)
  • Jonathan Sedar, Consulting Data Scientist at Applied AI Ltd
  • Shyann Seet, Independent Data & Analytics Advisor
  • Hilary Mason,  Data Scientist, Fast Forward Labs. (comment via E-mail).

Appendix 3 DataKind’s prototype data maturity model


Appendix 4 A selection of the models and frameworks we found

– Applied AI Data Science Maturity Model

– Gartner Master Data Management Maturity Model

– Gartner Business Intelligence and Analytics Model

– The Data Management Maturity (DMM) (see CMMI institute)

– Steven Mills, Chief Data Scientist, Booz Allen [PDF]

– Educause 2012

– Gapbridge Analytics

– Infofarm Data Science maturity model

– Dan Fisher, Data Analytics Maturity Model, 2014.

– Jay Zaidi, 2015 (AlyData): Data Management Maturity Model (DMM) developed by the Software Engineering Institute at Carnegie Mellon University.

– The Data Warehouse Maturity Model Business Intelligence Maturity Model [PDF]

– Big Data Maturity Model (2012), comes from an IT perspective. Advanced version  has more detailed emphasis on value creation, risk management, compliance, competency, architecture, policy, security, organization, audit.

– Comparative view by A McSweeney. McSweeney A., “Review of Data Management Maturity Models” 2013

Introduction to (Big) Data Science from InfoFarm


Appendix 5 Open Data Maturity Model

The Open Data Institute launched the first edition of its The Open Data Maturity Model in March 2015.

ODI Maturity Model: Guide – Assessing your open data publishing and use by Open Data Institute

They’ve built a ‘Map your pathway’ App which offers to help assess where you’re at, set goals and track progress towards them.  Benefits it promises include: Discover your organisation’s strengths and weaknesses; Identify areas of improvement to optimise progression; receive practical recommendations to help achieve your goal.

Appendix 6 Social Impact Data Maturity Model, University of Chicago

 

For more details see the Social Impact Data Maturity project site (we believe that the Data Maturity Framework resources are available under an open licence).

C:\Users\Datagirl\AppData\Local\Temp\Temp1_University_of_Chigaco_Social_Impact_Data_Maturity_Framework_4.28.16.ZIP\Page2.jpg

C:\Users\Datagirl\AppData\Local\Microsoft\Windows\INetCache\Content.Word\Page3.jpg

Context

Background

Our first step in the project was to work out what data maturity means and what models and frameworks already existed.   In May 2016 we published a short report on what we found.  

This led us to the conclusion that we needed to create our own framework and set out some important definitions around the research:

– Data – When we say data we have a broad definition. We include all the types of information an organisation might collect, store, analyse and use.

– Social Sector – In the context of this project we defined the social sector as being charities and social enterprises (businesses trading for social and environmental purposes).

– Data maturity – The journey towards improvement and increased capability in using data.

 

What we found out about Data Maturity in other sectors

Sian Basker and Madeleine Spinks, Data Orchard, 26th April 2016

Over the past couple of months we’ve been finding out what existing work has been done on Data Maturity and what we can learn from it. We’ve been trawling the internet, reading the literature, and interviewing some leading people with relevant knowledge and expertise (see Appendix 1 and Appendix 2 for details).

1. Data Maturity Frameworks

The concept of data maturity is relatively new and seems to be most widely used and understood in the data science community. Whilst there’s no listing for the term on Wikipedia, we found around 40-50 different models/frameworks and related theories. Indeed there are reported to be hundreds. Many of these focused on a particular industry or aspect/s of data: information maturity, analytics, business intelligence, data governance, open data, data warehousing , IT architecture, or big data. Various examples are listed in Appendix 4.

The earliest references we found to ‘data maturity’ were around 2005-2007 when both Gartner and IBM developed models for data quality and data analytics maturity. IBM built its first Analytics Quotient model in 2010 offering a quiz which identifies businesses as ‘novice, builder, leader or master’. Many other models emerged around 2012-13 when ‘big data’ first gained high media profile.  As knowledge and understanding has advanced more sophisticated/ updated versions have appeared.

Many data maturity models have been created by specialist vendors and consultancies as a means for selling products and services. They offer varying levels of simplicity/ technical detail to enable potential clients to understand where they are and where they might be going. Published examples include: an updated 2014 IBM model,Cardinal Path, Adobe Applied AI,  Accenture . Typically they offer an organisational diagnostic and, in some cases, an assessment report e.g. 2014 The Data Warehouse Institute analytics maturity tool assesses five dimensions (organization, analytics, data management, infrastructure and governance). Much of the published literature and resources are in fairly high tech language largely focused around data architecture and data governance. These tend to be aimed at more technical audiences within large enterprise environments.

In essence what many models explain is the journey from looking at retrospective ad hoc data to explain the past,  to a more continuous ‘current/real-time’ understanding of the here and now, a level of optimizing for efficiency and effectiveness, through to the ultimate state of predicting and creating the future.  Some models use analogies to human development pre-natal/infant/child/teenager/adult/sage; or (with reference to machine power) crawling/walking/running/riding a bike ; others focus on practical processes, tasks or action.

Most models we found were conceived and applied within private sector markets where primary drivers have been efficiency, risk management, maximizing revenues, and competitive advantage.  They tend to be aimed at large/very large enterprises where turnovers below $10M are regarded as small.  The sectors where data capabilities are being most rapidly and innovatively advanced appear to be in competitive markets: retail, technology, energy, health, insurance, and increasingly, banking and finance.

The only commercial sector benchmarking research around analytics maturity we found was published by Accenture in the Netherlands in 2015. Based on the DELTA model by Tom Davenport of the International Institute of Analytics, it had surveyed 250 companies in 2012 and again 2015 using key indicators along the themes of: Data, Enterprise, Leadership, Targets and Analysts. It identifies a move from the earlier use of analytics for improved efficiency towards a more recent development of data to support new and improved ways of working and decision-making. It also shows data analytics maturity strongest in sales and marketing roles.

Data Analytics has become an integral component of the service offer amongst leading business consultancies e.g. PWC, Deloitte, KPMG. Whilst these primarily serve the private sector, they also count government and charity organisations amongst their clients.

The public sector also has its data maturity story.  Whilst efficiency, security, and risk management were early drivers; increased transparency and public accountability are also key. The Environment Agency Data Maturity Model started in 2011 and now uses this alongside the 2015 Open Data Institute (ODI) model. The latter has been used to assess departments e.g. in November 2015 it published a scored DEFRA assessment against five themes. The primary departments involved in supporting this work appear to be: The Cabinet Office, DEFRA and BIS. Nesta is also currently undertaking research on data in the public sector.

To date we’ve found two models relevant to the social sector, both from the US. The first is Educause which has benchmarked analytics in hundreds of higher education organisations (many of them charities) since 2012. It offers some great resources and useful insights into how organisations are progressing  and not progressing. The second is data maturity model developed for social impact. Published in April 2016 by the Centre for Data Science and Public Policy, University of Chicago, it offers a framework around two aspects: data and technology; and organization readiness (Appendix 6). They plan to collect data from non-profit and government organisations and use this to benchmark.

There’s general agreement that whatever the sector, very few organisations are operating at the very advanced levels. Indeed most are at the early stages.

Much of the discourse and literature suggests the journey to maturity is fairly long and challenging, though there’s evidence to show it’s worthwhile in the end. The most recent well-rounded and applicable resource we found was a book called “Creating a Data-Driven organisation: practical advice from the trenches“, by Carl Anderson in 2015. Experts suggest it can take five years or more to develop, implement and reap the rewards of becoming a data mature and data driven organization. However, there seems to be no single ‘truth’ about data maturity. Those we spoke to had very different perspectives.

“I don’t see any use of formal rubrics inside of most companies. There’s a wide range of sophistication around how people document data, provide access, and think about data governance, but I don’t see a standard way of thinking about this. This is likely due to the different regulatory standards and norms in each industry, and each company has their own point-of-view around those norms. We work with folks in insurance, banking, and legal, and they are all very different.”

Hilary Mason, Data Scientist, Fast Forward Labs.

Whilst many of the maturity models are fairly simple, many recognize the complexity and interrelationships with other key aspects of organization development. Notably: leadership, business planning and strategy, culture, as well as the policy, security, data governance and underlying infrastructure digital tools and systems.  There’s no evidence to suggest how widely used the various maturity models are nor how useful. Indeed some commentators suggest they’re not worth doing at all.  However comparing with peers and market leaders  through benchmarking appears to be a popular approach to raising aspirations and understanding stages of development.

2. Do the existing frameworks concur with DataKind’s theoretical 5 stages?

DataKind’s theoretical model in Appendix 3 (based on one from Terradata via Duncan Ross, Chair of DataKind UK) suggests a pathway of: Nascent, Explanatory, Exploratory, Developing, Mastering.

Many of the other frameworks we identified had 3, 4 or 5 stages that are not dissimilar in principal.  However the detail and language does vary significantly and will need further research and development.    The typical model in the private sector is unlikely to resonate with the vast majority of charities and social enterprises. However many of the larger ones may already be operating at that level to some extent i.e. the 1.2% of charities and 6% of social enterprises with turnovers of £5m+.

3. What enables organisations to become more data driven

The seven key factors that appear most influential and effective in enabling organisations to grow and develop in their data maturity are:

– Data people at the heart/centre of the organisation, adjacent to leadership team

– Data recognized and valued as a key asset with and data culture established as a collective effort i.e. data is a team sport, not the responsibility of just one data person, data becomes intrinsic skill and asset for every team in the organization.

– Data must be accessible to many in the organisation. Therefore people need to be able to query, join/relate and share the data across the organisation.

– Quality Data: the organisation must be collecting the right data, relevant to the question at hand, and be able to trust it with confidence.

– Skills: people with the right skills to steward and query the data, asking the right questions.

– Time to absorb, discuss and challenge using data.

– Forward looking: moving from reporting on the past (what happened?) to the present (what’s happening now) to the future (extrapolation, modeling, recommended action, prediction/simulation).

According to Anderson, the top barriers stopping organisations making effective us of data are:

– Lack of understanding on how to use  analytics to improve what they do

– Lack of management capacity (competing priorities)

– Lack of internal skills

– Existing culture doesn’t encourage sharing.

(For full list of barriers: see chart on p. 16 of Anderson, 2015).

What next?

This research will be used by Data Orchard and DataKind UK as part of the Data Evolution project to explore social sector data maturity. The findings will be shared as a blog post and feedback and comments will be welcome.  Discussions about this research will also be held with social sector organisations at various events and workshops during 2016. More specifically the research will help shape our questions with charities and social enterprises as part of a national survey.

Contact

Appendices

View the appendices.

Why not stay in Hereford for our workshop?

Very old hadn-dran map on the left hand side in black and white juxtaposed with modern infographics

Hereford is easily accessible from Cardiff (1 hr on the train, 90 mins driving), Bristol (90 minutes either way), Birmingham (1hr 40 ish by train or car) and Manchester (2 hrs on the train, 3 by car).

Which makes our Hereford workshop the obvious choice for many charity and social enterprise leaders.

As our London workshop has sold out many people from further afield are thinking of making the trip. If you’re looking at a map and wondering whether it’s a good idea here are a few points we think may sway you.

Stay here

There is high quality accommodation on-site. En-suite, accessible rooms are available to book at Gardner Hall (where the workshop is being held). Residents also get use of the adjacent (fully accessible) Spa and Gym. If you fancy something a bit more rural, we have lots of accommodation really, lots of accommodation.

Plenty to do the night before

There are great (independent) places to eat and drink in the city like Shack Revolution, Beefy Boys, Rule of Tum (they were in the Observer), and the Cellar Door. Herefordshire also boasts the cheapest beer in the country which may interest some of you.

Plenty to do down the road

The Hay Festival is just down the road and starts the day after our workshop. So our workshop could be the start to a long weekend in the borders. And if literature isn’t your cup of tea. Try philosophy at the How the Light Gets In festival.

We know what you’re thinking. It sounds too good to be true. But it is true. Book now.

Photo credit: the image at the top of this post features an image of the Mappa Mundi used as it is in the Public Domain. The Mappa Mundi can be seen in Hereford Cathedral. .

Will you be our test pilot?

We would like to pilot our survey

We are busily working on developing an online survey to launch on 6 June 2016.

The way we think this is likely to work is:

  • On 6 June we will publish a short online survey that anyone from any charity or social enterprise will be able to take part in (and we hope loads of people will)
  • Then we will ask some of the people who took part in the short survey to encourage several people in their organisation to take part in a more detailed survey (and in return they’ll receive a report about their organisation benchmarked against other organisations

There are some options within this so we’d really like to test our initial survey to make sure it is easy to use, makes sense and gives us the information we need.

So we’re appealing for volunteers to help us test the survey.

It’s easy to help

You’ll need to take the pilot survey online (hopefully this will take no more than 5 minutes) and then talk to our research lead Sian Basker on the phone (or on Skype) about how the survey worked.

We hope to test the survey this week or next week.

If you’d like to be a test pilot (and we really hope you would) leave us your details below. Or drop an email to sian@dataorchard.co.uk .

Marketing update number 2

We’ve been getting the word out to charities and social enterprises across England and Wales. Well admittedly Wales is looking a bit sparse at present. So next week we’ll have a push there.

If you can help us tell people about the Data Evolution workshops or get people to sign up ready for our survey we’d really appreciate it.

We’re using Google Fusion tables to generate these maps. It’s surprisingly powerful, free and really easy to use. We’re thinking about a blog post explaining how we did this. Let us know if that would be helpful.

For comparison this is how things stood (marketing-wise) a week and a bit ago.

 

 

Progress (and data) on publicity

We launched the Data Evolution project last week. We’ve been writing to organisations, especially local infrastructure organisations.

We thought you’d be interested to see the distribution of charities and social enterprises we’ve written to so far. This distribution is based on the postcode of the head office. It reflects the fact that we’ve contacted organisations local to Herefordshire and London: where our workshops are taking place.

We really need your help to spread the word.

We’ve got resources online to make it really easy for you to do that.

Really great workshops: not all in London!

Vintage photo of a group of people being shown a model juxtaposed with modern infographics
This image uses a NASA photo uploaded to Flickr as part of the NASA Remix project and shared under CC-BY-2.0

Come and share

As part of the Data Evolution project we are running two workshops for leaders of charities and social enterprises. They will be participative and engaging. We hope that everyone who attends will have experiences to share and will learn from the experiences of others.

There is a small charge to cover costs.

London

One workshop takes place in London on 19 May 2016.

Herefordshire

And the other workshop takes place in Hereford on 25 May 2016.

(We’re sure you know this but just in case: Hereford has excellent rail connections from South Wales and North West England as well as the Cotswolds and central and south Birmingham).

We’ve arrived

Black and white picture of an old fashioned operator juxtaposed with abstract infographics

Hello world!

We’ve been waiting for a long time to say that.

In fact Emma Prest first wrote about the project back in December.

But finally the Data Evolution project is go.

Why should you care?

Data is important.

We don’t need to tell you that.

Across the social sector we know people recognise that data is an asset to their charity or social enterprise.

Every organisation is different, their data is different, their services are different.

Understanding where you are on a data journey (and so what the next step should be) is tough. And it’s tough to design projects to help charities and social enterprises without understanding the same issues.

This project is going to develop a framework to understand how different sorts of charities and social enterprises approach the use of data. We want to understand the journey (or probably “journeys”) that organisations take to make better use of their data.

This is not academic research. We are organisations that want to get stuck in and make a difference.

This is research that will lead to practical help for charities and social enterprises.

We need your help

We have a survey launching online on 6 June. It will take only a couple of minutes to complete. We really need you to take part in that and to encourage other charities and social enterprises to do the same.

You can sign up right now to be reminded when the survey launches.

If you have newsletters or networks of your own we’ve got some resources to make it really easy for you to spread the word.

If we get good participation from across England and Wales in this initial survey the rest of the research should be a breeze (that’s what we keep telling ourselves).

Your help could make all the difference.

Community

We also hope that the Data Evolution project will bring together the data community in the social sector. We’ll be sharing interesting things here on the blog and on our Twitter account.

Oh and we have some awesome workshops at the end of May. One workshop is in Herefordshire and one workshop is in London. Please come, or make your chief executive come.

We’ll be here all summer. Please keep in touch.

Hello World!

About us

Data Evolution is being delivered by

DataKind UK logo
Data orchard logo

With generous support from

Teradata
Nesta...
Esmee Fairbairn Foundation
Access The Foundation for Social Investment

To be successful the project will need the help and assistance of hundreds of charities and social enterprises right across England and Wales.