Data has always been essential to the running of any businesses.
Ancient man used clay tablets to record the numbers of animals bought and sold; our more recent forebears created ledgers and files of information stored on shelves and in filing cabinets. These days, of course, it’s all on a chip or in the Cloud. And, thanks to the data pulling power of developments like the Internet of Things, there’s much, much more of it than ever before.
Social media channels and sensors in everyday objects from buildings to cars to fridges, mobile technology and software logs are just a few of the ways massive data sets are constantly generated and updated. Information extracted from this data is increasingly used by science, government and business to deepen understanding and make critical decisions.
Emerging analytical and business decision support engines can make a real difference to your business. The key to exploiting it successfully is knowing what data to use – and what data is needed about your data.
“Experts often possess more data than judgement” – Colin Powell
Why should you consider this?
Information gathered from disparate data sets can provide detailed customer insight – who they are, what they do, what they like and don’t like, at levels that have not until now been achievable. It can allow a business to spot trends in social media and respond to them in real time. It can rapidly identify whether a product or service is a hit, or has reached its twilight and funding should be withdrawn. It can, quite simply, give a business the edge over its competitors.
A very literal example of this is how organisations such as Rolls-Royce and many Formula One teams, gather huge quantities of real-time telemetry, and use this to make immediate changes to how their systems (car plus driver) operate, as well as feeding this into future system developments.
Analysing the sheer volume of data in big data sets takes a lot of computing power and extracting insight is increasingly achieved through the use of artificial intelligence and cognitive computing. Computer systems that simulate human thought processes through computerized models, and self-learning systems that use data mining, pattern recognition and natural language processing to mimic the way the human brain works.
Cognitive computing may sound like science fiction, but the mathematics behind it was created in the 1980s and 90s. The main difference now, is that we have the computational power to cope with the workload, and it’s now more affordable and easy to procure (Amazon Web Services or Microsoft Azure, for example, sell it by the hour).
However, machine “learning” takes time, especially with unstructured data. When the machine starts to analyse, looks for patterns, and attempts to provide insight, it will only deliver results as good as the data provided. In order to succeed, a business needs to consider the quality, consistency, and compliance requirements of its data, to facilitate faster, and better informed decisions. But how is this done?
Closing the barn door after the horse has bolted!
In the early days of cognitive computing, I had a long conversation with someone involved in a trial to demonstrate the value of the artificial intelligence tool against years of existing customer service data. I asked them how the trial was going and the answer was, surprisingly “not well”. The reason was that the data provided by the client was largely unstructured and came from a number of systems and sources. The cognitive technology needed more data about the data, known as metadata, to allow the mathematical model to work.
Eventually, the pilot did generate reasonable results. However, a great deal of manual time had been invested to retrospectively add metadata to structure the existing data. Quite simply, this is completely uneconomical on anything other than small scale, ad-hoc demonstrators, so it is vital for a business to consider the data generated by its systems as early as possible.
The data explosion
Data volumes are exploding; it has been stated that more data has been created in the last two years, than in the previous history of human kind. By 2020, about 1.7 megabytes of new data will be created every second. According to Google, we perform 40,000 search queries every second on their platform alone.
“Picture a Megabyte as the equivalent of a small novel”
Most companies possess, or will possess, a mixture of structured, unstructured and semi-structured data. Unstructured data enters an organisation from many different sources, that are often ignored, are unmanaged, and not at all tied into the way it currently analyses and reports. Metadata, data which describes what data is, and what it relates to becomes increasingly important. An example of metadata for a document, or a scanned invoice, would be; file size, data created, source, author.
Metadata can be created manually, or automatically from source systems. Manual creation tends to be more accurate, but as we have discussed, the volumes are very large, and manual creation is no longer a practical option.
“Metadata management can be defined as the end-to-end process and governance framework for creating, controlling, enhancing, attributing, defining and managing a metadata schema, model or other structured aggregation system, either independently or within a repository and the associated supporting processes” – Wikipedia
In considering current and future business strategy, and the systems needed to support strategic objectives, it is essential to factor in data and metadata generation. From a business point of view, there are many advantages in getting this right upfront, and there are many disadvantages if the issue is ignored, such as:
- Rising costs and administrative overheads of managing this data, and no practical way of gaining any knowledge from it
- No clear idea of what to keep and what to archive
- The possibility of missing an emerging trend that could present itself as a lost opportunity
- The chance of overlooking a shift in the buying patterns of key customers
- Less accurate allocation of marketing and sales effort to the most promising product lines or services
- Less detailed historical data to model the effectiveness of planned marketing campaigns
In a world where more and more companies are investigating the possibilities offered by big data, a business that does nothing could be placed at a disadvantage.
The advantages of good data definition and management are clear. Companies can use data to provide meaningful insight into their business; helping them make better, and more informed decisions. Better information gives the opportunity to:
- Target marketing, using predictive analytics, at existing and potential customers who will be more receptive to a product or offer presented at the right time, at the right place to the right person
- Gain real-time feedback on products or services via social media feeds, and tailor the service or product to the most valuable customers
- Better manage risk, by using data generated by internal systems, linked to social media and other external feeds, to measure and analyse risk, in almost real-time
- Increase reliability of models using predictive analytics.
IT strategy starts with the business and its information
When creating or refining IT strategy, an organisation should use data requirements to shape and define elements of the overall IT strategy. It should consider the thought and planning that will shape its IT strategy, in terms of:
- What does it see as its differentiator, and what does it need to invest in, to create that differentiator?
- What systems does it need to generate the data it needs?
- What systems does it need to upgrade or replace, to provide the outputs it requires?
- What standards does it need to define and manage, to ensure compliance with, and support of its data requirements?
- What external services does it need to consider?
- How will it link its internal data with external sources such as Internet search engines and social media to spot side effects during drug trials? (See Wired for a real FDA example.)
- What differences are there in the organisation’s IT data requirements, and the requirements from a business or operational point of view?
- If a business-to-business organisation, what does it need to consider in the end-to-end hand-off of data to assure compliance in a highly regulated market such as pharmaceuticals or Meat Traceability Compliance?
When all types of systems are generating a variety of information, it will soon become apparent that missing metadata cannot be fixed retrospectively without incurring major expense. Just as automotive manufacturers find out, if a vital component is missing from a vehicle as it rolls off the production line, it’s a difficult and expensive process to recall all those vehicles and address the issue.
So, a clear strategy, to get the data right, is an essential part of planning, and a key consideration for the systems and solutions an organisation deploys.
In summary, data volumes and issues are only set to grow in size and complexity. This article has highlighted the importance of aligning business practice and strategic goals with data and information requirements. The opportunity exists to use these valuable assets, to innovate, and create value and genuine differentiation over competitors.
At Searchlight Consulting, we have used the knowledge across our 300-strong network of associates, to create a framework that helps align business, information and IT strategies and creates a supporting plan to deliver strategic business capabilities – digital or otherwise. We work with companies of all sizes to help to identify appropriate commercial solutions and appropriate service delivery models, including “cloud” and SaaS. We then work with clients to evaluate solutions and services, and commercially engage vendors. Our relationship continues beyond selection, through implementation to successful delivery.
We work to:
- Get the best out of Clients’ investment in Enterprise IT
- Shape, create and deliver IT-enabled, business change projects and programmes
- Provide independent, client-side advisors and supporting teams
- Help clients to deliver, meet their commitments, and build sustainable change and in-house capability
Our role is to provide clear guidance to our clients as they navigate their way through the complex and uncertain world of using IT, to transform their business processes, to enable smarter working and be more successful.
We do this by:
- Helping our clients align their IT strategy with their business strategy
- Helping our clients assess their IT capability in the context of their strategy, and identifying how to address any gaps
- Helping our clients implement the IT strategy agreed
- Working with all parties involved in IT-enabled business transformation, to deliver successful outcomes.
To discover how we can help your business, please contact firstname.lastname@example.org