Sometime in the early-1990's I had an epiphany and decided that I should write a book. In the course of doing my job as a software engineer in a software vendor corporation, I began to realize that we were dealing with lots of data, but trying to use it to make decisions for real life actions. I began to see that there was a progression from the data that filled our databases to the knowledge, and hopefully wisdom, that was used by various managers and executives to support and inform the decisions they made. Decisions that changed people's lives and made the business successful (or not).
I thought about the steps in the process and decided that there were five:
- Data
- Information
- Knowledge
- Understanding
- Wisdom
I never got around to writing the book I planned, but I never forgot the DIKUW process that I had "invented." In more recent years, with the advent of the web in general and Google in particular, I did a few searches to see if anyone else had discovered this process and written about it. Not surprisingly, many people have taken an academic approach to this very thing. I realized that I had independently invented a concept that had been invented previously by many others. There is quite a field in the study of the DIKW hierarchy, as it seems to be generally called.
When I discovered that, I sort of gave up on the idea of writing a book and pursuing this course of investigation. It didn't make sense to me to fish in the same waters where so many others had already been. However, as the years have passed, the same problems remain and it is not clear that those studying the field have in fact "fished the waters out." A recent quick review of the related literature shows that the field continues to be a source of papers, blogs, books, and general discussion.
The reasons I began to think about this topic so many years ago remain challenges. We have tremendous new tools at our disposal that were not even imagined 15-20 years ago. But, we have the same problems and many of the same inefficient uses of our data. Looking at the problems in the world today, it seems that we haven't figured out how to develop the knowledge and the wisdom that we need to truly solve the problems and "make the world better" based on the data we continue to gather in such large quantities.
With that in mind, I want to do a bit of a summary of the DIKW hierarchy so that it can be used as a model to inform the rest of this website. In my professional life, I am still trying to find better ways to solve these problems and provide access to the information that leads to knowledge and wisdom.
I would suggest reading a recent article in the Harvard Business Review title The Problem with the Data-Information-Knowledge-Wisdom Hierarchy for another summary and consideration of this topic.
Most people attribute the real origin of the DIKW in the information systems world to Russell Ackoff. I had never heard of Ackoff before beginning this research, but his reputation is immense. He died in 2009 and at least one article compared him favorably with Einstein for transforming our approach to problem solving. In his address to the International Society for General Systems Research accepting the presidency in 1989, Ackoff popularized the DIKW hierarchy.
Since it will be referred to often in this blog (I am presuming) and there are many places to read about other's ideas and definitions of this model, I won't go into great detail. However, I do want to share some personal thoughts about these steps in the hierarchy and how I see them applying in the information systems that we focus on today.
Data
When I first started working in this field, it was know by the pragmatic title of "data processing." That made sense, because we wrote computer programs to process the data that we collected. In those days, the programs I wrote were very simple by today's standards. We took simple databases, often flat files, of data and "processed" them into other flat files of data. Sometimes the output files were printed in semi-human-readable format for human consumption.
Data is simple to define and it is something all information/knowledge workers have worked with. Data are facts, numbers, symbols that have very specific meaning. For example, the population of the United States is 322,127,741 according to the US Census Bureau's U.S. and World Population Clock as of 10:05 PM EST on Nov. 9, 2015. That is a piece of data. We will assume it is true since it comes from a reliable source. By itself, this data does not have much meaning, but we could collect billions of facts such as that one. The same website shows the top 10 populous nations. The first five are: China (1,367,485,388), India (1,251,695,584), USA (321,368,864) (interestingly different than the population clock), Indonesia (255,993,674), and Brazil (204,259,812). Those are all pieces of data.
Information
Information, the next step up in the DIKW hierarchy, would be a way of working with the data to make it more valuable. Information, I will describe as combined data that has a value or a use. Based on the data shown in the above section, we can say that the population of India is about 4 times that of the United States. That is information that may prove useful for political and economic planning and prediction. That information can only be determined by gathering data and then processing that data.
Knowledge
The next step, knowledge, I will present as combining data and information to produce actionable plans. We have knowledge when we combine data and information with experience and develop a level of expertice that allows us to project that forward to making decisions. For example, with the information we produced above regarding the comparison of the US and India's populations, we might be able to come to the knowledge that the market for cell phones is much higher in India than in the US. By adding other data such as cell phone coverage in the two nations, cost of land lines, per capita income, market saturation and many other pieces of data and knowledge, we might decide that to grow our cell phone business we should try to break into the Indian market. This is knowledge that can lead us to making decisions.
Understanding
Understanding is a level that I included in my original data-to-wisdom process, but it doesn't seem to be included in most manifestations of the DIKW hierarchy. I thought of understanding as the application of knowledge with specific and related experience and focused research and thought. Understanding is that step beyond knowledge that is something not everyone will be able to come to with the same level of data and information. Given a "complete" set of data and information related to the US and India and the cell phone market, most intelligent people could process the input and come to similar knowledge. They can do the calculations, do the comparisons, ponder the meaning of the data and present a report of knowledge to a boss regarding the need or lack of need to penetrate the Indian cell phone market.
However, since I have never been to India, have never studied Indian culture, and know very little of the nation and the people, I don't have the understanding of India to make a good decision. It is possible, I have no idea, that Indians in large numbers are distrustful of US personal technology devices are are predisposed to avoid buying from US corporations. It may be that the cost of doing business in India is such that I can't sell cell phones at a profit.
This last item is something to consider. You might say, "Well, you could research that and get the data and the information to be able to know that. What's the problem?" The problem is that without the experience with Indian people and markets, I don't necessarily have the understanding I need to even know what questions need to be researched. I may be mislead to believe I know everything I need to know to make a good decision, when those with more experience and understanding would be able to give me a list of additional items to research. Since I don't know what I don't know, and I don't know which of these items are pertinent to my issue, I am likely to miss important data and information and am likely to make a bad decision.
Wisdom
Wisdom, the pinnacle of the hierarchy, is our real goal. We recognize wisdom as a valuable commodity, but what is it really. I think that is the most interesting question, but I am going to leave that for another time. I will define wisdom as the ability to make correct decisions for the circumstances presented and to properly carry out those decisions.
We will address that more in the future, but the pursuit of wisdom takes us far afield from data and information. Wisdom can only result from having proper goals and values. Wisdom allows one to make a decision against what data, information, knowledge, and understanding is telling one and to believe it is the right decision.
Research and analysis may tell me that if we attack the Indian cell phone market, we can double our revenues in two years and increase our profits threefold. But, if the goal of our company is to provide affordable cell phones to senior citizens throughout the US, then we will probably say "no" to the opportunity to go i nto the Indian cell phone market. Doing so will distract us from the goal of our company and likely mean that we will never achieve that goal. You may believe that is a silly goal, or that we should pivot the company to take advantage of this great opportunity. Wisdom allows me to remain focused on what I value and make decisions and carry them out despite distractions and information that leads me away from my goal.
Why is this Blog Called Information-to-Knowledge Quest?
It becomes apparent that I value wisdom very highly, why not call this blog Data-to-Wisdom, or something like that?
The reason is one of practicality. I believe that we are at the point with data and information technology that we can use our tools to solve many of the information-to-knowledge problems that we are faced with. We will try to understand more fully how to do that and investigate the trends. However, I don't believe we are now ready and able to move to the understanding or wisdom levels. We will talk about those and try to gain more understanding (no pun intended) of the processes, but we are not ready to automate those processes. We will never be able to automate those processes, because if we do something and automate the process of turning information into wisdom, we will have ceased to be human. We will have turned our freewill over to machines and we will have produced a world that science fiction has long warned us about.
I have gotten into the philosophical realm, and it is probably time to stop, so I will. I hope to look at some of the questions raised by this post over the weeks and months to come.