Information Scales

Published by Lex on

The conventional wisdom is that information scales well.
While data and information can scale, knowledge and context don’t scale at the same pace.

There is lag time when either the data changes or the context changes. How do you know if it is just a blip or if it is a meaningful change?

It takes time figure that out and no one really knows just how much time is needed. The time needed also varies a lot based on a number of unknowable factors.

Knowledge builds slowly, but information and data grow even more quickly. The cycle between information/data and knowledge is the process that AI is trying to accelerate.

The open question is how well does it do? Is it faster and better than humans? Does it lead to better decisions? And better by whose definitions?

Computers definitely process information more quickly, but it is governed the parameters that humans set up.

Increasingly, the processing of information is being outsourced to machines. These machines process information invisibly. We are not entirely sure of the inputs and we don’t know how the machines came up with the outputs.

This is not entirely different than how humans process information, but at least we might be able to reverse engineer the process to some degree to see how a conclusion and decision were made.

The fundamental gap is that data grows far faster than knowledge.

There is a desire to accelerate the growth of knowledge to keep up with data

There are a number of incentives to produce new knowledge based on new data.

And all this processing takes in place in an environment of confirmation bias. Human biases and human biases baked into algorithms.

In the big picture knowledge scales, but it definitely lags behind data and information.

So while the amount of information scales up, our ability to make sense stay just about the same.