What happens when evidence is everywhere and available to everyone? If we all have the same guides to the same destinations, what does that mean for where we are going and is it where we want to go? In this latest article in my series looking at evidence in 2024, I look at what it means to create in a world awash in data and evidence.
In the previous articles in this series, I introduce a general sense of challenges we face with the state of evidence in 2024. These are ones that are based on trends that I’m seeing and their implications for what it means to generate and use evidence to make decisions. In the last article, I looked at issues with evidence tied to AI and how its ‘black box’ of inputs, outputs, and decisions makes it risky for use as an evidence tool and adds many complications for how we incorporate it into our decisions. In this article, the focus turns to the human side of content creation and what evidence looks like when everyone can be in the evidence business and whether that is a good thing.
Evidence and Truth-Telling
Evidence is at the bedrock of our decisions to inform what we do as a society. (it’s not the only thing, but it’s one of the few things that we can generate that is systematic, transparent, and has some appreciable level of objectivity to it that can be shared by us all). Whether it’s the tools we use, the medicines and treatments we apply, or the lessons we teach and learn, evidence is at the heart of it all.
One of the least acknowledged aspects of evidence is the often over-reliance on an idea of a single truth. Whether this is a widely held idea of truth or not, the suggestion that evidence equals truth is what guides much of what we call evidence-based decision-making. This idea is tied to a model of knowledge and wisdom that is inflated and promoted in Western culture (but is present worldwide) of a canon. The canon is a collection of stories that serve as a voice of trust and authority. This voice provides truth criteria that are codified, canonized, and lionized.
This idea of the canon would be more comforting if it were true, but it is not — at least not as a the sole source of truth.
The canon was protected by scarcity. There was a time when the amount of effort it once took to create and publish a work — a poem, speech, book, film, or musical piece — was considerable and thus, there wasn’t as many such works. Anyone dedicated to study and inquiry was someone who could create evidence and, with some influence, inform what we do and what became known as truth.* The chance that one of these works was both good and widely accessed and shared by others was far lower than it is now when anyone can create something with little thought, skill, and access to networked technology. That is where we begin.
(*this wasn’t always the case, of course, and still isn’t)
When Everyone’s An Evidence Generator
If you want advice on anything — almost literally anything — you will find it in abundance online. The meme-culture that drives the sharing of quotes, tips and tricks, and lessons learned is overwhelming. Just like anything online, some of this content is useful. Some of it is also true.
Much of it is neither.
Derek Sivers wrote a clever book that focused on some of what is shared as sources of wisdom and truth contradict one another. In the book, he shows that much of what we consider true is a paradox such as: do things for love…and money, be and expert and embrace your inner idiot, or how original ideas are copies. The point Sivers makes is that there is some peril — or at least context — we need when we consider a truth or canon to be self-evident.
These are more philosophical examples, what about using empirical sources of truth?
Not much better.
Many of the platforms we use to create content and distribute it come with it the means of gathering, using, and generating evidence. Where we once had little idea of what kind of market we were working in and who, where, how and why people might access our content. Now, we have an abundance of evidence to guide.
The Case of Content Creation Distribution
Take a look at any publishing platform that can be monetized — blogs, Medium, Substack, newsletters, Instagram, YouTube or any other social content platform — and you will find a plethora of content creators that generate posts on how to grow your audience and make money. Many of these are based on experience, data, and detailed analysis of trends and consumer behaviour (and many are not). The central idea they market is: follow these pieces of advice and you can, with some persistence, generate an enormous list and many subscribers.
Search Engine Optimization and the myriad tools, technologies and courses available to expand your reach and influence algorithms all do the same thing.
Yet, what happens when everyone follows that advice? Every post gets programmed to send at just the right time — which often happens to be the same time — meaning there is a flood of messages that get sent and received at the same time. As both a content creator and recipient, this is maddening.
Yes, [insert time and day here] might be terrific for me to read a newsletter, but what happens when I get 10 of them at the same time because they all use the same guidance about “what works”? The use of evidence en masse nullifies the power of that evidence.
Having a video land in my inbox every morning at a certain time works, except when I am busy, and then the backlog starts. It’s like to anyone who’s got a subscription to The Economist: soon it’s about how many issues you’re behind, not what you have in front of you. In the case of marketing newsletters, the issue soon becomes one of having real-time evidence because once the widespread adoption of the existing evidence takes hold, that evidence is no longer valid.
Maybe it’s about making it more dynamic and real-time? Nope.
Is Real-Time Evidence The Solution?
Real-time evidence isn’t all that better, either in this case. We can see this when services like Waze, Google or any other GPS-enabled map services provide real-time guidance on traffic. Soon after a problem occurs on the road, the solution to a traffic jam is now a different problem because of the widespread adoption of the new strategy to avoid that traffic jam. This chaos has led to traffic becoming unmanageable.
The same is true for marketing and creating those kind of ‘sticky’ or attractive messages. If everyone is using the same click-bait phrasing, publishing scheme, image selection etc.. pretty soon everything looks the same. There’s no value and thus, the evidence doesn’t work by virtue of following it. Kirk Clyne recently posited the idea that soon we’ll have real-time data guiding our online marketing experience powered by AI. Our own ‘private’ internet will be designed to create a personalized experience. But what if what we might need is something that’s different from what we’ve had? Kirk’s model of reality that he outlines — which is an idea, not something he’s advocating for — isn’t tooled for that.
What I am saying isn’t universal, of course, but we are seeing more situations where the abundance of evidence, the increased access to it, and the means of synthesizing and distributing it is creating problems we’ve never had before.
The systems that are used to generate the evidence we use is guided by algorithms. When we don’t know what goes into them, how can we make sense of and use the evidence we have access to?
I’m publishing this when I’m done and ready to send it out. I used to try and land my messages when I thought my audience wanted to read them, but for the reasons outlined above, this no longer works because everyone else is doing the same thing. Now, I publish it when I’m ready — day or night, weekend or weekday. Thanks (or no thanks) to algorithms in the distribution networks I use from email to social media to search I’ve no reliable way to predict when people will see this or discover it. I’m OK with that now, but it means that the evidence I’ve been given is mostly useless.
Now what? That’s what we’ll need to explore and that’s where I’ll go next in this series. Thanks for reading — no matter how you got this :).
Photo by Nick Morrison on Unsplash and Photo by Austin Distel on Unsplash
Comments are closed.