Assessing editorial quality in a world of AI

James Watson

Marketers and content creators are under pressure to show greater efficiencies and productivity gains from AI, but they’re also anxious to avoid a slide in editorial quality. As the volume of AI-generated content proliferates, how do you avoid being put in the AI slop bucket - not least given there’s already a side industry of freelancers being paid to fix problematic GenAI content?

A key first step is to consider how we go about assessing editorial quality. For many, to paraphrase a famous 1960s US legal ruling on porn, good editorial quality is hard to define, but “we know it when we see it”.

The reality is that editorial quality assessments are always difficult to make, not just for writing, but for any creative endeavour. Take the Oscars as an example. Nearly every year, many column inches are devoted to the failings of the awards and how superior films lost out to inferior winners. Reasons vary. In some cases, these are driven by wider social issues at the time (2005’s Crash is a good example). In others, the filmmakers rode good timing and late momentum to the win (2022’s CODA is wonderful to watch, but remains a strange choice for best film). Sometimes, the critics are sniffy about a high quality crowdpleaser winning out over an art house favourite (think 2018’s Green Book trumping Roma). Readers are free to disagree with my assessments here, but the wider point is that quality is somewhat subjective.

What does quality mean for you?

For B2B brands publishing thought leadership content, their teams will rightly be paranoid about ensuring that they maintain a high standard of quality. If, as the industry widely agrees, good quality thought leadership helps ensure that a buyer chooses to work with you, then the converse is also true: poor quality content will deter your potential buyer.

A helpful step forward can be to establish a content quality framework to serve as your internal guiding star. It’s important to understand that all such frameworks are problematic in one way or another, but they nevertheless provide a helpful lens with which to consider your content. The reality is that it’s quite challenging to do in practice - and you should understand that your definition of what is good and bad will vary, depending on your organisation’s goals and approach to thought leadership.

With that proviso in mind, several such models exist to help provide a starting point. Ideally you should pick and choose from each to best align with your preferences - and your target audience. Some of the core points that you should ensure you consider would include:

1. Set out a clear purpose and audience

A first point is to establish a clear intent and purpose for your thought leadership. This is about ensuring audiences gain value and practical insights from your content. In the POETS model used at my prior company, FT Longitude, this is the “P”. You’re seeking to verify here that you’re providing useful and credible content for your target audience. In turn, this requires a very clear understanding of your target audience. All too often, the question of audience is dismissed as “we’re writing for the c-suite”, but it’s worth taking the time to define exactly who in the c-suite, what questions and issues they’re likely to be grappling with, and what kind of information would be most relevant and helpful to them.

2. Ensure it’s differentiated

A second consideration is about ensuring differentiation, which is one of the key pillars from the model used by Source, a consulting firm that provides an annual quality rating for thought leadership from professional services firms. In Source’s set of underlying questions, one of them asks “Is it revelatory?”. This is a difficult question to answer unless you’ve got a very clear view of your target reader and their expected degree of knowledge. Once you understand your audience, you can better ensure differentiation on the topic you’re focussing on, along with help from your in-house subject matter experts. Related to this, you need to do the due diligence in establishing what else has been published on the same topic, not least by your competitors, to ensure you’re not covering the same ground.

3. Back up your case

A concrete point to ensure is around having some kind of data to support your argument: you either have evidence to back up your case, or you don’t. This is the “E” in POETS, for Evidence. This may range from a primary survey (the go-to mechanism for most B2B thought leadership, despite the risks), through to secondary data collection, or the creation of a bespoke index of some kind, among other options.

4. Establish your expertise

A different consideration comes from a model in the world of search engine optimisation: E-E-A-T, or Experience, Expertise, Authoritativeness and Trustworthiness. And while GEO is now the new SEO, many of the same principles remain relevant for both. This model is focussed less on directly assessing the quality of the written output, but acts more as a guide about the attributes that would need to be in place in order for your content to be more likely to be picked up by Google or ChatGPT, which is also important to consider. And if you're seeking to avoid publishing slop, then these considerations will also help in your assessment. Two key takeaways: establishing your organisation’s first-hand knowledge of a particular topic (eg, personal anecdotes and practical insights from your subject matter experts); and confirming your expertise (from accurately explaining complex concepts in accessible ways, through to providing insight not available from basic sources).

5. Create a call to action

Another key element is about ensuring a clear call to action. In the POETS model, this is wrapped under the T, for Timeliness, ensuring that the insights bring urgency and topicality to your work, so audiences are compelled to act now. Source’s version considers this under “Prompting action”. Here, your approach will differ depending on your organisation’s approach to thought leadership, and where you prefer to be on the purist-to-commercial thought leadership spectrum. One of Source’s questions asks: “Does it give the reader a clear idea of how the consulting firm could help whilst avoiding being a thinly disguised sales pitch?”. This highlights one of the inherent tensions in thought leadership: a common goal for marketers is to link your brand to a certain theme so that people might consider buying related services from you, but without actually selling anything directly. But your mileage will vary: some companies choose to be more overly commercial in their messaging, while others go to great lengths to ensure that the messaging is wholly objective and non-commercial. Whatever you prefer, ideally give a clear signal to your readers too.

6. Define a style

In both the POETS model and the Source model, there is a consideration around style. Ultimately, there is going to be some subjectivity here. You might love Crash; I might hate Roma. Views will vary. The key point will be to define a distinct style for your company, and at the very least ensure that it is applied consistently. For those without such a guide, borrowing a third party one, such as The Economist’s long-running style guide, is not a bad place to start.

Model aside, it’s the process that really matters

These six points are a first step towards providing a clear and consistent way to assess the quality of your content output. Regardless of which others you prefer for your organisation, though, the most critical thing to get right is the editorial processes that are needed to underpin your content production.

These processes should span everything from how your proposed topics are conceptualised and defined for research, right through to the sub-editing, second checking, and fact checking processes that ought to accompany any serious B2B content creation. For anyone seeking to make use of AI within their content production, these processes should define where and how AI could be used–and where it shouldn’t.

Having a great quality framework by itself is nothing if you don’t have the underlying steps in place to underpin it. And no, you can’t delegate that to AI. But if you’re stuck, feel free to reach out.



Photo by Mirko Fabian on Unsplash

Share this post

Ready to lead with content that matters? Let’s make it happen.