From the event: Mindstone London May AI MeetupHow I created an AI viral scoring system for LinkedIn posts
View event

How I created an AI viral scoring system for LinkedIn posts

Introduction

So, my name's Jason Holloway. A few months ago, I set up my company for the second time round, this time focusing on AI security. It's called QL Security, and it's a sign of the times that when we do this, we have a presence on LinkedIn.

And like all good LinkedIn posters, I posted away and I found, as many who use LinkedIn find, that it's not easy to guess which post is going to go viral and which one is just going to die and die horribly. Has that happened to anybody else or is it just me?

Right. When I say LinkedIn, does Does everybody know what I mean by LinkedIn? Does anybody here not know what LinkedIn is? OK, that helps.

The problem: predicting what will work on LinkedIn

So I wanted to find out why some of my posts were working and some of them were not. And this is where AI came into it.

And I used AI not to help write the posts, but to tell me why some posts were scoring higher or getting more impressions, more followers, than others.

I wanted the AI to research the platform, to tell me what actually matters when posting on LinkedIn and what doesn't.

I wanted it to combine as many different research bits of data it could find and then tell me what it identified as being the key, I guess, categories of measurements that would determine what works and what doesn't.

And what it found was actually quite interesting. It wasn't what I was expecting at all.

What the AI research revealed about reach and engagement

I thought that all the posts that you see, like here, subscribe, I know that's YouTube, like below, et cetera, would actually help. turns out they don't.

Signals that help (and ones that don’t)

Saves, on the other hand, do. I didn't even know that LinkedIn allowed you to save a post. That was something I learnt.

And it used to be said that it was what I guess people reacted to in the first 60 or 90 minutes that drove the number of impressions. But unfortunately, that's no longer the case.

It's the first half an hour, perhaps even less. The amount of time that somebody actually spends watching your content really makes a big difference, the dwell time, as it's called.

Dwell time, saves, and the hidden cost of external links

And curiously, every time that you include an external link within your post, that actually has a negative effect on its influence and its reach. That I did not expect.

And I guess I knew about daily posting, thing but I didn't really know what the right number was. So I thought this was quite intriguing and so it started to create a nice summary for me of what actually mattered and what didn't matter

and some of the numbers that came up were staggeringly impressive in understanding why sometimes these posts go viral.

Comments, credibility, and conversation dynamics

What really matters is, for example, who responds to your post. If it's an expert in that field, it augments the reach.

If you can get multiple people commenting on your post, not just you and somebody else, it really does amplify the reach tremendously. tremendously.

So these little tips and tricks can actually be useful when you're writing a LinkedIn post if you can create a conversation amongst people.

How AI-generated content gets penalized

But what was also very staggering to find out is that the AI -generated posts get downvoted. They get squashed.

You could argue whether that's effective or not. My LinkedIn post seems to have a lot of AI generated content but perhaps that's another discussion.

What was also interesting though is that personal counts a lot more than company posts.

Turning insights into a LinkedIn post scoring framework

So I wanted to see what we could do with this information and so I actually asked Claude, that was the agent of choice that day, to actually build a scoring framework to be able to rate a post draft across all of these different areas and spit out a value from 0 to 100 as to how likely that would be to go viral and also to identify any weaknesses within it in the process so

How the scoring workflow works in practice

it did and I've been using it over these past few months to find out and it's a very straightforward process.

I write something, I effectively tell Claude to score it for me, it gives me a score. It also categorises the different elements and scores them individually and gives me a breakdown as to what worked and what didn't. And it even sometimes suggests different ways in which I could fix it, which is jolly nice of it.

And of course, if you can measure it before you publish it, then you can tweak it and tweak it until it becomes more, shall we say, publishable.

I tend to go for 70 or more as being my personal threshold. If it's above that, I can post it. If it's below that, that usually means I need to work on it a little bit.

When the platform changes: adapting to LinkedIn’s “360 Brew”

So this worked really well for the first month or two. And then, And of course, LinkedIn changed the game. They moved the goalposts. They came up with 360 Brew, which is the new algorithm.

And if you've been following the news on this, it's got over 150 different billion parameters involved in their AI model that actually replaces everything that used to rank the LinkedIn post. Which sounds great, except that it has altered how it scores.

stores and so if it spots AI generated content as I said it actually downgrades it even more than before reduces the reach but it also reduces the engagement which is a pain because obviously lots of people out there have been asking chat GPT and Claude and so forth to write their posts for them and can't understand why it's not working as well as they would like and that's because it actually understands the content.

If, for example, you're an expert on cars and you start talking about motorbikes or sailing, it's going to actually downgrade all of your posts because you're no longer a subject matter expert in cars, which you originally claimed to be. So this becomes really hard to understand.

Predictable templates and why they now hurt performance

The classic formula that AR uses, and we've all seen this, it's the hook, it's a list of bullets usually with emojis tied to it and then that question that rhetorical question almost at the end yeah we recognize it yeah it's

predictable and it's penalized so whatever you do try to avoid doing that it gets classified as likely AI as over half of the content currently is and it penalizes your reach so in order to avoid that I think it's also worth

Lexical and rhetorical “AI tells” to watch for

worth pointing out that there's some other AI tells. It's not just the Oxford comma and the use of the double M dash and similar.

These are some rhetorical figures that you often see in AI -generated copy. And if you look carefully at the LinkedIn posts, you will spot these.

It's the withheld opening, the game has changed, the short, punchy, but hollow content that you get there. There are two options, one obvious winner, and so on. I won't read them all, but just to point out, there are over 10 of those.

So, of course, I told Claude to go away and do a little bit more research and find out about 360 Brew and how to score the new system. So, he introduced the lexical tells to identify content that even I can write that sometimes sounds like AI, which is a little bit bizarre.

arms, spending too much time with Claude, obviously. But it also identifies those rhetorical patterns and will make them pretty obvious to me and also penalise the score on the back of that.

So how does it work? And does it work? Well, actually, yes.

Does it actually work? Iterations and outcomes

I've gone through three iterations now of this. We're actually on 3 .1. But nonetheless, it's shown up in

and a little bit of impressive content growth. I wasn't posting that regularly, and I wasn't very good at the posts I used to do.

And then when we launched the company, I started posting it. And that was the first version. And then, obviously, I updated it, and that's the second version. And now we're on the third version and beyond.

So it has worked for me. Now, you expect me to say, this is brilliant, this works.

Now, it doesn't sound like a lot, 53 ,000 impressions, but you've got to remember I'm only talking about AI security which is a marketplace that is minuscule at this moment in time and it's not the most exciting subject

so that's why I'm not talking about it tonight. But if you want to talk about AI security do come and see me afterwards. I'd love to have a chat around that.

But of course this is just me and I made the tool with Claude. Actually Claude made the tool, I'll take his bragging rights. so it's understandable that you would treat anything I say with a pinch of salt and rightly so

Independent validation and audience matching

but a contact I made the other day I was talking through this system with was interested in it and went off and actually tested it for himself and this is what he found he posted it he got more impressions than ever before certainly more reactions than ever before more comments than before and more profile views than ever before just on the back of one post which i take as a good

indicator that it worked but one of the things he said was actually quite interesting what i've actually built into the system is a way that actually finds out about you and what you are promoting and who you work for and what they're promoting and actually tries to understand the demographic intent behind your posts and then you actually see that reflected in

the scoring and this is what this gentleman found reflected in his post audience almost a one -to -one match with this target audience so he was pleased so who feels brave who would like to put this to the test are you sure because

Live walkthrough: scoring a real post

because I'm going to be absolutely honest about this. I'll ask you to type in your name in my LinkedIn so we can find you, and we'll take one of your recent posts, and then we put it through Claude.

Claude, I'm afraid, is a very nasty taskmaster and often rips these apart. Are you happy for that? You're still happy?

Okay, well, I've got two volunteers. I've got three volunteers now. Four, five, gosh, loads.

I think you, sir, were the first, So the honour goes to you. So would you mind just coming up and helping me with this, please? Please, congratulations.

Nice to meet you. Your name, sir? Raj. Raj.

Thank you very much. So if you step around here, let's make this bigger so you can all see this. Raj, if you wouldn't mind, go ahead and type in your name. Yeah, go on.

You can select yourself. And find a post that you've done recently. If you scroll down, usually, a little bit. Yeah, there you go.

Any of those that you want to try? Okay. Gosh, that's a fair bit there. So I'm not going to read it all out loud,

otherwise we'll be here for a while. But if I may, I'll just copy that. and I think it's only fair to say a big thanks to Raj for giving us this code.

Well done, Raj. I want him to be a long way away when I do this. So I'm going to paste this in, and as you can see, I'm not doing anything special.

I'm just saying a new friend of mine. I've never met you before, Raj, have I? I'm useless with names and faces. Apologies if I have.

have, but he's willing to give this a go. He's a friend of mine, so let's rate it. Now,

this will take a couple of seconds, and the reason it takes a couple of seconds is the skill that I'm using is actually reused across the team, so it's actually going to pull up information. Excellent.

So it's going to get it from Notion, which is our centralised knowledge -based system and off it goes and it's starting to score it right so category six is the applicability to the target audience it doesn't know that so it's going to score it without that information

Interpreting the score and improving the draft

and here we go can you all see that at the back shall I make it a little bit bigger would that help okay so congratulations 74 out of 100 well done so green ready to publish so it broke it down into the quality of the hook the content structure engagement triggers content value

technical optimization audience calibration which we just talked about and no ai tells in in the rhetorical sense. So obviously, it's given you some feedback

on the strengths of it. It's then suggested some improvements. And the biggest single lift it reckons is in the hook fold.

Basically, it's the title emoji line and the first real sentence are competing a little bit. It's given you some ideas on how to optimize it. it.

It's found a couple of minor AI tells. I don't know if you use AI for that one, Raj. Do you want to admit to that or not? Okay.

Fair enough. So you used it to embellish it a little bit. That's fine.

I completely get that. But you got a fantastic score. 74. It's

around. And so that's how I use it. And I go through multiple different iterations until I get to the one that I'm happy with.

And then I post it. Ah, right. So very good question.

So is it, you know, limited to myself? Is it something that's built into Claude, etc? Let me come back

to that, if I may, in a moment, as I think about to answer that question. So for me, though, the

The broader lesson: using AI to research and refine judgment

the real lesson here, yes, the real lesson here is I didn't use AI to write LinkedIn posts. I used AI to research the problem that I came up with. I used it really to plan, to think about it and

to help me think about it as an analysis really of the problem and how to improve my judgment as to what would work and what wouldn't. It changed how I thought about not only the problem, but also about the writing and about how I use AI.

Sharing the tool and how others can try it

And from that, I created this tool, which I've made public. If you want to have a go with this, please just go to this GitHub repository. It's got all of the research.

That is why, I guess, it scores it the way it does. It's also got the skill or the information that you need to download to make it your own in Claude, ChatGPT, or whichever AI system you prefer to use. And then it will help you.

The first time that you run it, it should ask you to create a strategy profile. But if you don't have one, it will need one. So just tell it, I don't have one. Help me create one. one, and it will create one that it will remember and then personalise it for you and your posts.

And if you find that useful, please leave a star on the fork or whatever in GitHub if you want to fork it, by all means do so, but I would love to know how you get on with that, and if you have any feedback, please let me know.

Conclusion

Thank you.

Finished reading?