The theory behind search engine ranking and search engine optimization (SEO) is pretty simple. Search engines like Google should rank the best and most relevant content that would serve the users best right at the top of the page 1 in their search results pages (SERPs) for each individual search query.

That would help give the search engine user the information they’re looking for, which is after all the job of a search engine. But does this actually work in reality? Does Google always rank the “best” content right at the top of their results?

It’s been an increasing complaint from more and more people in recent years that it’s getting harder and harder to find the information they’re looking for on Google. The quality of the search results seems to be getting worse and worse, with poor quality, non relevant search results appearing at the top. Users often find themselves having to trawl through a lot of results on the SERP before they find what they’re looking for (or perhaps not find it at all).

This doesn’t seem to square with what a search engine’s job is meant to be – give the people the information they searched for. And billions of people still use Google, so what’s going on? Does the “best” content always rank at the top of Google? Does the highest quality content always appear at the top of the Google SERPs for each query?

Google sometimes ranks the best content for individual search queries, but not always. They have made multiple tweaks to their algorithm in recent years which also incorporated other site-wide factors into individual ranking. This means that an individual piece of content from a website may not rank highly even if it is the “best” article on that topic, because of other factors on the site that pull down overall rankings even for high quality content.

In other words, no, the best content does not always rank at the top of Google. This has always been the case to some extent because of competition and site authority factors, but has grown even more common in recent years with various tweaks Google has made to the way it ranks sites.

Let’s dive into this issue in more detail.

How Content Ranking Should Work (Theory)

The basic theory of search engines like Google is that they will rank the articles that give the “best”  and most standout answer to any search query. This includes answering the question in a brief, summary way early on, and then going into more detail if needed about the nuance of the issues, including caveats, exceptions, different factors to weigh up, etc.

Under this theory, the content which ranks at the top of the search results for any given query should:

• Be high quality, useful, informative, well written content
• Give a precise, thorough and helpful answer to the question.
• Demonstrate expertise and trustworthiness on the topic.
• Compare and contrast different points of view when necessary
• Add original value and information beyond what’s contained in other search results
• Answer relevant obvious follow up questions the reader may have.
• Where relevant, help readers to make a buying decision by weighing pros/cons of products/services in a helpful way.
• Leave the reader satisfied in the sense that they got all the information they came to the article for (and preferably learned something new as well) and don’t need to go to any other articles to get any more information. That one article answered everything they wanted answered.

In other words, the “best” answer to a query “wins” that SERP by getting the top result. And as long as the content remains up to date and relevant, it should stay there as well.

However, one huge mitigating factor in this process that has been there for years is the authority/size of the website publishing the content, together with the competitiveness of the keyword. A brand new site publishing an article on a really competitive keyword with loads of websites already ranking for that term will never rank on the first page, even if it’s the “best” piece of content online for that query. Sites have always had to build up authority before they can rank for bigger, higher volume keywords, and in some niches, it’s very difficult or impossible without spending a lot on SEO.

But this has always been the case and taking out this factor of authority and competition, any website should be able to rank top for lower competition keywords (or at least a keyword size that matches the site’s size/authority) if they write the best piece of content for that topic.

In an ideal world, this is how search engines like Google should work, and indeed used to work until a few years ago. In fact, SOME Google searches still do work like this – you click on one of the top results, get a detailed, helpful, high quality piece of content that answers your question, and you leave satisfied and knowing everything you needed to know.

But a lot of Google users complain that this is NOT the experience they have now using Google. They say they can’t find what they’re looking for, the search results don’t give them the answer they want. That the SERPs are often full of non relevant, non useful information that doesn’t really answer the query they typed in.

Why is this? What’s going on? The main answers to this lie I believe in recent changes Google has made to their core algorithms, plus also some short-termist, spammy behavior from some website owners, that reduces the quality of search results and means the “best” content doesn’t always rank high on Google.

The Effect Of Recent Core Algorithm Updates

One major factor that’s jumbled up search engine results from what they might have been a few years ago is the core updates Google has brought in, which appear to have concepts in them that can and do conflict with ranking the most relevant and “on point” article for each given query.

Let’s cover a few of these factors.

Factor #1 – Topical Authority – Recent Google updates, especially in 2022, have placed more of an emphasis on websites having topical authority on the subjects they write about. In other words, having clusters or “silos” of content tightly grouped around one particular topic or sub-topic within a niche. Individual pieces of content, even if brilliant, in depth and realistically the “best” piece of content online for that topic/query, may not rank well now if they aren’t supported by a clustering of articles on the same general sub-topic (topical cluster or silo).

Factor #2 – Useful/Spam Content Updates – Similarly, if a website has a large chunk of articles that Google deems “unhelpful” or “spam” by their 2022/2023 updates and guidelines, they sometimes penalize ALL the content on that website (even the good content) if there’s what they deem as “too much” of this unhelpful content. In other words, even excellent, high quality, useful individual pieces of content that might rank #1 in terms of pure relevance and helpfulness, might get “de-ranked” or pulled down because of other content on the site.

Factor #3 – EEAT Considerations – Google has also placed an increasing emphasis on EEAT (Expertise, Experience, Authoritativeness, Trustworthiness) in recent years, especially for topics it considers YMYL (Your Money, Your Life – really mission critical, important topics it doesn’t want to get “wrong” in terms of the results it gives). This means again that an individual piece of content that might be brilliantly written and in reality give a good and “correct” answer, won’t rank in certain fields (eg. medical, finance etc), because the author/website does not display enough EEAT/credibility signals for Google to “trust” the content. Instead, they might rank inferior, less helpful content from sites that at least display more EEAT signal they’re looking for.

These policies are primarily discussed among bloggers themselves on how to improve their content and avoid being hit by algo updates. But if you think about it, the end users of search engines also get affected by these changes. Let’s cover why below.

How Does This Impact Search Engine Rankings?

Ever been searching for something online on Google in recent years and grown more and more frustrated, thinking something like “just give me the answer, will you!”. Well you’re not alone, and hopefully now you have some more explanations as to why.

Let’s draw out some of the things we stated above about Google updates, and take them on a few steps to see how they can affect the search results.

1. A blogger produces an article that is by all reasonable criteria, the “best” article on a given topic/query. It’s the most relevant, detailed, helpful and thorough article on that topic on the entire internet.
2. However, because of a lack of topical authority, or other “signals” or factors on their website, their content gets “de-ranked” or pulled down in the rankings. In some cases, not just the content Google deems “unhelpful”, but ALL content, even the high quality articles.
3. As a one off case, might seem insignificant.  But multiply this across all the hundreds of thousands or millions of websites similarly affected by recent Google algorithm updates, the quality of the search results must necessarily worsen, with the “best” results in terms of pure relevance not showing up near the top of the SERPs because of other site-wide factors on many websites that Google has decided they don’t like with recent updates.
4. Bottom line – with perhaps not all, but a large number of Google search results pages, it’s no longer about what’s the “best” individual page. Other factors are weighing in and influencing/tweaking the search results, pushing certain content further down than it should be in terms of the pure quality/relevance of that one page.
5. End results – bloggers/site owners suffer, but so do Google users – they get less optimal, less relevant search results than they would have done several years ago, because Google is focusing more and more on site-wide minutiae like “topical authority” and “helpful content”, and not just ranking each individual page on it’s own merits.

Other Factors That Impact The Quality Of Google Search Results

In fairness, it’s not all Google’s fault that the SERPs seem to be getting worse in a lot of cases. The world of internet marketing in general is full of shady, scammy types trying to make a “Quick buck”, and part of this involves trying to “trick” search engines into ranking poor quality content at the top of results pages.

Here’s some ways they do this:

1. Paid backlinks – There’s a whole industry around paying people to send backlinks to sites, which acts as a kind of authority signal and can help boost rankings. Essentially, paying to try and manipulate Google into thinking your site is “authoritative” by buying a load of backlinks from bigger sites to yours. Can push content up in the rankings higher than rankings, since a backlink can act as a kind of “upvote” or “thumbs up” for that piece of content
2. AI content – Some internet marketers are using AI tools to churn out masses of content on clusters of keywords to rank quickly on search engines. This can cause the SERPs to fill up with articles that are targeted at the keyword/query in question and have all the right words in them, but aren’t really high quality content and have just been churned out by an AI with no experience or real credibility on the topic. Users find themselves pivoting to forums like Reddit and Quora, where they at least know they’re interacting with a real person.

In both these cases, any short term successes are usually short lived and sooner or later the Google algorithm detects what’s going on and these sites “tank” in the rankings. But there’s always the next AI site to come along, plus this is happening so much now that it’s definitely affected the quality of SERPs, with poor quality AI or artificially backlink-boosted content appearing way higher up the search results than it deserves to, as in reality it’s not great content.

Will You Always See The “Best” Content Ranking High On Google?

To summarize this, if people are feeling an increasing sense of frustration with Google search, this is in a sense justified, because the recent changes they’ve made to their algorithm that specifically and explicitly states that search engine ranking is no longer necessarily purely about the quality/helpfulness/relevance of each individual piece of content anymore in all cases.

Sometimes it might be, and you might get the “best” piece of content as you might have 4-5 years ago. But a lot of the time, you won’t, as other factors are jumbling up the SERPs and causing less useful content to rank.

There’s a whole bunch of other on-site and off-site factors that go into ranking content now, that means that the “best” content for an individual query might no longer get the top ranking, or even get any space on page 1 of the SERPs.

“(A blogger) could write the best individual piece of content on the entire internet covering a specific search query, and never land anywhere near page 1 (of Google, if they ignore topical authority).

Ricky Kesler, Income School – see here

This statement is very true and also backed by my own experience.  There are now a lot more “hoops” that not just an individual piece of content, but the entire site it belongs to, has to jump through in order to guarantee that it will rank at the top of SERPs, even if it is by reasonable criteria, the “best” piece of content online for that topic.

Let’s summarize these criteria that need to be met:

1. Can the piece of content be judged to be “very high quality” or the “best” piece of content for a particular query?
2. If yes, does the site it was published on have enough basic authority, plus topical authority in Google’s eyes, to warrant ranking for that search query given it’s competitiveness?
3. If yes, and if the search query is in a more sensitive “YMYL” field such as medical/finance/law, does the site display enough “EEAT” signals (credibility and trust) to be considered trustworthy in Google’s eyes?
4. If yes, is the website relatively or totally free from what Google considers to be “spam” or “unhelpful” content that might pull down other content on the site?
5. If yes, then the content can rank at the top of the SERPs.

If an individual piece of content ticks ALL of these boxes, then yes, it can and should still rank at the top of the search results if it’s the “best” piece of content for that query. However, if the answer to ANY of these questions #2-4 is no, then it will probably not rank anywhere near the top of the search results, even if it could reasonably be described as the “best” piece of content for that topic in terms of quality, depth, helpfulness etc.

That’s a LOT more boxes that need to be ticked versus 5 or 6 years ago, which is part of the reason why you see a lot of mixing up the search results. Google no longer just ranks each individual piece of content on it’s own merits. There’s a lot more filters that content goes through now before it’s final ranking position is determined.

But what do you think of this change in the philosophy of how Google ranks content?

I thought the core job of a search engine was to provide users with the information they’re looking for, by ranking the most relevant, detailed and helpful piece of content for each individual search query.

 Do these algorithm changes to incorporate side-wide factors into ranking move Google closer to, or further away from, this core objective?

My opinion on this is clear, but I want to know what you think. Put your thoughts in the comments – are you finding the Google SERPs more or less helpful year on year? Are you finding the “best” content in the top few positions? Or more and more unhelpful content that does not belong there, and you’re finding better content further down on page 1/page 2 that should have been at the top? Should Google and other search engines go back to what they mostly did originally – rank each individual page on it’s own merits and by how much it helps the readers. Let us know your thoughts!