What is Index Bloat? — Whiteboard Friday

So in both of these cases, you can generate this very large number of URLs that are getting basically no traffic. So what might we actually do about this or decide whether we want to do something about this? 

1. Identify URLs with almost no traffic

So the first step I would take is identify URLs which have near as dammit no traffic. And a rule of thumb I’ve often used in the past is are they getting on average less than one click a month or something like this? You can draw a very low bar. And on sites that are affected by this in a big way, you’re still going to pick up a lot of pages that are getting literally zero traffic in most cases. 

Remember, by the way, if you’re looking at this from an organic lens, do check other channels as well. You don’t want to accidentally remove something that it actually isn’t really important to the social or email team or something like this. 

2. Improve any pages that are opportunities

Next up, improve any that are opportunities. So that’s kind of a big catchall statement. But if some of these pages, that you identify, perhaps you used to get a lot of traffic but have become out of date or something like that, or you think they do actually have quality content on them, maybe there’s a technical SEO issue that’s holding them back, find any that are actually worth doing something with. Perhaps they might have a lot of links, for example. You don’t want to just blanket wipe out this sort of latent value that you have. Do something with it if you can. 

3. Consolidate or cull pages you’re not able to improve

And then what’s left, you’ve got this big bunch of pages that get zero traffic that you don’t think are any kind of opportunity. So there’s a few different ways you can go, and you’re probably going to want to go and mix. 

So anywhere you have either existing or potential pages that match the intent or are very similar, you’re basically doing the same thing. So for example, if you’ve got one of these very specific product pages, but you’ve got a category page that’s about basically the same thing, and the product is no longer in stock, then you could consider a canonical or a 301. Obviously, a canonical if you still want that URL to be accessible, or a 301 if actually the page is totally redundant and you don’t need anyone to see that anymore. 

Again, this is if the intent and purpose and content of the page is going to be very similar. You could even if you think some of the content is worth consolidating, you could have that one page that you’re consolidating to have the best of the content from all of the sort of component pages. And you could choose this to be a new or existing URL. You don’t have to already have a good page. You could choose to make a new page that is really going to do well for this topic, rather than having all of these old pages, none of which were particularly worthwhile. 

For anything where you really just don’t serve this intent, it’s redundant, it never had any value anyway, you can just 404 or noindex. Again, 404 if you don’t need it to be accessible anymore. Noindex if you do, for example, it’s used by another channel or something like this. This is quite an extreme step. I would try to avoid this if you can. Google isn’t necessarily going to pass the full equity through a redirect or a canonical if the pages aren’t a good match, but with a 404, they’re definitely not. And with a noindex, eventually they’re not as well. Google eventually stops crawling noindexes. So yeah, this is something you want to avoid. But realistically, there probably will be some pages that fall into this bucket. 

So yeah, this is the sort of process I’ve followed myself in the past. It’s something I’ve seen good results with. I’ve seen a lot of other SEOs speaking about this, especially in the wake of the helpful content update and in the past around Panda, which I think probably worked quite similarly. 

So, yeah, let me know how you get on. And thank you very much.