Eli Schwartz

SEO – Page 6 – 📌Eli Schwartz


Home / SEO

Crawl Budget – It’s really a simple SEO concept

The phrase “crawl budget” is an SEO term that is frequently included in discussions about technical SEO, but it is typically used incorrectly. Most of the time when people refer to crawl budget they are considering it a technical SEO enhancement to improve the way Google understands a website. In fact, it is far simpler than that it is simply a budget.

The best way of understanding various aspects of Google’s algorithms is to view them from a financial standpoint. Crawling and indexing the web is a very expensive proposition and Google was able to beat out every search engine to dominance because they figured out how to do that before the money ran out.

While it would be ideal for Google’s crawlers to simply gobble up the entire web in one fell swoop that would be technically impossible. Crawlers need to literally crawl through the web discovering link after link and then as they land on a page they need to build a copy of the page into the database.

It’s about the Benjamins!

In the early days of search, while Google was still living on venture capital money, the engineers needed to come up with a way to efficiently crawl the web without going broke in the process. That way was to decide how much “budget” each site was allocated based upon its importance to Google and the web as a whole. That is crawl budget.

If a site is very important to the ecosystem, Wikipedia for example, Google would have wanted to allocate a lot of their hypothetical dollars to crawling as much of the site as they could. Alternatively, a brand new website with no authority on the web would be allocated a significantly smaller amount of budget.

New websites

This all makes logical sense. Taking this logic one step further, if the brand new website would have thousands of pages but only a few of them were valuable, it would have been very likely that their budget would have been eaten up by the crawler ingesting the lower quality pages without ever seeing the good ones.

The best approach for a website in this position is to simply declare – via no indexes or canonical tags – which are the lower quality pages and then the crawlers could just skip them.

A happy example

To illustrate this with an example, think about a website like a Happy Meal with a toy inside. You have a certain amount of daily budget to buy Happy Meals, but you only need the unique toys to complete a series. The only way you could find out whether the toy is unique is by buying the meal and opening the box. So, every time a Happy Meal is bought and the duplicate toy shows up, that days budget is wasted – unless you were very hungry. The most efficient way to do the toy collecting is simply to show the name of the toy on the outside of the box and then you would choose only that box.

Continuing this Happy Meal to website analogy, those no index directives and canonical tags are the best way of informing a search engine to ignore a particular box.  The crawler then has more awareness on how to most efficiently spend their limited budget.

Crawl budget summary

This idea of crawl budget applies to every website on the web regardless of authority, its just that more authoritative websites have more budget to be expended by the crawler. As websites gain authority, likely via links or other user engagement signals their budget will expand but without that there is no other way to get more budget.

Google refers to this as “crawl demand” and while they don’t specifically mention authority in their blog post on crawl budget, they sort of beat around it by calling it “popularity.”

Even if the crawl rate limit isn’t reached, if there’s no demand from indexing, there will be low activity from Googlebot. The two factors that play a significant role in determining crawl demand are:

  • Popularity: URLs that are more popular on the Internet tend to be crawled more often to keep them fresher in our index.
  • Staleness: our systems attempt to prevent URLs from becoming stale in the index.

This idea of budget was a key component of Google’s crawling algorithm and it is still exists today although the budget is vastly expanded. Google now has lots more money and resources to crawl the web, but the web is also bigger and more complicated.

Crawl budget today

One other change is that budget was likely initially calculated in small amounts of kilobytes which equated to a number of pages, that budget can be eaten faster if a site has dynamic scripts that are more expensive for the crawler to run.


Annual and quarterly planning for SEO

In addition to shorter days and colder nights (at least in the Northern Hemisphere of the world), September brings a special gift to the office which everyone claims to welcome – but in truth they despise: annual planning. Many large companies will have some sort of quarterly planning process where in the last couple weeks of a quarter teams will have to detail how they have progressed on the current quarter’s goals and what they hope to accomplish in the coming quarter.

Annual planning

While this process isn’t necessarily the most fun, it can be straightforward as it has the benefit of making predictions about that things that are already in flight. This can’t be said for an annual planning process which is typically conducted in the closing quarter of a year and requires a team to pick a goal for the coming year and negotiate for the resources to achieve said goal.

Winning at annual planning

Team that “win” the planning process  in the sense that their goals are accepted and their resource requests are met can in some ways rest for the next 12 months until this process begins again. In most organizations they will not be necessarily held to their goals because of course a lot can change over a year. Their real win is that they were given the resources (defined as budget and/or headcount) to meet their goal(s), and if they did not meet that goal for whatever external resources, they were still able to deploy their resources to make some other flashy business impact.

This business impact – even if it was not the one stated in the annual planning process – positions them in a very good light in the following year to again state lofty goals and request resources to help them get there.

Losing at annual planning

On the flipside of the teams that win the planning process are the teams that “lose.” These are teams whose goals are not adopted as goals worthy of being resourced and their requests will be turned down in favor of other teams. They will still have a goal for the coming year, but it will likely be somewhat watered down due to the lack of resources. This team could find themselves generally deprioritized and locked out of new headcount and budget for the entire next year. When it comes time to develop goals and make a request for resources in the following year, they start off with a significantly weaker hand as they don’t have the benefit of making a huge impact in the prior year.

Looking at planning in this light, the stakes could not be higher. Teams must win at the planning process and successfully pitch executives to get behind their goals or risk becoming a non-essential line item until something fundamentally changes in the business. The only exception to this favored/non-favored child reality is if there is a high-level executive backing a team or if there is a shift in power within a company like a re-org, fundraising event or new product need.

Due to the nature of where SEO teams sit within a large organization, they immediately begin this planning process behind the curve. In most companies, SEO either sits within marketing or product and this placement has a big effect on how they do in a planning process.

SEO in marketing

If an SEO team is a part of the marketing team, they likely report up to a marketing leader of some sort. This leader will be juggling budget and headcount requests from teams that can make far stronger 1+2 = 3 pitches. For example, a paid marketing team can show very clear math of how additional paid budget will impact acquisition, retention or awareness. They can also show how added headcount will improve the efficiency of their spend thereby adding more value to the organization.

The same argument might work for a content team that could show how output (the metric that content teams are measured by) will improve by a factor of how much more is spent on producing content.

Contrasted with the SEO team that has fuzzy math for how SEO works and even fuzzier math for what more spend might get, a pitch for resources is a losing proposition on a marketing team. This is why most SEO teams even in companies with large other marketing-subteams will still only consist of 1 or 2 people.

SEO in product

When an SEO team sits on a product team, the uphill climb to win resources is slightly easier but not by much. Rather than a pitch for just budget and headcount, the resource ask will also likely include engineering time. In a company that does not prioritize SEO, the engineering time request might get shunted aside in favor of building new products or improving existing products.

Logically, this makes a lot of sense as what product leader would rather have a roadmap full of improvements to existing products rather than one with all sorts of exciting new builds.

How to win resources

To win at the annual planning game, the SEO team is going to have to morph into the kind of team everyone loves to fund and support. This will apply regardless of where the SEO team sits in the org.

First and foremost, the whole idea of using wishy washy data to forecast SEO impact must be completely discarded. There is data within the company and Google Search Console which very clearly tells the story of how valuable SEO traffic is for top line acquisition. There may even be data on how it impacts the bottom line too. Get that data and use that as the guiding light within an annual goal.

Is the plan to increase SEO traffic by 10%? That’s not a good goal! Back into what a 10% organic traffic increase might mean to top line revenue and use that number as the goal. An increase of 10% of overall web revenue from organic sources sounds a whole hell of a lot more sexy than an unclear 10% increase of traffic.

Second, many other teams will pitch ideas that they don’t really know how they will implement but want resources to try. They certainly aren’t pitching the process on how they may or may not get there, rather they are saying they are going to build X and X requires engineers.  The SEO team should do the exact same thing.

Instead of asking for engineers to update a whole bunch of SEO requirements, instead ask for engineers (or content or money) to build X for SEO.

Third, and this is specific by company, there is a process for how every other product and marketing team pitches for resources. Make sure that the SEO pitch looks exactly the same and there is data to show even greater impact than other efforts. Keep the SEO jargon out of the pitch and use the same language that everyone else uses. The last thing an SEO team wants is to have their pitch stand out just because it was different.

Annual planning from a leadership standpoint

The prior advice mostly applies to SEO practitioners pitching for resources or those pitching SEO asks to the C-suite, but it can just as well be modified for leaders being asked for the resources. Know that SEO is incredibly important, and if the pitch for resources doesn’t have the clear 1+2 = 3 approach then push it back to the team for a revision, don’t just reject it out of hand.


Annual planning is a process no one really enjoys other than the people who run the process for everyone else. It is a necessary evil and it isn’t going to go away if it is ignored.  If an SEO team does not put their best foot forward they could risk losing an entire year with no impact efforts.  


PSA: Using rankings to track SEO success is dumb

It is amazing to me that in 2019 anyone would use still use rankings as a success metric for an SEO campaign. Rankings are a vanity metric and do not directly or even directly contribute to the success of a business.

Using a rankings report to as the only way to measure SEO progress is as asinine as using a paid marketing budget total as a metric of success. All a big budget shows is that someone can spend money, it says nothing about whether it is profitable. Rankings just exhibit the ability to be ON Google and not whether anyone even clicks or buys.

Rankings in the past

When I first started my career in search engine optimization the critical metric of success was ranking in search results, and most importantly it was how many number one positions one occupied. It did not matter whether those were useful positions or even if anyone clicked, just having them was a bragging right. As an added bonus, Google wasn’t the only search engine that anyone cared about, so having a top result on MSN.com or Yahoo also generated some SEO applause.

Keywords to rank in a top position were chosen using keyword research tools with preference given to exact match words that had high average monthly search volume with no heed paid to intent. Those keywords were then plugged in on-page meta, spun into content at a high keyword density and most significantly used in anchor text for deliberately built external links.

To keep track of these positions, SEO’s had to use a slew of tool’s whose primary function was to scrape search engines on a weekly, daily or even hourly basis for the latest rankings. Executives asked to see these reports and hence having a huge list of prominent top positions was a key component of an SEO role. This whole process worked and then it didn’t because things changed.

It wasn’t just one change, EVERYTHING changed.

For starters, Google became the dominant search engine if not the only search engine anyone cares about. Google earned this role by rapidly improving the search user’s experience which was a direct result of rooting out the kinds of practices that made pages rank undeservingly.

The list of changes on the search side over the last ten years are endless, but some of the key highlights are:

  • Panda – This Google update from 2010 deprioritizes thin content that is targeted at just generating search clicks. It used to be a separate algorithm that ran alongside the core algorithm, but now it just a part of the ranking algorithm.
  • Penguin –  This algorithm update from 2012 negates the impact of illegitimate links and penalizes sites that engage in these practices intentionally.
  • Local –  A national/global result is irrelevant if Google determines that there’s a local intent behind a query.
  • Google suggest – Google’s query suggestions continually to become richer as they use real time data and trends to direct people into a search funnel.
  • Intent matching – Results for synonyms, misspells, and pluralization are nearly identical provided that the intent behind the phrasing is the same. Choosing word order and pluralization is not a necessary SEO task as they will usually be similar if the intent is the same.
  • Entity matching – Google can parse the entity a user is seeking and will show relevant results. Additionally, the amount of entities that can be highlighted in search continues to grow.
  • Mobile – The rise of mobile search changes everything with regards to how people search with touch-only keyboards or voice search. Users with Android phones can already conduct searches with their cameras through Google Lens but expect the way that searches will be conducted to continue to grow as device technology improves.
  • Artificial Intelligence – Google is not just reliant on words that are in the query or even on the page. They can parse meaning and intent without any apparent match between content and query.
  • Artificial Intelligence and deep learning – Whether Google uses engagement metrics in rankings is up for debate, Google certainly has a significant amount of machine learning to know how content should perform in search results without even needing it to gather real engagement metrics.
  • Ranking signals – in years prior it was thought that content + metadata + links were the key components to ranking. They are still important, but Google claims to use hundreds of others so even with the most optimized content and links, its impossible to force a result.
  • Zero click results – To minimize on information arbitrage results and to provide an even better user experience Google will put answers directly into search results. This negates the value of a top ranking result, as many users will choose not to click any results at all.
  • Size of the web – for anything that is of high value there are now hundreds to thousands of sites chasing the same traffic. Generating search traffic requires being far more creative than just picking keywords off a list from a keyword research tool.

With all of this in mind, manipulating a particular ranking can be virtually impossible and even if one were successful all that effort might be for naught.

SEO success metrics

The primary success metric for SEO is and should always have been the same for every marketing channel: the amount of revenue, leads, visitors etc that the business needs to be successful. If every other marketing channel is contributing to the bottom line or at least the top line, and organic traffic is not, there is an issue that needs to be addressed. Patting oneself on the back for great rankings in this situation will be little solace if the business goes under for a lack of cash.

Some businesses, especially those with long sales pipelines,  may have attribution challenges in tracking channel performance back to organic traffic as typically search traffic will be mostly top of funnel. In this case, the fallback should be clicks from search engines, but effort should still be made to determine that the clicks are of value.

Even if it’s impossible to determine the business outcomes, the business should still be looking at the engagement rate – bounce rate, pages per visit, and time on site – from this source of traffic. If the engagement rates are too low to ever lead to a conversion event, there is an issue and the rankings leading to the clicks are of no value.  


Rankings alone as a KPI for SEO is vanity metric and it should never be used in budgeting, financial modeling or any other important business conversation. SEO should be judged in the same vein as every other marketing channel and if it can’t be then appropriate proxies that correlate to business KPI’s should be used. It is 2019, and Google has robotaxis on the road, we should stop pretending that they are the same search engine of 2009.


Big company vs small company SEO – enterprise SEO at scale

Search engine optimization is a straightforward process of aiding search engines in best understanding the content of a website so it can garner the maximum visibility in search results. Like anything in technology, there is a set of best practices which need to be implemented correctly to help the machines that power the crawling, ranking and discovery algorithms. In a small organization or on a simple website these best practices are easy to follow and implement. The person implementing the best practices might even have access to the content management system to do it themselves.

As an organization grows or a site becomes more complex, following one set of guidelines means there will be tradeoffs in another part of the company or on the website. This is what is typically termed enterprise SEO, but I prefer the more descriptive: SEO at Scale.

The rules and guidelines are the same for every site regardless of whether it has ten or a million pages. Every site, regardless of size, and in-fact every page needs to have optimized meta data, great content, and good backlinks that vouch for the site. A site like WashingtonPost.com doesn’t get a pass for thin content just because it has a great brand nor does Amazon.com get away with weak meta data.

What is SEO at Scale?

SEO at scale is really differentiated from plain-vanilla SEO in the execution of optimization efforts. Following best practices, implementing sitewide changes, and getting organizational buy-in is easy in small companies or on simple websites. (For this entire post, I am defining simple websites as a website that has a very clean architecture. A homepage with a few subpages that might have a few of their own subpages. A blog is a great example of a simple website: there is a homepage and then every subsequent page is organized by date with maybe a few tags thrown in.)

Quite the opposite is true at larger organizations and/or on complex websites. Making changes in this environment is akin to turning an aircraft carrier. From a company culture perspective, the keys to change may be shared within different departments and even small changes might require consensus across those various stakeholders. Something as simple as a title tag update might require the approval of various product and engineering managers, roles that if they existed in smaller organizations certainly wouldn’t have multiple people with that job title.

Once a change is approved, the actual rollout is directed in a standard engineering queue which is designed to keep websites and products bug-free by forcing all code changes into a process driven system. In this system there are many touchpoints which make SEO incredibly difficult and actually cede advantages on search to smaller more nimble organizations. (Tangential note: if you are competing on search with a large competitor, know that you have the advantage as they can never adopt change as fast as a smaller company).

Large company process

Large companies have things like dedicated sprints which hold back even small SEO changes from ever being released on the fly unless they clear the bar for a hotfix. There could be multiple layers of QA requiring the approval and understanding of engineers to even know what they are looking for in order to clear a change for release. However, for anyone working on SEO in a large organization nothing is as frustrating as the dreaded prioritization roadmap.

Any change, even a small one that requires engineering time, must beat out other engineering requests in order to make into a quarterly planning roadmap. This means that whomever is codifying that roadmap must believe that the SEO effort will be as impactful as something else that it might bump off of the roadmap. If the person making the SEO request can’t articulate the importance of that ask, it will never have a chance of making it into a roadmap. The only chance of a change happening will require an engineer having “extra” time, and no good engineer will intentionally allow that to happen!

In this environment, the key skill of an SEO cannot be just their creative and analytical abilities, but they also need to political and diplomatic. The person leading SEO-at-scale needs to have even stronger SEO abilities as they will need to articulate the what, why and how in a setting that could be potentially hostile to SEO asks.  They will need to know when an SEO best practice is really a key requirement or just a good to have. They will have to be a key player in the horse-trading, back scratching and negotiating that happens in any big company.


Depending on the culture, they may even need to be a data whiz who can participate in a data-driven conversation about what matters most to the organization. Further complicating this requirement, data might not even be readily available. Key data might be locked up due to security necessities and only accessible via a request process. A great SEO will know how to operate within the confines of data they can readily access at their fingertips while still knowing how to effectively manipulate the data that the organization requires. Small companies might have one system of record such as Google Analytics while larger companies will have multiple that each serve a different purpose.


On the people side, there are also some pretty big differences between big and small companies. Smaller companies will have direct points of contact for specific requests and when those contacts transition out of the organization, they will be replaced by another single individual. The same cannot be said at larger organizations where reorganizations might happen multiple times per year, people leave, and their responsibilities shift to multiple people or new people come in to roles that had not existed before. The owners and leaders of various functions might be in different offices that could even by a flight away, and building a relationship requires being a pro at virtual communication.

An SEO could take months selling an effort to an individual or whole team only to have that team no longer able to implement the requirements when the right time comes. Again, an SEO with strong soft skills will be able to thrive in this environment as they will find the new right contact and build a new relationship.


On the flip side, there are some amazing advantages to being able to do SEO at scale. At a smaller organization, the SEO may be capped in their career unless they move to another function while an SEO at a large company could continue to get promoted as their impact grows. If SEO efforts can grow in importance, the team focused on SEO will grow and the prospect of being in leadership becomes an option.  


 Larger organizations provide more opportunities for continuous learning and the SEO might find that they can explore new responsibilities without giving up their current ones. The greatest benefit is really the ability to get true SEO learnings not available at smaller companies. With a website that is relatively small, it will be nearly impossible to ever get statistical significance from an SEO test while larger websites afford many opportunities for testing and experimentation.

The complexity of a large website is really the best teacher of all. Knowing what the best practice around site hierarchy pales in comparison to actually learning what works best on a million-page website.  For a smaller website, international SEO might be limited to translating a contact page while at larger website it might mean translating a full website into ten different languages.  International opens up a whole new realm of testing and learning as the SEO team now needs to contend with keywords in languages they don’t know, competitors in other cultures, and a new set of rules around search engine discovery especially when it comes to content duplicated across languages or countries. For the right person, this is not a challenge, but an opportunity.


From a search engine perspective, SEO is the same for every website, but the effort that it takes to get there will vary widely depending on the complexity of the website and company. While many might think that brands get a leg up in search, and it’s very possible they do, that only happens when all other factors are even. Given the hoops that large companies need to jump through to produce the same result as a small company, this is far from guaranteed.

As anyone that has ever worked on SEO at scale knows, getting to the ideal state on all SEO factors is like climbing a mountain, at night, in a snowstorm. You know the peak is somewhere up there, and you just need to keep trudging forward and while it might take a long time to get there, if you stay the course you will get there. Everyone else might be on to the next mountain, but you feel a heroic sense of accomplishment at completing something that took monumental effort.


Zero click searches show the usefulness of search

Rand Fishkin recently published a blogpost referencing data that showed that Zero Click – defined as searches on Google that did not result in a click to any website at all – are now more than 50% of all searches.

In my opinion, searches where the user does not ultimately click to another website are actually beneficial for users because:

  • The user was able to satisfy their query without needing to click into one or multiple websites to find a quick answer
  • It makes users more reliant on search as a source of information – a place where next time around they might click.

Additionally, I am not convinced that any and all searches to Google properties are really a bad thing since for one the Google properties are so much better than their alternatives. Don’t believe me? When was the last time you used Mapquest to find directions or please share with me the URL of your favorite Youtube competitor.

Even in the places where Google was a bit more sneaky about cutting into competitors like say the restaurant review space, Yelp has so much richer content than 2 or 3 reviews you might find on a Google Local review. I think everyone can agree that users should end on the site or microsite of the best product.

The denominator is growing

Regardless, the true elephant in the room that might be leading to more zero click searches is that search has become so much better. Google suggest allows people to refine their searches even before they hit enter, but now there are so many other ways on the actual search result that Google helps users refine their search.

Anytime someone clicks one of these options – it is by nature a zero click search. Take for example this search:

A click into the next search page would be a zero click on the first search.

Or on many results there is a related search box like the one below:

A search for SEO tools brings up two complete sets of logos that would encourage a user to learn more about a company.

The net result of all these search improvements is there are MORE searches happening as Google reported in its last earnings report:

Our Google properties revenues increased $4,073 million and $7,757 million from the three and six months ended June 30, 2018 to the three and six months ended June 30, 2019, respectively. The growth was primarily driven by increases in mobile search resulting from ongoing growth in user adoption and usage, as well as continued growth in advertiser activity. We also experienced growth in YouTube driven primarily by video advertising, as well as growth in desktop search due to improvements in ad formats and delivery. The growth was partially offset by the general strengthening of the U.S. dollar compared to certain foreign currencies.

Without seeing the data from the Jumpshot study I can’t have an opinion on whether zero click search has truly grown, but even if it has I don’t think it’s a black and white negative against Google.


6 Things You Can Learn From Google’s Communication on Their Indexing Bug

In April of this year, Google experienced a bug where new pages stopped getting indexed. In a first for Google, they published a detailed synopsis of what happened.

Most of the time, our search engine runs properly. Our teams work hard to prevent technical issues that could affect our users who are searching the web, or webmasters whose sites we index and serve to users. Similarly, the underlying systems that we use to power the search engine also run as intended most of the time. When small disruptions happen, they are largely not visible to anyone except our teams who ensure that our products are up and running. However, like all complex systems, sometimes larger outages can occur, which may lead to disruptions for both users and website creators.

There are so many interesting nuggets of information in this debrief which supports some commonly held beliefs around SEO and debunks others. h

1) The indexation part of the algo is DISCONNECTED from the crawling algorithm. Clearly Google did not stop crawling new pages – they were just not getting pushed to the index.

2) The index is static – it needs to be “pushed” to datacenters and is not constantly in flux as many think. The index does not get updated on the fly, it is a static library that needs to be refreshed at intervals.

3) Search algorithms do change frequently far more than publicly disclosed or theorized on social media. Google was pushing an update on April 5th that did not coincide with a known algo update . Google says they update their algorithm over 500 times per year which is more than once per day. The fact that Google was “pushing” an index means that the algorithm that accessed that index would also have to have been adjusted.

4) Search console is a live look at data in the index – it wouldn’t have broken if it was disconnected from the index. Search console is one of my favorite search as it gives a look into data that we could not possibly see from any other source.

5) There is a “duplicate management system” as a part of the indexation algorithm NOT crawling. This explains why duplicate content with a canonical can rank for short periods of time if this process is not run in real time.

6) Google really does want websites to be successful in their index and to that end they try to give as much information as possible to optimize sites for Google search. In the battle for search visibility, there is no US vs THEM both Google and website owners want the same thing – to satisfy users.

SEO manager

The SEO Job Outlook: Everything you wanted to know

 Over the last year I noticed that the velocity of inbound recruiters pinging me about SEO roles had been increasing. I was curious whether there are actually more SEO roles or am I just “lucky.” To understand the total landscape a bit more I turned to LinkedIn Sales Navigator to do some deeper research on what recruiters are looking for and who might be qualified for these roles. 

Most of this post will be just facts and figures which underscores my gut hypothesis that there is suddenly a new found interest in SEO and the labor market is tight. 

Note: all numerical research is for the US only unless I specifically call out another location.

There are currently 47.5k job openings that match the keyword “SEO” and on the flipside there are only 909k people that match the same keyword. Taking these numbers at face value, this means that there is one job for every 20 people that match this keyword at the most broad level. 

To get a sense of the context of this ratio, I looked at the same metrics for some other job roles.

For “software engineers” the ratio is 10:1. 

For product managers, the ratio is 30:1.

For designers, the ratio is 23:1

With these numbers as a baseline, SEO appears to be high in-demand but not as in-demand as software which isn’t surprising. However, SEO is a very general keyword that could match anyone that remotely has anything to do with search engine optimization, and in reality the ratio could be even lower. 

SEO Specific People

Again using Sales Navigator, I broke down the people in the SEO category even further. First off,  of the 909k people that matched SEO only 26.5k had the words “SEO” in their title. Digging into just this subgroup, the results were quite fascinating:

  • 1,166 changed jobs in the last 90 days which a turnover ratio of 4%

Job role

  • 8,588 work in marketing and advertising
  • 3,677 work in internet
  • 2,002 work in information technology
  • 1,054 computer software
  • 806 writing and editing


The majority of these people (24k) work in marketing with the rest (1.6k) working under engineering.

Company size

The bulk of people are at smaller companies:

  • 864 are self employed
  • 4347  work in companies with 1-10 employees
  • 4954  work in companies with 11-50 employees
  • 3256  work in companies with 51-200 employees
  • 1801  work in companies with 201-500 employees
  • 1077  work in companies with 501-1000 employees
  • 1837  work in companies with 1001 – 5000 employees
  • 611  work in companies with 5001-10,000 employees
  • 1836  work in companies with more than 10,000 employees

I found the data around seniority and years of experience to be the most fascinating and where there is a real mismatch between hiring managers and the labor market. 


  • Owner 2,265
  • Director 2,138
  • Manager 1,405
  • Senior 11,611
  • Entry 16,081

While the prior data is titles which might not always align with skills, the years of experience actually requires passage of time. (I excluded people without a year of experience).

Years of experience

  • 6,550 have 1-3 years of experience
  • 3956 have 3-5 years of experience
  • 4862 have 6 -10 years of experience
  • 3731 have 10+ years of experience

Looking back at what recruiters are looking for, 12k of the current job openings want someone “mid-senior” level and above. Even if we assumed that anyone with 3 years of experience might meet that bar, the ratio of qualified candidates (based on the numbers above) to job openings is essentially 1:1! 

With this data in hand, this would definitely explain an uptick in demand for SEO. 

Here are some other interesting findings that came up from this research:

  • There are 5k people that have the title of SEO in California
    • Only 1,400 are in the Bay Area
    • There are 1,065 job openings in Bay Area!
  • There are 3k people in NYC that have the title of SEO 
    • There are 1,700 job openings
  • Here are some other US states and their population of people with SEO in their title:
    • Texas: 1,700
    • Florida: 2k
    • Illinois: 972
    • Washington:426
  • Internationally, here is the breakdown of people with SEO in their title
    • 8k in the UK
    • 8k in Pakistan
    • 57k in India
    • 4k in Spain

In short if you are looking for an SEO role and have experience, this is a sellers market. Know who your competition might and definitely don’t underprice yourself.

If you are looking to hire for SEO, you are up against many other companies that suddenly have the same need to fill a role. Prepare to break out all your tricks to recruit skilled SEO managers to your team.


Is SEO Marketing?

In most organizations, SEO is typically considered a marketing function but should it be? The answer to this depends on the SEO needs of the company.

In a smaller organization or a company who’s primary online presence has less than 100 pages SEO efforts will primarily focused on the content aspects to organic marketing:

  • Keyword research – the research and decisions on which keywords to use will drive a lot of the content and marketing on a website
  • Content creation which can span multiple areas
    • Blog posts
    • Marketing landing pages
    • Product updates – content about new products and created to promote these products
    • Feature pages – marketing the features of products
    • Home page content – typically a heavy branding effort, but requires keyword research to choose target keywords
  • Meta data creation – Using SEO best practices in the title tags, meta description and headlines. While it’s a key responsibility of an SEO team, the primary skill after keyword research is essentially copy writing
  • Backlink efforts /PR

All of these functions are very common marketing efforts and therefore make the most sense for SEO to sit in a marketing team. In this capacity the SEO individual (team) will be sit alongside content writers, paid marketers and PR. Reporting into a marketing leader, they will focus on the free search aspect counterpart to what the other team members do.

SEO for larger sites and companies

However for larger sites where much of the SEO will typically be product or engineering related, having the SEO team sit under marketing could make it challenging to both implement changes and be aware of challenges. The types of things the SEO team will work on in these companies will be more focused on the following than just content (which will also be a part of their job):

  • Migration planning and implementation – large sites will always be moving something as priorities change
  • Product build outs – these could consist of microsites, launches of new products or simply a constant cadence of refreshes and rebrands. In larger companies the product silos will be larger and every PM will have a preference to how they showcase their product
  • Architecture and taxonomy – the larger the site the more organization matters. Think of how clean the architecture of an ecommerce site might be and how terrible it could be without a lot of though paid to taxonomy.
  • International – This usually requires one person alone to focus on as there will be many moving parts that are driven by translations, currency, and even domains – all product related.
  • Competitive research and reporting – the more complex the company might be, the greater the need for customized SEO reporting might be.
  • Education – In larger organizations there is a need to have constant education as there will always be new people unaware of SEO requirements that have the potential to affect organic traffic.

In these larger sites, it might make sense for the SEO team to either be a part of a product team or even better on a growth team which sits between product and marketing.

From my experience, SEO has been most effective when it is in the same reporting line as product, but most companies actually put SEO within marketing.

In short, is SEO marketing? It really is contingent what the key levers the SEO team will be using to drive growth.


SEO Due Diligence for Investments and Acquisition – Lite version

It is prudent to conduct SEO due diligence when investing in (or acquiring) a business that relies on organic search traffic for a substantial amount of revenue. If the SEO foundation is somewhat tenuous the organic traffic is at risk of a decrease or even disappearing if something changes on the site, in the competitive landscape, or in a search engine algorithm update. 

It is important to keep in mind that while a site might have more ‘ownership’ of organic traffic than paid traffic –  which will just stop when the budget stops – the rankings and the traffic outcome are really at the whims of the search engines. That being said, rankings typically don’t disappear overnight IF best practices are followed and all SEO efforts are above board.

For the most part, any SEO due diligence projects I have conducted have been similar to an SEO audit, however the focus is usually on how stable the site’s traffic will be for the near future rather than what can be improved.  Anyone experienced with SEO could likely find obvious red flags in a site, but when investing a substantial amount of money into a site, you want to make sure you do this right.

Additionally, it is critical to understand what red flags are areas of opportunity versus which ones should cause the potential investors to turn and run.

Independent SEO Due Diligence

If you are in the early stages of looking at a potential website investment or acquisition, you might want to do some quick due diligence on your own.  To begin this process you are going to need access to their Google Search Console account, Google Analytics and server logs. You should also have a simulated web crawler like Oncrawl and a backlinks/traffic tool like Ahrefs on hand. 

The first step in this exercise is to see if there have been any negative SEO events in the recent past:

  • Check Google Search Console to ensure there has been a steady uptick in organic traffic
  • Check Google Analytics to see that there have never been any negatives drops in traffic since the site was launched
  • Check Ahrefs to see that there have never been any drops in visibility

If there have ever been any drops in visibility ask for clarifying details.

Technical due diligence

Next it’s time to start looking under the hood at the technical implementation of the site. Launch your crawl on Oncrawl and let it run without any filters at all. If the site is small you can use Screaming Frog’s free tool.

  • Look for large numbers of broken pages 
  • Look for chained redirects  (page → page → page) 
  • Look for duplicate content
  • Look for content that is no-indexed
  • Look for content that is indexed but shouldn’t be
  • Look for proper use of on-page meta tags
  • Drill into the internal linking architecture
  • Understand how pages link to each other and with what keywords
  • Manually look at the site for unindexable content 
  • Note the URL structure
    • Are there query parameters?
    • Are all pages connected to the root domain
    • Are there many subdirectories in the URL
    • Do they use subdomains?

Backlink Due Diligence

Moving on from the technical audit it’s time to start looking at the backlinks. Use the Ahrefs backlink checker to get a sense of the quality and quantity of backlinks.

  • Check for patterns of deliberate gray backlinking
    • Non-varied anchor text
    • Links that don’t seem to fit content
    • Links on non-relevant pages
  • Understand the quality of the links using a domain score metric
  • Dig into the links to reveal how they were obtained

Now let’s shift to the actual search results and start running some site queries.

  • Conduct a search on Google for the brand and make sure the homepage shows up first.
  • Notice how many pages are indexed in Google
  • Conduct a site: query with some common spam words appended to make sure that the site has not been hacked

Finally, we can assess the quality of the site itself

  • Analyze the quality of the content on the site. 
    • Is it in depth?
    • Are there typos?
    • Grammatical errors
    • Where is it located?
      • Blog 
      • Resource page
      • Homepage
    • What is the readership level
  • Using Ahrefs again look at what kind of pages are ranking on the site
  • Look at Google Search Console to see the kinds of terms that are ranking
  • Look at Google Analytics to make sure that time-on-site for content pages isn’t lower than the amount of time it might take to read the content.

Following the above steps should give you a good sense of the staying power for existing traffic. The next step would be to assess the conversion funnel of traffic that arrives from search engines. Regardless, the outcome of the quick SEO due diligence audit is to know whether you might want to spend more time researching the potential investment. This quick audit should in no way substitute for a full audit which must be done before any funds trade hands.

I will follow up on this post on how to do a truly in depth audit.

1 2 4 5 6 7 8