Eli Schwartz

Posts by

Eli

Home / Blog Archive
SEO

SEO tools: My full list

With the trillion dollar plus annual value of SEO traffic, there is no lack of tools that help to understand Google and optimize websites accordingly. This is not meant to be an exhaustive list of tools that can be used to help with SEO rather it is a list of tools that I personally use and find useful. I will continuously update this list as I discover more tools.

Crawling

When working on a large domain, understanding the size of a site and how it might be viewed on Google is critical. A good crawler will crawl a site similar to how a search engine would discover all the pages on a site. The crawl of the site should tell you any technical SEO issues that might exist and should be fixed. For crawling I have a handful of go-to options, each with specific pros and cons.

Screaming Frog’s desktop crawler should be the staple of anyone doing technical SEO. For smaller sites the free version should be perfectly fine, and for larger sites there is a nominal fee to access the pro version.  Screaming Frog allows users to crawl a site and then manipulate the data in a spreadsheet which is the preferable way, in my opinion, to handle large data sets.

My new favorite desktop crawler is SiteBulb which offers many of the same crawling capabilities as Screaming Frog, but also has some amazing visualization tools that negate the need to build charts in Excel. Additionally, it doesn’t have the memory leak issues of Screaming Frog and I have crawled hundreds of thousands of pages without needing to chunk out pieces of the site like I have done with Screaming Frog.

For cloud based crawling, I use both Oncrawl and Deepcrawl. They both offer similar capabilities and the decision over which one to use would be one of personal preference.

Backlink research

I learned SEO by doing backlink research on Yahoo Site Explorer and then replicating my competitors backlinks on my own site, so backlink research is an SEO process near and dear to me. Throughout my SEO career I have experimented with nearly every backlink tool from Moz to Majestic to SEMRush. My current favorite tool for backlink research (and many other functions) is Ahrefs. Although, again this is a personal preference as most backlink tools will have similar enough data to take action on for a linkbuilding campaign.

Competitive Research

Most link building tools will help with basic competitor research which will show the types of links that competitors have as well as the keywords they ranking on. For more specific competitor research I like to use Similar Web which gives me data around total traffic and the percentage of organic traffic. I also use Alexa.com (yes, it’s still around) to show keyword intersections between websites.

Keyword Research

When I first started in SEO there were only a few keyword research tools and none were very good. We are lucky that there are so many options to generating keywords that real users search. I typically try to optimize towards users and the queries I need for my sites rather than on search volume alone, but having this data can be very helpful for prioritization.

When I am working with a large site and a big Google Ads spend, I find the data in search query reports from Google ads and the Google Keyword Planner to be very useful.

I also use the competitive tools mentioned above to find new potential keyword ideas as well as keywords that competitors rank on.

For gathering Google suggest terms – which will be terms that users actually search and therefore Google suggests them, I use KeywordTool.io. This tool will also offer suggested ideas from other search engines including Youtube and Amazon.

To build huge lists of keywords, I have used Scrapebox, KeywordSheeter, and Kwfinder.

As people view Google more as a friend than a search engine, they ask more questions within search and AnswerthePublic is a great tool to get question based queries.

International

International SEO efforts will be similar to domestic SEO except for that it is focused on another country or language. Most of the SEO tools on this list will work for international SEO except the language or country needs to be changed.

For sites that need to build out Hreflang tags, this tool from Aleyda Solis can be very handy/ , but larger enterprises that might need help can use
hreflangbuilder.com.

Site speed

Site speed is a part of the  Google ranking algorithm, but not a major part. The main reason anyone should care about the speed of a website is because users will bounce if a site is too slow. For site speed, I usually use multiple tools because they are free and quick so more data can’t hurt. Pingdom.com gives a waterfall of how a page loads. Gtmetrix.com displays helpful information on what can be fixed to improve a page/site’s speed. I also use Google’s page speed tool which adds some additional info on mobile page speed.

My favorite site speed tool is simply to lower my mobile device to 3G and to see how fast a webpage loads. If it takes too long to load, then I can assume a real user would bounce on a similarly slow connection.

Rankings

As I have written previously, I am absolutely not a fan of ever checking search rankings because I think it is the wrong metric to look at. However, there are specific use cases where I find rankings to be very helpful. If a site made a number of changes, I might download a prior months queries and then put them into a rankings tool to see if there has been a massive shift from previously reported Google Search Console positions.

The two tools I use for rankings are Link Assistant’s Rank Tracker which runs ranking queries off of my desktop until I get captcha blocked by Google or my favorite: Rank Ranger. In addition to rankings, Rank Ranger has a ton of other features including competitive insights, schema creators, and social analytics.

Data

My go to SEO tool is Google Search Console which is free and everyone should be using even if they don’t trust the data as much as they should. Google Search Console data can be pivoted in multiple ways to find insights, but the UI is still a bit limited. The way around this is to pull down data via the API. To access the API, I use a Google Sheets plugin called the Search Analytics for Sheets. I have found that this tool has had issues recently with maxing out its API calls so I have had to fall back to building my own lookup tools in R Studio which you can do to if you follow this guide.

Optimization and testing

Most of my testing and experimentation happens manually because getting the keys to the kingdom that is the codebase of a website is challenging. If you are able to get access to either Cloudflare or another CDN, you can use tools like Distilled ODN and Clickflow to do SEO learning at scale. Absent that you should absolutely be testing by making single variable changes on multiple pages and then recording the clicks/impressions over a long time period to discover if there are any statistically significant learnings.

Enterprise

For SEO teams that need to have quick access tools that can do everything on all in one place they might want to use an enterprise SEO tool that can also help with writing bug requests, tracking and dashboards. Searchmetrics is the enterprise SEO tool I have been using for many years and they now also have tools which help content teams draft content that includes all the related keywords that are ranking on other sites.

The goal of any SEO tool is to make SEO less manual and more efficient. These are the tools that I use on most SEO projects and I love that new tools keep being produced to make SEO even easier! If there are any tools I missed or might no have heard about, please let me know.

SEO

SEO Personas as the foundation for keyword research

The idea of building elaborate customer personas is very popular on design and in various marketing teams, but so few people actually use them in their daily work life that the investment in creating them is hardly worth it. Many times, when companies build out these personas, they go overly deep into developing exactly who these customers might be and all of their character traits.

Personas for SEO

Personas might be passé, but when it comes to SEO I think that some sort of persona research must be the foundation of any good keyword research. Too many people begin a process of keyword research by firing up their favorite keyword tool and then picking keywords off the list with high monthly search volume that are relevant to their business.

Starting with keywords sorted by volume puts the emphasis on the wrong metric and leads to creating content that might not match the intent of a user or the needs of a website. In my opinion, it makes the most sense to prioritize exactly the kind of content that is needed to help a website monetize. Some keyword tools might show this after the content has already been written by doing a “keyword gap analysis” vs a competitor, but it is a lot easier to just determine what content is necessary at the outset.

The easiest way to figure out exactly what content is necessary is to go through a persona exercise to understand exactly how, why and what users want from the website. Only then once the users’ needs are taken into account does it make sense to distill those topics into precise keywords.

Persona research should answer questions such as where in the buying funnel a user might be. This will guide the depth of content a user expects to see. It is also important to understand the devices that a user will be using to access the website. Is it a desktop? A mobile device? Or maybe the user can be served just with a voice enabled device. Knowing this can quickly help decide whether long form or image heavy content is even appropriate.

Before embarking on this effort, it is worth acknowledging that existing personas likely will not be detailed enough for you to use for SEO and it is not a wasted effort to build personas from scratch just for SEO. The current personas that the company might be using will have details that are not necessarily helpful for SEO like age, gender, and career details.

Steps to build SEO personas

With that in mind, here are the best practices on developing personas specifically for SEO.

  • Identify all potential users of a website or product
    1. This is where keyword research as a start of an SEO effort typically falls short. Just because a website or product exists doesn’t mean that users will automatically want to search for it. Taking a step back to think about who the users of the website might be gives a good foundation for what kind of content and keywords to focus on. For example, an ecommerce website might want to target people that have a specific need and the focus of SEO should be on solving that need rather than just optimizing the product page. A SAAS product might have a similar phenomenon and targeting the problem rather than the solution would yield more search traffic.

  • Determine how the users might search based on where they are in the funnel.
    1. Again, traditional keyword research would only identify the popular terms for a vertical but not how the targeted users will search. Users very high in a funnel will be searching for a solution to a problem while users at the very bottom will be looking for the brand plus pricing info.


  • Slot them into the type of content that they might expect.
    1. There is a lot of advice around what kind of content is best for SEO, but none of that advice considers the granular needs of a specific user. If a user wants just a price or a list of features, they will be ill-served by a long form piece of content while a user that wants a detailed product review would similarly not be helped by a quick list of bullets.

  • Match them with a specific call to action that is relevant for the place they are in the buying funnel.
    1. Search traffic is a means to an end and is rarely the end itself. Even on a media site that targets readership, an increasing user count is of no benefit if the users don’t do a follow on engagement action. Where the users are in a buying funnel should determine the appropriate call to action (CTA) for the content. A reader that is very low in the buying funnel might be looking for a way to contact a salesperson while a user high in the funnel should be encouraged to just read more or maybe subscribe to a mailing list. When content is written for users rather than keywords it becomes a lot easier to have a targeted action type for users to take.

  • Classify the types of devices they would be using to access the content.
    1. While we constantly hear the refrain that the mobile web is dominant, this is not necessarily carried forward into executing SEO efforts. If it was, long form content would have fallen by the wayside in favor of short punchy shareable bits. Even though nearly every web user has a mobile device, there are some things that will always be done on a desktop. Buying business software or expensive shopping is probably going to involve a bigger screen somewhere in a buying cycle. Writing content to where the users are in a buying cycle should play a key factor into the screen size they will be potentially using to access the content.

  • As a bonus, pigeonhole them into a precise language or culture for internationalized content
    1. One last thing for sites that have international audiences, its critical to know what language they might expect to see content in and if there are any cultural nuances that should be addressed. What many people that have never done international marketing might not know is that its OK to have just English content for an international audience. They might not expect a translated page, so it is better to just give them content in English that contains the international options they need like shipping or currency. Understanding the users will prevent a website from creating language specific content unnecessarily.

With these best practices in mind, hopefully you will be able to develop SEO specific personas that will guide keyword research. Keyword research like everything in SEO should be targeted at real users – not search engines – and a persona exercise will go a long way into knowing who those real users might be.

SEO

Crawl Budget – It’s really a simple SEO concept

The phrase “crawl budget” is an SEO term that is frequently included in discussions about technical SEO, but it is typically used incorrectly. Most of the time when people refer to crawl budget they are considering it a technical SEO enhancement to improve the way Google understands a website. In fact, it is far simpler than that it is simply a budget.

The best way of understanding various aspects of Google’s algorithms is to view them from a financial standpoint. Crawling and indexing the web is a very expensive proposition and Google was able to beat out every search engine to dominance because they figured out how to do that before the money ran out.

While it would be ideal for Google’s crawlers to simply gobble up the entire web in one fell swoop that would be technically impossible. Crawlers need to literally crawl through the web discovering link after link and then as they land on a page they need to build a copy of the page into the database.

It’s about the Benjamins!

In the early days of search, while Google was still living on venture capital money, the engineers needed to come up with a way to efficiently crawl the web without going broke in the process. That way was to decide how much “budget” each site was allocated based upon its importance to Google and the web as a whole. That is crawl budget.

If a site is very important to the ecosystem, Wikipedia for example, Google would have wanted to allocate a lot of their hypothetical dollars to crawling as much of the site as they could. Alternatively, a brand new website with no authority on the web would be allocated a significantly smaller amount of budget.

New websites

This all makes logical sense. Taking this logic one step further, if the brand new website would have thousands of pages but only a few of them were valuable, it would have been very likely that their budget would have been eaten up by the crawler ingesting the lower quality pages without ever seeing the good ones.

The best approach for a website in this position is to simply declare – via no indexes or canonical tags – which are the lower quality pages and then the crawlers could just skip them.

A happy example

To illustrate this with an example, think about a website like a Happy Meal with a toy inside. You have a certain amount of daily budget to buy Happy Meals, but you only need the unique toys to complete a series. The only way you could find out whether the toy is unique is by buying the meal and opening the box. So, every time a Happy Meal is bought and the duplicate toy shows up, that days budget is wasted – unless you were very hungry. The most efficient way to do the toy collecting is simply to show the name of the toy on the outside of the box and then you would choose only that box.

Continuing this Happy Meal to website analogy, those no index directives and canonical tags are the best way of informing a search engine to ignore a particular box.  The crawler then has more awareness on how to most efficiently spend their limited budget.

Crawl budget summary

This idea of crawl budget applies to every website on the web regardless of authority, its just that more authoritative websites have more budget to be expended by the crawler. As websites gain authority, likely via links or other user engagement signals their budget will expand but without that there is no other way to get more budget.

Google refers to this as “crawl demand” and while they don’t specifically mention authority in their blog post on crawl budget, they sort of beat around it by calling it “popularity.”

Even if the crawl rate limit isn’t reached, if there’s no demand from indexing, there will be low activity from Googlebot. The two factors that play a significant role in determining crawl demand are:

  • Popularity: URLs that are more popular on the Internet tend to be crawled more often to keep them fresher in our index.
  • Staleness: our systems attempt to prevent URLs from becoming stale in the index.

This idea of budget was a key component of Google’s crawling algorithm and it is still exists today although the budget is vastly expanded. Google now has lots more money and resources to crawl the web, but the web is also bigger and more complicated.

Crawl budget today

One other change is that budget was likely initially calculated in small amounts of kilobytes which equated to a number of pages, that budget can be eaten faster if a site has dynamic scripts that are more expensive for the crawler to run.

SEO

Annual and quarterly planning for SEO

In addition to shorter days and colder nights (at least in the Northern Hemisphere of the world), September brings a special gift to the office which everyone claims to welcome – but in truth they despise: annual planning. Many large companies will have some sort of quarterly planning process where in the last couple weeks of a quarter teams will have to detail how they have progressed on the current quarter’s goals and what they hope to accomplish in the coming quarter.

Annual planning

While this process isn’t necessarily the most fun, it can be straightforward as it has the benefit of making predictions about that things that are already in flight. This can’t be said for an annual planning process which is typically conducted in the closing quarter of a year and requires a team to pick a goal for the coming year and negotiate for the resources to achieve said goal.

Winning at annual planning

Team that “win” the planning process  in the sense that their goals are accepted and their resource requests are met can in some ways rest for the next 12 months until this process begins again. In most organizations they will not be necessarily held to their goals because of course a lot can change over a year. Their real win is that they were given the resources (defined as budget and/or headcount) to meet their goal(s), and if they did not meet that goal for whatever external resources, they were still able to deploy their resources to make some other flashy business impact.

This business impact – even if it was not the one stated in the annual planning process – positions them in a very good light in the following year to again state lofty goals and request resources to help them get there.

Losing at annual planning

On the flipside of the teams that win the planning process are the teams that “lose.” These are teams whose goals are not adopted as goals worthy of being resourced and their requests will be turned down in favor of other teams. They will still have a goal for the coming year, but it will likely be somewhat watered down due to the lack of resources. This team could find themselves generally deprioritized and locked out of new headcount and budget for the entire next year. When it comes time to develop goals and make a request for resources in the following year, they start off with a significantly weaker hand as they don’t have the benefit of making a huge impact in the prior year.

Looking at planning in this light, the stakes could not be higher. Teams must win at the planning process and successfully pitch executives to get behind their goals or risk becoming a non-essential line item until something fundamentally changes in the business. The only exception to this favored/non-favored child reality is if there is a high-level executive backing a team or if there is a shift in power within a company like a re-org, fundraising event or new product need.

Due to the nature of where SEO teams sit within a large organization, they immediately begin this planning process behind the curve. In most companies, SEO either sits within marketing or product and this placement has a big effect on how they do in a planning process.

SEO in marketing

If an SEO team is a part of the marketing team, they likely report up to a marketing leader of some sort. This leader will be juggling budget and headcount requests from teams that can make far stronger 1+2 = 3 pitches. For example, a paid marketing team can show very clear math of how additional paid budget will impact acquisition, retention or awareness. They can also show how added headcount will improve the efficiency of their spend thereby adding more value to the organization.

The same argument might work for a content team that could show how output (the metric that content teams are measured by) will improve by a factor of how much more is spent on producing content.

Contrasted with the SEO team that has fuzzy math for how SEO works and even fuzzier math for what more spend might get, a pitch for resources is a losing proposition on a marketing team. This is why most SEO teams even in companies with large other marketing-subteams will still only consist of 1 or 2 people.

SEO in product

When an SEO team sits on a product team, the uphill climb to win resources is slightly easier but not by much. Rather than a pitch for just budget and headcount, the resource ask will also likely include engineering time. In a company that does not prioritize SEO, the engineering time request might get shunted aside in favor of building new products or improving existing products.

Logically, this makes a lot of sense as what product leader would rather have a roadmap full of improvements to existing products rather than one with all sorts of exciting new builds.

How to win resources

To win at the annual planning game, the SEO team is going to have to morph into the kind of team everyone loves to fund and support. This will apply regardless of where the SEO team sits in the org.

First and foremost, the whole idea of using wishy washy data to forecast SEO impact must be completely discarded. There is data within the company and Google Search Console which very clearly tells the story of how valuable SEO traffic is for top line acquisition. There may even be data on how it impacts the bottom line too. Get that data and use that as the guiding light within an annual goal.

Is the plan to increase SEO traffic by 10%? That’s not a good goal! Back into what a 10% organic traffic increase might mean to top line revenue and use that number as the goal. An increase of 10% of overall web revenue from organic sources sounds a whole hell of a lot more sexy than an unclear 10% increase of traffic.

Second, many other teams will pitch ideas that they don’t really know how they will implement but want resources to try. They certainly aren’t pitching the process on how they may or may not get there, rather they are saying they are going to build X and X requires engineers.  The SEO team should do the exact same thing.

Instead of asking for engineers to update a whole bunch of SEO requirements, instead ask for engineers (or content or money) to build X for SEO.

Third, and this is specific by company, there is a process for how every other product and marketing team pitches for resources. Make sure that the SEO pitch looks exactly the same and there is data to show even greater impact than other efforts. Keep the SEO jargon out of the pitch and use the same language that everyone else uses. The last thing an SEO team wants is to have their pitch stand out just because it was different.

Annual planning from a leadership standpoint

The prior advice mostly applies to SEO practitioners pitching for resources or those pitching SEO asks to the C-suite, but it can just as well be modified for leaders being asked for the resources. Know that SEO is incredibly important, and if the pitch for resources doesn’t have the clear 1+2 = 3 approach then push it back to the team for a revision, don’t just reject it out of hand.

Summary

Annual planning is a process no one really enjoys other than the people who run the process for everyone else. It is a necessary evil and it isn’t going to go away if it is ignored.  If an SEO team does not put their best foot forward they could risk losing an entire year with no impact efforts.  

SEO

PSA: Using rankings to track SEO success is dumb

It is amazing to me that in 2019 anyone would use still use rankings as a success metric for an SEO campaign. Rankings are a vanity metric and do not directly or even directly contribute to the success of a business.

Using a rankings report to as the only way to measure SEO progress is as asinine as using a paid marketing budget total as a metric of success. All a big budget shows is that someone can spend money, it says nothing about whether it is profitable. Rankings just exhibit the ability to be ON Google and not whether anyone even clicks or buys.

Rankings in the past

When I first started my career in search engine optimization the critical metric of success was ranking in search results, and most importantly it was how many number one positions one occupied. It did not matter whether those were useful positions or even if anyone clicked, just having them was a bragging right. As an added bonus, Google wasn’t the only search engine that anyone cared about, so having a top result on MSN.com or Yahoo also generated some SEO applause.

Keywords to rank in a top position were chosen using keyword research tools with preference given to exact match words that had high average monthly search volume with no heed paid to intent. Those keywords were then plugged in on-page meta, spun into content at a high keyword density and most significantly used in anchor text for deliberately built external links.

To keep track of these positions, SEO’s had to use a slew of tool’s whose primary function was to scrape search engines on a weekly, daily or even hourly basis for the latest rankings. Executives asked to see these reports and hence having a huge list of prominent top positions was a key component of an SEO role. This whole process worked and then it didn’t because things changed.

It wasn’t just one change, EVERYTHING changed.

For starters, Google became the dominant search engine if not the only search engine anyone cares about. Google earned this role by rapidly improving the search user’s experience which was a direct result of rooting out the kinds of practices that made pages rank undeservingly.

The list of changes on the search side over the last ten years are endless, but some of the key highlights are:

  • Panda – This Google update from 2010 deprioritizes thin content that is targeted at just generating search clicks. It used to be a separate algorithm that ran alongside the core algorithm, but now it just a part of the ranking algorithm.
  • Penguin –  This algorithm update from 2012 negates the impact of illegitimate links and penalizes sites that engage in these practices intentionally.
  • Local –  A national/global result is irrelevant if Google determines that there’s a local intent behind a query.
  • Google suggest – Google’s query suggestions continually to become richer as they use real time data and trends to direct people into a search funnel.
  • Intent matching – Results for synonyms, misspells, and pluralization are nearly identical provided that the intent behind the phrasing is the same. Choosing word order and pluralization is not a necessary SEO task as they will usually be similar if the intent is the same.
  • Entity matching – Google can parse the entity a user is seeking and will show relevant results. Additionally, the amount of entities that can be highlighted in search continues to grow.
  • Mobile – The rise of mobile search changes everything with regards to how people search with touch-only keyboards or voice search. Users with Android phones can already conduct searches with their cameras through Google Lens but expect the way that searches will be conducted to continue to grow as device technology improves.
  • Artificial Intelligence – Google is not just reliant on words that are in the query or even on the page. They can parse meaning and intent without any apparent match between content and query.
  • Artificial Intelligence and deep learning – Whether Google uses engagement metrics in rankings is up for debate, Google certainly has a significant amount of machine learning to know how content should perform in search results without even needing it to gather real engagement metrics.
  • Ranking signals – in years prior it was thought that content + metadata + links were the key components to ranking. They are still important, but Google claims to use hundreds of others so even with the most optimized content and links, its impossible to force a result.
  • Zero click results – To minimize on information arbitrage results and to provide an even better user experience Google will put answers directly into search results. This negates the value of a top ranking result, as many users will choose not to click any results at all.
  • Size of the web – for anything that is of high value there are now hundreds to thousands of sites chasing the same traffic. Generating search traffic requires being far more creative than just picking keywords off a list from a keyword research tool.

With all of this in mind, manipulating a particular ranking can be virtually impossible and even if one were successful all that effort might be for naught.

SEO success metrics

The primary success metric for SEO is and should always have been the same for every marketing channel: the amount of revenue, leads, visitors etc that the business needs to be successful. If every other marketing channel is contributing to the bottom line or at least the top line, and organic traffic is not, there is an issue that needs to be addressed. Patting oneself on the back for great rankings in this situation will be little solace if the business goes under for a lack of cash.

Some businesses, especially those with long sales pipelines,  may have attribution challenges in tracking channel performance back to organic traffic as typically search traffic will be mostly top of funnel. In this case, the fallback should be clicks from search engines, but effort should still be made to determine that the clicks are of value.

Even if it’s impossible to determine the business outcomes, the business should still be looking at the engagement rate – bounce rate, pages per visit, and time on site – from this source of traffic. If the engagement rates are too low to ever lead to a conversion event, there is an issue and the rankings leading to the clicks are of no value.  

Summary

Rankings alone as a KPI for SEO is vanity metric and it should never be used in budgeting, financial modeling or any other important business conversation. SEO should be judged in the same vein as every other marketing channel and if it can’t be then appropriate proxies that correlate to business KPI’s should be used. It is 2019, and Google has robotaxis on the road, we should stop pretending that they are the same search engine of 2009.

SEO

Big company vs small company SEO – enterprise SEO at scale

Search engine optimization is a straightforward process of aiding search engines in best understanding the content of a website so it can garner the maximum visibility in search results. Like anything in technology, there is a set of best practices which need to be implemented correctly to help the machines that power the crawling, ranking and discovery algorithms. In a small organization or on a simple website these best practices are easy to follow and implement. The person implementing the best practices might even have access to the content management system to do it themselves.

As an organization grows or a site becomes more complex, following one set of guidelines means there will be tradeoffs in another part of the company or on the website. This is what is typically termed enterprise SEO, but I prefer the more descriptive: SEO at Scale.

The rules and guidelines are the same for every site regardless of whether it has ten or a million pages. Every site, regardless of size, and in-fact every page needs to have optimized meta data, great content, and good backlinks that vouch for the site. A site like WashingtonPost.com doesn’t get a pass for thin content just because it has a great brand nor does Amazon.com get away with weak meta data.

What is SEO at Scale?

SEO at scale is really differentiated from plain-vanilla SEO in the execution of optimization efforts. Following best practices, implementing sitewide changes, and getting organizational buy-in is easy in small companies or on simple websites. (For this entire post, I am defining simple websites as a website that has a very clean architecture. A homepage with a few subpages that might have a few of their own subpages. A blog is a great example of a simple website: there is a homepage and then every subsequent page is organized by date with maybe a few tags thrown in.)

Quite the opposite is true at larger organizations and/or on complex websites. Making changes in this environment is akin to turning an aircraft carrier. From a company culture perspective, the keys to change may be shared within different departments and even small changes might require consensus across those various stakeholders. Something as simple as a title tag update might require the approval of various product and engineering managers, roles that if they existed in smaller organizations certainly wouldn’t have multiple people with that job title.

Once a change is approved, the actual rollout is directed in a standard engineering queue which is designed to keep websites and products bug-free by forcing all code changes into a process driven system. In this system there are many touchpoints which make SEO incredibly difficult and actually cede advantages on search to smaller more nimble organizations. (Tangential note: if you are competing on search with a large competitor, know that you have the advantage as they can never adopt change as fast as a smaller company).

Large company process

Large companies have things like dedicated sprints which hold back even small SEO changes from ever being released on the fly unless they clear the bar for a hotfix. There could be multiple layers of QA requiring the approval and understanding of engineers to even know what they are looking for in order to clear a change for release. However, for anyone working on SEO in a large organization nothing is as frustrating as the dreaded prioritization roadmap.

Any change, even a small one that requires engineering time, must beat out other engineering requests in order to make into a quarterly planning roadmap. This means that whomever is codifying that roadmap must believe that the SEO effort will be as impactful as something else that it might bump off of the roadmap. If the person making the SEO request can’t articulate the importance of that ask, it will never have a chance of making it into a roadmap. The only chance of a change happening will require an engineer having “extra” time, and no good engineer will intentionally allow that to happen!

In this environment, the key skill of an SEO cannot be just their creative and analytical abilities, but they also need to political and diplomatic. The person leading SEO-at-scale needs to have even stronger SEO abilities as they will need to articulate the what, why and how in a setting that could be potentially hostile to SEO asks.  They will need to know when an SEO best practice is really a key requirement or just a good to have. They will have to be a key player in the horse-trading, back scratching and negotiating that happens in any big company.

Data

Depending on the culture, they may even need to be a data whiz who can participate in a data-driven conversation about what matters most to the organization. Further complicating this requirement, data might not even be readily available. Key data might be locked up due to security necessities and only accessible via a request process. A great SEO will know how to operate within the confines of data they can readily access at their fingertips while still knowing how to effectively manipulate the data that the organization requires. Small companies might have one system of record such as Google Analytics while larger companies will have multiple that each serve a different purpose.

People

On the people side, there are also some pretty big differences between big and small companies. Smaller companies will have direct points of contact for specific requests and when those contacts transition out of the organization, they will be replaced by another single individual. The same cannot be said at larger organizations where reorganizations might happen multiple times per year, people leave, and their responsibilities shift to multiple people or new people come in to roles that had not existed before. The owners and leaders of various functions might be in different offices that could even by a flight away, and building a relationship requires being a pro at virtual communication.

An SEO could take months selling an effort to an individual or whole team only to have that team no longer able to implement the requirements when the right time comes. Again, an SEO with strong soft skills will be able to thrive in this environment as they will find the new right contact and build a new relationship.

Advantages

On the flip side, there are some amazing advantages to being able to do SEO at scale. At a smaller organization, the SEO may be capped in their career unless they move to another function while an SEO at a large company could continue to get promoted as their impact grows. If SEO efforts can grow in importance, the team focused on SEO will grow and the prospect of being in leadership becomes an option.  

Learning

 Larger organizations provide more opportunities for continuous learning and the SEO might find that they can explore new responsibilities without giving up their current ones. The greatest benefit is really the ability to get true SEO learnings not available at smaller companies. With a website that is relatively small, it will be nearly impossible to ever get statistical significance from an SEO test while larger websites afford many opportunities for testing and experimentation.

The complexity of a large website is really the best teacher of all. Knowing what the best practice around site hierarchy pales in comparison to actually learning what works best on a million-page website.  For a smaller website, international SEO might be limited to translating a contact page while at larger website it might mean translating a full website into ten different languages.  International opens up a whole new realm of testing and learning as the SEO team now needs to contend with keywords in languages they don’t know, competitors in other cultures, and a new set of rules around search engine discovery especially when it comes to content duplicated across languages or countries. For the right person, this is not a challenge, but an opportunity.

Conclusion

From a search engine perspective, SEO is the same for every website, but the effort that it takes to get there will vary widely depending on the complexity of the website and company. While many might think that brands get a leg up in search, and it’s very possible they do, that only happens when all other factors are even. Given the hoops that large companies need to jump through to produce the same result as a small company, this is far from guaranteed.

As anyone that has ever worked on SEO at scale knows, getting to the ideal state on all SEO factors is like climbing a mountain, at night, in a snowstorm. You know the peak is somewhere up there, and you just need to keep trudging forward and while it might take a long time to get there, if you stay the course you will get there. Everyone else might be on to the next mountain, but you feel a heroic sense of accomplishment at completing something that took monumental effort.

SEO

Zero click searches show the usefulness of search

Rand Fishkin recently published a blogpost referencing data that showed that Zero Click – defined as searches on Google that did not result in a click to any website at all – are now more than 50% of all searches.

In my opinion, searches where the user does not ultimately click to another website are actually beneficial for users because:

  • The user was able to satisfy their query without needing to click into one or multiple websites to find a quick answer
  • It makes users more reliant on search as a source of information – a place where next time around they might click.

Additionally, I am not convinced that any and all searches to Google properties are really a bad thing since for one the Google properties are so much better than their alternatives. Don’t believe me? When was the last time you used Mapquest to find directions or please share with me the URL of your favorite Youtube competitor.

Even in the places where Google was a bit more sneaky about cutting into competitors like say the restaurant review space, Yelp has so much richer content than 2 or 3 reviews you might find on a Google Local review. I think everyone can agree that users should end on the site or microsite of the best product.

The denominator is growing

Regardless, the true elephant in the room that might be leading to more zero click searches is that search has become so much better. Google suggest allows people to refine their searches even before they hit enter, but now there are so many other ways on the actual search result that Google helps users refine their search.

Anytime someone clicks one of these options – it is by nature a zero click search. Take for example this search:

A click into the next search page would be a zero click on the first search.

Or on many results there is a related search box like the one below:

A search for SEO tools brings up two complete sets of logos that would encourage a user to learn more about a company.

The net result of all these search improvements is there are MORE searches happening as Google reported in its last earnings report:

Our Google properties revenues increased $4,073 million and $7,757 million from the three and six months ended June 30, 2018 to the three and six months ended June 30, 2019, respectively. The growth was primarily driven by increases in mobile search resulting from ongoing growth in user adoption and usage, as well as continued growth in advertiser activity. We also experienced growth in YouTube driven primarily by video advertising, as well as growth in desktop search due to improvements in ad formats and delivery. The growth was partially offset by the general strengthening of the U.S. dollar compared to certain foreign currencies.


Without seeing the data from the Jumpshot study I can’t have an opinion on whether zero click search has truly grown, but even if it has I don’t think it’s a black and white negative against Google.

SEO

6 Things You Can Learn From Google’s Communication on Their Indexing Bug

In April of this year, Google experienced a bug where new pages stopped getting indexed. In a first for Google, they published a detailed synopsis of what happened.

Most of the time, our search engine runs properly. Our teams work hard to prevent technical issues that could affect our users who are searching the web, or webmasters whose sites we index and serve to users. Similarly, the underlying systems that we use to power the search engine also run as intended most of the time. When small disruptions happen, they are largely not visible to anyone except our teams who ensure that our products are up and running. However, like all complex systems, sometimes larger outages can occur, which may lead to disruptions for both users and website creators.

There are so many interesting nuggets of information in this debrief which supports some commonly held beliefs around SEO and debunks others. h

1) The indexation part of the algo is DISCONNECTED from the crawling algorithm. Clearly Google did not stop crawling new pages – they were just not getting pushed to the index.

2) The index is static – it needs to be “pushed” to datacenters and is not constantly in flux as many think. The index does not get updated on the fly, it is a static library that needs to be refreshed at intervals.

3) Search algorithms do change frequently far more than publicly disclosed or theorized on social media. Google was pushing an update on April 5th that did not coincide with a known algo update . Google says they update their algorithm over 500 times per year which is more than once per day. The fact that Google was “pushing” an index means that the algorithm that accessed that index would also have to have been adjusted.

4) Search console is a live look at data in the index – it wouldn’t have broken if it was disconnected from the index. Search console is one of my favorite search as it gives a look into data that we could not possibly see from any other source.

5) There is a “duplicate management system” as a part of the indexation algorithm NOT crawling. This explains why duplicate content with a canonical can rank for short periods of time if this process is not run in real time.

6) Google really does want websites to be successful in their index and to that end they try to give as much information as possible to optimize sites for Google search. In the battle for search visibility, there is no US vs THEM both Google and website owners want the same thing – to satisfy users.

SEO manager
SEO

The SEO Job Outlook: Everything you wanted to know

 Over the last year I noticed that the velocity of inbound recruiters pinging me about SEO roles had been increasing. I was curious whether there are actually more SEO roles or am I just “lucky.” To understand the total landscape a bit more I turned to LinkedIn Sales Navigator to do some deeper research on what recruiters are looking for and who might be qualified for these roles. 

Most of this post will be just facts and figures which underscores my gut hypothesis that there is suddenly a new found interest in SEO and the labor market is tight. 

Note: all numerical research is for the US only unless I specifically call out another location.

There are currently 47.5k job openings that match the keyword “SEO” and on the flipside there are only 909k people that match the same keyword. Taking these numbers at face value, this means that there is one job for every 20 people that match this keyword at the most broad level. 

To get a sense of the context of this ratio, I looked at the same metrics for some other job roles.

For “software engineers” the ratio is 10:1. 

For product managers, the ratio is 30:1.

For designers, the ratio is 23:1

With these numbers as a baseline, SEO appears to be high in-demand but not as in-demand as software which isn’t surprising. However, SEO is a very general keyword that could match anyone that remotely has anything to do with search engine optimization, and in reality the ratio could be even lower. 

SEO Specific People

Again using Sales Navigator, I broke down the people in the SEO category even further. First off,  of the 909k people that matched SEO only 26.5k had the words “SEO” in their title. Digging into just this subgroup, the results were quite fascinating:

  • 1,166 changed jobs in the last 90 days which a turnover ratio of 4%

Job role

  • 8,588 work in marketing and advertising
  • 3,677 work in internet
  • 2,002 work in information technology
  • 1,054 computer software
  • 806 writing and editing

Function

The majority of these people (24k) work in marketing with the rest (1.6k) working under engineering.

Company size

The bulk of people are at smaller companies:

  • 864 are self employed
  • 4347  work in companies with 1-10 employees
  • 4954  work in companies with 11-50 employees
  • 3256  work in companies with 51-200 employees
  • 1801  work in companies with 201-500 employees
  • 1077  work in companies with 501-1000 employees
  • 1837  work in companies with 1001 – 5000 employees
  • 611  work in companies with 5001-10,000 employees
  • 1836  work in companies with more than 10,000 employees

I found the data around seniority and years of experience to be the most fascinating and where there is a real mismatch between hiring managers and the labor market. 

Seniority

  • Owner 2,265
  • Director 2,138
  • Manager 1,405
  • Senior 11,611
  • Entry 16,081

While the prior data is titles which might not always align with skills, the years of experience actually requires passage of time. (I excluded people without a year of experience).

Years of experience

  • 6,550 have 1-3 years of experience
  • 3956 have 3-5 years of experience
  • 4862 have 6 -10 years of experience
  • 3731 have 10+ years of experience

Looking back at what recruiters are looking for, 12k of the current job openings want someone “mid-senior” level and above. Even if we assumed that anyone with 3 years of experience might meet that bar, the ratio of qualified candidates (based on the numbers above) to job openings is essentially 1:1! 

With this data in hand, this would definitely explain an uptick in demand for SEO. 

Here are some other interesting findings that came up from this research:

  • There are 5k people that have the title of SEO in California
    • Only 1,400 are in the Bay Area
    • There are 1,065 job openings in Bay Area!
  • There are 3k people in NYC that have the title of SEO 
    • There are 1,700 job openings
  • Here are some other US states and their population of people with SEO in their title:
    • Texas: 1,700
    • Florida: 2k
    • Illinois: 972
    • Washington:426
  • Internationally, here is the breakdown of people with SEO in their title
    • 8k in the UK
    • 8k in Pakistan
    • 57k in India
    • 4k in Spain

In short if you are looking for an SEO role and have experience, this is a sellers market. Know who your competition might and definitely don’t underprice yourself.

If you are looking to hire for SEO, you are up against many other companies that suddenly have the same need to fill a role. Prepare to break out all your tricks to recruit skilled SEO managers to your team.

SEO

Is SEO Marketing?

In most organizations, SEO is typically considered a marketing function but should it be? The answer to this depends on the SEO needs of the company.

In a smaller organization or a company who’s primary online presence has less than 100 pages SEO efforts will primarily focused on the content aspects to organic marketing:

  • Keyword research – the research and decisions on which keywords to use will drive a lot of the content and marketing on a website
  • Content creation which can span multiple areas
    • Blog posts
    • Marketing landing pages
    • Product updates – content about new products and created to promote these products
    • Feature pages – marketing the features of products
    • Home page content – typically a heavy branding effort, but requires keyword research to choose target keywords
  • Meta data creation – Using SEO best practices in the title tags, meta description and headlines. While it’s a key responsibility of an SEO team, the primary skill after keyword research is essentially copy writing
  • Backlink efforts /PR

All of these functions are very common marketing efforts and therefore make the most sense for SEO to sit in a marketing team. In this capacity the SEO individual (team) will be sit alongside content writers, paid marketers and PR. Reporting into a marketing leader, they will focus on the free search aspect counterpart to what the other team members do.

SEO for larger sites and companies

However for larger sites where much of the SEO will typically be product or engineering related, having the SEO team sit under marketing could make it challenging to both implement changes and be aware of challenges. The types of things the SEO team will work on in these companies will be more focused on the following than just content (which will also be a part of their job):

  • Migration planning and implementation – large sites will always be moving something as priorities change
  • Product build outs – these could consist of microsites, launches of new products or simply a constant cadence of refreshes and rebrands. In larger companies the product silos will be larger and every PM will have a preference to how they showcase their product
  • Architecture and taxonomy – the larger the site the more organization matters. Think of how clean the architecture of an ecommerce site might be and how terrible it could be without a lot of though paid to taxonomy.
  • International – This usually requires one person alone to focus on as there will be many moving parts that are driven by translations, currency, and even domains – all product related.
  • Competitive research and reporting – the more complex the company might be, the greater the need for customized SEO reporting might be.
  • Education – In larger organizations there is a need to have constant education as there will always be new people unaware of SEO requirements that have the potential to affect organic traffic.

In these larger sites, it might make sense for the SEO team to either be a part of a product team or even better on a growth team which sits between product and marketing.

From my experience, SEO has been most effective when it is in the same reporting line as product, but most companies actually put SEO within marketing.

In short, is SEO marketing? It really is contingent what the key levers the SEO team will be using to drive growth.

1 2 4 5 6 7 8 15 16