Eli Schwartz

Eli – Page 4 – đź“ŚEli Schwartz
Posts by


Home / Blog Archive

HTML Sitemaps for SEO – 7 reasons you need one

A sitemap guides your website visitors to where they want to go. It’s where they turn if they haven’t found what they are looking from those dropdown menus.

Beyond helping your visitors navigate your website, which should be the primary focus of any marketing effort, there are many other reasons to use a sitemap.

First, it’s important to understand that there are two types of sitemaps:

  • XML sitemaps
  • HTML sitemaps

What Are XML Sitemaps?

XML sitemaps help search engines and spiders discover the pages on your website.

These sitemaps give search engines a website’s URLs and offer data a complete map of all pages on a site. This helps search engines prioritize pages that they will crawl.

There is information within the sitemap that shows page change frequency on one URL versus others on that website, but it is unlikely that this has any effect on rankings.

An XML sitemap is very useful for large websites that might otherwise take a long time for a spider to crawl through the site.

Every site has a specific amount of crawl budget allocated to their site, so no search engine will simply crawl every URL the first time it encounters it.

An XML sitemap is a good way for a search engine to build its queue of the pages it wants to serve.

What Are HTML Sitemaps?

HTML sitemaps ostensibly serve website visitors. The sitemaps include every page on the website – from the main pages to lower-level pages.

An HTML sitemap is just a clickable list of pages on a website. In its rawest form, it can be an unordered list of every page on a site – but don’t do that.

This is a great opportunity to create some order out of chaos, so it’s worth making the effort.

Why You Should Leverage HTML Sitemaps

While you may already use an XML sitemap – and some insist that an HTML sitemap is no longer necessary – here are seven reasons to add (or keep) an HTML sitemap.

1. Organize Large Websites

Your website will grow in size.

You may add an ecommerce store with several departments or you may expand your product portfolio. Or, more likely, the site just grow as new people are added to a company.

However, this can lead to confusion for visitors who are then confused about where to go or what you have to offer.

The HTML sitemap works in a similar way to a department store or shopping mall map.

The sitemap is a great way for the person maintaining the sitemap to take stock of every page and make sure it has its rightful home somewhere in the site.

This is the directory for users that can’t find the pages they are looking for elsewhere on the site and, as a last resort, this should help them get there.

2. Serve as a Project Manager & Architect

Think of the HTML sitemap as an architectural blueprint for your website.

The sitemap becomes a project management tool. It oversees the structure and connections between pages and subpages.

It’s also a forcing function to make sure that you have a clean hierarchy and taxonomy for the site.

A good sitemap is like a well-organized daily schedule.

As any busy person knows, there’s a big difference between an agenda that has every meeting popped on at random or those that are themed and organized around time blocks.

In either case, an agenda is still an agenda but an organized one is far more useful for everyone.

3. Highlight the Website’s Purpose

As a content-based document, the HTML sitemap serves as a way to further define your website’s specific value.

Enhance this benefit by using SEO to identify the most unique and relevant keywords to include on the sitemap.

Anchor text is a great way of creating keyword relevancy for a page and for pages without many cross-links, a sitemap is an easy alternative to use choice anchor text.

To understand the power of anchor text alone, look at the search results for the query “click here”:

7 Reasons Why an HTML Sitemap Is a Must-Have

4. Speed the Work of Search Engine Crawlers

You want to help those search engines out in any way you can and take control where you can. The assistance includes finding your content and moving it up in the crawl queue.

While an XML sitemap is just a laundry list of links, HTML links are actually the way search crawlers prefer to discover the web

The HTML sitemap helps call attention to that content by putting the spotlight on your website’s most important pages. You can also submit the text version of your sitemap to Google.

5. Increase Search Engine Visibility

With some websites, Google and other search engines may not go through the work of indexing every webpage.

For example, if you have a link on one of your webpages, then search bots may choose to follow that link.

The bots want to verify that the link makes sense. Yet, in doing so, the bots may never return to continue indexing the remaining pages.

The HTML sitemap can direct these bots to get the entire picture of your site and consider all the pages. In turn, this can facilitate the bots’ job and they may stay longer to follow the page navigation laid out for them.

Not only does a taxonomy and hierarchy help users find themselves, but it’s incredibly important for search crawlers, too. The sitemap can help the crawlers understand the website’s taxonomy.

There is no limit to how big a sitemap can be and LinkedIn even has a sitemap which has links to all of their millions of user pages.

7 Reasons Why an HTML Sitemap Is a Must-Have

6. Enable Page Links in a Natural Way to Drive Visitors

Not every page will connect through a link located in a header or footer.

The HTML sitemap can step in and find these ideal connections that address how visitors may look for things.

In this way, the HTML sitemap can reflect a visitor’s journey and guide them from research to purchase. In doing so, this benefit of HTML sitemaps can raise the organic search visibility of these linked pages.

In this instance, the sitemap is the fallback that ensures that there is never a page on a site that is orphaned.

I have seen huge gains in the traffic of sites that had issues with deeper pages not receiving many internal links.

7 Reasons Why an HTML Sitemap Is a Must-Have

7. Identify the Areas Where Site Navigation Could Improve

Once your website grows and you develop more pages, there may be duplicate data, which can be problematic for a search engine.

But, after mapping everything out, you’ll be able to use the sitemap to find the duplication and remove it.

As an aside, this only works if there is an owner of the sitemap that is looking at the sitemap on a semi-regular basis.

Also, when you apply analytics or heat map tools, it may conclude that more visitors are using the HTML sitemap than use navigation.

This is a clear signal that you need to reassess why this is happening if the current navigation is missing the mark.

It’s important to determine how you can change the site architecture to make it easier for visitors to find what they need.

For all these benefits, you’ll want to maintain an HTML sitemap. These benefits save resources (time and money). They also deliver an effective way to guide your website visitors to what they need and help close those sales.

Getting Started

If you don’t have an HTML sitemap but do use a platform like WordPress, I recommend one of the many sitemap plug-ins. The plug-ins automate much of the sitemap development and management process.

For larger sites, it might take running a web crawl like:

The output of this web crawl should then serve as the basis for organizing all of a site’s pages around themes.

After developing the HTML sitemap, don’t forget to put a link on your website that is easy to find.

You can either put the link at the top, as part of a sidebar or in a footer menu that continues to be accessible as visitors move from page to page.

However you look at it, an HTML sitemap is an easy way to get huge benefits without a lot of effort.


The power of backlinks for SEO

In the early days of the Internet and search, Google differentiated themselves from the other search engines by focusing on quality signals to determine relevancy for a query. Amazingly, the other engines – and yes there were lots of other engines – completely ignored quality and looked at keyword matches to pages in the index.

The primary quality signal that Google used to determine quality is the value of the links that pointed to a specific page or website. The value passed by those inbound links is calculated by the value of their own links and on and on it goes. From Google’s perspective, the Internet is a true web of pages linking to each and connecting all pages together.

Linking understood

Google modeled their ranking algorithm like a traditional academic authority model. An academic paper with a new idea is considered to be more authoritative if it has a large quantity of citations discussing it. At the same time the quantity of those citations has to be qualified by the quality of the citations, so a paper cited by a Nobel laureate would be more valuable than one cited by a high school senior.

Moving this model over to the web Google used the same calculation to value the web. A website that has a link pointing to it from Stanford University would in theory be more valuable than one that only has a link from Kaplan University. It’s not that Google recognizes the Stanford is a highly reputable university with a higher caliber of education than Kaplan because of the Stanford “brand”, rather Stanford has more authority because it has a higher quality of other websites that link to it than Kaplan.

Furthermore, quality is not created by a website alone, the linking page also must have its own authority which is calculated by the internal link flow as well as any external inbound links. From this respect, a link from the Kaplan homepage to a website is likely more valuable from a link standpoint than a private student’s blog on the Stanford domain.

Viewed holistically in this manner, the idea of .edu or .gov having more link authority than a .com is completely false as every domain has to stand on its own within the web based on its own backlinks. It is likely that a .edu or .gov will have more link value to share, but there is no guarantee. Just to underscore this point, Google knows that whitehouse.gov is the most valuable government website not because it is the website but because it has the highest value of incoming links.

Manufactured linking

While Google claims to view hundreds of factors in determining rankings, links have always had a very prominent part of the calculation. The non-link factors are lot more mysterious, but on its face, this link calculation algorithm seems very simple to manipulate. High value links will pass an extraordinary amount of value and help the linked page rise in search rankings.

As a result, almost from the day Google launched its index, huge economies sprung up to help marketers manipulate their rankings via artificial rankings. On the cleaner end of things, there were reporters or websites willing to accept compensation in exchange for a link placement while on the dirtier end there were botnets designed to hack websites just to place links.

In between these two options, there were brokers that assisted websites in finding the perfect place to purchase a link on a permanent or even temporary basis. Up until 2012, all of this link manipulation was remarkably effective and websites that spent vast sums on link building saw their websites dominate valuable positions in Google. But this is not the way Google had been conceived to work. Websites were not supposed to just be able to spend their way to the top of rankings when Google really wanted its index to focus on user experience and relevancy.


In 2012, Google released their Penguin algorithm update whose sole purpose was to identify manipulative linking schemes and demote the recipients of the links. When possible, Google nuked entire link networks bringing down sites that linked as well as the sites receiving the links.

For the first few months and even years after Google unveiled this algorithm update, websites were terrified of having their previously undiscovered link building efforts be revealed and lead to a penalty. Sites frantically posted disavow files to Google where they disclosed the shady links they may have had a role in acquiring. Out of fear, websites even disavowed links proactively that they had nothing to do with. This algorithm update gave rise to the concept of negative SEO where a malicious person could point dirty links at a website and then watch Google penalize the receiving website for having dirty inbound links. (Note: Google claims this is not possible, but there are many case studies of negative SEO working).

It has now been almost 8 years since this algorithm update and link buying activity is once again picking up. Websites have become more confident in their abilities to evade Google and use these links to accelerate their SEO efforts. This time it is now called guest posts or sponsored posts rather than outright paid links

Google is smarter than you think

In my opinion, most of the time this is a completely wasted effort not because they will be caught by Google, but because the links just don’t work. What many might forget about Google is that it is a company driven by machine learning and artificial intelligence. Outside of search, Google’s Waymo has driven more autonomous miles than anyone else working on self-driving vehicles. To date, in the 5 million miles driven by Google,  we have not heard of any serious injury or fatality caused by Waymo which means that Google has AI that is good enough to make life and death decisions. This challenge is light years more complex than ranking search results.

In many instances a human reviewer can quickly identify a pattern of artificial links which means that Google’s AI can likely do the same. A website might not get penalized when their artificial links are discovered, but the links themselves will just be discounted from the ranking algorithm. The net result is that any resources expended in acquiring the links was completely for naught.

Link solution

If links are an important component of SEO and they can’t be manipulated, this might seem like a dead end for a website looking to increase rankings. Fortunately, there is a solution and it is one that Google recommends: Build a brand. Brands don’t build links, they get links.

Brands in search

Google has been accused of favoring brands in search, and that should be true simply because users favor brands! Just like in a supermarket we gravitate to the branded products the exact same happens on a search results page. As in the earlier explanation about the value of university links, Google doesn’t give a brand extra credit for being a brand; rather, they recognize brands because they have brand characteristics.

Building a brand on the web is not an easy feat, but the first step is to think like a brand. A brand like Coca-Cola doesn’t seek websites to link to them because they know if they create well designed products, refreshing beverages, and launch campaigns the media will talk about them. A brand focuses on its core product offering first and then seeks to get attention. A non-brand seeks to get attention so they can one day have a great product.

Focus on the right goal

Focus on the product and let marketing tell that product story. The byproduct of that story will establish the brand and lead to links which will reinforce the brand. This does not have to be done without help. Brands use PR agencies to tell their stories, and any company aspiring to be a brand can do the same.

There are amazing PR agencies that are familiar with SEO who can ensure that there are links within promotional campaigns, but the PR is the focus not the link.

Links are and always will be a part of the ranking algorithm, but think of the algorithm like the smart human Google intends it to one day be. If a human could easily detect an unnatural link the algorithm likely could too.

Instead of using precious resources to build those unnatural links instead deploy that effort to build a brand which attracts the links that lead to rankings. A clever infographic designed to inform, a media campaign, a billboard or a unique approach to data can all be used to generate the buzz that leads to links and eventually coveted rankings. Don’t focus on the means to build a brand on search – and instead view the links as the byproduct of brand building Google has always intended for it to be. Links are just a piece of the algorithm designed to inform Google about authority that should already exist.



The tactics and strategies for SEO are very similar whether the target customer is a consumer, business, non-profit or government. In all cases, the goal is to maximize organic visibility for whomever is looking for this particular website or content. The means to achieve this goal are always the same.

The real difference in strategies between these various end users is the expectations of how SEO will perform and what kind of content should be optimized. Generally, SEO is much higher in the buyer’s funnel than other sources of traffic, however for consumers it is far more likely to have a conversion happen in the same session as the organic click.

This is the breakdown in the types of content that should created and the expectations for each type of user. All of these buckets will be fairly broad as a consumer could be a teenager looking for an idea for a high school paper or a high net worth individual seeking a financial advisor. The same principles apply to B2B that is a sole proprietorship buying for the business as a Fortune 10 company.

With that in mind these are the buckets:

  • Consumer – Usually buying for themselves so there will be less decisionmakers and therefore the buying process is quicker. The consumer wants their information they were seeking and if there is purchase intent, they want to be reassured that the purchase is worthwhile. Content for a consumer should be conversion oriented.
  • Business – At any medium or large company there will be lots of decisionmakers so the goal of the content should be to get the search user to become aware of the brand. Content should be written to get the user to search more or share information to be added to a database. The SEO efforts may have to aim a bit lower to have users join a webinar rather than to buy products or follow the brand on social.
  • Non-profit – a non-profit functions like a business in their buying behavior except they may be more budget conscious. Keep in mind that a non-profit cold range from a small local PTA to a global organization like the Red Cross. Don’t make any assumptions.
  • Governments – are like businesses with the clock rewound to a century ago. The content for government buyers has to establish the business as an entity worth continuing to explore and should focus on building internal advocates. Governments can range from local cities all the way up to national federal agencies. Understand the buying process and target those users.

Having the right expectations before embarking on any optimization effort will better help all stakeholders manage their time and resources as they invest in SEO. Too often B2B SEO campaigns fail because there was an expectation of instant conversions. Knowing that SEO for B2B, government or non-profit is there only to assist other channels be more successful will go a long way in having SEO efforts that everyone is on board supporting.


Competition and SEO

Competition and competitors are always a touchy subject in any business strategy, but things could get really heated in the digital marketing realm. Unlike in the offline world where attacking a competitor costs a pretty penny and is very visible, online there are many areas of opportunity to unseat a competitor without breaking a sweat.

In the bucket of fair competition, websites can create and promote head to head comparisons, bid on a competitors brand name or call out their competitors in their paid marketing. But, then there are there are the dirty tactics too like negative SEO attacks, snitching on competitors to Google, clicking their ads to drive up their marketing or scraping their content.

All of this adds up to really nasty competition to a site which in many cases is just a faceless entity. This aside, I think many sites approach competition the wrong way with regards to SEO. While websites might think of their competitors as those that have similar product offerings to them, for SEO it is really any site that is targeting the same search terms for whatever purpose.

Who is the competition

In this sense, Wikipedia is just about everyone’s competition even if Wikipedia isn’t selling any products. Similarly any other site that is providing information that might satisfy a user’s query intent should be added to the competitive set. All sites that are in this competitive set should monitored or at least observed on occasion to see how they are growing and any specific tactics they are using to drive growth.

Learning from the competition

Rather than focus on destroying competition, I think it is best to learn from competitors. If there is something that is working for them learn how to do it better. If they are generating traffic from a specific query set but not effectively answer the query’s intent, this is an opportunity to create better content.


In the same vein, if a competitor has created content within a specific topic but left open large gaps in their coverage of the topic, this is an opportunity to do it better. A competitor might have a broad approach that can be better capitalized on with a far more narrow strategy. For example, a pest control product site might be able to beat a competitor by having detailed how to’s on using a product rather than general awareness content.


Observing how and why a competitor receives links is another great strategy for growth. Provided that a competitor is accruing links in above board fashion, trying to understand the intent behind why someone might link to them can lead to even better ideas on where to find new links for your own site. If a competitor is using illegitimate link tactics this might be reassuring that they are probably similarly weak in other areas of the business.


To dig and learn from competition, I use three primary tools.

  1. Google search – search their site on Google using site queries (site:domain.com) to see how they come up in Google. Observe how many pages they have, title tags, meta descriptions, rogue pages they probably did not want indexed, images and content strategy.
  2. Backlink tool – Any tool will suffice, but I like Ahrefs. Use this tool to dig into their backlinks, top keywords driving traffic, recent trends on performance and any similar sites to them you may not have been aware of before.
  3. A crawling tool – Any cloud or desktop tool will do, but this is where you really learn about how their site is structured and any deeper learnings you could not find on Google yourself. Note, that many sites do not appreciate being crawled for this purpose, so do this at your own risk.

For SEO, competition is not a bad thing at all and it is innate in how search works. Even if you are the only one that has ever offered your service you will inevitably face competition since there will never just be one result on Google. Use competition to shorten your learning curve and develop better strategies. Viewed in this light competition should be welcomed not avoided.


SEO project management and workflow

Managing SEO as a product means that SEO asks will have to fit within a typical product prioritization process. In many organizations, product requests must be accompanied with detailed information that would allow a product manager to stack rank any request against any other priority.

The stack ranking will typically have the ask as well as details which would help them to calculate the resources and time they might need to complete the request. While every organization might have its own format,  here’s a format I have found to be incredibly useful.

Spreadsheet explained

  1. The first column has a quick summary of the ask
  2. The second column goes into a bit more detail on why it needs to be fixed. This should be explanatory enough that someone could understand it just by reading the spreadsheet.
  3. This column should explain the fix as well as any alternative options. This will give the product manager all the information they need on how to go about assigning the request to any stakeholder
  4. This column scores the impact of the fix on a scale of 1 -10 with 10 being the most impactful
  5. Next I score the effort related to making the fix on a scale of 1 -10 with 10 being the lowest effort.  A ten might be a quick text fix while a one could be a full rebuild
  6. As with anything SEO related, there is a certain amount of guesswork that goes into planning so in this column I score the confidence of the impact and effort on a scale of 1-10
  7. After all these three columns we can get to the stack ranking by adding up the scores. A higher score is the most impactful, low effort and high confidence of success.
  8. The next few columns will be for tracking and coordination. Column eight will have additional notes not captured previously.
  9. This column will record a bug ticket, so anyone looking to follow along on progress can know where to look.
  10. Column ten will have the date it shipped to engineering which is very helpful for bunching work into quarters.
  11. This column shows the assigned person so anyone checking on progress knows who to talk to.
  12. The rest of the columns are additional notes that are very helpful for future tracking
    1. Completed dates
    1. Notes

Too often SEO requests are ignored or not assigned because there is not enough clarity on what is being requested. Using a detailed spreadsheet like this or whatever is more comfortable for company culture will ensure that SEO asks follow the same model as anything else that will come in front of engineering or product teams.

Having a detailed spreadsheet such as this one are also very helpful to have handy when there’s a sudden need to share progress with an executive and there’s a clear list of what has been accomplished or where things are in the pipeline. Additionally, this document is great to hand over to other employees when the SEO person moves on from a company.


Product led SEO

This blogpost was written in 2019 and I have expanded the thesis of this blogpost into my book with the same title: https://www.amazon.com/gp/product/B091D572ZL

The idea of “product led growth” upends the whole premise of  marketing a product to promote adoption and instead focuses on getting a great product into the hands of users who become the marketing agents.

In this paradigm, users of the product will the love it so much that they will share it with others in the company and/or their social circles. There may be innate triggers within the product that encourage sharing and thereby force the hands of the original user to share with their networks.

Notable recent examples of companies that were incredibly successful at this are Slack, Dropbox and Zoom which become more useful for individual users as it is adopted within a group. Each of these companies deprioritized traditional marketing and instead focused on building an amazing product that users would just have to share with their networks. This idea has worked amazingly well as each of these companies grew themselves exponentially without spending any substantial funds on early marketing.

When product led growth is successful, the company will have acquired a large segment of users to learn from and then build marketing strategies around. The company can learn what caused the product to be adopted by other teams within a company – naturally and then use marketing to encourage this adoption process to be more deliberate.

SEO led product

I am a proponent of using the same approach for SEO efforts. Too often, SEO efforts begin too simply with just a group of keywords. These keywords are developed by the marketing team or founders based on their own knowledge of the product. These keywords become the stems of keyword research input into any keyword research tool and the output is ideas of related keywords.

This new longer keyword research list becomes the seed for content ideas that will be written and posted on the website. The keyword list becomes a sort of checklist and content road map which doesn’t change much based on actual performance or real time metrics.

Keyword research and SEO efforts don’t end with just a content plan, but it also is incorporated into the product. The very names of the products are determined by keyword research on what a user would be most likely to insert into Google as a query in search of a product. On many occasions using keyword research for this purpose can be very inadequate especially when demand is low or not yet existent for the product, but still it is forced to suffice.

The downside to SEO in this process is that there’s no room for a user feedback loop and much of the content creation is all too manual. Content intended to match keywords is written in somewhat long form and the full library will only scale as fast as the content producers can write. The greatest gap in this approach is that the SEO strategy is laser focused on specific keywords and an expectation of generating a high position on just those keywords. Ranking on targeted keywords, aside from the vanity aspect, is aspirational and may never even be achieved.

Product led SEO

Flipping this script to product led SEO, instead of using SEO to market the product – the product becomes the SEO driver. Many of the most successful websites on the Internet have achieved their organic dominance through this approach. Rather than relying on keywords and content as the bedrock of their SEO efforts, they used a scaled approach which relied more on product and engineering than marketing. Amazon, TripAdvisor, Zillow and even Wikipedia are great examples of product led SEO, but there are thousands of others.

For each of these four companies, before they developed and propagated their product on the Internet there wasn’t even keyword research for them to rely on.

  • Amazon focused on building a great architecture to support a well-indexed site even before the idea of SEO existed. Their site has grown into the SEO magnet it is today by scaling that initial iteration of a well developed product that fit with SEO principles. Had they relied on keyword research to launch the site in the early days of the Internet, they may have over-prioritized the adult keywords that were so popular at that time to the detriment of book ecommerce that no one was yet searching.

  • TripAdvisor didn’t start by creating a “blog” of reviews of the most popular hotels with the most search volume. Instead they built an architecture that could scale and host reviews for every property in the entire world. It may have taken years before they outranked the individual travel blogs and sites that ranked highly on search for the most popular properties, but their reward is that today they rank in the top five of results for every hotel in the world.

  • Zillow didn’t focus all of their SEO efforts on trying to rank for the popular keywords in their space which may have been words like “home value” or “online realtor.” Instead they poured their efforts into building a colossal site which has a page for every single address in the United States. At the time, that may have seemed like a foolish approach as a) no one looked for specific addresses b) they would be competing with Google maps or even Mapquest which was still a thing. Looking at their footprint now where they are visible for every address in the country, they could not have made a better bet.

  • Wikipedia didn’t set out to be the online encyclopedia for what most people were looking for; rather, they set out to be an encyclopedia of everything. Early in their process, having an entry for everything might have seemed like an absolute impossibility. They disregarded the naysayers and built the product that could continuously scale into a repository of everything in every language. There are gaps in their knowledge base and there probably always will be, but they have been undeniably successful at achieving the goal.

Scaling product led SEO

Just like product led growth, the feedback loop drives future iteration of the product. Knowing what resonated on search with both search engines and users will dictate the future roadmap of improvements and adjacent products.

Amazon does not need to make the same SEO leap of faith as they enter new categories as they can be very confident that they will get organic traffic on anything they launch. TripAdvisor’s success in hotel reviews gave them a playbook on how to launch attractions and things to do products. Zillow’s dominance in address search opened up the pathway to organic visibility in the mortgage vertical which had always been one of the most competitive categories on the web.

The twenty year winner

With the clarity of hindsight, product led SEO will always be the clear winner, but it is undoubtedly challenging to envision the success you might see from this approach when first starting an SEO effort. It is my opinion, that there can be a product led SEO angle in every vertical and niche.

Aside from the likely monumental work by engineering teams to build a product that will eventually become an SEO juggernaut, you will need a great degree of patience and even faith. At the outset it will not be clear that there will be demand for the eventual product but remember there was no data that supported the eventual SEO goals of Amazon, TripAdvisor and Zillow either. The best selling point for product led SEO is that after companies are successful with their product first efforts they will eventually also dominate organic results for the most coveted category keywords too.


Duplicate content is not a penalty

In the realm of worlds where people think that Google is out to get them, there is a popular idea of a duplicate content penalty. As a result, there is a level of unjustified paranoia around ensuring that all content is unique and even efforts made to spin duplicate content into something else. There is an even an obsession with blocking pages to Google’s crawlers if content is not unique.

In fact, duplicate content is an issue but there is no actual penalty applied to anything that is deemed to be duplicate. From a user perspective, Google wants to make sure that all content in the search results is completely unique to other results, so a user doesn’t see a results page with 7-10 listings of the exact same content. This could be content from the same site or even across different sites.

Therefore, when Google identifies duplicate content, they have an algorithm which determines the canonical version of that content. In their analysis they take into account any canonical listings in the source code, but there is no guarantee that they will agree.

As they determine canonical levels of content, they look for authority, user experience and what algorithmically seems like best overall fit.

Provided that the content is not a doorway page intended to trick Google into ranking a page undeserving of being ranked. (City/state pages for example) then duplicate conent is not harmful.

Having duplicate content on a site is in no way an issue that could hurt a website and it should not be avoided. Duplicate content can come in many forms and in many cases it can be very valuable for users. For example, product descriptions are usually sourced from manufacturers and are duplicated across all websites that sell that product. There is no reason to avoid hosting this content or go through the extra effort of changing a few words so it is unique.

As another example, wire news services like the Associated Press or Reuters have their news syndicated across many media sites. If a website such as CNN.com or the NY Times would not include this content, they would be doing their users a disservice.

When it comes to how Google ranks this duplicate content in both of these examples, they will choose the website that best matches the user’s query and allow the duplicate content to rank on the query. Depending on the query, a user may see a product page on Amazon while another user would see Walmart.com in the first position for their query. Query modifiers like “near me”, “reviews” or “free shipping” could be determinants that drive that visibility.

In short, duplicate content if it otherwise fits the overall purpose of a website and was created for users does not need to be avoided. As with everything related to SEO the overarching principle should be whether something is good for users and if it meets that bar it is  perfectly safe to use it.


Interviewing SEO candidates

Once you have decided what type of SEO person you want on your team, you will have a better sense of the skillset you want to ensure they have. With this in mind you can begin crafting an interview process and questions to ask during the interviews.


Any SEO candidate should always meet with all of their potential counterparts even if they are not on the same team. Interviewers from these respective teams should assess both hard skills – can they do the job and soft skills – will they be able to work with them. The number of actual interviewers will depend on the norms for a company, but if a large number of interviews is standard, here is my take who should assess SEO talent.

  • Product – Meeting someone on a product team is mostly applicable to hires who will focus on product and technical aspects of SEO. For these hires, product will usually act as a hub between all the different stakeholders and therefore the product person should interview the SEO candidate as if they are being added to their team. They should probe on a variety of soft and hard skills.
  • Business intelligence or data science – this is the team that will be responsible for reporting out on SEO metrics and more than likely will have to build measurement tools. This team should assess the SEO candidates analytical abilities as well as whether they will be easy to work alongside.
  • Marketing counterparts – It’s always helpful to have potential team members confirm that they are able to work with a new hire and those that will be working the closest with them should participate in the interview process. If the company is large and having many marketers meet with candidates is not feasible, at a minimum the person responsible for paid marketing should meet with all potential SEO hires. SEO and paid marketing are very similar from a performance standpoint and the paid marketer would be best positioned to assess their search skillset.
  • Content – Depending on the company, an SEO team might own the content team, be on the content team or just work adjacent. Regardless of whether the SEO candidate a technical, product, content or link hire, writing is a critical skill. The interviewers should assess the candidate’s writing abilities and soft skills on how they communicate with writers.
  • Senior executives – This is not applicable for all hires, but senior hires should be assessed on whether they have executive presence. Will they be able to communicate with executives directly or do they need to go through their manager? Most importantly around the executive interviews is that SEO should be considered a mission critical process within a company and having executives sign off on SEO hires keeps that sense of mission on the radar at the highest level.
  • Engineering – front end – Front end engineers will be building and fulfilling the requests from the engineering team. These interviews will determine the candidates abilities to communicate their requests to engineers and their abilities to make complete asks that don’t require engineers to continuously request more details. All SEO candidates except those on the link building side should meet with engineers.
  • Engineering – back end – Back end engineers are responsible for building server side code as well as handling any redirects. Only the technically minded SEO candidates need to be assessed by back end engineers and the questioning should probe how much they understand about various technology and tools.
  • QA – Any technical or product centric SEO needs to be detailed oriented and no one is better at fleshing out details than quality assurance employees. The interview should focus on whether they are a big picture thinker or can they think very granular about how things might work.
  • Design – Meeting with design is more of a soft skill interview to see whether design can get along with SEO. Rarely will SEO hires have design talent or designers have SEO talent, so they will need to work together very closely as they build sites and pages.
  • Sales – for an SEO hire that is primarily going to be focused on link building, it would be ideal to have someone from sales assess whether the candidate actually has strong sales abilities and instincts. This can be a very short interview and this can be a quick first impression on whether they have the communication abilities to make people act.


As these are a lot of people to meet, the process should be broken into at least four stages.

  1. Recruiter screen – a recruiter can ask questions about resume experience and ensure that the candidate uses the right language to proceed through the hiring process
  2. Hiring manager screen – the potential manager of the SEO hire should talk to the candidate either in person or on the phone. Aside from questions around how they will work together, the hiring manager should determine whether the candidate will be able to add value with their SEO skillset or will the manager need to fill in a number of gaps.
  3. Provided that the candidate passed the first two stages they should now be brought onsite. On site they should meet with product, content and an engineer.
  4. If they pass the first three interviewers, they should then meet with second round which should another engineer, marketing counterparts, design and business intelligence.

Stages three and four can come on the same day, but if possible, they should be split to save time for both the hiring organization and candidate if there isn’t a mutual fit. This gives the first round of interviewers time to circle up and discuss the candidate before forcing them to spend even more time with a company that may not hire them.

A bad hire is always more costly than not hiring someone, so although this may seem like a lot for one hire, SEO can end being responsible for most of a company’s revenue and the right fit will make all the difference in the world.


SEO reporting – Google Search Console is all you need

The primary success metric for SEO is whatever the business uses to judge any other marketing channel. Therefore, regarding reporting on SEO, there doesn’t need to be a specific SEO tool to prove that SEO is working. Either there is sales/revenue/leads attributed to an organic channel or there isn’t.

There should of course be an analytics tool in place which can show whether traffic is contributing the goals of the business, but this tool should already be in place for every other channel. Ideally, the tool should use weighted multichannel attribution as in our fast-paced world a customer will always have multiple touches with a website and credit should be shared across the whole journey.

Even without a full multichannel attribution setup, there should still be indicators that SEO is driving the business forward. If it is too difficult to attribute a sale to an organic source, the lead source can be tracked back to organic touches through a tool like Marketo. If that is also too challenging a simple analytics report on pages per visit or bounce will show if visitors to the website are engaged or if they just leave without doing anything.

Using traditional analytics or business intelligence is the best option for reporting SEO progress on existing SEO efforts, but this will not work as well when a campaign is brand new.

Reporting on early progress

For brand new SEO efforts, my favorite solution is Google Search Console. Google Search Console is a peek under the hood as if you can see Google’s own analytics. This is the only tool that shows queries that a website is receiving impressions on even if the searcher does not click. This is a far better option than any SEO tool which makes a best guesstimate on SEO visibility based on the millions of keywords that they crawl. Google Search Console is not guessing, these are real words that users type to see a site.

Using Google Search Console, you can see early progress on SEO even as the pages are just getting crawled by Google. Newer pages and sites not be positioned very highly in Google, but even at lower positions they will be getting eyeballs from Google’s billions of users. At those low positions they may not get meaningful clicks, but even so I think just seeing the impression count grow is the best early indicator of SEO growth.

3 stages of SEO

I like to think about SEO performance in terms of three levels.

  1. Impression – this is the first level of SEO growth, each eyeball on a URL in Google search is considered an impression. Obviously at higher positions the impression number on a keyword will be higher, but even at lower positions there will still be users that do search deeper into Google and will see the website’s listing. Impressions are indication that a website or page is in a consideration set for search and growth in this metric is positive and declines are very negative.
  2. Clicks – A click is when a user clicks a search result and go through to a website. (This number may not always match visit data in analytics or logs packages as Google is just measuring how many clicks there were on a specific listing regardless of whether the webpage even loads. Short visits will not be picked up by script based tracking systems). Clicks are a factor of impressions as well as clickthru rate. As clickthru rate from search improves, the clicks/users will grow without any subsequent change in impressions.
  3. Conversions – this is the final and most important result from SEO traffic and is how SEO campaigns should be judged. If clicks are arriving to a website from search but not converting, this is when a conversion rate optimization effort would come into play.

In summary, these are all stages in an SEO effort. Impressions means a website is on the field, and eligible to play. Clicks are hits and progress towards an ultimate goal, but what really counts is the winning that happens from conversions.

In addition to search reporting, there are other great features in Google Search Console.

  1. Coverage – there are many gaps in this reporting, but it is the only source to know how many pages of a website are included in Google’s index. On the inverse when pages are being dropped out of the index from an error or other issue this is the place to find that out.
  2. URL lookups – On a URL by URL basis, Google Search Console has the option to see whether a URL is indexed, whether Google accepts a canonical suggestion, and you can even see how the page was rendered by crawler.
  3. Data comparisons – There are now 16 months of data in Google Search Console, so there are many possible comparisons you can make with data. For example, traffic can be compared year over year or week over week and you can even drill into specific URL’s and queries.
  4. Filtering – This leads into one of the best primary features of Google Search Console. You can compare sets of URL’s, keywords, devices, countries and many more options. You no longer need to just trust aggregated charts, but you can dig into the numbers behind the graphs

With these reports, here are some important things to look at in Google Search Console.

  1. Brand vs non-brand – many people might assume that they are doing well in SEO based on just looking at the total number of organic visits coming to their site; however, what they may miss is that much of this could just be branded traffic. While brand traffic is great, it does not indicate SEO success. The click for brand arrived organically simply because the user clicked through from Google rather than directly typing the domain name into their browser. Growth of branded traffic will plateau at the natural penetration level of the brand. Branded SEO  traffic will only grow at the same rate that a brand expands its awareness. On the other hand, non-brand traffic could grow infinitely if a company continues to imbue creativity in their SEO efforts.

Knowing the ratio of brand to non-brand traffic is critical to assessing current progress on SEO efforts. For companies that have not yet invested in SEO, its not unreasonable that their brand vs non-brand ratio would be 90/10 whereas a company that has smaller brand footprint investing in SEO for many years might be closer to 20/80.

  • Comparison report – Using the comparison tool, it’s important to do frequent year over year checks on important pages to ensure that they are continuing to accrue more traffic than the prior year. The same should be done for brand and other important queries. There may not always be something to do about it, but it’s important to at least know.
  • Canonicals – As a canonical link is only a suggestion to Google, knowing whether those suggestions have been accepted by Google is very useful information. It is helpful to sort through URL’s receiving traffic to ensure that they match the expected URL’s and if there is a canonical issue.
  • Errors – Unfortunately there are many errors that Google Search Console reports on which aren’t really concerning, but there are issues that are worth addressing. Some of the ones that are important are schema, crawling and definitely anything related to robots files.

In reporting on anything related to a business, you always want to connect the outcomes to the input. HR is not measured on the quality of sales produced by the employees they hire, while sales people are not compensated directly based on the performance of the accounting team. Likewise, SEO should not be measured on how well sales teams close the deals created from organic traffic sources, but they should be tied to goals on how that organic traffic performs once it arrives on the website.

Tools which show the growth of traffic as well as the next point where that traffic goes is really all that is necessary in order to show success at SEO. Extraneous tools like keyword research, testing, and crawling tools are helpful but they don’t truly elucidate the performance of SEO.   


Site updates and migrations

Much like any offline business that likes to refresh the paint, reorganize a storefront, or renovate the same is applicable to online businesses or just the web presence of a company. However, when reconfiguring an online presence there is a lot more that must be considered especially when it comes to SEO.

Choosing a new technology vendor, folder structure, or just a homepage update will most likely have an impact in the way that Google or other search engines perceive a site. This isn’t to say that change should be avoided at all costs just to not upset the apple cart; rather, certain precautions should be included in the plan.

The biggest concern when updating a site is that search engines will no longer be able to find the old pages where they used to be and will also have a harder time finding the new pages. This will have a double impact of lost visibility on old pages and not recovering that visibility on new pages. Therefore, the goal in any update is to maintain the structure of the old and nimbly pass users and crawlers on to the new.

The best practice to achieve this goal is to set up permanent redirects from the old page location to new URL. Technically, this is referred to as a 301 redirect which will force browsers as well as crawlers to update the cache for the new URL. This is in contrast to a 302 redirect which is considered to be just a temporary redirect. Temporary redirects are very useful in passing users to a new URL as a result of a particular state (login cookie, time based, location) but the primary URL still remains the same in the memories of a browser and search engine.

Due to the complexities that might arise with a permanent redirect, a temporary redirect ends up as the default redirect option in many popular CMS (content management systems) tools. As a result, setting up this permanent redirect must be a deliberate exercise.

In theory, this redirect will also pass forward the earned equity that is acquired from external sites linking into a URL as well as internal links that used to link to the URL. Additionally, when a site is rolled over a new website or merged into an existing site this redirect should also help pass forward users as well as authority.

Steps to avoid breaking redirects

Regardless, even with this best practice followed to the letter there will still inevitably be complications with the redirects. Many of the issues stem from missed redirects which end up as broken pages. The most sure way to avoid missing any pages is to have a comprehensive list of every URL on a site placed  in a row on a spreadsheet and in a parallel column there will be a row with the new URL location.

Since the redirect file will not be a 1 to 1 old to new map and will likely use algorithmic rules on redirection, pages can still be missed. Before launching the new build of a website, the entire site should be crawled and verified that old URL’s correctly redirected to new URL’s.

Even with the best laid plans, there can still be issues with how redirection plays out. When a redirect instruction is given to a search engine, the engine is being requested to consider the new URL as being equal to the old URL. In practice, that decision is completely up to the search engine like all things related to their indexes.

As the redirect is a change being introduced to the search engine, it is possible that the request is not adopted, and the previous authority is not passed to the new URL. This is a not an insignificant risk and therefore complete site restructures and site migrations should not be taken lightly.  A full site migration should only be undertaken when absolutely necessary for legal purposes or branding needs. In these instances the primary business need will overshadow the potential loss that could come from a migration that leads to a loss in traffic from the migrated URL’s. If the business need could not trump a migration that leads to a loss in traffic, alternative plans should be considered.

I have overseen redirects that have managed to maintain traffic to a new location exactly as it was before, but I have also worked on projects where there was a 50%+ loss in traffic after the redirect. There is no real way to know whether the redirect will be accepted until it is rolled out.

Even redirects within a site have the potential to cause unfavorable adjustments in rankings and a full site rearchitecture should be undertaken carefully. Again, if traffic loss is an unacceptable cost to an overarching business need, then alternatives should be found. An option in this scenario is to use a staged approach where parts of the site are redirected and once traffic has stabilized, the next tranche of the site will be redirected. This process can continue tranche by site until the full site has been updated.

One major consideration to keep in mind with all redirects, is that the redirects likely have to be maintained in perpetuity. For as long as there are backlinks or users that might find the old URL, the redirects have to remain in place to avoid sending users and search crawlers to the wrong location.

These best practices to proceed cautiously when doing redirects and to maintain the redirect mapping forever will apply in any scenario where URL’s are changing, including a site update that just moves a handful of pages.

Even with all the associated risks, change should not be avoided just out of fear. Even if there are temporary drops in traffic, traffic may recover slower or there will be an even more substantial growth in traffic due to a better site structure. The primary takeaway on updates and migrations is that they should be done carefully and slow

1 2 3 4 5 6 15 16