I have dedicated my book Product-Led SEO, to Dave Goldberg who passed away in May of 2015. At the beginning of the book, I mention what an amazing person Dave was, but this is the longer story of why I am so grateful to him. Dave’s influence on me and my career enabled me to be who I am today and is why I am honored to dedicate my book to his memory.

During my first couple of years at SurveyMonkey, I came into frequent contact with Dave because Dave believed in staying close to his team. I attended a meeting with Dave at least once or twice a week, and I found him remarkably kind and easygoing. Every question he asked was prefaced with “have you thought of” or “that’s great, but how about…”

He was an absolute savant when it came to numbers and metrics, but he never pounced when people stumbled. The first time I was put on the spot with a number I was astounded when Dave — the CEO — helped me think through my numbers, then praised me for coming up with the right logic!

Dave was one of the kindest human beings I have ever met in my career, but I only really learned what a truly special person he was shortly before his untimely passing.

After spending a couple of years in traditional growth and SEO, I got an idea in my head that I should move overseas to Singapore to get more international experience. When I tried to leave the company to follow this dream, Dave wasn’t having any of it.

Within 20 minutes of hearing from my manager that I planned to take a new role in Singapore, I was sitting across a table from Dave in his conference room. He asked me to share why I wanted to move overseas, and then he agreed with me that it was a great idea. He discussed ways that I could achieve my dream of living overseas while also continuing to work at SurveyMonkey. The sticking point was that SurveyMonkey did not have an office in Singapore nor did they want one.

After this meeting, when Dave and I could not come to an agreement on how I could both stay with the company and move to Singapore, I thought this had been decided. I would go with the option that I waiting for me in Singapore. I appreciated Dave’s efforts, and I even felt sad that I was unable to continue working with him.

The next day, Dave emailed me that he had found a way to make it happen. He could place me at the office of one of our investors in Singapore and I could write myself a new job description. He even emailed the investor with me on cc, so I would see that I was welcome at their office.

But, Dave didn’t just send me to Singapore to help me to achieve a personal dream — he insisted that it be on behalf of the company with all the benefits accorded any employee. With only a few weeks before my move date, Dave made sure that I was moving with the full company assistance on a company sponsored visa, foreign health insurance and relocation guidance.

And he did one more thing, that I only learned about when I returned. He was adamant that I remain an employee, so I could continue to vest my stock. He set up a full Asian subsidiary of SurveyMonkey which would allow me to continue to vest any shares that I was granted.

Dave’s personal interest in my dream didn’t end there. Dave made my goals his too. He asked me to present to him every month my learnings from Asia, and one of those meetings was the week of his passing.

Those two years of experience in Singapore has changed my life in so many ways but none of it would have happened if not for Dave. After Dave’s passing I made three personal changes:

1. While I was interviewing for SurveyMonkey I had read Dave’s article in what was then called BusinessWeek about how he left work at 5pm to spend time with his family. I remember that when I had read that I scoffed at the idea that any CEO could truly leave the office that early and shut off, but when I got to know him and interacted him, I discovered how true it was. For a time I sat near Dave’s desk and while he didn’t always leave right at 5, he left pretty close to that time. When it comes to work, there is always tomorrow, but there might not always be a tomorrow for family time. During his lifetime, Dave made it clear that his priority was to his family, and I have done my best to internalize this lesson and do the same.
   
2. Dave’s level of respect for other people was immense and it was the characteristic most discussed after he passed. He took a personal interest in the people that asked him for help and I know from other colleagues how much he cared about their lives and careers.  I know I will never be like Dave, but I always try to help mentor anyone that asks me for help.
   
3. Even further, the idea of moving to Singapore was motivated from deep within my gut, and no amount of discussion was going to change my mind. In my conversation with Dave about moving, he tried valiantly to change my mind – but then when he realized how deep my desire to move was, he flipped the script and  asked how he could help. Dave didn’t just support me in the way he thought I should be supported, he helped to fulfill a dream in the way I wanted it to happen. It takes superhero strength to support someone in doing something you might disagree with, but that was Dave. When I am in a position to help, I think of Dave and try to be helpful for the recipient in the way they want support, not in the way I want to give it.

I can never be the person Dave was, but by using Dave as my compass I will always be striving to be a better human being. If there is anyway I can ever help you, I can be found on Linkedin or on my personal website. And, please don’t thank me — thank Dave.

The root of technical SEO is a deep understanding of a website’s architecture and how Google relates to the pages on the site. In my experience, the best way to gain this level of knowledge on any individual site is through deep log file analysis of Googlebot and user access logs.

Google Search Console reveals some of their crawl issues but it’s always at a very high level and aggregated. For a highly trafficked site, it would be nearly impossible to find the specific pages with issues.

On a site with many Googlebot entries, the number of rows will easily overwhelm your computer’s ram if you try to do this in Excel. Just opening the file will cause the system to slow and this is without even trying to run any queries.

Therefore, my favorite tool for this task is Splunk. For the unfamiliar, Splunk is a fantastic big data tool which allows you to parse large amounts of data quickly and easily to make important decisions.  Splunk even has a free version, which allows you to index up to 500MB per day. For many websites, this free version should be more than enough just to upload and analyze your access logs.

Here are the top 3 ways I use Splunk to help me with technical SEO efforts.

Find 404 pages generated by a Googlebot visit

404 pages (not found error) are wasted visit for every bot or human visitor. Every time a user hits a 404 page instead of the page they meant to see you are missing an opportunity to show them the correct content and at the same time they are having a subpar experience with your site. You can always proactively find 404’s with a crawling tool like ScreamingFrog, DeepCrawl or Oncrawl but if you have a lot of broken URL’s fixing all of them might not be a realistic goal.

Additionally, this doesn’t help you find a 404 resulting from an incorrect link on someone else’s site. When discovered by Googlebot, these links will send Googlebot to a non-existent page.

This is where log parsing becomes very helpful as you can discover 404’ed URL’s that are frequently accessed by users/bots and choose to either fix them or redirect the traffic to a working page.

Once you have your data imported into Splunk, here’s how you set up the query to find the 404 pages:

1. First choose your time period. For this type of query, I usually use 30 days, but you can choose whatever you want.
2. Type the following into the query box.

Index = {the name of your index} status = 404 | top limit = 50 uri

Your limit can be whatever you want, but I like to work with 50 URL’s for 404 pages to make sure I don’t miss any. Once this query completes, click on the statistics tab, and you will see all the URL’s that you need to urgently address laid out in a table.

Google expects 404 errors on every website, so the existence of them isn’t necessarily an urgent issue. However, some 404 URL’s could be the result of an unintended error or a valuable link (internal or external) pointing to the wrong page.  Running this analysis will allow you to make an uneducated decision.

Calculate the number of pages crawled by Google every day

If you use Google’s Search Console, then you are probably familiar with the screen where Google shows how many URL’s they crawl per day. This data may or may not be accurate, but you won’t know until you look in your logs to see how many URL’s Google actually crawls per day. Finding the daily crawl amount is very easy in Splunk once your data is uploaded.

1. Choose a time period of 30 days (or 7 if you have a lot of data)
2. Type the following query:

index ={name of your index} googlebot | timechart count by day

Once the query completes, click on the statistics tab, and you will have the true amount of pages crawled by Googlebot each day. For added fun, you can check out the visualization tab, and see how this changes over the searched time period.

This is more of an FYI than an urgent fix, but it is helpful to know if Google is now picking up new categories on a site or slowing a crawl. If either of these are true, it could be time to dig into the data.

Find rogue URL’s wasting crawl budget

As most marketers (should) know, Google allots a crawl budget to each site based on their Page Rank – not the visible one, but the real one in the Google black box.

If Googlebot wastes some of your valuable budget on URL’s you don’t care about, it obviously has less bandwidth to use on more important URL’s. Without knowing where Googlebot is spending time, you can’t know if your budget is being used effectively.

Splunk can help you quickly discover all the URL’s Googlebot is crawling which will then give you the data to make a decision about what should be added to your robots.txt file.

Here’s how you find the URL’s that Googlebot is crawling:

1. Choose your time period. This can be any amount of time, and you should keep trying different time periods to find problematic URL’s.
2. Type in the following query:

index={name of your index} googlebot uri_stem=”*”| top limit=20 uri

You can set the limit to whatever you want, but 20 is an easily manageable number. Once the query completes, click on the statistic tab, and you will have a table showing the top URL’s that Google is crawling. Now you can make a decision about any pages that should be removed, blocked by a robots file, or noindexed in the head of the page.

I use Splunk in over a dozen different ways to help me accomplish various SEO tasks, and these are just three of my most common uses.

Search console is one of my favorite SEO tools as it provides some of the best data directly from Google search database. The drawback to the Search Console data is that it is sanitized and normalized; however, without precise data it is a great source of directional insights. One challenge that most people will face with Google Search Console is that there is too much data, but here a few ways you can easily filter the data for actionable results.

1. If you have built a solid brand, many of your bigger keywords will be branded. If your brand is somewhat complicated to spell, you will invariably have misspells. Use the filter below to put the primary words in your brand filter out most brand results from queries.

   

2. In the current iteration of Google Search Console it can be difficult to find precise queries for a given page, here are the steps to dig up page level queries
   1. From the page report click into a specific popular page.
      
   2. Once that page is in the pages filter, click queries and you will see popular queries
3. Find pages that rank for the same queries
   1. Search the keyword in the queries filter
      
   2. Then click into pages and you will see all pages that ever show up for phrases containing that query
      
4. Find pages with queries that have a seemingly low CTR
   1. Click into a popular page
   2. Check off CTR in the metric portion of the page
      
   3. Look at the query list for that page to find queries with CTR’s that seem out of the standard range.

 

I will eventually expand on this post with other ways to use GSC so check back again!

Google Express originally called Google Shopping Express quietly ended their subscription pricing on August 23rd. Up until today, Google charged an annual subscription just like Amazon Prime. The Google Express news of the day was dominated by the announcement that Google and Walmart would be partnering on voice search to combat Amazon, but there was little attention paid to the change in their pricing model.

Here’s what the Google Express pricing page says now.

What happened to memberships?
The Google Express membership program ($10 for monthly members and $95 for annual members) has gone away in August 2017. If you have a membership, you’ll be refunded for the remaining time.

Why we’ve made the change:
We’ve recently welcomed new stores to broaden our product assortment. Many of these arrange for their own deliveries and returns, or have listed the same shipping fees and per-store minimums for everyone, regardless of membership status. Delivery speeds also vary, ranging from one day in some areas to one week or more with some specialty stores. Learn what stores are available on Google Express.

You’ll see free delivery when you meet the per-store minimum ($25 or $35 with most stores) – no membership required.

This is a really big deal since Google has significantly lowered a barrier to using their home delivery product and with the addition of Walmart allows them to beat Amazon on price and speed. I assume that Google will eventually have to make a bigger splash about their no membership required delivery product since this is just too big to bury in a quiet change.

In 2016 Google told Search Engine Land that trillions of searches are conducted per year on its search engine and most people conduct dozens of searches per day. Google produced a great microsite which explains how Google search works, but in a nutshell, Google archives the entire web into a database and every search is a query into that database.

Every time a Google search is conducted, Google does its best to determine the intent behind the query, and then shows the most relevant matching results. Since Google’s search experience is just an algorithm, it may not always guess the right intent, so you want to give Google more clues about what you are looking for. Other times you might be looking for results within narrow parameters or on a specific type of site. To conduct these more specific searches on Google you must use search modifiers. Here are the top 10 search modifiers which can help you level up your Google queries:

 

1. site: this query is very helpful when you are looking for results only on a specific website or type of website. Examples of this query: blog site:wikipedia.org means you only want to see results about blogs from Wikipedia.
2. Another way of modifying the site query is to add a specific kind of web TLD (top level domain) like site:.gov. Example: blog site:.gov means you only want to see result about blogs from website that end with .gov
3. Quotation marks around a query “example” means that the results must contain the word and not a related version of the word.
4. A dash before a word means you only want results that DO NOT have words related to that word. Example website -blog means you don’t want to see anything related to the word blog.
5. A plus sign before a word means that you want Google to take into account a word they may have discarded as a stop word. Example: +The long road home means that I want Google to consider the to be an equal part of my query to the other words.
6. Adding the word OR between two queries means that you want results related to either the first word or the second word. Example: blog OR website  means you want to see results that are relevant to either query
7. Similarly, the word AND between queries requires that results be related to both queries. Example: blog AND website  means that results have to be related to both blog and websites
8. A tilde before a query means you want Google to also bring back results that are synonyms of your query. Example: ~blog  brings back results for all words that are synonyms of blog
9. Filetype allows you to specify the type of result you want to bring back from Google. Example: filetype:.pdf means you only want to see results that are PDF pages

Don’t worry if you can’t remember all this since you can always conduct what Google calls an advanced search with the link hidden under the settings.