SEO Metrics, Rogue Crawlers, And Website Speed


SEO Metrics, Rogue Crawlers, And Website Speed
SEO Theory Hangout from March 14, 2016

Michael Martinez of Reflective Dynamics, Inc. talks with Kent Yunk of Roaring Pajamas SEO about search engine optimization.

* How does Google index new content from existing sites?
* Barry Schwartz misattributed John Mueller’s comment to support of “domain authority”
* The concept of domain authority is pseudoscience that has misled the Web marketing industry
* Third-party metrics are not computed from Google’s data
* Toolbar PageRank has not been useful for at least 12-13 years
* Toolbar PageRank is being discontinued by Google
* Web marketers continue to look for quality metrics but none of them are relevant to Bing and Google
* SEO tool vendors need a way to benchmark things over time
* Only reported rankings and referral traffic are usable as progress metrics
* How well can third-party metrics help Website owners benchmark performance?
* Spammers easily influence the third-party metrics
* SEO tool vendors do go to great lengths to filter out bad data
* Tool vendors do not confirm that links they include are indexed in Bing/Google and passing value
* No one can reliably determine that a link is helping or passing value
* Even experiments that show promise cannot be scaled up
* SEO tool vendor metrics are not accurate and are not relevant to search engine data
* There are no reliable alternatives to using search referral traffic as a benchmark
* There is no way to reliably predict what SEO performance will be
* Anyone can spam a site to the top of search results but how long will it stay there?
* Many SEO tool vendors run rogue crawlers
* Rogue crawlers ignore robots exclusion standard
* Ethical crawlers honor wildcard directives
* Webmasters do not owe anything to tool vendors
* SEO tool vendors are just a small fraction of companies running rogue crawlers
* Crawlers can be downloaded from Internet and run from cloud services
* Many “business intelligence” and “brand monitoring” companies run rogue crawlers
* These crawlers are just looking for mentions of brand names
* Rogue crawlers tie up TCP connections and prevent legitimate people from visiting Websites
* Current methods for speeding up Websites are using bad ideas
* Adding prefetch directives to Web pages increase load on servers without bringing real traffic
* Developers are the wrong people to lead the discussion in how to speed up Websites
* Server administrators often have to correct or fix bad developer ideas
* Bots are inflating traffic data for a lot of Websites
* Many business owners don’t realize the bot traffic is fake
* Analytics data can be very misleading to business owners
* Google Analytics filters are not very good
* SEMalt is the most well-known traffic faking company and they sell “analytics tools”
* Bots are also filling in lead generation forms
* Some bots are trying to hide their activity on a small number of sites by afflicting many other sites
* People create rogue bots for many reasons and they will be around for a long time
* Early Web metrics in the 90s were measured in “hits”
* “Hits” was just the number of objects being requested by clients
* Pages with more images and other elements were getting higher hit counts per visit
* Hitcounter service confused the discussion around “hits” because they only reported 1 hit per page view
* Marketers shifted away from page views to visits
* Counting “visits” is also problematic. When does a visit start and end?
* If you visit a site at the same time from two different devices through 1 router, is that 1 visit or 2?
* 1 corporate router can handle traffic for hundreds of people surfing the Web
* You have to filter Google Analytics traffic two ways: as it comes in AND through the reports
* Many people are only working with one filtering method
* Reflective Dynamics uses multiple analytics packages per site
* In Google Analytics, create an Admin filter to block traffic from known spam domains
* In Google Analytics, use advanced filters in reporting to remove spam traffic already recorded
* On some sites, it is easier to do filters on the fly
* Michael often just looks at the Google / Organic channel
* SEO Theory Premium Newsletter recently published a case study showing why Google’s analytics data is not reliable
* You need to “flatten” the signal in the data as much as possible in order to identify legitimate traffic spikes and dips
* Some large volume Website owners have whole teams that manage analytics data and fight rogue crawlers
* Michael estimates that 40-50% of all traffic on the Internet is fake
* Small SEO agencies are at a disadvantage compared to large in-house IT departments
* Adjusting Google Analytics data is more than a “band-aid” solution
n_iuzLr7ks0

Thanks! You've already liked this
No comments