One day you note a fall off in the traffic Google sends your website. As Google is the main source of your traffic, as is the case for many websites, alarm bells naturally start ringing. Investigating, you realize that the site does not appear at all in Google or has poor visibility at best in search results. What is a poor site owner to do? Did someone say panic?

Understand why the site disappeared from Google

There are several reasons why a site no longer appears in typical Google search results.

[Read more →]

The other evening Camillo Di Tullio, a.k.a. Dr Who, asked me via IM if I was having problems accessing highly trafficked social media websites like Facebook or LinkedIn. In that particular moment, I wasn’t, but his question stuck a particular cord. We’ve seen many downtime issues with major Internet sites lately.

Website downtime, planned and unplanned, presents a company with a reluctant marketing opportunity. After all, investments in search engine visibility and other website traffic drivers are all for naught when a site is no longer reachable. The best a company can do is acknowledge the issue and, where appropriate, attempt a dose of humor while working frantically behind the scenes to insure the problem doesn’t occur again.

[Read more →]

Several months ago a client inspired me to write a comprehensive guide to keeping website content out of search engines. Usually website owners are focused on the opposite side of search engine optimization, insuring web content is well indexed. Yet, as many can attest, search engines can be all too efficient at finding documents they shouldn’t. Thus, the need to understand what options exist, how they work and which search engines support them.

One problem with the techniques available up until now is that options for digital media have been limited. The official way to keep video, audio and pdf files out of search engines was through the robots.txt protocol, not a very efficient tool when setting indexing options on a file level.

[Read more →]

Rare is the web professional who doesn’t know that building a great website isn’t usually enough to guarantee its success. Sites have to be visible in search engines for the keywords and phrases web navigators are most likely to associate with the site’s content. An entire industry has grown up around SEO, search engine optimization. Yawn, you say.

What about the reverse side of the coin, keeping content out of search engines? Should be easy, no? Maybe not. In February, we looked at 5 ways to stop Google and the other search engines from downloading and indexing a website’s pages.

[Read more →]

There are occasions when some content on a web page just shouldn’t appear in search engines. The most frequent example is repeating header and footer details, such as site copyright information. This site uses the hcard format to provide contact information site visitors can save in a vcard for use with a PIM such as Thunderbird or Outlook. Yet some of the information required for a detailed vcard is not really appropriate for a search engine’s index. Historically, the best solution was to place such content on a page using JavaScript as search engines have avoided indexing JavaScript (they probably do analyze it). A JavaScript approach to keeping some page information out of search engines isn’t perfect – not all visitors will have JavaScript enabled.

[Read more →]

While it may seem paradoxical, there are many occasions where you may want to exclude a website or portion of a site from search engine crawling and indexing. One typical need is to keep duplicate content, such as printer friendly versions, out of a search engine’s index. The same is true for pages available both in HTML and PDF or word processor formats. Other examples include site “service pages” such as user friendly error message and activity confirmation pages. Special considerations apply for ad campaign landing pages.

There are several ways to prevent Google, Yahoo!, Bing or Ask from indexing a site’s pages. In this article, we look at the different search engine blocking methods, considering each method’s pros and cons.

[Read more →]