So many aspiring SEOs! – the SEO Quiz results are in

Share

15 questions, 5 weeks and 5 books: almost 700 people took the 2008 SEO quiz challenge.

Note to the reader: this article was originally posted on our Italian blog on December 2^nd. The quiz targeted an Italian audience; we’ve published this translation in order to allow a wider audience to follow search marketing developments in Italy.

Why a SEO quiz

The idea of the quiz came from reflections on the state of SEO knowledge and usage in Italy, observed from the perspective of a SEO practitioner.

Search engines, with Google in particular (question 1), are the gate keepers between us and the net. We use search engines not only to search for information that we imagine is out there somewhere, but also to navigate to a specific site, such as Fiat, or to perform a task, such as buy a ticket for a Tiziano Ferro concert (question 15).

Several years ago, in a previous job, I began to realize how SEO knowledge was rather limited even among web professionals. A representative for a well known information technology company said at one point “you just have to pay”, as if a bribe would do the trick. The year before, a “SEO” company had implemented a solution based on doorway pages on web sites created for the purpose, already an outlaw tactic at the time.

I believe that times have improved. But it is still necessary to dispel many common SEO myths that are hard to die, ranging from the importance of the meta tag keywords (question 7) to how blogs are already optimized for search engines. I’m even more puzzled by how little investment is currently made in search engine marketing, particularly for visibility in organic results (~80% of clicks) ¹.

John Wanamaker² noted that the results of traditional advertising are notoriously difficult to measure when he observed ironically,

“Half the money I spend on advertising is wasted; the trouble is I don’t know which half“

The beauty with search engine marketing is that this problem disappears – we can measure almost everything.

In times of crisis, or at least the perception of crisis, is it not appropriate to privilege media where we can measure results?

But who deserves the blame?

If overall SEO knowledge, and consequently its application, is quite low (and not just in Italy), the primary responsibility most likely rests with SEO professionals. Our role is to credibly inform and train people regarding SEO’s potential. I’m passionate about search engines and web analytics. Through this SEO quiz and my various presentations in public, I hope to share some of this passion together with a greater knowledge of how search engines work.

The SEO quiz results

Tough questions

The multiple-choice questions were difficult as revealed by the answers. At the same time, with more than 200 factors that could affect the results of a search query (question 4), it is necessary that a true SEO expert possess a vast panorama of the state of search engines yesterday, today and tomorrow. There is no pretext that the questions are comprehensive: 15 questions are too few in comparison to the more than 200 factors considered by Google. Some important topics, such as keyword selection and link management, are not even mentioned. They are definitely topics for likely inclusion in the next edition of the SEO Quiz, 2009.

Your comments

As indicated in the Italian writeup of the SEO quiz after the first week, most of the comments are of the type “Question 5 is not quite clear…” or “complex and interesting enough as a test. Many questions may have more than one right answer. Congratulations”. To these we can add “I don’t know a thing about SEO ” and “Some questions are poorly conceived: If I read «… could affect ranking…» I think could? SEO is made of elements and data which are certain … in my opinion questions that can not be verified with a test should be avoided, even if they are part of commonly accepted wisdom, i.e. «everyone says»… just my two cents“.

The last comment is particularly revealing as it highlights a significant issue in search engine optimization. SEO is composed of factors that can be isolated and verified. The answer regarding the meta keywords tag (question 7) is a good example. Yet SEO also depends on the search engine algorithms, algorithms that are not published and are subject to change over time. That these algorithms are applied to the entire world wide web means it is not always possible to isolate and test a search engine optimization hypothesis. Consequently, a part of SEO is subjective as some have noted. The important point here is subjective from what basis? Rumors? Or are opinions derived from experience in the Information Retrieval (IR) field and an ability to think like Google, i.e. …if I were Google…. Concrete experience does make a significant difference. I prefer that my medical doctor or airline pilot have decades of experience in the field to augment text book theory. It is the experience that helps us to decide how much weight to give to each factor. I can share your frustration at having to deal with some potentially ambiguous questions, but SEO is also about the uncertainties that are part of the game. Precisely for this reason the quiz was launched in a light hearted manner:

It seems that the summer fun is now over, but not so fast: it’s time to check, just for fun, your SEO knowledge!

The official answers

For each question I have indicated the official answer and, when appropriate, a source for the answer. It should be noted that in a multiple choice test, the goal is to choose the best answer – that doesn’t mean that the other answers are totally wrong. Now that you see the answers, you can form a definitive opinion on whether the questions were sufficiently clear (without the wording of a question implicitly giving away the answer!). In the event that you believe there is a better answer to the official one, fine – we’ll use the “wisdom of crowds“ to improve. Just explain your reasoning and cite third party sources to justify your position.

1	Which is the most important search engine in Italy? Arianna (Libero) 11 answers; 2% Google is the most important search engine in Italy. A sample source: (with many reservations) Nielsen, the top business sites in Italy, July 2008. For further information, see the related article: Who’s using Google for their organic and paid search results? Verdict: Google is currently the only significant search engine in Italy. Google 604 answers; 89% MSN/Live (Microsoft) 7 answers; 1% Virgilio (Alice/Telecom) 48 answers; 7% Yahoo! 6 answers; 1%
2	In how many search engines should a new website be submitted? 0 171 answers; 25% One could submit a site to one or more search engines, but it would be time wasted. Major search engines (for example Google, Yahoo, Microsoft, Ask) use incoming links to discover new sites and content and to determine the ranking of sites in search results. One’s time is better spent ensuring that a site has inbound links from other similar sites. Verdict: it is no longer worthwhile spending time to submit websites to search engines. 1 102 answers; 15% 2 77 answers; 11% 5 225 answers; 33% 100+ 101 answers; 15%
3	In what year was the service to insert new sites suspended for one or more European Yahoo! directories? 2000 47 answers; 7% The Yahoo directory closed to commercial sites in several Europe countries in October 2004 but definitive closure to sites of all types took place, at least in Germany, in 2006 (The link to submit a site, Web-Site vorschlagen, disappeared between 15 and 17 February 2006). I have confirmation from unofficial sources that the same thing happened in Italy, but there has never been a formal announcement and the link to suggest new sites remains, but the directory isn’t updated. To explore this topic further, see: Are directories still useful for SEO and in increasing web site traffic? Verdict: Directories edited by hand are not as important as they once were. 2002 114 answers; 17% 2004 244 answers; 36% 2006 204 answers; 30% 2008 67 answers; 10%
4	How many signals, or factors, does Google use in determining search results ranking? 1 22 answers; 3% For a long time Google indicated more than 100 factors, such as in the webmaster help, retrieved April 24 2006 and still visible in the online AdWords training course (section 5a), but Google notes more than 200 factors in a Google Press Day presentation (May 10 2006), slide 17, and webmaster information now speaks of more than 200 signals. Verdict: search engine visibility has a certain complexity… avoid the trap of expecting a valid SEO analysis on the fly! 5 172 answers; 25% 65+ 134 answers; 20% 100+ 109 answers; 16% 200+ 239 answers; 35%
5	Google allows me to see the PageRank of my site or part of my site in how many different places? 1 287 answers; 42% The public PageRank is visible in the Google toolbar, in the Google directory and, just for one’s own site, in Google Webmaster Tools. The value shown in the toolbar is not actually used by Google. It is static – it is only updated every 3-4 months (question 6). The value could fluctuate while Google data centers are synchronized but such fluctuation doesn’t last long. There is no official interface (an API) to access this information – the various extensions made by third parties for Firefox are tolerated but not encouraged. The same PageRank is visible in the Google directory for sites listed in the open directory. The width of the image pos.gif divided by 4 gives the PageRank of the pages listed in the directory. In the Google Webmaster tools there are 2 references to PageRank: the page with the highest PageRank in recent months and the PageRank distribution (High / Medium / Low / Not yet assigned) of pages in the site. In the comments some of you have noted frustration with this question, perhaps too ambiguous. The problem was how to formulate the question without giving too many clues to the answer, but I accept the criticism. If you answered “4″ counting the 2 options in Google Webmaster Tools, or you answered “Over 10″ having in mind all data centers that provide the toolbar data, and you’ve thought of the Google directory and the Google Webmaster Tools, you can consider your answer correct (I have doubts that many have thought of all 3 places offered by Google). Verdict: It seems that the public PageRank serve mostly to convince people to supply Google, through the toolbar, the addresses of all the sites they visit. Also, it is worth noting that there are many cases where the visible toolbar PageRank has fallen yet the traffic from Google has not diminished. With more than 200 factors affecting ranking (question 4), don’t give too much weight to Google’s public PageRank. 2 83 answers; 12% 3 70 answers; 10% 4 40 answers; 6% 10+ 196 answers; 29%
6	How often does Google update the PageRank displayed in the Google toolbar? In continuation 178 answers; 26% Ref: Matt Cutts, Google. Verdict: The public PageRank in the Google toolbar is not used by Google… it is worth pointing out that PageRank is not an accurate indicator of a site’s performance in Google. Every hour 38 answers; 6% Every day 120 answers; 18% Every 3-4 months 334 answers; 49% Every year 6 answers; 1%
7	Which search engines sometimes use the keywords meta tag for ranking? Ask, Google, MSN/Live, Yahoo! 358 answers; 53% Ref: Danny Sullivan, meet the crawlers follow-up. The keywords meta tag has been so abused that for years it has not been a good indicator of neither the content nor the value of a web page. For this reason it is mostly ignored by search engines. Few search engines consider them, and then only in rare cases. Verdict: the recomendation is to avoid using this tag. It isn’t necessary that you provide a gift to your competitors, giving them the results of your keyword research. Ask, Yahoo! 183 answers; 27% Google 43 answers; 6% Google, MSN 54 answers; 8% Yahoo! 38 answers; 6%
8	For a multinational company with a presence in Poland, is it better to use a specific domain, to use separate folders on a generic top-level domain or to use sub-domains? Generic top-level domain with folders, i.e. www.antezeta.com/pl 230 answers; 34% A national domain is usually the best answer, but the best choice also depends on other factors, i.e. if there is a unique relationship between geography and language. For further details, see our related article Top level domains, subdomains or directories for Search Engine Optimization of multilingual websites?. Verdict: SEO is a combination of marketing, technological and lingustic factors. Internationalization adds complexity to this mix. National domain, i.e. www.antezeta.pl 300 answers; 44% Sub domains, i.e. pl.antezeta.com 146 answers; 22%
9	Who provides natural and paid results to Arianna – Libero? Arianna – Libero 134 answers; 20% Google provides organic and sponsored results to Libero, Kataweb, Virgilio and others. To explore this topic in greater depth, see Who’s using Google for their organic and paid search results?. Verdict: often search engines are only nominally what they appear to be. The actual service is provided by one of the majors, generally Google or Yahoo. Google 370 answers; 55% MSN/Live 53 answers; 8% Yahoo! 119 answers; 18%
10	Google may ban a site from its index, which has: Code or text entered by hackers 32 answers; 5% Google is a private company and does not need to provide the reasons for inclusion or exclusion of a site in its search engine. Verdict: Even if Google is very benevolent, its power and potential for its abuse led to the start of an alternative project in France, Quaero, while Germany is working on Theseus. Breached Google guidelines 281 answers; 42% Used IP delivery techniques 53 answers; 8% Reasons known only to Google 24 answers; 4% All of the above 286 answers; 42%
11	Usage of Google Analytics could affect the ranking of a site in Google? No 438 answers; 65% Ref: Google Analytics Help, Danny Sullivan, and various interviews with Brett Crosby, Google Analytics, such as this. Consider also my observation made after the release of Google Analytics in November 2005: Google has access to a subset of data regarding our web site visits through people who use the Google toolbar. Verdict: The official answer is no, but the only certainties in life are death and taxes – a yes answer is understandable. At the same time, Google already has good insight into network traffic thanks to the popular toolbar. Problems can arise in the context of AdWords and conversion, or if something “underhanded” is going on. Yes 238 answers; 35%
12	Spends on Google AdWords advertising can benefit natural visibility in Google? No 370 answers; 55% Ref: Google AdWords Help. Verdict: Google today is led by very enlighted people. They know that the success of the organic results is critical for the AdWords cow. Yes 306 answers; 45%
13	Which search engines accept payment for consideration (but not the ranking) of pages in their organic results, so-called paid inclusion? Ask 174 answers; 26% Yahoo Search Submit Pro. The concept of a payment to obtain quick merit-based visibility derives from the days when you had to wait a long time for inclusion consideration in a manually compiled directory. The same logic doesn’t apply to search engines driven by automatic algorithms. Verdict: Keep your wallet closed so as not to waste money. Google 46 answers; 7% MSN/Live 67 answers; 10% Yahoo! 97 answers; 14% All of the above 292 answers; 43%
14	Using the Google Toolbar could help the visibility of my site in Google? Maybe 145 answers; 21% If display of the toolbar PageRank is enabled Google is notified of any page viewed by the toolbar user. So far there has not been a clear message from Google regarding current or future usage of data on the web browsing collected by the Google toolbar – a rather different stance relative to that concerning AdWords and Google Analytics. For this reason, the official answer is “maybe”. Skeptics will note that the experience with the Alexa toolbar and others earlier taught us how easy it is to manipulate data collected from a browser toolbar. Google does have the benefit of strong experience in the filtering of “invalid” clicks from AdWords collected data. I strongly suspect that Google is not blind to this signal – the amount of data is enormous and too interesting to ignore. The alternative would be to purchase similar data from ISPs, as does Hitwise. For now this is just an hypothisis – I have yet to set up a test to verify the validity of this suspicion. Verdict: Here I turn to the “wisdom of the crowd“: have you ever seen a quote on this topic by a Google employee? No 484 answers; 72% Yes 47 answers; 7%
15	According to an Altavista researcher, later at IBM and now at Yahoo!, what are the three types of queries asked of search engines? Automatic, Interactive, Interrogative 227 answers; 34% Ref: Andrei Broder et al, A taxonomy of web search. Verdict: Search is not just activity to find information that one imagines is available online but also to find specific sites such as the Fiat 500 one, or to carry out some specific tasks, such as booking a flight from Milan to Rome. Random, General, Targeted 185 answers; 27% Navigational, Informational, Transactional 223 answers; 33% Profound, Thoughtful, Superficial 41 answers; 6%

Table 1: The best answers to the SEO Quiz

The grades

Perhaps the questions were difficult, but your answers were on the whole very encouraging. From the chart below note that the vast majority of respondents got at least one third of the questions right. More than 20% of you have responded correctly to at least half of the questions. Not bad considering that most respondents are not SEO professionals (there aren’t many of us!). The picture is even more positive if we drill down into the details. Many of you may have chosen a wrong answer, but one very close to the right answer (question 3 (2004) and question 4 (100)).

Other questions, though having an official answer (Usage of Google Analytics could affect the ranking of a site in Google?), by their nature are difficult to verify independently. As indicated in the comments to the answers, a more skeptical response is not always completely wrong.

Grade	No. of correct answers	%Totals (676 respondents)
80%	12 4 respondents (1%)
73%	11 13 respondents (2%)
67%	10 27 respondents (4%)
60%	9 31 respondents (5%)
53%	8 85 respondents (13%)
47%	7 100 respondents (15%)
40%	6 108 respondents (16%)
33%	5 118 respondents (17%)
27%	4 96 respondents (14%)
20%	3 56 respondents (8%)
13%	2 26 respondents (4%)
7%	1 10 respondents (1%)
0%	0 2 respondents (0%)

Table 2: Distribution of grades

Who participated?

Participants are divided about fifty-fifty between those who come from business / marketing areas and those with a technology background, responsible for developing and managing websites. I was also glad to see some SEO colleagues among the participants.

Five lucky people received a copy of Marco Massarotto’s Internet PR

A copy of the book Internet PR was awarded to

• Pietro Casaluci
• Laurent Belin
• Bernardo Manzitti
• Aldo Fontana
• Laura Bolletta

Hearty congratulations to each and a special thanks to Marco Massarotto for having worked with us on this initiative.

Next steps

A personalized email

All those who provided an e-mail address will receive the table of answers with their responses highlighted. If you participated and the e-mail doesn’t arrive within 24 hours:

1. check the “spam” folder using webmail and your email client if you’re using one (you can add sean -a+t- antezeta.com in your contacts to avoid this problem)
2. Send me an e-mail with your full name, e-mail and, if you kept it, your unique confirmation code. In case you have not provided your contact details in advance, you must supply your confirmation code.

Subscribe to the rss feed

Consider subscribing to this blog’s rss feed to stay updated on developments in search engine marketing. Periodically you’ll find a new article which could be of use in a current or future project. We also have a partial list of previous SEO and Web Analytics articles. You can also subscribe to our newsletter that is sent out from time to time.

Follow search engine developments as seen by a very authoritative source

The articles published at Search Engine Land, particularly those written by Danny Sullivan, are worth your attention. Danny has a journalistic background and has been objectively analyzing search engines since 1996.

Submit a question for the next edition of the SEO Quiz, 2009.

The SEO quiz has been a lot of fun and its time to start preparing for next year. But now it is up to you to suggest (sean -a+t- antezeta.com) a question, possibly providing the answer and authoritative sources to justify the official answer.

Attend a SEO course

If time is limited, a seo course might be a good solution. One common key to the success of leading companies is their investment in quality training. They know that education is inexpensive relative to the value of an employee who has been provided the appropriate tools needed to be proactive and fully operational.

Start a SEO project for your online presence

If you want to make the most of your online presence, let’s talk. We offer consulting on search engine optimization and the other elements of web marketing, web analytics included.

Share your participation in the SEO quiz

Feel free to add the quiz badge to your blog or website.

<a href="http://www.antezeta.it/blog/quiz-seo-esito-2008/"><img alt="Il quiz seo" src="http://www.antezeta.it/i/ho-fatto-il-quiz-seo.gif" style="border:0;width:220px;height:60px;" /></a>

Thanks to everyone who participated in the quiz!

Similar Posts:

• 15 questions, 5 weeks, 5 free books: the SEO quiz is here
• SEO for a Blog – All roads lead to RomeCamp 2008
• Google’s SearchWiki, Customized & Personalized Results
• Yahoo Web Analytics (ex IndexTools) soon in no man’s land?
• Are you an accomplished woman vying for a place in President Obama’s Cabinet?

If you're new here, you might subscribe to my feed by Email, RSS feed and/or follow me on Twitter, which is updated on a more frequent – and more meaningless – basis in English and Italian. Finally, if you're a Sphinn user, Sphinn love is welcome :-). Thanks for visiting!

Share