On Search Engines and Dynamic URLs

GilesGuthrie

Staff Emeritus
11,038
United Kingdom
Edinburgh, UK
CMDRTheDarkLord
There is a belief that search engines which 'crawl' sites are unable to handle dynamic links, which are the saviour of active page coders (basically, create one page, pass the page a parameter, such as a pointer to a record in a database, then the page goes and gets the record, and shows the whole page to the browser as if it was static).

This basically means that if you run a database-driven site, which GTP is, and also giles-guthrie.com is, you're stuffed on the search engines, because they'll never find your content. This was behind Jordan's recent switch of all the GTP Links (he discusses this in the PHP Forum, Hard to Learn thread).

However, the times they may be a-changing. I found this on the WebProNews site:

Question:
Google does not index asp generated sites well in their index. That was why we originally sought help from this SEO company. As an ecommerce site, 90% of our content is dynamically generated. Are their shopping cart solutions or services that have addressed this problem of search engine visibility in a legitimate "Google friendly" way? Are their optimization companies that only use Google-friendly techniques?

Answer from Google:
That's not true any longer.

Writing dynamic urls as if they were static used to be the "right way" to present dynamic urls, but that's changing, at least for Google. Google is getting better about crawling dynamic urls, and we'd prefer to see dynamic urls in all their glory instead of written as if they were static. You'll see an increasing number of hosts where we could crawl deeply into a site through all kinds of dynamic urls. Google was pretty much the first to crawl dynamic urls, and we want to do it right without causing webmasters to rework their site. Of course, if you have a url with 15 parameters and only two of them actually mean anything, it's always a good idea to shorten a url whenever possible by trimming out the unneeded parameters. Changing dynamic urls to appear static will be less important over time as Google crawls dynamic urls better. We also estimate host load by taking account whether a url is dynamic or not. Keeping dynamic urls written as dynamic will help us to estimate the load for your server and keep a bot from hitting your server too often. Hope that makes sense. It's a change from what we've been recommending to webmasters, so I wanted to give people a heads-up.
 
So you mean all that .htmaccess stuff Jordan was talking about won't effect the search results you get on Google eventually?
 
Eventually, that's right. Google does list the page extensions that they crawl, but omits PHP, which is a shame for me and Jordan. However, when I typed in my own name (narcissistic, I know!!!), it had indexed several PHP files on the site.
 
Back