Determine if User Agent is a crawler?

microsoft.public.inetserver.asp.general

13 Dec 2005 7:36 PM

Bill

Is there a fast way, with Classic ASP, to determine if a user agent is a search engine
spider?

I know that ASP.NET has Request.Browser.Crawler, I'm looking to see if classic ASP has
something built in, or do I have to look for strings in the user agent.

Thanks!

13 Dec 2005 9:36 PM

David Morgan

Check Request.ServerVariables("HTTP_USER_AGENT") for crawler, spider and
bot. Most search crawlers have something like that in their user agent
string.

Show quote

"Bill" <Bill***@yahoo.com> wrote in message
news:e6BJUyBAGHA.292@TK2MSFTNGP14.phx.gbl...
> Is there a fast way, with Classic ASP, to determine if a user agent is a
> search engine
> spider?
>
> I know that ASP.NET has Request.Browser.Crawler, I'm looking to see if
> classic ASP has
> something built in, or do I have to look for strings in the user agent.
>
> Thanks!
>
>
>