Why not register?


Post new topic Reply to topic  [ 10 posts ] 

All times are UTC [ DST ]

Author Message
PostPosted: Mon Aug 18, 2008 8:51 pm  Post subject: BOTs on Dead Donkey
User avatar
Offline

Dead But Dreaming
Joined: Sat Sep 09, 2006 3:07 am
Posts: 337
Location: Lost in space and time
What are the Yahoo BOTs and others that I see logged in? Are they
like a web crawler indexing the site?

_________________
Image


Top
 Profile  
PostPosted: Mon Aug 18, 2008 9:19 pm  Post subject: Re: BOTs on Dead Donkey
User avatar
Offline

The Ancient One
Joined: Sat Oct 20, 2007 10:31 am
Posts: 3354
Location: Last house on the Right
I guess so.


Top
 Profile  
PostPosted: Mon Aug 18, 2008 11:02 pm  Post subject: Re: BOTs on Dead Donkey
User avatar
Offline

Site Admin
Joined: Sat Nov 02, 2002 1:35 am
Posts: 19625
Location: En España
they are exactly that, instead of appearing as an unregistered guest they appear as xyz Bot

_________________
Mouse nipple for the win! Trackpoint or death!
Image


Top
 Profile  
PostPosted: Wed Aug 20, 2008 7:57 am  Post subject: Re: BOTs on Dead Donkey
User avatar
Offline

Dead But Dreaming
Joined: Sat Sep 09, 2006 3:07 am
Posts: 337
Location: Lost in space and time
So I'm curious... Did they all have to sign up for the site or did
you (Spud) create them to have Dead Donkey to be more searchable?

_________________
Image


Top
 Profile  
PostPosted: Wed Aug 20, 2008 9:57 am  Post subject: Re: BOTs on Dead Donkey
User avatar
Offline

The Ancient One
Joined: Sat Oct 20, 2007 10:31 am
Posts: 3354
Location: Last house on the Right
Oberon wrote:
Did they all have to sign up for the site or did
you (Spud) create them to have Dead Donkey to be more searchable?


Google does not support "warez" in contrast to Baidu for instance. If they knew what this site is about they would remove it from their databases. :)

By the way I've just read a book about Google. They store everything you search for several years. They use it (as they state) to improve and personalize their search results and addvertisement "products" like AdWords and AdSense. In the future (and thanks to paranoid Mr. Bushnutte) this information could be used by the government or police authorities. Just make sure you don't enter something like "sharp knives" into Google before you plan to kill someone. :D Google denies that it is giving the database content to third parties but their cooperation with the Chinese government considering the indexing of their content and several other reports pretty much proove they do. And they have to if a shitty paranoid goverment creates shitty paranoid laws anyway. :watchingyou:


Top
 Profile  
PostPosted: Wed Aug 20, 2008 11:10 am  Post subject: Re: BOTs on Dead Donkey
User avatar
Offline

Site Admin
Joined: Sat Nov 02, 2002 1:35 am
Posts: 19625
Location: En España
They have to get a court order at the least, and several governments in south america have asked for the info from google and google, very nicely, told them to go f*ck themselves.

China has got small sets of data from them, but nothing on the scale of yahoo and aol has handed over... yet.

@registering, no browsers identify themselves with a http header that identifies their 'agent'. So when it see's agent as Googlebot it identifies them as a bot. it then turns off things like sessions, login checks, etc so its easier on the site. you can turn it off and it will just appear as another unregistered 'leech' :lol: , however its not as efficient.

Basically, google and other search engines, for better or for worse, have clusters of machines that 'spider' sites... that is go through, consume lots of bandwidth and index an entire site by virtually 'clicking' every link in and out. It does things like calculate a site ranking based on content hits and incoming and outgoing links.

So the best thing you can do for dead-donkey.com is stick it in your signature at other sites, because when they are indexed it will see a link to dead-donkey and say its a better site based on the number of links. :beerchug:

_________________
Mouse nipple for the win! Trackpoint or death!
Image


Top
 Profile  
PostPosted: Wed Aug 20, 2008 8:25 pm  Post subject: Re: BOTs on Dead Donkey
User avatar
Offline

Dead But Dreaming
Joined: Sat Sep 09, 2006 3:07 am
Posts: 337
Location: Lost in space and time
@chupa: I had never heard of Baidu. I went there and of course it is all in Chinese. You can see the 'English'
text in the link area down at the bottom of the browser. It looks very google-like to me.

@spud: Are they always logged in? Wouldn't that start to eat up your bandwidth (assuming you have
a monthly limit)? BTW, thanks for answering all my questions! I really appreciate it.

:beerchug:

_________________
Image


Top
 Profile  
PostPosted: Wed Aug 20, 2008 10:21 pm  Post subject: Re: BOTs on Dead Donkey
User avatar
Offline

Site Admin
Joined: Sat Nov 02, 2002 1:35 am
Posts: 19625
Location: En España
Quote:
@spud: Are they always logged in? Wouldn't that start to eat up your bandwidth (assuming you have
a monthly limit)? BTW, thanks for answering all my questions! I really appreciate it.


They do consume a hell of a lot of bandwidth, and there's regularly complaints and rebellions by sites but essentially, traffic defines a site and google brings in traffic:
http://www.google.co.uk/search?q=google ... +bandwidth

Yes, we have to pay to feed the bot :(

_________________
Mouse nipple for the win! Trackpoint or death!
Image


Top
 Profile  
PostPosted: Thu Aug 21, 2008 8:00 am  Post subject: Re: BOTs on Dead Donkey
User avatar
Offline

Dead But Dreaming
Joined: Sat Sep 09, 2006 3:07 am
Posts: 337
Location: Lost in space and time
spudthedestroyer wrote:
They do consume a hell of a lot of bandwidth, and there's regularly complaints and rebellions by sites but essentially, traffic defines a site and google brings in traffic:
http://www.google.co.uk/search?q=google ... +bandwidth

Yes, we have to pay to feed the bot :(


Nice link. I read through a couple of horror stories of Googlebot eating up 1-2GBs of
bandwidth in a short period of time. Assuming bandwidth is at a premium (which I don't
know if it is for you) is it worth it? I see what you are saying about traffic and hits but
if 30%-50% of traffic is from a bot(s) then that seems like a really high price to pay. From
what I've been reading, Googlebot scans on a daily basis? That also seems a little extreme.

_________________
Image


Top
 Profile  
PostPosted: Fri Aug 22, 2008 9:09 pm  Post subject: Re: BOTs on Dead Donkey
User avatar
Offline

Site Admin
Joined: Sat Nov 02, 2002 1:35 am
Posts: 19625
Location: En España
I think its an unfortunately byproduct, its a very useful tool and a "click income". If bandwidth was tight, I'd restrict the traffic but we have plenty.

_________________
Mouse nipple for the win! Trackpoint or death!
Image


Top
 Profile  
Display posts from previous:  Sort by  

All times are UTC [ DST ]

Post new topic Reply to topic  [ 10 posts ] 


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Frontpage / Forums / Scifi


What's blood for, if not for shedding?