I have no reverse proxy installed. In terms of Apache config…I’m running four VHosts on the same Apache instance. The only other “strange” thing is I’m using that I can think of is mod_deflate.

This is what I’m seeing in the Bad Behavior log table for Feedster:

http_headers:
GET /feed/ HTTP/1.1
TE: deflate,gzip;q=0.3
Accept-Encoding: gzip, deflate, compress
From: webmaster@feedster.com
Host: http://www.kgadams.net
If-Modified-Since: Sun, 21 May 2006 21:26:38 GMT
If-None-Match: "87af99f4cc3f479eaf691ac5697be113"
User-Agent: Feedster Crawler/1.0; Feedster, Inc.
 
denied_reason:
Header 'TE' present but TE not specified in 'Connection' header

As you mentioned, I’ve observed a couple of rejections for Opera of which the following is an exemplar:

http_headers:
 
GET /?p=2 HTTP/1.1
Host: http://www.kgadams.net
User-Agent: Opera/6.03 (Windows 2000; U) [en]
Pragma: no-cache
Accept: */*
Referer: http://www.kgadams.net/?p=2
Max-Forwards: 10
 
denied_reason:
 
Header 'Pragma' without 'Cache-Control' prohibited for HTTP/1.1 requests

Thoughts?

As for Feedster, it is using TE correctly, according to my logs. If you’re seeing it blocked, check to ensure your site isn’t using a misconfigured HTTP accelerator (reverse proxy). This is usually squid, tux, or things along those lines, but some unusual Apache configurations might do it.

If Feedster is being blocked at your site, it’s likely that so is the Opera web browser, so it’s definitely something you’ll want to work with your web host to get fixed, or if they’re unwilling to fix it, to find a new web host.

Thank you for developing Bad Behavior. When I read about how it works, it seemed to me that it would frustrate spammers a great deal more than simply having their spam filtered. My hope is that in combination with Akismet it will make the spam problem largely invisible to me and, at the same time, disadvantage the folks generating the spam enough to perhaps convince them to back off.

One thing I’ve noticed in my logs is that it appears Feedster’s crawler (or at least something identifying itself as Feedster) is being blocked for incorrect use of the TE attribute. My next step should probably be to track down a Bad Behavior discussion/support forum so I can find out more about whether this is expected/wanted behavior.

The problem is that not enough people are using it. Which is why I’m developing version 2 to reach more platforms and stop even more spam on even more blog, wiki, forum and guestbook software packages.

In the meantime, while I haven’t driven the spammers out of business yet, I’ve at least been able to provide virtually spam-free blogging. Enjoy!