Antispam

Effort to Neutralize Online Opinion - Russia uses DDoS

Anatoly Lubarsky — Sat, 26 May 2007 07:14:00 GMT

China uses Great Firewall in order to neutralize critical online opinion. Sites such as Wikipedia and NY Times are blocked in China.

Russia apparently has returned to the soviet 70s during the last several years. I think such period is called "Restauration" in history... Well it seems like Russia uses DDoS to shut down opposite online resources. See for example massive DDoS attacks against Estonia. LOL, in 70s it was all about producing noise on short wave frequencies in order to neutralize foreign radio broadcasts.

Recent problems with livejournal - with effort to shut down entire communities.

My personal message to the people responsible for these attacks is this:

............./´‾/)
............/...//
.........../...//
....../´‾/..../´‾\
.././.../..../..../.|_
(.(....(....(..../.)..)
.\................\/.../
..\................. /
....\..............(

Use Hardware Firewall to Block Spam

Anatoly Lubarsky — Tue, 27 Mar 2007 19:10:00 GMT

As I started to use dedicated hosting for my sites - I realized for sure that hardware firewall is the ultimate solution to prevent comment/referral/trackback spam. Social media common-used antispam solutions that exist are passive and lack of power. In the long run help escalating spam attacks and make them more sophisticated. All existing solutions try to fight against people while social media is dealing with sophisticated spam software solutions.

The cost of common antispam tactics

Common-used antispam solutions like:

deleting spam
captchas
banned keywords
nofollow attribute
etc

are not effective in the long run and help escalating spam attempts. Spam comes from spam software. Common spam solutions just need a list of keywords and some ad content to be configured initially. Afterwards the machine is able to query search engines for keywords and flood the network with millions of spam attempts while the post requests come from various public IPs and proxies. When it gets http response 200 (nofollow, banned keywords and captchas) - the software believes the attempt was success and will continue and escalate its attempts to spam.

Get hardware firewall

Seems like no-win situation for the blogging world and social media solutions out there ? Not exactly. If it is possible try to get and configure hardware firewall to get rid effectively from comment spammers. It will help make your servers more secure and finish with spam. You will still need to add captchas or some other application-level antispam tactics but it would serve only as additional protection level.

How to configure firewall

I'll try to demonstrate how easy it is to ban thousands of spammy IPs with the help of hardware firewall (I use Cisco PIX Firewall). It is less trivial to get real spammers IPs and distinguish them from public proxies, but I will save this info for another post. Initial configuration (images are clickable):

Create a group called something like "Banned group" for the "outside" network.
Configure firewall hosts/networks. Put all suspicious IPs on the "outside" network:
Configure firewall access rules to deny traffic for this group via http, https:
That's it. Next time you want to ban some IP - add it to the "banned" group.

Blog Spam: NewsGator Hacked ? (Bug in Spam Software)

Anatoly Lubarsky — Sat, 06 Jan 2007 05:29:00 GMT

Recently I'm getting reports on my site of spammy attempts to make GET/POST requests to the page called:

/ngs/siwin.aspx

All requests come from various IP addresses and various user-agents. All of them end with 404 - Page Not Found, since I really don't have such file. At first I thought that some spam software has bug in attempting to figure out the blog engine and post comment page url. Googling does not seem to help.

But recently after some investigation I realized that newsgator has such url structure and somehow is related to it. I'm subscribed to newsgator but don't use it too much...

So, I think: it is either a bug in newsgator or some doorway hacked and used by blog spammers.

Interesting, what people behind newsgator think about this.

P.S. : I think it is spam because these requests usually come in bulks (tens of them at once)...

Update: seems like some spam software checked for keywords in the content of the site to find out what blogging software is used in order to use the right protocol posting. Seems like newsgator subscribe button caused this - spam software mistakenly supposed that "subedit" blogging tool used (if I'm correct ?). And tries to post comments to the according comment page. As soon as I removed newsgator's subscribe button I stopped getting these error messages.

Antispam: WebaltBot = Dirty Spammer

Anatoly Lubarsky — Tue, 26 Dec 2006 21:21:00 GMT

Lately I posted some thoughts regarding blocking comment spam attempts on blog engines - Blocking Comment Spam Attempts. It is a problem since there is no single efficient technique right now to prevent POST attempts by blatant spammy robots. All techniques, except blocking by IP prevent the spam just to get into the database and that's it.

Something called Webaltbot is a blog spam robot that is able to POST via random IPs. GET requests come from the single original IP, so this way I was able to spot him (72.232.83.106). It uses the following user-agent string:

Mozilla/5.0 compatible WebaltBot/1.00 (i686-pc-linux)

Webaltbot is very blatant producing tens of POST attempts per minute. Take care...

Thoughts on Comment Spam

Anatoly Lubarsky — Sat, 16 Dec 2006 18:26:00 GMT

Wanted to speak about comment spam in blogs. While bloggers are aware of comment spam by the end of 2006, there are still many popular blogs flooded with spammy comments, for example del.icio.us blog. There are some popular solutions that proved to be not effective:

Comment moderation - not scalable, one cannot moderate hundreds of comments per minute. Other cons: this way HTTP 200 response is returned to spammer, which makes spam software believe it was success. And it will continue with spam attempts.
Closing comments on older posts. This one is lame, imho.
Content filtering - easy to work around. It is also usually a lot of overhead.
rel-nofollow - returns HTTP 200 to spammer, so it does not prevent flooding.

So it leaves us with CAPTCHA which proved to be the most effective. While it also has some cons:

It reduces dramatically the number of normal comments, since users have to do more work.
There are solutions that beat CAPTCHA. While these solutions exist, I can say it is not productive for spammer to distribute such solution as a software, because it is not trivial to implement and customize for each and every CAPTCHA solution. So deny access by IP address can usually solve the problem.
Running long time with CAPTCHA can cause DoS, because there can be significant number of unsuccessful attmepts. So here I think need to combine it with IP blocking maintaining some sort of spam IP black list. Or customize CAPTCHA in a certain way to not return 200 HTTP response status. Which is difficult to implement since it can be normal user, who made a mistake reading CAPTCHA.

Lately I had as much as thousands comment attempts per day on this blog which nearly caused DoS. So I:

Started to monitor all attempts.
Collected most blatant waves of spam IP addresses.
Ranked the IP addresses list.
Banned spammers with most attempts by IP. BTW, having this list in front of me I don't think that most spammers use public proxies to post (which is a comon belief). Most of them just spam from their own computer/s. There are very few that use routing to public proxies IPs, so I don't care about them right now (not to mention it will play against them in the long run).

Human Comment Spammers (Distraught Visitors)

Anatoly Lubarsky — Mon, 20 Nov 2006 22:46:00 GMT

My blog is rather protected in terms of comment and referral spam. See for example:

Antispam part 1 Fighting weblog Comment Spam
Antispam part 2 Anti Referral Spam Patch

But today I had one of those distraught visitors on my blog. Fighting CAPTCHA he succeeded to post approximately 50 comments in 2 hours simply copy-pasting "good" and "ok". It didn't take me that much to delete his comments but bothered me since I get all comments mailed to my inbox. Because of this I hope Chinese Communists will take him one day (IP address: 222.130.199.37).

Speaking of comment spam I don't use "nofollow" attribute since I don't like it (not to mention it is not effective and only escalates spammy attempts). Also I don't close comments for older posts.

Anti Trackback Spam Patch

Anatoly Lubarsky — Mon, 13 Jun 2005 14:22:00 GMT

Following Referral Spam Patch implemented here 2 months ago, I patched the weblog engine today against the trackback spammers. This is the most hi-tech spam attack, but it was very easy to block. Using the same blog_BannedURLs table. Now my weblog is fully protected against spammers -- comment spam, referral spam, trackback spam.

Anti Referral Spam Patch

Anatoly Lubarsky — Sat, 26 Mar 2005 02:29:00 GMT

CAPTCHA control helps me to fight comment spam. It is very effective. Nevertheless, I get lots of referrer spam each day during these months. Even though the most aggressive IPs are banned on the hosters level, I still get each day about 100 referrer spam records in the database. Usually I delete them and have already created a T-SQL script, that does the work. There are not so many anti-referrer spam solutions for .Text based engines. Almost all of them use instead of trigger that work around the original .Text T-SQL procedure to patch the table. I think, that instead of trigger is not a good solution because of performance overhead and trigger maintenance. Instead, my solution uses 2 simple patches to stored procedures.

Step 1 -- Create a banned keywords table and fill it with keywords (like poker). Something like that:

CREATE TABLE blog_BannedURLs
(
    theWord VARCHAR (255) NOT NULL,
    CONSTRAINT [PK_blog_BannedURLs] PRIMARY KEY  CLUSTERED 
    (
        theWord
    ) ON [PRIMARY] 
) ON [PRIMARY]
GO

Step 2 -- Patch blog_GetUrlID procedure

ALTER PROC dbo.blog_GetUrlID
(
     @Url   NVARCHAR(255)
    ,@UrlID INT OUT
)
AS
BEGIN
    SET NOCOUNT ON

    SET @UrlID = -1

    IF ((@Url IS NOT NULL) AND (LEN(@Url) > 0))
    BEGIN
        SELECT @UrlID = UrlID 
          FROM blog_Urls WITH (NOLOCK) 
         WHERE Url = @Url

        IF (@@ROWCOUNT = 0)
        BEGIN
      
            DECLARE @BannedWordsCount INT
            SELECT @BannedWordsCount = COUNT(theWord) 
              FROM blog_bannedURLs WITH (NOLOCK)
             WHERE CHARINDEX(theWord, @URL, 0) > 0
      
            IF (@BannedWordsCount) > 0
            BEGIN
                SELECT @UrlID = -1 
            END
            ELSE
            BEGIN
                INSERT INTO blog_Urls 
                VALUES(@Url)
      
                SELECT @UrlID = @@IDENTITY
            END
        END
    END

    SET NOCOUNT OFF
END

Step 3 -- add (@UrlID > -1) to the IF in InsertReferral

ALTER PROC dbo.blog_InsertReferral
...
...
IF ((@UrlID IS NOT NULL) AND (@UrlID > -1))

It's ready. Enjoy :)

Updated 2005-04-02

x2line Antispam Updates - Implementing nofollow

Anatoly Lubarsky — Fri, 21 Jan 2005 01:05:00 GMT

From now on x2line.com site supports automatic rel nofollow tag for links in comments. I suspect "nofollow" will not prevent comment spam. Nevertheless I think that Google is planning to develop nofollow feature towards something big and conceptual, we are just at the beginning of some process, so I don't want to be left behind. Anyway CAPTCHA control is rather more powerful.

One sucker is killed, more to come:

address: Singel, 258 1016 AB Amsterdam The Netherlands; phone: +31 20 535 4444; fax-no: +31 20 545 4445

He was a Weblog Spammer No 1 here. Posting from lots of IPs. BTW, all comment spammers post using Windows Server 2003. All of them. Think about that.

And one another useful thing I did lately is preventing "Ask Jeeves" bot. It is very rude and should be prevented.

Update: nofollow support was removed from this site.

Starting to Fight Comment Spammers. Initial Black List

Anatoly Lubarsky — Sat, 25 Dec 2004 18:42:00 GMT

Some statistics since integrating CAPTCHA control for ASP.NET. There were more than 2000 attempts to post a comment spam on this site. All failed. Interesting that half of them came from .info domain. Here is a complete list of spammers ads urls obfuscated (just remove "aa" to read):

'%.iaanfo%'
'%teaaxas%'
'%caaasino%'
'%moaartgages%'
'%diaaet-pills%'
'%phaaentermine%'
'%poaaker.mall%'
'%inaasurance.mall%'
'%haaandmade2000%'
'%12aa.163.72.13%'
'%maaaloylawn%'
'%muaasic.us%'
'%acaars.us%'
'%heaarmosa.us%'
'%s.aaco.uk%'
'%twaaist.co.uk%'
'%traaavel.com%'
'%moaanavaletoys.com%'
'%caaarlson.com%'
'%reaatreat.org%'
'%caaarolinas.org%'
'%doaalphins.org%'
'%heaalps.org%'
'%foaaundation.org%'
'%vaaatives.org%'

Update (this one was hard to find):

'%hoaasting4u%'

Update 2:

'%4faaree%'
'%weaab4u%'