Web 2.1

I made a major change to TwitBlock the other night. The change was made to protect people who are heavily blocked, but are not "spam". Of course that depends on your definition. (A topic for another day)

Originally each block on account would yield 10 points. Then I became aware of just how murky this issue is. Barack Obama is blocked by many accounts (Republicans no doubt) plus some people with extreme right wing views were being blocked heavily. Then the complaints started. People whose businesses survive on a huge Twitter following accused me of destroying their reputations, and generating further blocks on their account by showing the number of existing blocks.

So now two things have changed for the time being:
1. Clicks on "not spam" are deducted from blocks;
2. Blocks are diluted by the size of a user's following. 10 points are added for every 1%. So, if you're blocked by 40 people, but are followed by 8,000 this will only yield 5 points.

Although this has stemmed the complaints, the scanner is less aggressive and lots of real spam accounts are not showing up with high enough scores. I am struggling to find the balance in the face of all of this and may have to tweak it again.

I just received an email that I thought would be of interest to everyone. I have removed people's names for reasons of privacy, but I have left the spelling mistakes in for a sense of realism. My open reply to the author follows at the bottom. Read the rest of this entry »

Constructive criticism of TwitBlock seems to have quite rapidly turned into some quite aggressive complaints including several demands for immediate closure.  This is not intended to be a malicious project, but with a sudden rush of large numbers, I am experiencing a minefield of issues that I did not predict. I am doing my best to fight these fires, but please bear a few things in mind a few things when feeding back to me:

First and foremost, this is alpha software.
I did not expect 7,000 visits a day after just three weeks. (I can thank Mashable for that overhead). I am constantly thinking of ways I can please everyone and still keep the application doing what it needs to do, which is help people remove illegitimate followers.

Read the rest of this entry »

- or – "I told you it was in Alpha"

I've rolled out an experimental TwitBlock feature designed to reduce "false positives" for legitimate accounts that are being blocked. Whitelist entries are now subtracted from blocks. i.e. accounts marked as "not spam" will have their blocks counteracted on a 1:1 basis. If this feature is abused, it will be removed. It survives on the premise that the spam bots are not capable of whitelisting each other.

Here's the full story:
Read the rest of this entry »

As we approach 3,000 TwitBlock users, we know of over 100,000 blocks and have stored 20,000 profile pic checksums. I figured it was time to start crunching some numbers.

The first of many reports shows the top 20 most duplicated avatars that we know about.

Many spam accounts use identical avatars across hundreds of accounts. TwitBlock uses this fact as an indicator of a likely spam account. This report just shows the top 20 that we've identified, but there are many more.

This indicator is one of the best ways Twitter could prevent spam accounts from signing up in the first place. Clearly bots have been developed that continually generate new accounts and Twitter does not seem able to prevent this despite the most prolific accounts displaying such identical properties.  With a tiny 0.01% of Twitter accounts authenticated with TwitBlock one can only imagine how many of these accounts are out there.

The list of Twitter accounts below all have something in common – They all have an identical profile image, which looks like this:

At the time of writing none of these accounts have been suspended. Whether they are breaking any laws or not I don't know, but it is clearly a syndicate whichever way you look at it. The profiles all point to a Korean-registered "Cash generator" website, which [I would hazard a guess] is a con.

TwitBlock unearthed this statistic from a list of only 18,000 100,000 blocked accounts provided by under 400 3,000 TwitBlock users . When you consider the size and growth of Twitter, you can well imagine that there are far more than 120 288 profiles in this syndicate. You also have to wonder how much of Twitter's growth figures can be attributed to this junk.

[ UPDATE:  18 Aug ]
Many of these accounts have been suspended, but TwitBlock is discovering new ones each day – currently 248 accounts known with this image.

[ UPDATE 19 Aug ]
I've produced a report of the top 20 most duplicated profile pics identified by TwitBlock

Read the rest of this entry »

A detailed explanation of the scoring mechanism used by TwitBlock.

Some people have complained that they get a high spam score and point out that they are not spammers. There are a number of important things to note about this.

• This software is in alpha – these indicators and the scoring mechanisms attached to them will change.
• As the system gathers data it will rely less on heuristics and more on cross-referencing (e.g. how many people have blocked an account)
• Some of these tests are only indicators of automation, not specifically of malicious behaviour.
• The spam rating has no limit – Scoring 40 may be high for a "legimate" account, but in a list with real spammers scoring 300+ you'll be way down the bottom.
• If you display characteristics of a spammer then perhaps this amounts to the same thing as being a spammer. Most normal users score zero.

Roughly in order of accuracy, here are the 8 tests currently performed in the standard TwitBlock scan.

Read the rest of this entry »

The day a thousand apps stool still

I noticed some weeks ago that Twitter's OAuth implementation didn't appear to be verifying signatures. I knew this because I purposefully set an invalid access token which was accepted unconditionally. I thought this was odd, but as a newbie to OAuth I was just happy that my app was working, so I filed the problem at the back of my mind under "deal with it if it becomes a problem". Today (the week I release by beloved TwitBlock app) it very suddenly became a problem.

Read the rest of this entry »

A bulk blocking and spam filter tool for Twitter

www.twitblock.org

I've finally got round to building the Twitter app I've been thinking about for months. While everyone else is preoccupied with making fun, or cool apps, I've been thinking about the increasing problem of spam and junk followers on Twitter. I won't go into why I think this is such a problem right now, plenty of time for that later.

This is just a quick announcement to say that I've released an early alpha version of a tool that I hope to develop into something genuinely useful. Currently it's a simple scanner that analyses your followers for signs of "spammy" behaviour. I'll post more details about these indicators soon, and I'll also share some of the interesting discoveries I've been making about Twitter spam as I go on my mission.

UPDATE: I have posted about these indicators

Read the rest of this entry »

If you know about the Qwitter service, then you may also know what people say about it – that it plain doesn't work. So for my first Twitter app, I decided to make one that does.

I have been made aware since then that there is also Twitdiff, although I haven't tried it at time of writing.

If you don't know about Qwitter, it's a service that monitors your Twitter followers and emails you if someone unfollows you. My app currently tweets the notification instead, so everyone will know you've been qwit.

I'm not offering my app as a public service [yet] I knocked it up in 2 hours and if you know what you're doing with a LAMP set-up you can download it and run it yourself.

» Download qwitter 0.1.2
Requires PHP >= 5.2.x  + json extension, MySQL >= 5.0.45

Read the rest of this entry »