Twitter Grader and Making Twitter Better

Written on August 30, 2008 by Adam Kmiec in Marketing & Advertising

Hubspot, the maker of Web Site Grader, has introduced a new product caller Twitter Grader. Twitter Grader seeks to quantify a person’s Twitter profile. Specifically, Twitter Grader, indicates they use the following information to arrive at a score:

The number of followers you have
The power of this network of followers
The pace of your updates
The completeness of your profile
…a few others

I think this is a GREAT first step in figuring out how to indicate a person’s value to the Twitter community. Admittedly, Twitter Grader is limited by the information that is readily available. Essentially, that can’t leverage an input for their algorithm that does NOT exist.

The one thing missing from Twitter Grader’s methodology is content quality. Unfortunately, there really isn’t a simple metric they can pull from because Twitter doesn’t offer one. I’ve asked Twitter to add in functionality that allows people to “thumbs up” or “thumbs down” a Tweet based on the value the reader believes they are/are not getting. I think this feature is a critical add on for a variety of reasons.

It’ll keep people honest. Tweeters will actually have to consider what type of content they publish.
It takes volume of Tweets out of the equation or at a minimum reduces its value. The current Twitter Grader model looks at frequency of posts. Well hell, that’s not a real determining factor of value. If I had no life I could just tweet non stop for 24 hours and overtake the current Tweet leader.
You’ll generate more Twitter members. The singe biggest gripe I hear about regarding Twitter is the signal to noise ratio. Many people complain to me that Twitter is filled with a lot of garbage and it’s too challenging to find the diamonds in the rough. I couldn’t agree more. Even if you follow, so-called industry leaders, you end up with a lot of shameless self-promotion.

The need for a quality scoring system in Twitter is no different than when we needed a better way to calculate search engine result relevancy. For those of you who can remember the web circa-1996, people could defeat the search engines by using a keyword terms over and over in the meta tag data or by repeating the word over and over on a page. The search engines looked at that page and said, “wow, these guys must know a lot about keyword term X since that keyword term comes up a lot.” Of course, we needed to evolve. We needed better quality and better relevancy for end users. The search engines got smarter and fixed the equation.

We need Twitter to fix the equation. If they don’t, they won’t attract new users, and the Twitter Sphere will simply be riddled with lots of Tweets by people about what they just ate.

Tags: Best Of, Hubspot, Opinion, Twitter, Twitter Grader

Carlos Abler

Adam.

This article is very well laid out. I think I have a suggestion that might augment the thumbs up/down approach, although it is still imperfect. As I agree with everything you are saying I think that we should have these mechanisms to support discourse and feedback and hopefully that will increase the overall value of Tweets.

First let me blab about some challenges. (If you just want to know my suggestion then scroll to bottom)

Conflicting Paradigms of Criteria.
If I really like random bitching or hearing about what type of sandwich someone is eating, or like reporting and hearing about all the fluctuations of emotions and digestion, and there are others who like the same, then that is the perfect content and venue for me. Likewise if all I want to do is talk and read about high culture and and there are others who want to do the same, then that is the perfect content and venue for me.

The above paragraph brings up a couple of things. 1) The challenge of irreconcilable paradigms of judgment in a single judgment scheme, 2) the complexification of community.

With irreconcilable paradigms, an arbitrary or narrowly reductionist choice must be made by a person or algorithm must be made. Voting does not do it, because democracy is not and indication of quality, it is an indication of popularity. In some venues votes to actually highlight better content. But the masses can be wrong. Like in my job, there are some things that I understand that i might be the only person in the room educated enough about a given issue to make the best judgment because of my training, research and experience. For example with usability and interaction design best practices. Just because 5 other people give the thumbs down, does not mean that a product based on that judgment will be of better quality. It will just have been the most popular in the room on that day. This is why the Senate exists.

With community complexification, this is in a way Twitters built in method of handling the quality filtration issue. Twitter does allow for me to follow or not follow someone, so there is already a built in mechanism for quality filtration based on subjective preference.

However, to the point of your tweets and blog, the overall quality, maturity and evolution of Twitter will be aided by mechanisms and discourse venues like what you call out. We must however be wary of anything automated in this respect, and we must beware that certain feedback types are not misapplied. (like if finability becomes a function of popularity we will have to dig through a lot of Britney Spears to find the Economist).

My suggestion? If the thumbs were not just a button you could hit and be done, but forced you to to leave a comment, then the votes would contain a lot more data to support the context of the judgment.

On the thumbs down side, if someone writes “I can never understand what the hell you are saying”, the lesson learned there is very different from “Your politics suck”. One the thumbs up side, if someone writes “I think you are brilliant” or “I think you are hot”, then we can see what the agendas of the thumbs up are. With thumbs up it is like the Peoples Choice award at the Webbies. Just because a designer on one project has a million friends they can spam to go and vote online, does not indicate that their site was a better quality site that the one that got the vote of the Digital Arts and Science Academy. (Not just using this example because my project came in second on the People’s Choice. OK, maybe i am)

Keep the pressure on brother.

Carlos
Carlos Abler

Adam.

This article is very well laid out. I think I have a suggestion that might augment the thumbs up/down approach, although it is still imperfect. As I agree with everything you are saying I think that we should have these mechanisms to support discourse and feedback and hopefully that will increase the overall value of Tweets.

First let me blab about some challenges. (If you just want to know my suggestion then scroll to bottom)

Conflicting Paradigms of Criteria.
If I really like random bitching or hearing about what type of sandwich someone is eating, or like reporting and hearing about all the fluctuations of emotions and digestion, and there are others who like the same, then that is the perfect content and venue for me. Likewise if all I want to do is talk and read about high culture and and there are others who want to do the same, then that is the perfect content and venue for me.

The above paragraph brings up a couple of things. 1) The challenge of irreconcilable paradigms of judgment in a single judgment scheme, 2) the complexification of community.

With irreconcilable paradigms, an arbitrary or narrowly reductionist choice must be made by a person or algorithm must be made. Voting does not do it, because democracy is not and indication of quality, it is an indication of popularity. In some venues votes to actually highlight better content. But the masses can be wrong. Like in my job, there are some things that I understand that i might be the only person in the room educated enough about a given issue to make the best judgment because of my training, research and experience. For example with usability and interaction design best practices. Just because 5 other people give the thumbs down, does not mean that a product based on that judgment will be of better quality. It will just have been the most popular in the room on that day. This is why the Senate exists.

With community complexification, this is in a way Twitters built in method of handling the quality filtration issue. Twitter does allow for me to follow or not follow someone, so there is already a built in mechanism for quality filtration based on subjective preference.

However, to the point of your tweets and blog, the overall quality, maturity and evolution of Twitter will be aided by mechanisms and discourse venues like what you call out. We must however be wary of anything automated in this respect, and we must beware that certain feedback types are not misapplied. (like if finability becomes a function of popularity we will have to dig through a lot of Britney Spears to find the Economist).

My suggestion? If the thumbs were not just a button you could hit and be done, but forced you to to leave a comment, then the votes would contain a lot more data to support the context of the judgment.

On the thumbs down side, if someone writes “I can never understand what the hell you are saying”, the lesson learned there is very different from “Your politics suck”. One the thumbs up side, if someone writes “I think you are brilliant” or “I think you are hot”, then we can see what the agendas of the thumbs up are. With thumbs up it is like the Peoples Choice award at the Webbies. Just because a designer on one project has a million friends they can spam to go and vote online, does not indicate that their site was a better quality site that the one that got the vote of the Digital Arts and Science Academy. (Not just using this example because my project came in second on the People’s Choice. OK, maybe i am)

Keep the pressure on brother.

Carlos
http://justinkownacki.blogspot.com/ Justin Kownacki

I agree, but (as my recent tweet — which led me here — explained) I think that rating system needs to be divided between:

* the relevance of each tweet TO EACH USER, and

* the relevance of each tweet AS A COMPOSITE WHOLE.

For example, someone might post a link to the all-time greatest SEO article in the history of the web, but if you only use Twitter to keep in touch with 3 people, that article means nothing to you. Thus, your thumbs-down shouldn’t deter an SEO expert from being able to see it — nor should their thumbs-up force that article onto the minimalist’s page.

Also, a thumbs up or thumbs down is too limiting. A scale of 1-5 or 1-10 stars will give far more accurate results over time. Relevance isn’t an either-or thing; it scales.
http://justinkownacki.blogspot.com/ Justin Kownacki

I agree, but (as my recent tweet — which led me here — explained) I think that rating system needs to be divided between:

* the relevance of each tweet TO EACH USER, and

* the relevance of each tweet AS A COMPOSITE WHOLE.

For example, someone might post a link to the all-time greatest SEO article in the history of the web, but if you only use Twitter to keep in touch with 3 people, that article means nothing to you. Thus, your thumbs-down shouldn’t deter an SEO expert from being able to see it — nor should their thumbs-up force that article onto the minimalist’s page.

Also, a thumbs up or thumbs down is too limiting. A scale of 1-5 or 1-10 stars will give far more accurate results over time. Relevance isn’t an either-or thing; it scales.
Pingback: Twitter Needs A Thumb
Pingback: Whiskey Notes and Parachuting Fish » Hey Twitter, Grade This