Forums : Ohloh General Discussion

Hi,

I'd like to point out to two accounts, dpavlin and kost. While I fully respect dpavlin, deducting from what his account page says about his work (Svaka čast, Pavlin!), and I don't want anyone to take this personally, but their example makes me question the quality of Ohloh's KudoRank calculations.

As you can see, kost has not reported work on any projects. He has zero activity on Ohloh. Yet, he gets KudoRank of 8. From Ohloh documentation: KudoRank 8 is relatively uncommon, with about 6% of the population receiving this rank.

So, a person with (as far as the system is concerned) zero activity whatsoever gets KudoRank of 8? I understand that he inherited it from dpavlin, however I don't understand how can dpavlin have so much influence on kost. I have over a year activity opensource reported to Ohloh, and I have a KudoRank of 1.

What is wrong here?

Ivan Vučica about 18 years ago

Hi ivucica,

Some explanations will probably help here.

Stacks are important for KudoRank. The more people who include your software in their stack, the higher your KudoRank will become. That's the primary reason why dpavlin has such a high KudoRank -- he is a very active contributor to FUSE, which is a very popular project.

Kudos come from people, not from coding. Ohloh only awards Kudos for development work you have done when other people add your code to their stacks. Not to take anything away from your work, but because no one else has added your work to their stacks, you are not receiving any Kudos for that work. If you would like to see your KudoRank rise, you need to convince other people to use your software. Until then, your KudoRank will remain at 1.

Regarding kost, yes, it is a bit odd that a single Kudo vote can elevate his rank so much, but it is a single vote from a very highly ranked developer. Also, dpavlin has been very judicious with his Kudos, and he has not handed out Kudos to very many people, which makes his vote very powerful.

It looks like you have received some new Kudos and stacks recently, so things might improve with the next KudoRank calculation tonight. Be aware, however, that Kudos from empty accounts are not going to count for much.

I hope this helps,

Robin

Robin Luckey about 18 years ago

Thank you for quick information.

I believe I understand the kudo system better. And perhaps the reduction of strength of vote should be in order -- I do not mean significant, but at least something reasonable (definitely not more than 5 or 6).

However, I'd like to point out that team of opentibia does not have any kudos, and opentibia itself doesn't have stacks (I've began a promotional campaign in the community so people are beginning to register and stack it) yet the kudo rank of the long-term developers over there around 5 or 6, and has been for very long (when the stack count of opentibia was around 1).

Ivan Vučica about 18 years ago

It would be more useful if new users' initial Kudos weren't assumed to be more heavily weighted, because they haven't had a chance to spread Kudos around. There's no sense assuming they're prudent when they haven't yet proven themselves one way or another.

One easy way to do it would be to only increase the weight of someone's opinion for each day they go without giving away any Kudos. That way, you still reward people who are indeed careful with praise, while still being skeptical of users who haven't established themselves.

See my blog post on the topic, and please tell me if I'm wrong.

Marty Alchin about 18 years ago

Wouldn't that mean weighing people most heavily who are there for the long term?

And creating the danger that only longterm active ones can have a high Kudos rank?

It should be weighted to also avoid concentrating all kodu might on established people.
But it should as well take the activity into account, which is done through stacks, though. If you're active and people value your project, then you'll have high impact. And the chance of people stacking your project is higher when you've been active for a long time.

Arne Babenhause... about 18 years ago

Hi Gulopine,

KudoRank posts always become much longer than I intend. Perhaps this is a sign that KudoRank is too mysterious for its own good. In any case, I hope you'll forgive the long post.

All of the kudos and stacks on Ohloh form a giant graph of interpersonal trust. Ohloh simply assigns KudoRank to people based on their positions within this graph.

At first, new Ohloh members are completely disconnected from this graph because that have no stacks or kudos. Because they are disconnected, we can't draw any conclusions about them, and we display them with a default minimal KudoRank of 1.

Then, abruptly, Ohloh members get their first kudo or stack, and suddenly they're in. When this happen, Ohloh is able to position them in the graph, and assign the first real KudoRank. It's a crude approximation based on a single data point, but it's better than nothing. There is necessarily a discontinuity at this point -- from nothing to something -- which you observed when Jonathan gave you your first Kudo.

Forgive me for trying to distill your thoughtful blog post into a single sentence, but you seem to argue that KudoRanks would be better served if Ohloh waited to act on this new information. Let me respectfully disagree for a few reasons.

KudoRank is a perhaps foolish attempt to be a reflection of real people and the work they have done. Reputations, skills, and development work are real things, outside of Ohloh, and as much as possible we want try to use only the real things to gauge KudoRanks. You and Jonathan have clearly known eachother since before yesterday, and the fact that the data was entered onto Ohloh yesterday is poor measurement the value of your relationship, not based in reality.

Also, if nothing changes in the future, Ohloh would eventually decide to award you the KudoRank 7 you've deserved all along. What has been served by misrepresenting you before this time? You must admit that 7 is a much better approximation of your real skills and reputation than 1, by a large factor. To change KudoRanks slowly is to be wrong on purpose, and the stability would merely be an illusion. Personally, you aren't really changing from a 1 to a 7 in your skills; our KudoRank was simply wrong before, and now it's less wrong. It's not a flaw that we corrected ourselves so quickly.

Over time, the stability in KudoRank you are seeking will naturally be realized. The single thread that now connects you to the graph will eventually expand into a web (sign up your coworkers!), and you'll become less dependent on a single data point for your rank. Over time, more developers will join Ohloh (I hope!) and the increased connectivity will make the KudoRanks for everyone more stable and accurate.

Robin Luckey about 18 years ago

Thanks for taking the time to read my post and respond so thoroughly. I do understand where you're coming from, but I'd like to correct just a couple things.

First, I hardly think that Ohloh would wait to act on the information. I simply think it shouldn't act quite so enthusiastically that quickly. But that's largely irrelevant anyway. I do realize, like you said, that KudoRank is a crude approximation of how the world really works, and we can never really expect it to be accurate. I just figure that if we try, we should really try. :)

Second, I guess I'm of the mind that I don't deserve a KudoRank of 7. Not yet, anyway. It's not supopsed to be based on my skills, but my reputation. Just like the real world, you have to prove yourself any time you go into a new environment, regardless of how good you are. Sure, more highly skilled people can usually do that more quickly, but it still needs to be done.

Maybe I should just be glad that Ohloh's more friendly than the real world.

Also, as a side note, are there really so few Kudos being awarded that simply being added to the graph puts me ahead of almost 90% of Ohloh users? Maybe Ohloh's just too new, I suppose. Maybe just having more people with Kudos would solve the problem.

Actually, as a side-side note, the 74619 uses mentioned in the KudoRank thing: are those actual Ohloh users, or are unclaimed usernames from version control included in that number? If it's all the names, that alone would explain a lot, really.

Marty Alchin about 18 years ago

Well, I very very quickly grew to rank of 9 but, not that I complain, but I don't think I deserve it. However I fully understand the difficulty in accurately ranking people, and thank you again for developing Ohloh as good as it is (it's great!)

Ivan Vučica about 18 years ago

I think a Kudos rank doesn't try to simulate you proving yourself in your new environment, but to deduce your already existing reputation from kudos and stacks.

You don't begin a completely new project in Ohloh, so why should your reputation start at 0?

Arne Babenhause... about 18 years ago