Sign up! Waking Dreams Secret Mixter
Home » Forums » Bugs » Concern about ratings

Concern about ratings

permalink   Mon, Apr 16, 2007 @ 6:10 AM
I wasn’t sure where to put this, but ‘bugs’ is close enough I think…

Looking over the first page of new remixes (right now) you can see an odd rating pattern: 5 songs rated at 5 stars, and the rest totally unrated. Next page? 4 rated at 5, and one rated at 4.

Now, it’s always good to be optimistic and share the love and all, but am I the only one who thinks this defies the point of having ratings at all? (unless the quality really HAS gone up that much in the last few months, which I doubt a bit) Is this something to do with having to leave reviews with ratings - that is, people don’t want others to know they just gave a low rating?

One solution, I think, would be to allow, or force, reviewers’ names to not be shown (eg, anonymous to all but admins and the reviewer themselves) so that they can leave negative reviews without fear of backlash or whatever.

Or perhaps give a weighted percentile that takes into account the number of ratings as well or some such zaniness.

(Hahah, and I really hope this post doesn’t result in a flood of 1-star ratings with text along the lines of “So I heard you think the ratings are too high”)
permalink   Sat, Apr 21, 2007 @ 8:12 AM
The review/ratings system currently in place seems to work well for the most part; I can understand concerns about “1-star trolls” as well as the perception of clique-ish “5-swapping”. I’m guilty of the same thing as ASHWAN, just hitting the stop button and moving on w/o a rating or review if it ain’t my cuppa tea; I don’t know about others, but 1st thing I often check out when I visit ccmixter is “recent reviewers” which gives me a better picture of tastes and attitudes than simply trying to wade through endless contest entry-remixes. I’m a little paranoid about taking up MB w/ stuff unworthy of publication, and as there is no way of telling whether an upload has been listened to X number of times w/o being rated/reviewed, I kinda just assume if something’s been up for a while w/ no feedback, it’s a case of “if you can’t say anything nice, then…” and I’ll delete the upload. Any criticism I’ve gotten has been polite, constructive and definitely appreciated- my only other thoughts on the ratings would be; w/o sub-division into categories such as “production quality”, “originality” etc. any explanation of a rating has to be in the review, where I think people often try to work around the limitations and subjective interpretations of 20%-increments; for some, 3/5 means “not-bad, average” whereas others might take it as “needs major fixing before suitable for upload”. I also try to review (and remix)stuff that I think might be missed otherwise.
Other than that, of course higher visibility and positive attention are goals for any of us or our work; but I NEVER approach my use of the site from a self-promotion-strategy viewpoint, what love I get or give should be honest and not some cynical manipulation of a community; otherwise eventually the original spirit of the site is lost and all the great people and music found here would evaporate from disillusionment.
Alright, I’ll shut my pie-hole now ;)
permalink   Fri, Sep 7, 2007 @ 2:55 PM
I posted about this very same subject about a month ago, and then turned right around and deleted it because I thought it might sound a bit whiney. I was just tired seeing all these tracks with no ratings, and a few that are 4 or 5 stars. It just seemed like if the track wasn’t worth a 4 or 5, it wasn’t worth reviewing/rating to anyone. I see that I was mostly right :)

But, as tacet said, what are we actually rating on? I mean, I try to rate based on the overall quality of the track, how well it was put together. If it’s a genre I don’t like (not many of those), or just doesn’t seem to click for me, I’ll usually just skip over it as well… (yes I’m just as guilty)…

Oh well. I dunno… like it was said above, I think we should all try and balance things by rating or at least reviewing tracks no one else will touch :)
permalink   Mon, Apr 16, 2007 @ 6:11 PM
Yeah, some nice ideas here. no system will ever be perfect when you are asking for judgment on art work though.
i think one of the other points (and this is the way I approach things) is that generally if I listen to something that I am not really feeling, I will just click the stop button and then move on. not bother to comment or rate. therefore when I leave a rating it is uaually a 4 or 5. i will generally only comment on something i don’t like if i feel i really have something positive to contribute or if a piece really offends me.
if it happens whereby you can remain anon, just be careful when you cuss Victors work, he will find you out! haha.
permalink   Wed, Apr 18, 2007 @ 8:42 AM
You raise an interesting point, and there is no easy answer.

We used to have fully anonymous ratings, and that actually worked less well than the current ratings system.

This is because the board always acquired a few trolls who essentially gave a “1” out of “5” to anything not within their own narrow genre ideas. On the one hand, it was healthy, because some folks who might have felt restrained felt able to give low ratings when appropriate. On the other hand, it proved unsuccessful, because some folks just love to try to give low ratings.

My own work sometimes deserves a 1 or 2, and even when I like it, it is sometimes is targeted to a genre not all folks like, resulting in low ratings. Therefore, in light of the two understandable reasons for low ratings,ratings didn’t make as much difference to me. I know at least one of my collaborations on this board which resulted in a fun set of music arose because neither my collaborator or I sounded much like anything else on the board. Nothing right or wrong about that, but we would have been ill-served to worry about ratings.

But some of the mixters who posted work which was fine or even quite good still got a percentage of needless 1 ratings.

I think that ratings only tell a limited amount about a work.

I try to review things by folks who don’t post often here, so that we don’t have so many unreviewed posts. although my review count is rather high, I’ve only reviewed a fraction of what is here. The contests always mean so many more things are posted than I can get to reviewing.
Also, I confess a mild bias that contest entries interest me just slightly less than the other postings.

Your ideas are good ideas. I prefer the current system. But rest assured that Victor has listened and thought through and continues to be open on such points. He’s proven that by experimenting with the ratings system, and I’m sure he’ll continue to work with everyone to hunt the right system.

I think that the editors’ pick system works better, although as with any such thing they pick things I might not pick, and don’t pick things I would pick. But that’s the purpose of a subjective “pick” system—so that the editors can pick what they really like.
I’ve never known a bad song to picked for editors’ pick, and all the very best songs have gotten the designation thus far.
permalink   Fri, Apr 20, 2007 @ 2:58 AM
Many other sites have gone with the “dig” concept instead of stars. The stars give an opportunity to say how much you like something, but then there’s a review option for that too.

I often also wonder what we’re rating things on? Musical structure? Production? Quality of recording? Presumably it’s a mix, that’s what I try and do personally.

I think the reason we’re getting clusters of 5’s in the reviews is that folks bounce-review each other. You give me five, I give you five, also some people email friends to announce a new remix.

Nothing wrong with any of these things, it’s what the community is about, it should happen more :)

The obvious solution is that we all do our best to review those which have no reviews, in the hope that we’ll be that spark of encouragement they need to continue.

Be honest, be fair and be constructive….oh and be happy ;)