Suggestion: rating system

There is a thread about about underrated tracks [url=http://www.wikiloops.com/forum/viewthread.php?thread_id=963] here [/url].
I believe that some of the problem partly is that wikiloops is growing fast and that it is very easy to miss out the real gems. But the number of users could be an advantage if used right. One way of making use of this would be to change the rating system:

When a track gets rated the tracks forming the base also get rated. What i have in mind is a system that requires a fractional point system (the rating can still be shown as integers by rounding before displaying). The points given should be based on the reciprocal of the "mixing distance" for example:
[url=http://www.wikiloops.com/backingtrack-jam-48341.php] #48341 [/url]gets thumb. That means that [url=http://www.wikiloops.com/backingtrack-jam-48222.php] #48222[/url] gets half a tumb and [url=http://www.wikiloops.com/backingtrack-jam-45012.php] #45012 [/url]gets 0.333 (1/3) thumbs.

Not only increases this the "fairness" of the system, it also performs the more important function of helping users to find really good tracks.

One thing that also could be considered is to let the number of listens (by playlists, albums and radio) influence the rating

---

I agree Nilton, I had the same thoughts. If a song at the end of the tree is good and gets lots of thumbs, then the template should get it too.

Also, if there was a way to see if a song has been listened to lets say 20 times for at least 3/4 of the length, it should generate an automatic like. However the drawback of this is that some people may be tempted to abuse this system.

Abuse is always an issue, no matter what system you have. But my idea was not people boosting their ego with likes but to establish a mechanism to help others find good tracks by turning the growth of wikiloops from a disadvantage to an advantage

---

Hey Nilton,
First of all, +1 for taking this discussion away from Slims' thread, it was getting too far away from his initial idea there :)

Now, this is a very, very complex matter, and I do like such tasks a lot.
Since you asked for it, let my try to take the whole thing apart for you.

Basis of our discussion is the feeling of "I might be missing some great stuff here", which lead to the thought of "maybe a changed rating system could help me not miss good stuff".

One way to change the rating system has been outlined by yourself above, Lairdy actually suggested removing the rating system and I believe the current rating system is good as it is.

Let me explain.

First of all, lets look at the feeling of "I might be missing some of the good stuff".
I would like to stress the point that no one said "I am having trouble finding any good stuff", so that does not seem to be our problem.
So, the problem is rather that we all know there is a lot of good stuff, and we know how to find it, but we have the feeling there might be more good stuff than what we found so far... ok, then.

Lets talk about the "good stuff", the "hidden gems" - what are their common aspects?
I believe, a track with 120 thumbs and 23 remixes is not perceived as one of those "gems" any longer, we are wanting to stumble on those forgotten, unremixed and underrated tracks with 4 thumbs on a magnificent track, right?
Mind, these criteria a very, very much influenced by our knowledge of wikiloops - if the average track received 120 thumbs, our criteria would look different.

Besides that, our idea of what "good stuff" is is highly individual! Are we talking about "good stuff with vocals" (for the listener), "good stuff to improvise on" (ideally missing your instrument), or what?

So, whatever we do, we need to keep those personal preferrences open for you to choose, or our tracks to offer will not match your desires.

Our gem-search would work like this:
Find [my desired instruments & genres] which are [not totally popular] and - most importantly - [not known to me so far].
These criteria are all met by the "order by date" function - the latest jams are most likely to be unknown to you, not overly popular yet and are filterable in any way you like.

I know this does not make you all happy, hang in there. I know its complex.

Before going on, let me introduce my bold anti-thesis to our problem:
It goes like this: "If you invest some time spotting "good stuff" in the list of latest contributions and follow some people you found worthy, you will not be missing (m)any gems at all."
Why?
Let's have ourselfs an example.
So, we have track X which you have missed, and which for some wicked reason only got four thumbs while it was among the "latest 10".
You might think this track would vanish into the heap of tracks, but:
wikiloops has some kind of swarm intelligence going on.
It only takes one person with some followers who creates a remix of track X, and you will get to hear it when you are notified about the remix by newsfeed.
I have discovered quite a lot of interesting tracks and musicians this way, and "unknown template X" can end up with a huge remix tree that way.
You are however correct in your observation that track X may be stuck with 4 thumbs, while it has remixes with 30+ thumbs. I will get back to that a little later.

The second reason why track X will not vanish besides getting remixed by someone with a wider follower base is this:
Given - our track is not among the latest jams any longer, it "only" attracted a few thumbs and has not been remixed at all. Where would we find this track?
Possibly in the best rated tracks list. Surprise, surprise.

The label on the search-filter does say "best rated", it does not say "most thumbed",
and the best-rated-filter is based on a quite complicated algo which calculates an index value based on something like (thumbs+downloads+remixes) / listeners -
a completely "missed" track with 4 thumbs and 45 listeners will rank a lot better than a track with 12 thumbs and 3.000 listeners.
This explains why the "best rated" list is slowly, but constantly changing - and the fact that an instrumental track without any lady singing is on position 1 today prooves that the algo is doing quite a good job in evening-out the "unfair" thumb-effects some complain about.

What you need to understand that the working rating system is already trying to show you some real jewels, but is also designed to exchange them with others who also deserve some attention. If there was no such mechanism at work, the top listed tracks would stay forever, just because they are more likely to attract even more thumbs than the tracks on page 2.

Bottom line is - there we have the two layers of safety which do help to fish out any gems we are affraid to loose. If they don't get remixed by one of your buddies, you will see them return via the "best rated" lists if they deserve to show up there.
Of course you are welcome to "dig" a little by browsing results page 100 and beyond, where you'll find some good stuff on its way to fame :)


Now, let me get back to your idea of a changed rating system once more before I end.
As you will hopefully understand now, the effect of user ratings / thumbs on a tracks presentation is a lot smaller than you may have expected, so the need to change the system may not seem as urgent.
The problem about giving fractures of a thumb to prior mixing steps is that we'd definetly see the template tracks benefit from this, while later remixes will win less or have no benefit at all.
Right now, templates and early mix-stages already benefit from getting more points for remixes and downloads, so I don't see why they should receive additional points - as you will see when looking at the search results, there are few tracks with many participants listed there, the "best funk" 1st page features results ranging from 2-5 participants with only two tracks with 4 or more... that is quite balanced (mind, one-instrument templates are excluded on purpose!).

I could go on for long telling you about the different special algos used to present the best possible selections of tracks, it is an interesting thing to think about for sure.

IF I had to improve or change anything about the current system,
a filter showing me a list of new members contributions-only OR a filter to show the "best rated of the last three months" might be nice to have to go gem-hunting -
to change the rating system would lead to such a huge scoring difference between older and newer tracks that it would be almost impossible to even that out by math... and I'm carefull to fix something that actually works quite decent :)

Ok, i was under the impression that ratings = thumbs. So i was partly kicking in open doors, sorry 'bout that.

But, i cannot get rid of the feeling that with the current rate of uploads i (and therefore others) AM missing out gems. And some of them can be found by the "order by date" function. I tried it as you suggested and it helps some, thank you. But then again, I still believe there is a (large and growing) gap between the different search results. Especially if you, like me, don't give too much for genres or instrumentation and often define fields pretty loosely. But you are absolutely right, a different rating system will not help here at all.

Maybe some different search options...?

---

I guess there are many ways to explain that feeling you (and many others) express, and I do share it myself.
I had to let go of the idea "I know who is playing what on the 'loops" quite some time ago, and with 50k tracks in the database, I must admit that I have come to terms with the idea that I will not be able to listen to each of them, accepting that I will miss some fine tracks.
Once you start thinking about it, we all are missing stuff all the time - can't possibly not miss everything that is out there. You'd go nuts trying to not-miss anything on youtube...
Maybe we are not used to wikiloops being that way, and the feeling does only apply to a group of users who remember the small community we were some time ago, where it used to be quite easy not to miss anything. Maybe that is so, it is hard to tell.

Another approach on the topic is the simple-logic one:
"If the rate of uploads has increased, you need to increase your time checking out fresh uploads if you want to make sure you don't miss anything." - it's silly, but it does show that this goal can't be solved by smarter presentation.

There is however certainly things that may be improved to adopt the changing situation :)

As you can imagine, I have spent quite a lot of time thinking about this, and for the sake of the fun of it, let me share some sketched ideas of what could be done - as you will see, most of these ideas do not lead to anything good at some point.

Let's start with the "ignorance is bliss"-model.
I could start to analyze your browsing behaviour, and if you have listened to five tracks of user X and have never interacted in any way (no comment, download, thumb, not listened at least for X seconds...), I could simply hide this users contributions and any tracks this user is part of, without any further notice towards you. That might lead to a "cleaner" list of tracks to find gems in, making you feel less lost looking at the list of new uploads. I believe the downside of not seeing the whole of wikiloops this way is somewhat scary.
Today, your newsfeed is representing this idea - it only shows tracks of people you like and tracks you are involved in. But you still have the choice to increase your horizon with the other options to browse around.

Second, why not do a "Tracks nilton will like"-list?
First, let me analyze which tracks you have remixed or downloaded. Aha.
So, you like drum & bass templates from various genres, not including ABC.
So, thats what I'll show you, but let me exclude tracks by people you already follow (= you had them on your newsfeed), and tracks you are involved in or have reacted on before.
You will never stumble on gems from other genres, or featuring a different lineup than what you have preferred in the past - such a self-learning semi-intelligent thing will at best show you some nice stuff you have not heard before, it will never show you that "wow, what a gem!" unexpected thing you had not thought of at all. I'm really not convinced it would be worth trying.

So, lets do a "things people like nilton like"-List
Another idea, similar to amazons "people who bougth this also liked..."-concept.
I could try to find out if you are part of a distinguishable group of users with a certain common taste, p.e. "those who never like pink sock funk", or whatever the common denominator may be. Doing this without any referrece to your list of followers and people you are following would be an interesting experiment- I do have very strong doubts about the quality of results, tho.
If users would mark tracks with a deliberate "this is the kind of stuff I'm after"-button-click to teach such a system, the results may get better the more data is collected,
to do this by analysis of the existing data would not lead to great results, especially not if you have a diverse taste in music and are open to give a thumb on a somehow cool track which you would have never even thought of joining...

All of the above have a more or less obvious "messing with your head & spying on you by data analysis" character about them which leaves a certain taste in my mouth.
Interesting ideas, nice experiments... but not what we really need, right? Needless to say this kind of technique is present on a lot more websites than you'd expect.

The only thing I feel really sure about as far as needed search options are concerned is:
We need a "show tracks with well-rated chord descriptions" filter for people looking for practise tracks & in need of chords. That is indeed a missing and simple feature about the search page...

Looking foreward to ideas and remarks, it's always interesting to fiddle with ideas :)

Interesting ideas, certainly. And i totally agree about the taste in the mouth.

And yes, a more detailed description would open up for very interesting search options. But some difficulties here.

1) The current description system leaves some things to be desired
2) People (me among them) are reluctant to use it
3) Even if the first two issues were removed, the usefulness would still be limited

IMO a search system would contain information of
A) Key and mode
B) tempo
C) Rythm, not only beat but also for example timeline and cycle
D) Chord sequence independent of A) like nashville number system or similar
E) Form and structure

For me it is quite evident that such a system must be automated since many users will not enter this information for various reasons. I do not fully know what technologies exist but i am pretty sure that most musical classifiers produce data that are hard for humans to interpret. I also do not know to what extent these technologies are available outside the research or strictly commercial domain.

---

Edited by nilton on Januar 24 2016 21:13

---

sorry guys,
you may call me a nihilist, but I don't understand this thread:

I mean: What are you talking about? As far as I understand it,
the first theme is:
you are talking about "thumbs" and I my impression is, that these "thumbs" are important.
But what can I do with (lets say) 100 thumbs!
Can I make music with them, can I use them as a blanket, can I paper a wall of my studio with them? ...... Can I even exchange them into bitcoins or any other currency?
My perception is: No you can't!
If you could do any of the points, mentioned beyond with any number of thumbs, you perhaps would have reason to think about a marketing strategy.

But luckily you don't need to consider this, cause:
This is a website for noncommercial music.[/b] (isn't it?)

Secondly
you talk about the difficulties to find good music cause the rating system does not support a time saving search in a sufficient way.

My experience in this regard:
I often like music far away from the mainstream. You may say: "Then it must be easy to find music which you like: Just search for music which nobody likes!"

But be sure: Also for people like me it is not so easy to find music to which I really would like to play to.

What I want to say: No matter which music style you prefer, you will never be able to create a rating system, which allows everybody, not to spend no more time for listening to music of the less perfered kind.

That's life!

Thanks for your kind attention
Will

Dick just spent quite a bit of time reading this thread and must admit I'm having trouble digesting it all ! However there seems to be quite a few flaws in what I believe you are saying about a few things here ! So I will try to put it in point form ! I must say that for all these points I have used my own tracks for reference as these are the ones I know the best and no other reason .

1) Track searching !
I must admit I haven't used this feature a lot but after reading your message decided to test it out !
I searched blues tracks with vocals and to sort by date !
The results ) only about 50 % of my tracks appeared in the search ?

2 ) Thumbs or likes ?
As you are well aware I am not a great fan of the like button ! This is not because I don't think I get enough thumbs as on average I get more than a lot of others and similar to most on here !
Yes your point that getting likes makes one feel good is correct but on the other side of the coin not getting thumbs can be very depressing especially if you think your track has been overlooked for whatever reason .
Also by having a ( personal ) like system it has the feel of this is a competition !
Surely a system that gave the thumbs to an overall jam rather than an individual could be a much fairer system ( that is if you must have them at all ) I think if there was a way to promote more comments rather than thumbs would be a move in the right direction !

3) The Albums
I think this was a very initiative and great idea , but again some sort of favouritism seems to be in play here !
I would have presumed that the albums would rotate from top to bottom on a daily basis which would be very fair to all !
However the same album has been in the number 1 position for weeks ! ( why is this so ) ?
It is obvious that an album in the number 1 spot is going to get many more listens than number 30 and if this is worked out by thumbs or the amount of listens then other albums will get forgotten and lost !
It maybe nessasery to also limit the amount of albums to each person or there are going to end up being thousands of them and an album will disappear as quick as the jams do ! Maybe so many per musician and to put up a new one you need to take one down .

Although I have many other things I would love to talk about ( like supporting members verses non supporting ) I think this is enough for one session !

Thanks Dick I know it must be hard having your job !
Keith Laird ( concerned member )

It were thought similar to this that led me to start this post. But as Dick explained thumbs/likes are only a (small) part of the rating system. And as he explained it it actually works quite well. I think the core here is that thumbs/likes perform two functions:
1) Being a instant feedback mechanism
2) Providing input to the rating system together with number of listeners and downloads.

So we are talking about two different issues here:
A) The user getting, or not getting, feedback by likes and feeling somehow aggrieved. A small issue in a sense since there are well known ways around that. Dick has mentioned them like providing feedback yourself and the album feature both of them working as well or better as could be expected.

B) The possibility to find "Gems" or the optimal jam-track for the occasion. Which is different from A) and a much more important issue, at least for me.

---

Yes I'm aware of what your intention was Nilton I don't think that Dicks answer for me anyway was convincing and if they are such a small part why are they needed you still get to see the overall jam stats ! I do know a lot of musicians get downhearted and leave I can't say for sure if this is to do with the like system but I know I gave chatted with a lot of musicians on here that think the system is wrong !

---

Edited by onewholeft on Januar 25 2016 12:06

Which means that you not simply can take it out of the equation without replacing it with something else.

I, at least, appreciate a rating system which helps me find what i want using a minimum of time, effort, frustration and other resources. I was not fully aware of its inner workings when i started this thread, but after carefully reading the explanations it makes more sense.

If there is a possibility to introduce a (automated) classifier which helps find tracks that could complement or even replace the rating system

---

Edited by nilton on Januar 25 2016 13:29

---

Yes mate but we are talking about 2 different things - you are talking about the searching system I am talking about people's feelings ! And what the thumbs system does to deflate their feelings ! And I believe ( I'm not saying I'm right ) that the thumb system does more harm than the few egos it encourages !

---

Edited by onewholeft on Januar 25 2016 12:47

Thanks for trying to single out the issues here, nilton :)
I believe enough has been said concerning the (initial) thumbs topic. The relevance of thumbs is smaller than you may have thought, so lets leave it at that for now.

I'd like to comment on Lairdys experience when using the earch engine:
The "order by date" function orders tracks from new to old. If you do not set any other filters, you will see that all used track IDs are displayed in descending order.

This still works fine if you filter by genres.

However, as soon as you pick "wanted instruments", like "Vocal" in Lairdys case, there is a logical conflict which arises from the undetermined question:
are you looking for "vocal only" templates, or are you looking for "any track which includes vocals".
What the search engine will do is show you a mix of both, and it will sort some "vocal only"-templates to the very top page, even tho they are way older than the latest track which also contains vocals.
So, the "order by date" function is "disturbed" by the attempt to present the most exact match to your search request. The perception "only 50% of my tracks are listed" is a result of this effect, and it is not correct. All your tracks are listed, but in a different order than you may have expected.

The whole thing is desiged to serve you the best backing track matching your criteria, if you look for drums and bass combos, it would be of no use to show you all tracks including drum and bass, so a rule of "preferr exact lineup-matches over tracks which contain additional instruments" does make some sense.
The ideal place to find your own contributions will be your profile page, I guess :)

The criticism on the album search page is valid, the sorting of the albums displayed there is based on album-thumbs and does need a similar mechanism / scoring system as we have on the tracks listings to ensure there is some movement. This will be improved as soon as some other important tasks are taken care of (got a load of translations going on...).
The "featured album" on the wikiloops homepage just displays the latest-published album, btw, no manual fiddling about that either.

What strikes me about Lairdys post is the sentence "some sort of favouritism seems to be in play here", which is a good way of describing the "I'm being cheated"-feeling some seem to have.
I have rejected any kind of functionality that would favorize any individual or musical piece in any way other than evaluating the interest of the average wikiloops visitor (who, btw, does seem to have a quite different taste in music than me), and you will have a very hard time prooving me wrong on this one.
Even the former "top-100-radio"-streams have been renamed and randomized to avoid any feelings of competition... there is little more which I could possibly do :)
Feel free to tell me if I'm missing something.

@uncle will_c: right you are in many ways, ye olde nihilist :P

Dick I think you are missing a lot ! But I'm way to tied to do this now after completing a 16 hr day at work ! Tomorrow is a Holliday so I will try and post again then - but on the search issue I searched exactly as you said - blues jams with vocals by date ! So are you saying now I have to enter say guitar as well ? I will try it now I know they all have guitar in them !

---

Edited by onewholeft on Januar 25 2016 13:05

Hi Dick so after a night sleep I know see what you are saying about the search but would have thought you would be abl to search tracks with vocals by date !
As I said earlier I don't use that feature very much anyway .

I will say Dick that regardless of anything said in the above thread about Albums Thumbs Searching or anything else for that matter . This is still the best jam site online by a LONG LONG LONG way and I'm sure over the years to come will only get even better !

As for you ! You have done and keep doing an amazing job mate ! And for that I say thank you !

Now I'm off to listen to and play some music ! Keith Laird

A new idea!!

This discussion got me thinking a little. Can the fairness of the rating system be improved if we take into account the (average) percentage of a track that is listened to??

If the average percentage is low this will probably mean that a great deal of the listeners abort prematurely. This means that the rating function can be changed to

(w1*average_percentage + w2*Thumbs + w3*downloads + w4*remixes)/listeners

Or even
average_percentage((weighted) thumbs+downloads+remixes)/listeners

To me any of these functions will improve the fairness as mentioned above. But it is only a suggestion and i am aware of the difficulties implementing it and also that there may be higher prioritized changes pending

---

I have a question: Why is it nearly impossible to get a thumb after some days? If a ratingsystem should messure the quality, why does the quality decrease but the track rest the same?

Please let me be a bit sarcastic today: Today someone could copy a musical idea, playing it in his style for the peargroup behind and getting a little wave of thumbs. Not a thousend or million but quite a few...

There will be other ways than ratings for different but explizit goals!

For sure there is no lack of musical emotion if a strange listener has other intentions.

---

Edited by Neronick on Februar 12 2016 15:28

@Neronick: I have some answers:
I know a guy, who is absolutely silly and ruthless and another guy who is smart and emphatic.
The silly guy is rich and get's a lot of material thumbs each minute of his life and the smart guy lives in poverty.
Conclusion (1): The world is not fair!

I further know a guy who was very attractive when he was young and he was sought after by many beautiful women. Now, when as an old man nobody regards him as sexy.
Conclusion (2): The sex-appeal decreases while the time passes by.

And I know a guy, who loved to make music when he was young and now, more than 50 years later, the same guy still loves to make music, no matter whether anybody listens.
Conclusion (3): It is possible to take the creative process itself as a reward for itself.

Recommendation:
Take conclusion (3) as a guideline. This will help, to accept conclusion (1) and conclusion (2).

With the best wishes for the year of the Fire-Monkey which has just begun. B)
will_C

---

Great info here I now understand the way the whole thing works for me I don't really take the thumb system to heart. I listen and play just to do so not to see if I can get the most thumbs or listeners. All the musicians here are great in there own way and I try to listen to as much as I can and also for me I do have certain musicians I click with some and some I don't. It's like starting a band you can play with some and others you can't. I'm on the wikiloops for the fun and not all the praises by the amount of thumbs. Honestly I really don't pay them much attention on the tracks I play but I do thumb if I like something. So Jam on everyone and have fun.