New reviews/feedback feature?

43 Votes Replace the Service Reviews forum with a new Reviews feature. It'd be easier to use and will help cut down on drama.
105 Votes Don't need a new Reviews feature, it'll be overly intrusive. Just keep the Service Reviews forum.
12 Votes Remove the Service Reviews forum and don't replace it with anything, it's harmful and PSAs should stay in bulletins.
16 Votes Other

I've been looking over this forum recently and it's not really what I imagined when I first put it up... I do like the idea of a public space for transparent discussion of business transactions, but it feels counter-intuitive (eg. not everyone'll check for a PSA before doing business with a user) and at times overly hostile and personal. 

I originally got the idea for a reviews forum from BJD trading forums, where negative reviews are just a thread you make when you receive substandard service or products from a user (or don't receive it at all), and they were more like small posts about single transactions, rather than big compilations of evidence about why the viewer shouldn't do business with so and so. I think it'd be healthy to try moving towards a model where users feel comfortable just commenting on individual good or bad experiences you've had without having to make a case against a user's character. If a user has a large quantity of a certain type of review from a bunch of people, a viewer can more or less imagine what working with them would be like and draw conclusions from there, and it prevents the responsibility of managing a large PSA from falling onto a single individual. 

To that end, I'm currently fiddling with the idea of replacing the reviews forum with a reviews feature attached instead to each individual user's page, where users can leave reviews and comments. This would be intended to be a replacement for the Service Reviews forum. It has some upsides and downsides, so I wanted to hear everyone's thoughts. 


Basic outline of the feature:

  • When posting in the commerce section (Art Marketplace, Design Marketplace, Adoption Center), or when sending a PM, a user's review ratio is shown next to their username badge (this would be a percentage, 0% for all negative reviews, 100% for all positive reviews). This links to their reviews page.  
  • Any user (blocked or not) can leave positive or negative feedback. 
    • All reviews have to be directly relevant to a single business transaction (maximum of one review per business transaction). Some form of proof has to be provided (eg. a link to the completed product / traded character, or a link to a thread, or screenshot of the transaction or receipt).
    • Only the reviewer and the reviewee can reply to each review, but all replies are publicly viewable. 
    • Reviews can be deleted, closed, or edited by the user who posted the review.
Pros:
  • I feel like it makes more sense than PSAs; it's immediately visible as a warning when doing business with a user, but also doesn't bring additional outside attention to the matter like PSAs do, which can lead to witch hunts or harassment. 
  • Easier to browse feedback attached to a specific user (instead of having to search the Reviews forum), and encourages independent users to come forward with their own feedback, rather than having a single person take on the responsibility of collating a large PSA. 
  • Would encourage people to leave positive reviews which is nice... positivity is good...
Cons:
  • Wouldn't support off-site users.
  • I'm not sure if having a rating attached to an account would make people uncomfortable; this is the reason I specifically don't want it visible as part of a user's profile page, and want to keep it to specific areas of the site. I understand that it could create stress for users who don't want to see it, but being able to turn off the feature defeats the point of having it at all, since people with poor reviews could just turn it off. 
    • There could be an option to hide ratings from the site so you don't have to see or think about them, but other users would still be able to see your rating? 

Regarding Moderation:

I've received some requests lately for reviews/PSAs to be more strictly moderated, which I imagine will also apply to any new review feature, so regardless of outcome, I'll be looking at revising the rules for reviews.

From the threads I've reviewed, the main issue seems to be people who present one business transaction with evidence, then use it to piggyback irrelevant accusations, such as personal disagreements, proof-less transactions, or speculation. This is toeing the line on breaking the rules, so I'll be adding another clause clarifying that every section/paragraph of any PSA must be directly relevant to a business dispute with proof, and not just arbitrarily tacked onto the end of a PSA. 

If you'd like to see additional moderation for PSAs or reviews, please suggest any additional rules you think would help improve the quality of the forum (or any future features we may add). 


Please do chip in with any thoughts or suggestions, either via the thread or by filing a ticket if you're not comfortable speaking in public. Thanks as always for your patience and help. 

filthdipper

i'd personally prefer to not have a reviews section tied to each individual user, as not only does that not help against people who are not on toyhouse, it also seems very similar to the karma system on a lot of older forums. in my experience, karma/upvote/review systems like that that display next to people's names tend to lead to favoritism - people upvoting their friends and treating people with very few karma (reviews) as untrustworthy. it also tends to be easy to abuse - an unpopular, but not otherwise shady user getting mass poor reviews because of a falling out with a friend or so on, and then never being able to climb back out of that hole would be an issue.

 it may also just lead people to account hop to remove their review score, which defeats the whole thing. 

the service reviews forum has had a bit of an issue of not being about business transactions, but i think that could probably be fixed up with a little more moderation rather than just scrapping the entire thing - making it more similar to artist's beware on livejournal where there is a review process before the post goes up might work.

nitocris

Gotta agree with January, if the rules were enforced more effectively, then everything would be gucci.

FreeFallingUp13

If blocked and unblocked users alike can make reviews, then there's a possibility of people sabotaging others maliciously with doctored screenshots. And there is a huge possibility of unsavory users attacking those who leave negative reviews, since those are public, and view-able without an account, I'm assuming. People have block-evaded by signing out of the site and/or using a different device. This also puts more pressure on every individual user to leave a review, somewhat like ebay. This IS an obtrusive practice on ebay, with sellers basically begging users to give them full 5 star reviews because "it could hurt their score" etc. etc.

I voted on the poll before reading the forum, but I actually agree with everybody else on here saying that the forum itself seems to be more effective. While there is drama, it allows several people to collaborate and post the service review and other abusive or manipulative behaviors associated with the services. Plus, people can react in the comments to discuss and ask questions about specifics.

PicklePantry

I feel like that new system of individual ratings will be abused very quickly. For Amazon that system is fine because of how huge it and the amount of users using it, but Toyhouse is an incredibly small community where people will be more inclined to vote out of personal feelings instead of how their transaction went. We have these past few PSAs as example. The addition of letting blocked users leave their own review leaves a bad taste in my mouth.

SquidsInTheAttic

I'm on board with everyone so far.  I really don't like the idea of having a rating next to my name.  I find the idea alone anxiety inducing and would likely drop out of the community side of toyhouse and use the site solely for storage.

More moderation and enforcement of the rules seem more the way to go.

Trowa

From what i've seen, people are pretty decent with policing when posted forums are either toeing the line or just blatantly a personal issue and not a business issue.

I have to agree with the people that have spoken about just needing a better enforcement or a review system before posting. Just like before a thread can be posted on the forums, it has to go through a review by staff. Which means we have less people hastily throwing stuff together and having to go back and edit 6 or 7 times for accuracy when they should have done it the first time. The amount of boards where they start out "i just threw this together real quick so forgive me for errors! i'll fix them later" is kinda bogus but i imagine with just a review system, even that can be cleaned up.

ChickieDee

I'm gonna put my two cents in and say no to this as well, for the reasons of it being a highly abusable system.

I want to spitball here and maybe instead there can be a system of pre-review before a PSA goes live? Like once the user submits it, instead of immediately posting, it gets reviewed, and if it follows the PSA etiquette, THEN release it to the forum?

Of course, I have no idea about running a site or what kind of coding/time this would take, so this is totally just an idea to throw out in there. But I really think the ratings system is an idea which, as of how it's being pitched right now, is so, so, SO easy to take advantage of.

HellMilk

I agree with everyone above about this point system in the end being abused and having a negative impact on the community/forums as a whole. 

Also agree with the need to just enforce the rules on those especially who comment and add nothing to the PSA but accusations with no proof and speculation . As well as reviewing a PSA to avoid those who slap one quickly together (some times not even posting any proof or screen shots just 'this user did this and I'll eventually post proof' and also the Bewares that become personal or have nothing to do with some sort of exchange/ transaction 

Wicked

i get where you're coming form with the BJD trading forums thing... many moons ago i used to buy and sell lolita clothes in the egl sales group on livejournal and they had the same system. it was a pretty nice system that allowed ppl to know you're legit - when i sold things on other sites i'd like to my egl sales reviews to prove i was a dependable seller LOL

with that being said, i dont think that kind of system would work on TH. The community here is different & i can see people abusing it 'n' I think the idea of PSAs having to be reviewed by a person before being posted is the best solution.

your original idea of just having it be a place to review single transactions is nice & it'd be need if the forum could be... pushed in that direction? but this forum has become so associated with PSAs i'm not sure if the shift is possible.

perhaps make a seperate subforum for PSAs, and anything posted there has to be reviewed by some type of mod. and then for service reviews post guidelines on what you intended it to be- i think when the forum was created people didn't quite catch on the original intention.
I know in the EGL feedback site they give a specific format that you must post your reviews in; if you're looking to set up this forum to be more of an individual transaction type of thing, giving guidelines like this may help steer it in the correct direction !

Kadrina

I kind of like the idea of a hybrid? Have basic star reviews with maybe like a 500 character limit for quick look up (similar to what sites like Amazon has where only people who actually purchase can submit a review and you can appeal a review with evidence), but keep the possibility of full scale bewares. I get what a lot of people are saying by worrying about scores, but at the end of the day, it holds people accountable. At the very least bewares should be heavily moderated and reviewed by either the admin or by an objective* panel, tbh.

*meaning your friend isn't the one agreeing to it.

I will be happy with any sort of change, though, as both sides tend to pile on - especially with younger users, who don't necessarily know any better because they see others write similar PSAs. Heck, I have noticed that depending on who writes the PSA depends on if they get a free pass or not. Tbh, I have gotten in the habit of blocking the really outrageous ones.

Edit: Thinking about it... I wanted to add, I don't like making PSAs strictly a bulletin thing, because it gives people with larger following a bigger platform, while people with less subs don't have a bigger impact. There is also no good easy way to report it if it devolves into accusations like those that have popped up the past week.

Sheep

Is it possible to use a review function only after a transaction has been completed?
So the website will only ask you to review a user after you have transferred a character?
There is still room for abuse here (friends trading characters back and forth to leave good reviews) but I assume there can be an algorithm in place to catch people doing something like that.
The reviews can stay in tact even if the character is deleted. It can just be viewable under history like "X user traded X to Y user. No problems"
If there is a problem (like someone back out of a trade) then users can just support a ticket and it needs to be reviews by the moderation team (maybe the same standards Artist Beware holds up to)

That is my suggestion. That way positive reviews can go in quickly because most trades appear to be that way
An problems can be reviewed by a moderation team and it isn't just arbitrary


I like a rating system because it gives a slightly more objective view of someone. Maybe someone who had problems in the past is trying to change, but is always plagued with that one bad beware (and it has happened on artist beware that users have changed their tune and are totally 100% good to work with now).

Motherbeast

Sent in a ticket with my full feedback but the general idea is that I agree with everyone on here that this isn't a good system to implement.

  • It allows anyone who holds a review too much power and has artists at the mercy of the reviews
  • It can allow anyone who has any malicious intent to abuse said power
  • It doesn't solve the problem of the recent PSAs we've been getting, it only hides it under a review system
  • It can prove to be a headache for the staff to moderate, seeing as how it goes from one public forum to semi-public minor instances scattered through all of the site's users
Reinforcement of the rules, stricter modding and generally better management would be a better solution imo. I had also suggested either of the following:
  • Someone in the Suggestion Forum iirc had the idea of having PSAs submitted through something, being taken in for a brief and unbiased review (does it follow the rules, does it contain evidence, etc etc etc) and then being properly posted on to the forums. We can try to pursue that type of thing instead.
  • Or, if we really want to try for this idea to work, the absolute last idea I can think of is to also have a "Reviews" link for everyone as well that links to the reviews they've left behind a la eBay; that way artists can avoid potential customers who are spitefully leaving behind reviews as how customers can avoid artists with a lot of bad reviews.