Taking Feedback/Suggestions for Service Reviews

Posted 5 years, 6 months ago by admin

New reviews/feedback feature?

297 Votes Replace the Service Reviews forum with a new Reviews feature. It'd be easier to use and will help cut down on drama.
221 Votes Don't need a new Reviews feature, it'll be overly intrusive. Just keep the Service Reviews forum.
98 Votes Remove the Service Reviews forum and don't replace it with anything, it's harmful and PSAs should stay in bulletins.
44 Votes Other

I've been looking over this forum recently and it's not really what I imagined when I first put it up... I do like the idea of a public space for transparent discussion of business transactions, but it feels counter-intuitive (eg. not everyone'll check for a PSA before doing business with a user) and at times overly hostile and personal. 

I originally got the idea for a reviews forum from BJD trading forums, where negative reviews are just a thread you make when you receive substandard service or products from a user (or don't receive it at all), and they were more like small posts about single transactions, rather than big compilations of evidence about why the viewer shouldn't do business with so and so. I think it'd be healthy to try moving towards a model where users feel comfortable just commenting on individual good or bad experiences you've had without having to make a case against a user's character. If a user has a large quantity of a certain type of review from a bunch of people, a viewer can more or less imagine what working with them would be like and draw conclusions from there, and it prevents the responsibility of managing a large PSA from falling onto a single individual. 

To that end, I'm currently fiddling with the idea of replacing the reviews forum with a reviews feature attached instead to each individual user's page, where users can leave reviews and comments. This would be intended to be a replacement for the Service Reviews forum. It has some upsides and downsides, so I wanted to hear everyone's thoughts. 


Basic outline of the feature:

  • When posting in the commerce section (Art Marketplace, Design Marketplace, Adoption Center), or when sending a PM, a user's review ratio is shown next to their username badge (this would be a percentage, 0% for all negative reviews, 100% for all positive reviews). This links to their reviews page.  
  • Any user (blocked or not) can leave positive or negative feedback. 
    • All reviews have to be directly relevant to a single business transaction (maximum of one review per business transaction). Some form of proof has to be provided (eg. a link to the completed product / traded character, or a link to a thread, or screenshot of the transaction or receipt).
    • Only the reviewer and the reviewee can reply to each review, but all replies are publicly viewable. 
    • Reviews can be deleted, closed, or edited by the user who posted the review.
Pros:
  • I feel like it makes more sense than PSAs; it's immediately visible as a warning when doing business with a user, but also doesn't bring additional outside attention to the matter like PSAs do, which can lead to witch hunts or harassment. 
  • Easier to browse feedback attached to a specific user (instead of having to search the Reviews forum), and encourages independent users to come forward with their own feedback, rather than having a single person take on the responsibility of collating a large PSA. 
  • Would encourage people to leave positive reviews which is nice... positivity is good...
Cons:
  • Wouldn't support off-site users.
  • I'm not sure if having a rating attached to an account would make people uncomfortable; this is the reason I specifically don't want it visible as part of a user's profile page, and want to keep it to specific areas of the site. I understand that it could create stress for users who don't want to see it, but being able to turn off the feature defeats the point of having it at all, since people with poor reviews could just turn it off. 
    • There could be an option to hide ratings from the site so you don't have to see or think about them, but other users would still be able to see your rating? 

Regarding Moderation:

I've received some requests lately for reviews/PSAs to be more strictly moderated, which I imagine will also apply to any new review feature, so regardless of outcome, I'll be looking at revising the rules for reviews.

From the threads I've reviewed, the main issue seems to be people who present one business transaction with evidence, then use it to piggyback irrelevant accusations, such as personal disagreements, proof-less transactions, or speculation. This is toeing the line on breaking the rules, so I'll be adding another clause clarifying that every section/paragraph of any PSA must be directly relevant to a business dispute with proof, and not just arbitrarily tacked onto the end of a PSA. 

If you'd like to see additional moderation for PSAs or reviews, please suggest any additional rules you think would help improve the quality of the forum (or any future features we may add). 


Please do chip in with any thoughts or suggestions, either via the thread or by filing a ticket if you're not comfortable speaking in public. Thanks as always for your patience and help. 

Ariicchi

I agree with those who say to keep it as it's the only way to know who to not do business with. However that being said, I think there should be a clear cut guideline that breaks down what is and isn't allowed. For example, if you want to post a PSA you need to have clear evidence. No more of these posts that are strictly hearsay. There needs to be very clear and hard facts. Paypal receipts and chargeback screenshots.

Perhaps building a template in which people are to format their PSA's would also be very helpful. I know the Livejournal account has a template.

WHO:

WHERE: List business related websites only.

WHAT: Was it traditional art, digital art, sculpture, fursuit, jewelry, etc.?

WHEN: Start date of transaction to current.

EXPLAIN: Tell only your side here. Do not include other clients having issues. Be clear, concise, and no overly hostile text or name calling. Please include screencaps of proof as you go, or number claims where relevant. Start from the beginning until now.

PROOF: No copy pasted text will be accepted. Screencaps only. Ensure all real life names, email addresses, and unrelated third parties are censored. Please leave out unrelated chit-chat.
Do not insert images directly into your post. Link to them only.

Another thing the LJ does that is pretty nice is "We do not accept group posts or group warning posts. Please make individual posts." This I know would end up cluttering the forum, but at the same time people will have different negative interactions with perhaps the same people, but it would make editing posts easier. I noticed some of the large PSA, with more than one person, some information is wrong but it takes forever to edit / fix and it becomes a mess.

I don't believe getting rid of the Service Reviews is a good thing, because a lot of these posts are very helpful and well warranted and needed. I feel there needs to be a stricter guideline and must haves. Another idea is locking comments all together and having admin approve comments based on their content. This will help de-clutter the forum with unnecessary "WOW I WONT BE COMMISSIONING THEM." posts and allowing people to show their own proof of, yes this actually happened. Which would be a tad different than group PSAs as it's just someone adding additional proof without it being a group PSA in which only one person has access to editing.

Ariicchi

Then perhaps : Submissions are to be transaction based. Art trades, game currency, or items for art trades, are acceptable as well as monetary exchange. (Copied again from the AB live journal.)
To touch on what Forsaken mentioned, all art forum game PSAs should be bulletin only. 

dollete

I think that maybe PSA's should be moderated by admins before being released to the public, so it doesnt overlap with the service reviews forum, just create another specific PSA forum, and if you wish to post in that forum, upon posting it will get sent for review to moderators to decide if it is important for the public to be aware of. The admins can either message the PSA creator to say if they need to change anything, or if and why they got their PSA denied. PSAs that the public should know of, would be unhidden and released to the forum as any other thread would.

Idk if it has to be exactly like this, but I think some sort of PSA whitelist system would work better than reviews on profiles, because it just sort of defeats the purpose, especially since some PSA's are about popular users who aren't on Toyhouse.

Cornspiracy

Going to be frank. I think service reviews is an unneeded forum that is too easily abused, and that any poor experiences should be sent to Artist Beware. I also just wanna say that there wasn't much thought put into how service review works.

I dunno if I'm the only one, but I've always had an issue with how intense the block system, and how bad it works with the system reviews section. People could look here, and not even know there's been a review made about them because they're blocked. If they do find it out, they can't comment to defend themselves nor explain the situation considering you also can't comment on forums of those who've blocked you.   

Just one reason out a million, but anyway, I really think Toyh.ouse is not the place for service reviews.

Also want to add that PSA/Callouts as a forum is a terrible, terrible idea that repeats the block issue, and only supports people bringing their small personal drama to the public. You can review a callout all you want, but you're only getting one side. Not to mention that if the ones reviewing the PSA feel its unreasonable, they will only get dragged down, and possibly be accused of being problematic individuals themselves.  I beg of you dear mods, please do not. HAHA.

Pepperly

I think mods and a tighter rule-set should "fix" this, as Arriichi posted above a couple times. I highly agree with "transactions only", and not personal threads like 'this user is transphobic and mean' or 'maybe will steal our stuff', but scammed money and trades only. Also extenuating circumstances like how people have been abusing the art threads, since that is scamming people out of work. Though, evidence of multiple instances of flaking turning into an art-game forum ban would help a lot.

Honestly, a mod reviewing the OP before posting combo'd with immediate locked comments should fix everything. If someone wants to tell the OP something, take it to PMs. Have the OP edit and bump the thread. Most of the issues I see are 90% in the comments, anyway.

As an artist, I'm not comfortable with the idea of reviews like ebay. I'm pretty on the ball, and I'm sure I'd have mostly positive reviews, but if blocked users (which I admittedly have quite a few of for various reasons, ranging from personal to known scammers) can make bad reviews on me because i didn't tell them why I blocked them, I'm not cool with that. I just can't think of a fair way to balance user reviews without using -- you guessed it -- mods, which would have a smaller load by only modding this forum. Checking a zillion user reviews just seems overwhelming.


On the off chance you nuke this forum, please move Positive Service Reviews first; I love that thread ; w; maybe to Art Marketplace?

Peony

Oh shoot I just realised I misread the poll.

I support the opinion on just having a more moderated Service Reviews forum. Though I do see the pros and cons to having user reviews, I do not know if this community is more likely to abuse them or to actually follow through with them, and I'd rather not have the risk. I don't want the PSAs to stay in Bulletins though since that can only reach a small amount of people, so having a forum like this still has its purpose on warning the public about scammers and the like.

Ellteo

Originally I thought the feature sounded alright, but after thinking about it and seeing people raise questions - I'd have to agree that maybe it'd be best not to do away with the service review forum.

I do think the Service Review forums can be pretty bad - the mindset of a lot of the threads often enough turns to the same kind of Callout Culture you see on tumblr or DA, mostly in the comments. But it serves it's purpose in warning those who buy art often who will or will not treat the transaction in a way they approve of.

I think (as others have suggested) a rough template would work wonders, as well as auto-locking the comment section. That way there's less legwork for the mods going over entire threads, and if things need to be added, people can contact the OP? It would help curb the negativity and mud slinging that sometimes happens in the comments, while still giving people a platform to inform others of negative experiences. 

I don't think there's going to be a perfect answer here, but tighter rules and the above suggestions would be a good start. I think an entirely new feature similar to what ebay has for sellers would be doing a disservice to a lot of people.

I appreciate you guys trying to problem solve and find new ways to improve things! Just, if the service review forums DO get taken down....please save the positive service review thread, it's so wholesome sdkjsfhd

Titheling

Honestly I think the service review DESPERATELY needs looked at. Recently there have been a ton of posts in this forum which are unmerited and some outright dangerous towards the people they're directed at. I think that the reviews really need to go through a process of being accepted - that there needs to be a hardline requirement for what can and cannot be posted in the thread. I really think it needs more heavy monitoring more than anything else suggested. Potentially this would look more like submitting a review and then it getting accepted by a mod before it's ever posted in the thread itself. The piggybacking of irreverent and personal accusations is certainly a problem - though I've also seen whole PSA's posted which are riding on personal and completely unbusiness related accusatory and speculative evidence. When this behaviour is called out or opposed it simply leads to a lot of childish back and forth trying to validate things that should never be on this thread in the first place.
I think the moment a thread is made with evidence that is personal in nature it needs to be taken down and the person asked to resubmit it when it's only business related. I also think there needs to be a pretty zero tolerance policy in relation to this. Most of the threads that have either partial or mostly personal 'evidence' tend to lead to this long string of comments by others which creates a sort of validation of these accusations. It also, frequently, appears to lead to a lot of speculation and assumptions. This can be incredibly dangerous when we're talking about an artist whose l ivelihood is the work being suddenly put into a PSA spotlight in relation to something completely not to do with their business transactions.
I don't think it's 100% necessary but using a form would make things easierin tandem with being required to submit potential PSA's. It would make it easier for both the people reading it to see what the problem is - as well as would speed up the process of anyone 'reviewing' them. I do think that if a form is used that there needs to be an extra section for anything else - that there needs to be a space for information that might not fit perfectly in a form but still is vaible to the actual PSA.

I also wonder if linking to Personal PSA's from this thread should simply be not allowed. I've seen this as well to skirt around business only PSA's (usually when called out) and it doesn't negate the fact that these are being directed FROM a thread where that is rule breaking.

Honestly with the review rating I'm really not in favour - I can see this being a bit dangerous and outright abused very easily...Ontop of that, if someone has only done 4 commissions and one person who they were doing art for was unreasonable/demanding/a bad customer overall and then left a negative review despite perfectly reasonable work then the percentage is going to show up as pretty godawful. The same thing stands with someone doing a ton of good work and then just starting to do things that are outright terrible - but they have 200+ good reviews so the 5 reviews are hardly going to appear visible even if they all happened within the last month or two. As well, a lot of people don't care to leave reviews unless they've had a particularly bad experience - think of YELP for instance. And if the person doesn't do a ton of work on this site but is perfectly alright elsewhere that also stands. I've also seen plenty of completely unreasonable customers upset because of completely unreasonable things - demanding things or being really unfair in their assessment of the artist with nothing to back 'this person was awful don't ever commission them'. People are petty and take advantage of systems like this for petty reasons. Unlike a site where physical goods are being bought and sold, leaving reviews is potentially less easy to regulate. I am actually pretty opposed to this for that reason - even as someone who this honestly wouldn't effect much at all given how little I engage in even trading. As well, in a community like this I can easily see this feature turning into a weird popularity contest and that would straight up drive me away if it became anything at all. Basically I can see this system working well for other types of communities but I just outright don't want to deal with it on here.

Along with this, some PSA's are pretty business related but at the same time really wouldn't be possible to see in a simple review. There is a bit more complexity - trading a stolen character (I've seen this happen where the person sends the entire character via discord which would remove any trade having happened onsite), a customer demanding a refund after work is completed, unreasonable demands where someone is asking for changes that are above and beyond what would be acceptable to ask, a commission being dragged along for ages, people changing accounts or having multiple accounts to get around certain things, actual tracing and theft. I think it's better to have it so things can be laid out clearly like they are now. Easy 'immediate' feedback seems good in theory - but when it's percentage based well that tends to mean that there's a heavy disadvantage to those who don't do a ton of business on here and how damaging one bad review even if unmerited can be. This would also make me really worried about asking for changes/having a problem with anything I received commission wise if it was something which could go both ways - I mean someone who gets a really bad review is likely to just give one in turn, right? A lot of the stuff on this review forum also doesn't simply fall under 'I bought a commission' category - there are reasonable reviews for chargebacks via customers, theft when there was no actual buying/selling taking place, stolen things for sale/trade and other similar topics. Honestly, more moderation would be ideal along with the recent change in clause. I also believe that this needs to be something required within the comments themselves - that when things start to slide into hearsay and speculation the moment this happens the comment needs to be removed.

I think any sort of rating system is going to lead to drama in different ways...

mothgf

honestly my opinion is starting to sway into the same way as it is for wicked. I'm just not seeing the use of these threads anymore. PSAs should be posted somewhere meant for PSAs, not for a place with little moderation and wasn't ever meant to be for PSAs in the first place.