Crowdflower

[Yazoo63]

Has anyone ever actually had their completion percentage adjusted upwards after contesting a "wrong" answer ?

This seems to happen to me time after time.  I submit a task and am told that one of my answers was incorrect (when I KNOW it was correct). Then I click on the box for "that's unfair", and submit my proof in the box.  After that, nothing happens.  I never hear back from them; my completion percentage is never adjusted upwards as it should be.  

Eventually, my completion percentage falls below the level that they consider to be acceptable, and I get banned from doing that particular task ... all because they incorrectly graded one or more of my answers.

I'm just wondering if it's even worth the time to contest a wrong answer.  It seems like a waste of time, since whatever you say to them or do, it's not going to change anything.  It's like nobody is listening on the other end.

I'm not slamming Crowdflower.  I really enjoy their tasks.  This just seems to be an unresolved issue that has been occurring for a long time and it can be really frustrating at times.  

[dbcomputers01]

I used to say a question was unfair, but after a few times, they haven't done anything, so I just don't contest, and move on.

[skippymcnapper]

I always fill out the "that's unfair" box also and I agree it does feel like nothing ever comes of it. If someone is examining all those and then changing the "incorrect" answer to be correct based on our reasoning, it doesn't change your score. You still got below the needed percent and banned. I like to think that I'm helping out a fellow FCer who might work on the task later, but who knows.

[kingreyam24]

I would click on the unfair block and would never get a response either. Then, I started clicking on the help in the upper right hand corner and emailing them through there and then they emailed me back and said they were sorry. Unfortunately, the survey was over by the time they got back to me. I have seen other people say that they were able to get their score changed though. I guess it just depends on how fast they get back to you and if the survey is still available or not.

[kapeh12]

I answer the "unfair" section of the answer key if I truly feel the answer was inconsistent with their instructions or flat out wrong.  But I learned to follow up by submitting a Feedback comment (as someone mentioned above, you get the link by clicking on the Help button on the upper right corner).

When I submit feedback, I usually try to be professional, stating my reasons and specifics about which incorrect answers I felt were unfairly marked wrong, and ultimately lead to my disqualification.  I usually receive responses that the group will pass on my information to the task programmers, and if the task is still running, they do reset my accuracy.  On a few occasions recently, where the task was ended, they forced payment for the task.  I believe the forced payment on the failed tasks - in my case - is due to me having worked some of those tasks for over a year and a half and submitted countless feedback to assist them in improving the answer keys.  I wouldn't expect the payment, but you should get the accuracy adjusted if the task is still running.

I don't always hear back timely on my feedback.  Sometimes I hear back within 24 hours, other times it may be a week or so.  I'm guessing the timeliness depends on their support staff and how many feedback requests are sent in.  If they are running some tasks with some major answer key issues, you can probably assume that help desk department is probably wading through a big queue of feedback requests.  So if you do submit feedback - do it for the purpose of assisting Crowdflower in improving their answer keys for the next round (accuracies rest when the task is re-run).  If they happen to get your feedback and the task is still running - you may get another shot at it.

The important thing - be respectful in the feedback.  That will improve your chances of a favorable reply if it's possible.

Good luck!

[Administrator]

I asked CrowdFlower about this and they mostly echoed what kapeh12 posted.  The "Send Feedback" link is the best way to report an issue.  The response time for those can vary, but they are working on getting them answered faster.  Hope this helps!