Surpassing human-level performance

Machine learning progress gets harder as you approach or even surpass human-level performance

Example:

Let’s say you have a problem where a team of humans discussing and debating achieves 0.5% error, a single human 1% error, and you have an algorithm of 0.6% training error and 0.8% dev error.

type	err	err2
Team of humans	0.5%	0.5%
One human	1%	1%
Training Error	0.6%	0.3%
Dev Error	0.8%	0.4%

avoidable bias	0.1%
variance	0.2%

Once surpass the human error, then making progress on the machine learning problem are just less clear.

Problems where ML significantly surpasses human-level performance

There are many problems where machine learning significantly surpasses human-level performance.

Online advertising, estimating how likely someone is to click on that. Probably, learning algorithms do that much better today than any human could
Making product recommendations, recommending movies or books to you.
All logistics predicting how long will take you to drive from A to B
load approves predict whether someone will repay a loan, and therefore, whether or not you should approve a loan offer.

All four of these examples are actually learning from structured data.

These are not natural perception problems, so these are not computer vision, or speech recognition, or natural language processing task.

All of these are problems where there are teams that have access to huge amounts of data

speech recognition systems
computer vision
medical image diagnosis