Why Machine Learning Strategy

Machine learning is the foundation of countless important applications, including web search, email anti-spam, speech recognition, product recommendations, and more. I assume that you or your team is working on a machine learning application, and that you want to make rapid progress.

Example: Building a cat picture startup

Say you’re building a startup that will provide an endless stream of cat pictures to cat lovers.

You use a neural network to build a computer vision system for detecting cats in pictures. But tragically, your learning algorithm’s accuracy is not yet good enough. You are under tremendous pressure to improve your cat detector. What do you do?
Your team has a lot of ideas, such as:

• Get more data: Collect more pictures of cats.
• Collect a more diverse training set. For example, pictures of cats in unusual positions; cats with unusual coloration; pictures shot with a variety of camera settings; ….
• Train the algorithm longer, by running more gradient descent iterations.
• Try a bigger neural network, with more layers/hidden units/parameters.
• Try Adam istead of gradient descent
• Try dopout
• Try a smaller neural network.
• Try adding regularization (such as L2 regularization).
• Change the neural network architecture
  • activation function
  • number of hidden units, etc.

If you choose well among these possible directions, you’ll build the leading cat picture platform, and lead your company to success. If you choose poorly, you might waste months. How do you proceed?

Machine learning strategy is changing in the era of deep learning because the things you could do are now different with deep learning algorithms than with previous generation of machine learning algorithms.