Whether to use end-to-end deep learning

Pros and cons of end-to-end learning

Pros:

• Let the data speak: If you have enough X,Y data then whatever is the most appropriate function mapping from X to Y, if you train a big enough neural network, hopefully the neural network will figure it out. By having a pure machine learning approach, your neural network learning input from X to Y may be more able to capture whatever statistics are in the data, rather than being forced to reflect human preconceptions.
• Less hand designing of components needed

Cons:

• May need large amount of data
• Excludes potentiallty useful hand-designed components.

Key question: Do you have sufficient data to learn a function of the complexity needed to map x to y?