[Solved]: How do you determine the inputs to a neural network?

ignougroup
Problem Detail: I’m looking at this tutorial on neural networks. The data that is given from the UCI study includes various attributes, such as “mean x of on pixels”, “total # on pixels” etc, which are taken as input to the neural network. However, it seems to me that this choice of input parameters is somewhat arbitrary, and, in this case, would only apply specifically to the black-and-white letter recognition problem, and maybe could be extended to some other image recognition problems. As a result, my questions are:

  1. How do you determine what inputs you should use for a neural network?
  2. Are more inputs necessarily better?
  3. Does it matter if inputs are linearly independent?
  4. And, finally.. Could one build a neural network that could determine its own inputs for an arbitrary problem and raw data set?
Asked By : Teofrostus

Answered By : Kyle Jones

How do you determine what inputs you should use for a neural network?

Experimentation. Initially you use a priori knowledge and intuition to guess the features will be useful for classification or prediction or whatever it is your network is doing. Then you test the network and observe its error rate. Based on the results you make adjustments; add/discard features, regularize, add/discard layers, adjust the learning rate, and so on.

Are more inputs necessarily better?

No. You’re trying to find the features that matter to what you’re trying classify/predict and discard unrelated features. Processing input/features that don’t matter to the computation just wastes CPU time.

Does it matter if inputs are linearly independent?

If inputs are linearly dependent then you are in effect introducing the same variable as multiple inputs. By doing so you’ve introduced a new problem for the network, finding the dependency so that the duplicated inputs are treated as a single input and a single new dimension in the data. For some dependencies, finding appropriate weights for the duplicate inputs is not possible.

Could one build a neural network that could determine its own inputs for an arbitrary problem and raw data set?

This is essentially what neural networks that deal with raw data only must do. The hidden layers tease features out of the raw data and those features are gradually refined by the feedback that occurs during training. This is an approach you might use if you have no idea how to proceed given some classification problem. But usually you have some idea what features are both useful and difficult for the network to compute for itself, and you provide those features as additional input.

Best Answer from StackOverflow

Question Source : http://cs.stackexchange.com/questions/54634 3.2K people like this

 Download Related Notes/Documents

Leave a Reply