Decision Trees

• A decision tree consists of interior nodes and leaves, as with any tree:
  • Interior nodes contain a single feature which can either be true or false.
  • Leaf nodes contain a single category label. (Multiple leaf nodes can correspond to the same category.)
  • When an example is presented to an interior node:
    • The feature value for that example is checked.
    • If it is true, send the example to the left child.
    • If it is false, send the example to the right child.
  • When an example is presented to a leaf node:
    • The algorithm returns, with the leaf's label as the example's label.
• Inductive bias of decision trees:
  • Occam's Razor:
    • "Plurality must not be posited without necessity."
  • For decision trees, avoid a "plurality" of nodes.