Decision trees

• Can be applied to regression problems

Decision tree - Baseball Salary Example

Let's walk through the figure

• This is using the Hitters data from the ISLR 📦

Let's walk through the figure

• This is using the Hitters data from the ISLR 📦
• I fit a regression tree predicting the salary of a baseball player from:
• Number of years they played in the major leagues
• Number of hits they made in the previous year

Let's walk through the figure

• This is using the Hitters data from the ISLR 📦
• I fit a regression tree predicting the salary of a baseball player from:
• Number of years they played in the major leagues
• Number of hits they made in the previous year
• At each node the label (e.g., $X_j<t_k$ ) indicates that the left branch that comes from that split. The right branch is the opposite, e.g. $X_j\ge t_k$.

Let's walk through the figure

• This is using the Hitters data from the ISLR 📦
• I fit a regression tree predicting the salary of a baseball player from:
• Number of years they played in the major leagues
• Number of hits they made in the previous year
• At each node the label (e.g., $X_j<t_k$ ) indicates that the left branch that comes from that split. The right branch is the opposite, e.g. $X_j\ge t_k$.
• For example, the first internal node indicates that those to the left have less than 4.5 years in the major league, on the right have $\ge$ 4.5 years.

Let's walk through the figure

• This is using the Hitters data from the ISLR 📦
• I fit a regression tree predicting the salary of a baseball player from:
• Number of years they played in the major leagues
• Number of hits they made in the previous year
• At each node the label (e.g., $X_j<t_k$ ) indicates that the left branch that comes from that split. The right branch is the opposite, e.g. $X_j\ge t_k$.
• For example, the first internal node indicates that those to the left have less than 4.5 years in the major league, on the right have $\ge$ 4.5 years.
• The number on the top of the nodes indicates the predicted Salary, for example before doing any splitting, the average Salary for the whole dataset is 536 thousand dollars.

Terminology

🎋 The final regions, $R_1,R_2,R_3$ are called terminal nodes

Terminology

🎋 The final regions, $R_1,R_2,R_3$ are called terminal nodes

🎄 You can think of the trees as upside down, the leaves are at the bottom

Interpretation of results

• Years is the most important factor in determining Salary; players with less experience earn lower salaries

Interpretation of results

• Years is the most important factor in determining Salary; players with less experience earn lower salaries
• Given that a player is less experienced, the number of Hits seems to play little role in the Salary

Interpretation of results

• Years is the most important factor in determining Salary; players with less experience earn lower salaries
• Given that a player is less experienced, the number of Hits seems to play little role in the Salary
• Among players who have been in the major leagues for 4.5 years or more, the number of Hits made in the previous year does affect Salary, players with more Hits tend to have higher salaries