Home / Summaries / Introduction to data mining / nodes-child-comparable

Classification: Basic Concepts, Decision Trees, and Model Evaluation

Q: Within a decision tree we distinguish tree types of nodes. Which are they?

Root node. The very first node of the tree (i.e., the starting point). Represents the entire dataset being split based on the most significant attribute. Internal node. Any node between the root and leafs. Each represents a test/decision on a feature. Internal nodes keep splitting the data until the stopping condition is reached. Leaf Nodes. The endpoints of the decision tree. From this point, no further splitting happens. Represents: In classification: the final class label. In Regression trees: a value prediction .

Q: What is a child node?

Child nodes are the new nodes you create when you split a node based on some attribute. You can distinguish between parent and child nodes: Parent node: The node you split. Child node: the resulting subgroups from that split.

Q: What are the four major advantages of classifying based on a decision tree?

Inexpensive. Inexpensive to construct. Fast. Extremely fast at classifying unknown records. Easy Interpretable. Easy to interpret for small-sized trees. Comparable Accuracy. Accuracy is comparable for other classification techniques for many simple data sets.

Q: What are the three core practical issues of classification?

Underfitting and Overfitting . Underfitting vs Overfitting. Underfitting: Model is too simple . Doesn't capture the underlying patterns in the data. Performs poorly on both training and test data. Overfitting: Model is too complex (than necessary). It memorised the data instead of generalising. Performs very well on training data, but poorly on test data. I.e., Cannot asses model performance by training error. Missing Values . Affects decision tree construction in several ways: How to distribute instance with missing value to child nodes. How impourity measures are computed. How a test instance with missing value is classified. Cost of Classification .

Q: What is a confusion matrix and how is it used to evaluate a classification model?

Confusion Matrix A confusion matrix shows the results of a classification model that is applied to a test set . It compares the the class the algorithm predicted with what the object's actual class was for an attribute. Layout A confusion matrix is displayed as f(a,b). a shows the actual class the object belonged to. For a binary attribute, (1) presence vs. (0) absence. b shows what the model predicted the class would be. When the values for a and b align, the model made a correct prediction.

Q: When you use a learning algorithm to induce a decision tree for a classification model, which two issues must be addressed?

When to Split? Each step in the tree must select an attribute test condition to divide the records into smaller groups. I.e., Based on which attribute's criteria might be best split the group of all our records(/data objects). E.g., You want to find out who is likely to buy a product. Default = no, you want to find the groups that have yes. Would you start splitting on age-groups, gender, income first? When to Stop? You can go on until every object is successfully classified. Although, sometimes early termination might be more useful.

Q: Occam's Razor, or the Principle of Parsimony, is a strategy to prevent model overfitting. What is the definition of Occam's Razor?

As Simple as Possible . When two models have the same amount generalization errors, the simpler model is preferred over the more complex one. Accounting for Serendipity: The reason for this is, that complex models have a greater chance of being fitted purely by chance. Equal Generalization Errors . Occam's Razor only applies for models with the same amount of generalization errors. 2 Methods for Incorporating Complexity . Pessimistic Error Estimate. Minimum Description Length Principle.

8 important questions on Classification: Basic Concepts, Decision Trees, and Model Evaluation

Within a decision tree we distinguish tree types of nodes.
Which are they?

Root node.

The very first node of the tree (i.e., the starting point).
Represents the entire dataset being split based on the most significant attribute.

Internal node.

Any node between the root and leafs.
Each represents a test/decision on a feature.
Internal nodes keep splitting the data until the stopping condition is reached.

Leaf Nodes.

The endpoints of the decision tree.
From this point, no further splitting happens.
Represents:

In classification: the final class label.
In Regression trees: a value prediction.

What is a child node?

Child nodes are the new nodes you create when you split a node based on some attribute.
You can distinguish between parent and child nodes:

Parent node: The node you split.
Child node: the resulting subgroups from that split.

What are the four major advantages of classifying based on a decision tree?

Inexpensive.

Inexpensive to construct.

Fast.

Extremely fast at classifying unknown records.

Easy Interpretable.

Easy to interpret for small-sized trees.

Comparable Accuracy.

Accuracy is comparable for other classification techniques for many simple data sets.

What are the three core practical issues of classification?

Underfitting and Overfitting.

Underfitting vs Overfitting.

Underfitting:

Model is too simple.
Doesn't capture the underlying patterns in the data.
Performs poorly on both training and test data.

Overfitting:

Model is too complex (than necessary).
It memorised the data instead of generalising.
Performs very well on training data, but poorly on test data.

I.e., Cannot asses model performance by training error.

Missing Values.

Affects decision tree construction in several ways:

How to distribute instance with missing value to child nodes.
How impourity measures are computed.
How a test instance with missing value is classified.

Cost of Classification.

Can you describe the common reasons for Overfitting and how we can fix this?

Noise: Exceptional cases or outliers in the training set can lead to the model making wrong classifications, when you generalise it to unseen data.

We should accept that errors due to exceptional cases are unavoidable and establish the minimum error rate achievable by any classifier.

Lack of Representative Samples: If you don't have sufficient training samples, or they are not representative for the group you want to generalise to, overfitting might occur.

What is a confusion matrix and how is it used to evaluate a classification model?

Confusion Matrix

A confusion matrix shows the results of a classification model that is applied to a test set.
It compares the the class the algorithm predicted with what the object's actual class was for an attribute.

Layout

A confusion matrix is displayed as f(a,b).
a shows the actual class the object belonged to.

For a binary attribute, (1) presence vs. (0) absence.

b shows what the model predicted the class would be.
When the values for a and b align, the model made a correct prediction.

When you use a learning algorithm to induce a decision tree for a classification model, which two issues must be addressed?

When to Split?

Each step in the tree must select an attribute test condition to divide the records into smaller groups.
I.e., Based on which attribute's criteria might be best split the group of all our records(/data objects).
E.g., You want to find out who is likely to buy a product.

Default = no, you want to find the groups that have yes.
Would you start splitting on age-groups, gender, income first?

When to Stop?

You can go on until every object is successfully classified.
Although, sometimes early termination might be more useful.

Occam's Razor, or the Principle of Parsimony, is a strategy to prevent model overfitting.
What is the definition of Occam's Razor?

As Simple as Possible.

When two models have the same amount generalization errors, the simpler model is preferred over the more complex one.
Accounting for Serendipity: The reason for this is, that complex models have a greater chance of being fitted purely by chance.

Equal Generalization Errors.

Occam's Razor only applies for models with the same amount of generalization errors.

2 Methods for Incorporating Complexity.

Pessimistic Error Estimate.
Minimum Description Length Principle.

The question on the page originate from the summary of the following study material:

Introduction to data mining | 9780321321367 | Pang Ning Tan, et al

View summary

A unique study and practice tool
Never study anything twice again
Get the grades you hope for
100% sure, 100% understanding

Remember faster, study better. Scientifically proven.

Classification: Basic Concepts, Decision Trees, and Model Evaluation

8 important questions on Classification: Basic Concepts, Decision Trees, and Model Evaluation

Within a decision tree we distinguish tree types of nodes. Which are they?

What is a child node?

What are the four major advantages of classifying based on a decision tree?

What are the three core practical issues of classification?

Can you describe the common reasons for Overfitting and how we can fix this?

What is a confusion matrix and how is it used to evaluate a classification model?

When you use a learning algorithm to induce a decision tree for a classification model, which two issues must be addressed?

Occam's Razor, or the Principle of Parsimony, is a strategy to prevent model overfitting. What is the definition of Occam's Razor?

Summaries related to Classification: Basic Concepts, Decision Trees, and Model Evaluation

Introduction to data mining

Exploring Humans Philosophy of Science for t…

Arts and Culture: Pearson New International …

Sociology

Sociology of the Arts Exploring Fine and Pop…

Thinking Art

Statistics for the Behavioral Sciences

Principles of Marketing

Global Marketing, Global Edition

Studyguide for Database System Concepts by A…

Cultural and Creative Industries

Contemporary Approaches in Cultural Sociology

Within a decision tree we distinguish tree types of nodes.
Which are they?

Occam's Razor, or the Principle of Parsimony, is a strategy to prevent model overfitting.
What is the definition of Occam's Razor?