Coding With Fun
Home Docker Django Node.js Articles Python pip guide FAQ Policy

What is the difference between nested and non-nested tests?


Asked by Anahi Brown on Dec 08, 2021 FAQ



Nested means here that all terms of a smaller model occur in a larger model. This is a necessary condition for using most model comparison tests like likelihood ratio tests. In the context of multilevel models I think it's better to speak of nested and non-nested factors.
And,
Nested versus non-nested can mean a whole lot of things. You have nested designs versus crossed designs (see eg this explanation ). You have nested models in model comparison. Nested means here that all terms of a smaller model occur in a larger model. This is a necessary condition for using most model comparison tests like likelihood ratio tests.
Also Know, More to the point, under the null hypothesis (and given certain moderate assumptions), the difference in goodness-of-fit between two nested models follows a known distribution, the shape of which depends only on the difference in degrees of freedom between the two models. This is not true for non-nested models.
In fact,
Nested CV estimates the generalization error of the underlying model and its (hyper)parameter search. Choosing the parameters that maximize non-nested CV biases the model to the dataset, yielding an overly-optimistic score. Model selection without nested CV uses the same data to tune model parameters and evaluate model performance.
Moreover,
Non-nested factors is a combination of two factors that are not related. Say you study patients, and are interested in the difference of age and gender. So you have a factor ageclass and a factor gender that are not related. You should model both age and gender as a main effect, and you can take a look at the interaction if necessary.