Calculates the proportion of deviance explained by one or more models relative to a reduced model (a pseudo-\(R^2\)), and the gain from each added term or component, such as a spatial random field.
Arguments
- object
Output from
sdmTMB(). The most complex model. Its shape parameters (see Details) are used for all models.- ...
Other
sdmTMB()models fit to the same response with the same family, ordered from most to least complex. The last is the reduced model, usually intercept-only with no random effects or random fields. Name the arguments to label the output rows. Names are arbitrary and only to help reading the output. These models can be set up withdo_fit = FALSEand are then fit here. Alternatively, they can be fit first, in which case they are refit (if needed, see below) starting from the previously estimated parameter values for speed.
Value
A data frame with one row per model with columns model,
deviance, deviance_explained (relative to the last model), and gain
(over the next simpler model). Any fixed shape parameter values (on their
internal scale) are in attribute "fixed".
Details
Deviance explained, \(1 - D / D_0\), is a common pseudo-\(R^2\) for generalized linear models (Cameron and Windmeijer 1997) and is what mgcv reports. It reduces to \(R^2\) for a Gaussian linear model.
Sequences of models. gain is the increase in deviance explained over
the next simpler model. Like sequential (type I) sums of squares, it depends
on order: a spatial field added after covariates usually gains less than one
added to an intercept-only model, since both can explain the same spatial
pattern.
Shape parameters. For some families, the deviance depends on parameters
other than the mean: phi for negative binomial, compois(), and
lognormal(); p for tweedie(); and phi and Q for gengamma().
Deviances computed with different values of these are not comparable, so
the other models are refit with them fixed at their values in object, as
mgcv does for its null deviance. Previously fit models are refit
starting from their estimates for speed. Reordering the models changes the
answer because it changes the comparisons.
Random effects and random fields. The deviance is conditional on the
predicted random effects, so deviance explained measures in-sample fit of
the fixed and random effects together. Because it does not penalize
complexity, more flexible fields almost always increase it. Do not use it to
choose among random effect structures or meshes; use cAIC(), AIC(), or
sdmTMB_cv() instead. For families with shape parameters, fixing phi can
shift variation between random effects and observation error in refit
models, so gains from random effects are approximate.
For delta models, the deviance is summed over both components.
References
Cameron, A. C., and Windmeijer, F. A. G. (1997). An R-squared measure of goodness of fit for some common nonlinear regression models. Journal of Econometrics, 77(2), 329–342. doi:10.1016/S0304-4076(96)01818-0
Examples
fit <- sdmTMB(density ~ depth_scaled + depth_scaled2,
data = pcod_2011, spatial = "off", family = tweedie())
fit0 <- sdmTMB(density ~ 1,
data = pcod_2011, spatial = "off", family = tweedie())
compare_deviance(fit, fit0)
#> Refitting the other models with Tweedie p fixed at the value from `object`,
#> since the deviance depends on it.
#> model deviance deviance_explained gain
#> 1 fit 14590.31 0.1770922 0.1770922
#> 2 fit0 17730.19 0.0000000 NA
# Not comparable, since each model estimates its own Tweedie p:
1 - deviance(fit) / deviance(fit0)
#> [1] 0.114655
# \donttest{
# Gain from adding a spatial random field after depth:
mesh <- make_mesh(pcod_2011, c("X", "Y"), cutoff = 20)
fit_sp <- update(fit, spatial = "on", mesh = mesh)
compare_deviance(fit_sp, depth = fit, intercept = fit0)
#> Refitting the other models with Tweedie p fixed at the value from `object`,
#> since the deviance depends on it.
#> model deviance deviance_explained gain
#> 1 fit_sp 11880.74 0.3664711 0.1863758
#> 2 depth 15375.90 0.1800953 0.1800953
#> 3 intercept 18753.27 0.0000000 NA
# Models after the first can be left unfitted
# with `do_fit = FALSE`, since they are fit here with
# Tweedie p fixed anyway:
fit_sp_only <- update(fit0, spatial = "on", mesh = mesh, do_fit = FALSE)
compare_deviance(fit_sp, spatial = fit_sp_only, intercept = fit0)
#> Refitting the other models with Tweedie p fixed at the value from `object`,
#> since the deviance depends on it.
#> model deviance deviance_explained gain
#> 1 fit_sp 11880.74 0.3664711 0.1036665
#> 2 spatial 13824.83 0.2628045 0.2628045
#> 3 intercept 18753.27 0.0000000 NA
# }
