- Select Pearson, Spearman or Kendall based on the form of the game, normality and size campionaria.
- Per variabili categoriali usa chi-quadro/Fisher e misura l'associazione con V di Cramér o φ.
- Valuta confondenti con correlazione parziale and supports the analysis with graphics and diagnosis.
- The relationship algebra (insiemi, funzioni, composizione) structures the pensiero and the application in the database.
Capire che type di legam c'è tra due variabili è una delle domande più frequenti in statistica applicata, scienza dei dati e ricerca sperimentale e ricerca quantitative . The other way around the relationship is linear and always part of the cogliere, another way back is monotonous but non-linear, oppure coinvolge category and non numeri, the anchor depends on a third variabile that confonde le acque: insomma, non è una passeggiata.
In this practical and theoretical guide in Italian, we see in organic mode the main tip of mathematical relationships between variables , such as visually diagnosing and with a formal test, qualitative coefficients used (Pearson, Spearman, Kendall), with multiple variables (chi-quadro di indipendenza, test esatto di Fisher, V di Cramér, coefficiente di contingenza, phi), as misurare I agree to value with the Kappa, and how to manage the variability of the agreement to process the correlation between parziale. Inoltre, apriamo una finestra sull' algebra delle relazioni (in siemistic and computational sense), essential for formalizing concepts such as domain, composition, inversa and funzioni.
What we intend for “relazione” due to variability
In mathematical language, a relationship R between A and B is a sottoinsieme of the Cartesian prodotto A×B : the copy ordinate (a, b) apparent to R indicates that the element a di A is “in relationship” with the element b di B. In termini applicati, a relationship può essere deterministica (funzione) oppure probabilistica (associazione stochastica): in the first case ad ogni a corresponde esattamente un b, in the secondo we served a tendency (più or less strong) and not a legam uno-a-uno.
All internal algebra relations (in an insiemic sense) are interesting properties such as riflessività (ogni a è in relation to this relationship), symmetry ((a, b) implies (b, a)), antisymmetry (if (a, b) and (b, a) then a = b) and transitività (da (a, b) and (b, c) follows (a, w)). This property is crucial: if a relationship is complex, symmetrical and transitive , it reveals a relationship of equivalence that “partitions” A in classi di elementi equivalenti. If invece is riflessive, antisymmetrical and transitive, we abbiamo una relazione d'ordine.
In this analysis, we translate this idea in practice: evaluating the type of variability (quantitative continua, ordinali, nominali/dicotomiche) e la classificazione nalle statistiche e scegliamo strumenti adeguati per misurare la relazione, knowing that “relazione” does not always mean “cause-effect”.
Visual ispezione: eat riconoscere pattern and direction
Prima di buttarsi nei numeri, un graphico a dispersione è spesso la demossa più intelligente. A cloud all along a straight line suggests a linear relationship (positive or negative); structure the curve indicating non-linear relations (but monotone). If you don't intravede a pattern, the linear relationship may destroy the assent.
With variable values, the scatter plot pannelli (and example in R with pairs(dati) ) and smussate trend line (geom_smooth in ggplot2) can help to note associations, clusters and outliers . Attenzione: gli outlier podeno influenzare moltissimo i coefficienti, specie Pearson.
For example in R you can start like this: pairs(dati)
# Matrice di grafici a dispersione
pairs(dati)
# Correlazioni grezze (matrice)
cor(dati, use = "pairwise.complete.obs")
A first visual screening, corrected by the number, helps to monitor the test based on shape, monotony and apparent linearity.
Check its normality: when serving davvero
The parametric test (as well as the Pearson test) assumes, in some cases, normally or with a symmetrical structure without heavy code. Per controllare l'ipotesi, si parte spesso da strumenti grafici como istogrammi con densità e QQ-plot.
In R, a minimum battery may include:
# Istogrammi con densità e curva normale sovrapposta
par(mfrow = c(2, 2))
plot_hist <- function(x) {
hist(x, prob = TRUE)
lines(density(x), col = "red")
curve(dnorm(x, mean(x), sd(x)), add = TRUE, col = "blue")
}
plot_hist(dati$GASTEDU)
plot_hist(dati$GASAUDE)
plot_hist(dati$GASLAZER)
plot_hist(dati$IDADE)
To compare the observed quantities with a normal standard, the QQ-plot is uninstitutional:
par(mfrow = c(2, 2))
qqfun <- function(x) {
qqnorm(x, main = "", xlab = "Quantili teorici N(0,1)", pch = 20)
qqline(x, col = "red", lty = 1)
}
qqfun(dati$IDADE)
qqfun(dati$GASAUDE)
qqfun(dati$GASLAZER)
qqfun(dati$GASTEDU)
In view of this, if I can not use the formal normality test for accumulating evidence: Kolmogorov–Smirnov with statistical parameters, Lilliefors, Cramér–von Mises, Shapiro–Wilk, Shapiro–France, Anderson–Darling and the Pearson (chi-quadro) normality test.
normalita <- function(x) {
t1 <- ks.test(x, "pnorm", mean(x), sd(x)) # Kolmogorov–Smirnov
t2 <- nortest::lillie.test(x) # Lilliefors
t3 <- nortest::cvm.test(x) # Cramér–von Mises
t4 <- shapiro.test(x) # Shapiro–Wilk
t5 <- nortest::sf.test(x) # Shapiro–Francia
t6 <- nortest::ad.test(x) # Anderson–Darling
t7 <- PearsonDS::pearson.test(x) # Pearson chi-quadro di normalità
pv <- c(t1$p.value, t2$p.value, t3$p.value, t4$p.value, t5$p.value, t6$p.value, t7$p.value)
data.frame(p_value = pv, row.names = c(t1$method, t2$method, t3$method, t4$method, t5$method, t6$method, t7$method))
}
normalita(dati$GASAUDE)
If I dated back to normal and the relationship is linear , the Pearson coefficient is a good scelta prima; otherwise it is prudent to prefer to mix basate sui ranghi with Spearman or Kendall.
Relationship between quantitative variability: covariance and correlation
The covariance between The problem is the covariance depending on the mixed unit , which is little comparable.
The Pearson linear correlation solves the standardizing problem for the standard deviation: the coefficient r varies between -1 and 1, giving values ranging from ±1 indicating a strong linear relationship (positive or negative) and 0 suggesting linearity. Attention: r = 0 does not exclude non-linear relations.
Calculation per correlation in R:
# Coefficiente di Pearson e test di significatività
cor(dati$GASTEDU, dati$GASAUDE, method = "pearson")
cor.test(dati$GASTEDU, dati$GASAUDE, method = "pearson")
When the relationship is monotonous but non-linear or when it is not normal (the outlier), there is sense in using Spearman (ranghi; robust and adapted to medium-large campioni) or Kendall τ (basato su concordanze/discordanze; preferibile su campioni piccoli or con molti pareggi):
# Correlazioni non parametriche
cor.test(dati$GASTEDU, dati$GASAUDE, method = "spearman")
# Su un sottoinsieme più piccolo, meglio Kendall
dati2 <- head(dati, 20)
cor.test(dati2$IDADE, dati2$GASAUDE, method = "kendall")
An idea useful: the coefficient for determining R² in a linear model without restituting the variable rate of Y spiegata linearly from X. In R:
summary(lm(GASAUDE ~ ESTCIVIL, data = dati))$r.squared
For dichotomy vs quantitative variability, the point-biserial coefficient coincides with the Pearson calculation with a coded variable 0/1; in practice, cor(dichotomous, quantitative) with method = “pearson”.
Variable categories: indipendenza, strength of association and piccoli campioni
When the variability of nominal or ordinal sleep enters , the relationship is studied with a contingency table (frequency with doppia classification). The typical ipotesi is: H0 = indipendenza (necessary association), control H1 = dipendenza.
The difference test is the indipendenza chi-framework . If your sleep is frequently lower than that (typically < 5), if you use the Yates correlation for 2×2 or pass the Fisher test , particularly indicated with campioni piccoli.
# Tabella incrociata e chi-quadro
xtabs(~ PROFI + ESTCIVIL, data = dati) -> tab1
chisq.test(dati$PROFI, dati$ESTCIVIL) # p-value non significativo => indipendenza plausibile
# Campioni piccoli: test di Fisher
chisq.test(dati$PROFI, dati$RENDA)
fisher.test(dati2$PROFI, dati2$RENDA)
To quantify the intensity of the association in the I×J table if you use: V di Cramér (0–1), contingency coefficient (0–1, limitation), and in 2×2 il phi (φ) , which is formally a Pearson r application to the binary table.
# Misure di associazione per tabelle
library(vcd)
xtabs(~ PROFI + RENDA, data = dati) -> tab2
assocstats(tab2) # riporta V di Cramér, coeff. di contingenza e test
# Attenzione: cor() su codifiche numeriche di categorie non è equivalente a φ in generale
If the interest is to compare groups with quantitative variables (reddit per profession), if the test is valuated , ANOVA and the rispettive alternative non-parametric, there is no confidential interval. In general, it is consistent with the type of variability and method is fondamentale.
Accordo tra valutatori: Kappa di Cohen
When due (the più) giudici classifies gli stessi oggetti in category, it is not enough to mix the percentage of agreement, perché a quota può essere dovuta in this case . Il Kappa di Cohen quantifies the correct agreement for the casual agreement: I value it further to 0 indicating the agreement pari al caso, I value it maggiori suggest in the increasing agreement (interpretazioni comuni: scarso, discreet, moderato, good, molto good).
In R it is possible to estimate Kappa anche in versione ponderata (per category ordinali):
set.seed(1)
val1 <- sample(0:1, 10, replace = TRUE)
val2 <- sample(0:1, 10, replace = TRUE)
# Kappa non ponderato
fmsb::Kappa.test(val1, val2)
If the category is not unordinarily natural (and sometimes, mild/moderate/severa), Kappa considers the penalty less in disagreements “vicini” and less “lontani”, resulting in a much more informative one.
Relazioni e variabili di confusione: la correlazione parziale
Può capitare che due variabile In question, if you calculate the correlation between X and Y controlling for Z.
In R, with ggm::pcor si ottiene la correlazione parziale di primo ordine e si può testne la significatività. It is useful to compare classical and parziale per capire as Z “spiega” del legam grezzo.
library(ggm)
# Correlazione parziale tra GASLAZER e GASAUDE controllando GASTEDU
rp <- pcor(c("GASLAZER", "GASAUDE", "GASTEDU"), var(dati))
# Correlazione grezza
r <- cor(dati$GASLAZER, dati$GASAUDE)
# Test della correlazione parziale (1 variabile di controllo)
pcor.test(rp, 1, length(dati$GASAUDE))
# Confronto R^2 grezzo vs parziale
data.frame("Senza_controllo" = r^2, "Con_controllo" = rp^2)
If the spigot “crolls” in control, allora Z was an important partner . When the control variable is category (es. professione), I can not use similar approaches or models that include the category as a factor.
Limit and good practice in the use of the correlation
The Pearson coefficient mixes only linearity . If the relationship is non-linear (quadratic, sponenziale, logarithmic), it can result in 0 pur in the presence of a strong dependency. In this case, Spearman values , transforming the non-linear model accordingly.
Gli outlier podeno distorte parecchio il resultato. For this reason, first of all, it is convenient to check the graph and, if necessary, carry out robust analysis or check the sensitivity of the result when eliminating extreme points.
Another classic: correlation ≠ causality . Due to variability I can not correlate perché sleep influenzate gives a third fattore, merely coincidence or per effect of an inverse mechanism. Servono disegni sperimentali, strumenti causali o modelli appropriati per parlare di causa-effetto.
Type of relationship: linear, monotonous, null, categorial
This is often mixed (quantitative vs dichotomous/nominal/ordinary) if it becomes a test of comparison between groups, correlations biseriali/point-biserial or a model (and example regression with predictive factori). With variabili enterbe categoriali , ci si affida a chi-quadro/Fisher e alle misure di strength dell'associazione già viste.
Funzioni eat special relationships
A function is a relationship that, per one element of the domain, closely associates one element of the codomain. In the end, it is a sottoinsieme di A×B tale that ogni a in A compare a sole volta with the first component. Other important operations in the algebra of this relationship are: union (R1 ∪ R2), intersezione (R1 ∩ R2), composition (R2 ∘ R1) and inverse (R^{-1}, which is a simple copy of (a, b) in (b, a)).
Questi mattoni conceptuali tornano utili sia nella matematica pure (equivalenze, ordini, classi di equivalenza) sia in applicazioni concrete dove serves formalizzare relazioni fra entità.
Algebra of relationships and science I gave
In the world of the database, the relative algebra is the theoretical structure of SQL: operations such as selection, proiezione, join, unione and intersezione versioni pratiche di operazioni su relative-insiemi. This formalization is a powerful weapon for describing, optimizing and manipulating things quickly.
In the context of artificial intelligence and machine learning, the capacity of modeling relationships between entities allows for cost-effective construction of knowledge, systems for raccomandation and pipeline for integration; This value is valid in economics, biostatistics and social science, it is almost always “ chi è legato a chi, e in che mode ”.
Classical examples of relationships in theory of insiemi
Relation of equivalence : realistic, symmetrical and transitive sleep. Esempi tipici includono “avere lo stesso valore assoluto” sui reali o la congruenza modulo n sugli interi. Ogni relation of equivalence induces classi di equivalenza that ripartiscono l'insieme in blocchi disgiunti.
Relazioni d' ordine : riflessive, antisimmetriche and transitive. Esempi: ≤ sui numeri naturali, l'inclusione ⊆ sull'insieme delle parti P(X), o “divide” (|) sui naturali. Sono fondamentali per definee ordinamenti totali o parziali e per ragionare su massimi, minimi, catene e antichains.
Order lessicography : what I said. It is an order in a sequence (stringhe, number written in sequence) that compares element by element; There is an important noise in this structure and I neglect algorithms.
This section is useful perché cirda that “relazione” in mathematics is a major concept of the sole correlation : include uguaglianze, ordini, composition and so much more, all indispensable for modellare sistemi complessi.
Practical workflow: control it there results
It is possible to flow the work without losing it through construction and sometimes:
- Graphic inspection: scatter plot, smoothing, istogrammi, QQ-plot. Occhio a outlier e forma non lineari.
- Verifications: if it works, test normality; Controlli di omoscedasticità if a linear model is foreseen.
- Scelta del coefficiente: Pearson (lineare, dati circa normali), Spearman/Kendall (monotona/non normale/piccoli campioni).
- Categorical: chi-quadro/Fisher + misure d'associazione (V di Cramér, coeff. di contingenza, φ). For group comparison, t-test/ANOVA or alternative.
- Confundenti: correlate parziale or multivariate model (including category fattori).
Qualunque sia la scelta, interpretae nel contesto è la chiave: grandezza dell'effetto, significatività statistica, ampiezza del campione e qualità dei dati contano como (se non più) del numero finale.
Esempi di codice R ricorrenti
In short, you can use this method to make it more accurate in analyzing the relationship between variables, and then adapt it to your dataset:
# 1) Correlazioni
cor(x, y, method = "pearson")
cor.test(x, y, method = "spearman")
cor.test(x_small, y_small, method = "kendall")
# 2) Grafici (base e ggplot2)
plot(x, y)
# ggplot2: geom_point() + geom_smooth(method = "lm" o se = FALSE)
# 3) Normalità
shapiro.test(x)
nortest::ad.test(x)
# 4) Tabelle e test
xtabs(~ a + b, data = dati) -> tab
chisq.test(tab)
fisher.test(tab) # campioni piccoli o celle con attesi < 5
vcd::assocstats(tab) # V di Cramér e coeff. di contingenza
# 5) Kappa
afmsb::Kappa.test(giudice1, giudice2)
# 6) Correlazione parziale
ggm::pcor(c("x", "y", "z"), var(dati))
Practical note : le scelte “giuste” dipendono dai dati. It's worth validating the hypothesis, comparing the method and, when it works, constructing a piccolo script for diagnosing the question quickly in a reliable way.
Abbiamo seen that “relazione” can mean more often than not: the linear form with Pearson, the monotony with Spearman/Kendall, thin all'association within the category with chi-quadro/Fisher and misure with V di Cramér and φ; senza dimenticare the agreement between giudici (Kappa) and the management I gave confondenti with the parziale correlation. Therefore, the relational algebra fornisce i mattoni concettuali per defining domain, inverse and composition, between the relational algebra guides the mode in which I archive and interroghiamo i dati nei database. The chiave idea is scegliere lo strumento in base al type di variabili, alla forma del legam e agli obiettivi dell'analisi , ricordandoci che nessun coefficiente, da solo, può raccontare tutta la storia.
