Correlations and discoveries¶

Candidates¶

The engine creates every pairwise combination of enabled analytical numeric metrics:

Boolean,
Status,
Scale,
Duration,
Number,
virtual event metrics.

Free text and choice fields are not correlated. Identical series and series without variance are discarded.

In addition to same-day pairs, targeted next-day rules exist:

Signal today	Signal on the next day
Exercise duration	Stress
Exercise duration	Mood

Spearman rank correlation¶

For \(n\) complete value pairs \((x_i,y_i)\), ranks are calculated first:

\[ R_i=\operatorname{rank}(x_i), \qquad S_i=\operatorname{rank}(y_i) \]

Tied observations receive their mean rank. The Spearman coefficient is then the Pearson correlation of these rank sequences:

\[ \rho_s= \frac{ \sum_{i=1}^{n}(R_i-\bar R)(S_i-\bar S) }{ \sqrt{\sum_{i=1}^{n}(R_i-\bar R)^2} \sqrt{\sum_{i=1}^{n}(S_i-\bar S)^2} } \]

Here \(\bar R\) and \(\bar S\) are the means of the two rank sequences. The range is \(-1\leq\rho_s\leq1\): positive values describe a monotonic relationship in the same direction, negative values one in opposite directions.

The frequently cited shortcut

\[ \rho_s=1-\frac{6\sum_{i=1}^{n}d_i^2}{n(n^2-1)}, \qquad d_i=R_i-S_i \]

is exact only without ties. Because Steady handles ties through mean ranks, the general correlation formula above is authoritative.

Significance approximation¶

A t-statistic with \(n-2\) degrees of freedom is calculated from \(\rho_s\) and the number of complete pairs \(n\):

\[ t=\rho_s\sqrt{\frac{n-2}{1-\rho_s^2}}, \qquad \nu=n-2 \]

The two-sided p-value is determined from Student's t distribution. This test assesses compatibility of the observed relationship with the null hypothesis \(\rho_s=0\); it does not prove causality.

Minimum requirements and multiple testing¶

A result is considered only when:

at least 14 complete value pairs are available,
the absolute Spearman coefficient is at least 0.25,
the Benjamini-Hochberg-corrected q value is at most 0.10.

For \(m\) hypotheses tested at the same time, p-values are sorted in ascending order, \(p_{(1)}\leq\dots\leq p_{(m)}\). Corrected values are:

\[ q_{(i)}= \min\!\left( 1, \min_{j\geq i}\frac{m}{j}p_{(j)} \right) \]

The Benjamini-Hochberg correction limits the expected false discovery rate across all metric pairs tested together. At most 20 discoveries are returned, ordered by absolute strength and sample size.

Below these strict discoveries, Steady shows a separate Emerging patterns category. A pattern in this tier has at least 20 complete value pairs, an absolute Spearman coefficient of at least 0.45 and a corrected q value of at most 0.30. Patterns already shown as discoveries are not repeated in this tier. This list is also limited to 20 items.

Presentation and uncertainty¶

Every discovery and emerging pattern contains:

direction and strength,
number of complete pairs,
number of missing pairs,
same-day or next-day relation,
comparison of the lower and upper third of X values,
evidence level based on pair count,
an explicit note that association does not prove causation.

The card remains visible when no robust discovery is found or calculation fails; the UI then shows an appropriate empty or error state.