View source for Unit Perception Tests and Empirical Transition Structures

= Unit Perception Tests and Empirical Transition Structures =

== 1. Unit Perception Test (UPT) ==

A '''Unit Perception Test (UPT)''' is a single, local test of synchrony between two timeline-referenced events.

* We have two time-indexed streams, ''A_t'' and ''B_t'', defined over a shared timeline index ''t'' (the ``tNdx`` in the Timeline Paradigm).
* A UPT evaluates a synchrony predicate ''S'' at a specific pair of time points, for example:
** <math>S(t) = \mathbf{1}\{A_t \in C_A \wedge B_t \in C_B\}</math>
** where ''C_A'' and ''C_B'' are value categories and ''\mathbf{1\{·\}}'' is 1 if the condition holds and 0 otherwise.
* Each UPT is therefore a binary observation: "synchrony detected" (1) or "no synchrony detected" (0).

Because of noise and limited data, any single UPT is unreliable and should be treated as a noisy measurement, not as ground truth.

== 2. From UPTs to State Space ==

To move from isolated tests to a dynamical view, we define a '''state space''' over which the process evolves.

Examples of possible state definitions:

* Binary synchrony state:
** <math>X_t \in \{S, \neg S\}</math>, where ''X_t = S'' if ''S(t)=1'', else ''\neg S''.
* Multi-valued state including lag or pattern class, for example:
** <math>X_t \in \{\text{no synchrony}, \text{lead A}, \text{lead B}, \text{locked}\}</math>.

The Timeline Paradigm provides the indexing substrate: each state ''X_t'' is attached to a specific timeline index ``tNdx``, and can be derived from the underlying category structure (rank array, category bins, and ''hOccurs'' pointers).

== 3. Markovian Iteration and Empirical Transition Matrices ==

Once we have a sequence of states ''(X_t)'', we can treat it as a Markov process and empirically estimate how often the system moves from one state to another.

For a finite state set ''S = {s_1, …, s_m}'':

* We observe a long sequence:
** <math>X_1, X_2, \dots, X_T</math>
* For each ordered pair ''(s_i, s_j)'', we count how often we see a one-step transition:
** <math>N_{ij} = \#\{t \in \{1, \dots, T-1\} : X_t = s_i,\, X_{t+1} = s_j\}</math>
* The empirical transition probability from ''s_i'' to ''s_j'' is:
** <math>\hat{P}_{ij} = \frac{N_{ij}}{\sum_k N_{ik}}</math>

Collecting all ''\hat{P}_{ij}'' into a matrix ''\hat{P}'' gives the '''empirical transition matrix''' (also called the state transition matrix).

== 4. Averaging Away Randomity ==

Why this "averages away" noise in individual UPTs:

* Each UPT contributes a single bit of information (synchrony vs no synchrony) at one time step.
* Random false detections tend to be uncorrelated over time and across many trials.
* True dynamical relationships (for example, persistent synchrony, characteristic lags, stereotyped transitions) manifest as stable patterns in the transition counts ''N_{ij}''.

As ''T'' grows:

* The empirical matrix ''\hat{P}'' converges (under standard assumptions) toward the underlying transition probabilities.
* The Markovian iteration (repeating UPTs across many steps) yields a '''perceptive envelope''':
** a stable map of which states tend to follow which, with what probabilities;
** robust to individual noisy UPTs because it aggregates many of them.

== 5. Relation to the Timeline Paradigm ==

Mapping back to the Timeline Paradigm Matrix implementation:

* The append-only timeline index ``tNdx`` defines the order of observations.
* Category bins (the ''cats()'' ragged arrays) and the rank structure provide efficient access to:
** which times ''t'' are in category ''C_A'' (stream A) or ''C_B'' (stream B);
** where synchrony predicates ''S(t)'' hold.
* A UPT is an evaluation of ''S(t)'' (or ''S(t, t')'' if lags are allowed) at a specific timeline index (or pair).
* The Markov chain ''(X_t)'' is built from these UPT results, and the empirical transition matrix ''\hat{P}'' is the quantitative '''perceptive envelope''' over the dynamic under study.