QK-NLP Lab

Quaternionic embeddings + metric-based attention, running live in your browser.

Weights are random and untrained, so similarity scores show the geometry at work, not learned meaning. Words that are shared between sentences get identical embeddings, which is what drives the similarity number.

Controls

Metric Curvature κ: 1.0 Quaternion channels C Seed

Sentence A

Sentence B

Attention weights: sentence A

Attention weights: sentence B

Sentence similarity

–

Cosine of mean-pooled outputs (A vs B)

Rotation test

Random rotation r Max score change Max |f(rx) − r·f(x)|

Left-multiplying every token by a unit quaternion r leaves attention scores unchanged and rotates the outputs, because weights multiply on the right.

Parameters (Q, K, V projections)

Quaternion linear (3 × 4C²) Real linear, same width (3 × 16C²) Ratio

Weight sharing in the Hamilton product gives 4× fewer parameters per linear layer at equal width. This is not the blanket "50%" in the repo READMEs.