Factor analysis of the adapter Gram

Principal axis factoring with oblimin rotation on the 134x134 trait correlation matrix; parallel analysis chose 9, five were also extracted because five is the hypothesis, and no Goldberg congruence clears 0.85.

Factor analysis of the adapter Gram

What is factored

qwen35/analyse_fa_qwen35.py reads results/gram_sweep.npz - the exact 134x134 Gram - and factors the trait correlation matrix derived from it. Two matrices are run throughout:

• uncentred: R = G / outer(d, d), plain cosine.
• centred (ipsatised): the grand mean removed by double-centring, H = I - 11^T/p, Gc = H G H, then Rc = Gc / outer(dc, dc).

Off-diagonal statistics (qwen35/results/fa_qwen35.md, section 2):

Off-diagonal agreement between the two: r = 0.9556. Uncentred condition number 49.4. Ipsatisation removes exactly one dimension, so R_centred has rank p-1, its smallest eigenvalue is 0 and SMC is identically 1, which is why the centred run starts PAF from ridge-SMC (ridge = 0.001).

The 34 Lexicon traits are factored but score 0 in every congruence target, so congruence is judged on the 100 Goldberg markers only (fa_qwen35.md, section 6 target definition). See Trait provenance.

The file's own verification section reports all_ok = True across seven checks (hand eigendecomposition, SMC vs explicit OLS, varimax recovering known structure, two varimax algorithms agreeing, oblimin on known oblique structure, Tucker hand case, PAF recovering a known model).

PCA versus PAF: two different matrices

The blog page draws the two decompositions side by side and refuses to put them on one axis. qwen35/build_blog_page.py, scree_svg() docstring:

LEFT is PCA of the adapter cloud: eigenvalues of the double-centred Gram of the 134 weight updates, in per cent of total variance. ... RIGHT is the factor-analytic question, which is a different matrix and a different quantity: eigenvalues of the 134x134 trait correlation matrix with the diagonal REDUCED to communality estimates ... Principal axis factoring partitions common variance, not total variance, so its eigenvalues are smaller, can go negative, and its elbow is not the PCA elbow.

The PCA side is PCA, the scree curve and the two nulls.

How many factors

Reduced (SMC-diagonal) centred eigenvalues, ranks 1-15 (fa_qwen35.md, section 3): 16.514, 14.180, 6.367, 4.859, 3.336, 2.668, 1.969, 1.710, 1.402, 1.284, 1.200, 1.148, 1.104, 1.031, 1.006.

• Kaiser (unreduced eigenvalue > 1): 16 uncentred, 18 centred.
• Reduced eigenvalues > 1: 9 (uncentred).

Horn's parallel analysis, 95th percentile, first crossing, 500 reps per cell (fa_qwen35.md section 3; the same grid is qwen35/results/fa_qwen35.json#n_factors.parallel_analysis_uncentred.grid):

#n_factors.reference_N = 1528. The chosen number is stated in the report as

Chosen k = 9 (k = Horn's original (unreduced, 95th-percentile) parallel analysis on the ipsatised matrix at N=1528. ... N=1528 is carried over from sweep100's lower estimate (m in [1528, 5809]). ... Solutions are also extracted at exactly 5 because 5 is the hypothesis.)

There is no true N: these are 134 weight updates, not 134 questionnaire respondents, and the reference figure is an effective dimensionality carried over from an earlier sweep with no reseed controls here to re-estimate it. The blog page states the consequence plainly: "Five factors is a choice, made so the solution can be compared with the Big Five, and the data does not pick it."

Note for anyone rebuilding the figure: the blog page's right-hand panel reads n_factors.parallel_analysis_uncentred even though the chosen-k rationale in fa_qwen35.md is phrased in terms of the ipsatised matrix. At N=1528 both give 9 unreduced, so the marks drawn ("5 extracted", "12 retained") are unaffected.

The k=5 solution

Four solutions were extracted (centred_k5, centred_k9, uncentred_k5, uncentred_k9), each unrotated, varimax and oblimin. PAF converged in 8-12 iterations with 0 Heywood cases in every one (fa_qwen35.md section 4). The solution the project uses is centred_k5, oblimin (direct oblimin, gamma = 0); its factor intercorrelations are small (largest |Phi| 0.282, mean 0.120, section 5).

Five factors, ordered by SS loading, with the project's names (qwen35/analysis/fa_summary.json#centred_k5.factors[].name, mapped to the blog page's display titles by FA_KEY in qwen35/build_blog_page.py):

(fa_qwen35.md section 6, centred_k5 oblimin table, and section 7 headers.)

The blog page renames "Fearful withdrawal" to "Approach and Avoidance" in display only; the underlying direction, npz slug and steering job are all FA_FearfulWithdrawal.

The full congruence row for that solution (fa_qwen35.md section 6, "Which Goldberg factors clear the thresholds?"):

centred_k5, oblimin -- best congruence per Goldberg factor: E 0.539, A 0.655, C 0.574, ES 0.405, I 0.682. Clearing 0.85 (fair): none. Clearing 0.95 (equivalent): none.

The same holds for centred_k9 (E 0.406, A 0.644, C 0.556, ES 0.487, I 0.616), uncentred_k5 (0.329 / 0.536 / 0.490 / 0.393 / 0.575) and uncentred_k9 (0.374 / 0.629 / 0.536 / 0.464 / 0.636). No solution clears the conventional "fair" congruence bar against any Goldberg factor. The recovery is real and ordered but it is not equivalence, and any wording that implies the Big Five was reproduced should say so.

A note on the maintainer's recollection: the figures recalled as "Warmth~A 0.66, Competence~C 0.57, Imagination~I ~0.6, Fearful withdrawal ES 0.41 / E 0.34, Arousal E 0.54 / ES -0.35" are right except for Imagination, which is +0.682, not about 0.6. The paired secondary values are also in the table: Fearful withdrawal loads E +0.338 alongside ES +0.405, and Arousal loads ES -0.348 alongside E +0.539 (fa_qwen35.md, centred_k5 oblimin).

Does an evaluative factor survive rotation

fa_qwen35.md section 8. The "Eval" target is +1/-1 by keying for every primary trait. A perfectly clean Goldberg factor already scores 0.447 on it by construction.

An evaluative ("good trait / bad trait") factor is visible before rotation and falls below the construction baseline after it.

Communality

fa_qwen35.md section 9: mean communality h^2 is 0.317 for centred_k5 (u^2 0.683, range 0.051-0.516), 0.360 for centred_k9, 0.352 for uncentred_k5, 0.402 for uncentred_k9. There are no reseed controls in this sweep, so the noise-floor comparison an earlier sweep ran cannot be reproduced and the communalities have no reliability ceiling to be read against.

The k=9 solution and the steering hand-off

qwen35/analysis/fa_summary.json#centred_k9 stores nine factors with name = null - the k=9 factors were never named. Their SS loadings and top traits: 6.43 (untalkative, disorganized, extraverted, haphazard, quiet), 5.61 (timid, shy, bashful, guilty, weak-hearted), 5.47 (neat, unintellectual, negligent, casual, conscientious), 5.00 (vigorous, bold, practical, gruff, efficient), 4.89 (warm, kind, mothering, cooperative, engaging), 4.22 (impractical, unimaginative, imaginative, creative, innovative), 2.75 (self-pitying, moody, touchy, emotional, melancholy), 2.73 (uninquisitive, deep, fretful, careful, uncertain), 2.26 (envious, selfish, jealous, crooked, distrustful).

fa_qwen35.md section 10 records that steering.oblimin_loadings holds the nine-column oblimin pattern of centred_k9, and that steer134_on_modal.py's fa_coeffs turns column j into the fa{j} steering direction (mean-centred, unit norm in the double-centred Gram metric). Best Goldberg match per column: fa1 = E (0.41), fa2 = E (0.36), fa3 = C (0.56), fa4 = E (0.28), fa5 = A (0.64), fa6 = I (0.62), fa7 = ES (0.49), fa8 = I (0.43), fa9 = A (0.44). So the fa* directions that appear in the steering corpus are the nine-factor solution, while the five named factor cards on the blog page are the five-factor solution - a naming collision worth knowing about when reading Steering the base model along weight-space directions.

Qualitative reads

qwen35/analysis/qual_fa.json and qual_fa_notes.md hold a blind read of the steered transcripts for the five factors plus PC4-PC6, from phase10_runs/steer_results_fix2.json, judged with phase10_runs/judged_steerfix23.json. Both record that about 59% of responses end mid-sentence (512-token cap) so nothing there claims anything about how a response concludes, and that zero false identity claims occur across all eight directions at any alpha. qual_fa_notes.md also reports damage-corrected slopes: seven of the eight directions keep their leading judged scale on the subset of prompts that never loop; the one that does not is FA_FearfulWithdrawal.

Related: PCA, the scree curve and the two nulls, Weight-space geometry of the 134-adapter zoo, Polarity, bipolarity and the trait graph, History of the Big Five, Steering the base model along weight-space directions, Glossary.