<- Back
Comments (12)
- brumarHow do we measure and compare prediction quality in a symbolic/deterministic context that also punish for complexity? I struggled on this in my phd thesis and had to brew my own Minimum Description Length implementation because I found nothing applicable. The statisticians I interrogated were like "don't try to do model selection on symbolic models, make them probabilistic and work from there"
- saithoundPySR itself is still alright, mostly because of previous development. But not for long. Quality control has collapsed, and e.g. the examples in the repo are now embarrassing vibeslop.Pangram reports 100% AI on most comment blurbs in the examples directory. That in itself is not the problem, AI-generated code could still be valid. But in this case, the AI-generated examples have the characteristic shape of slop: plausible sounding technical jargon, combined with zero actual understanding of either the target domain, or even evolutionary search (loss function design). They were vibed by people who had no idea what they were doing.I suggest taking a look at the "evolved PRNG" example [1].First, its loss function is basically designed to reward discovering linear feedback shift registers (LFSRs) only. It deliberately gives a massively lower loss to LFSRs than most other functions.The AI slop phrased this in terms of "certifying the period", but the period certificate used only identifies LFSRs of max period, is unsound on the rest of the search space, and gives a meaningless very high loss to most non-LFSRs. So the regression will almost always just return an LFSR.Whoever designed the example (Codex I suppose) also knew that most LFSRs are defective, and make poor random number generators. Thus, they included a "shear" penalty to detect such defective LFSRs, but of course they slopped it together.The 360 correlations they measure should have been added together (having just one strong correlation is bad, and having two strong correlations is twice as bad as havin one). Instead, they take the MEAN of the correlations, so a perfect correlation contributes only 1/360=0.00278 to the loss.And indeed, the result of their example run, reported in the docs [2] has such a perfect correlation: bit 29 is copied to bit 31 every two iterations.[1] https://github.com/astroautomata/PySR/blob/master/examples/p...[2] https://pysr.ai/v2.7.0/examples/objectives#inventing-a-pseud...
- JanisErdmanisIs the Julia part here compiled via JuliaC and if not what would it take to do so?
- owls-on-wiresrelated: https://github.com/owls-on-wires/keplearnA deterministic SR engine in Rust, written for SRBench. Happy to see the field getting some front-page attention!
- jgalt212Has anyone thrown the n-body problem at this?
- verdvermrelated: https://github.com/verdverm/pge-jaxA deterministic algorithm for Symbolic Regression, which I recently ported to JAX
- thatsabadlookYay now when you load your python project you have 2 interpreters, 2 gcs, 200mb of ram overhead, and nondeterministic performance due to joat.