Epsilon & Unsociable
Epsilon Epsilon
Hey, I've been tinkering with neuroevolution to design ultra‑compact LSTM cells for edge deployment—ever tried evolving architectures instead of hand‑tuning them?
Unsociable Unsociable
No, I haven't. The search space is huge, so you need a really good fitness function and a pruning strategy. It can get messy if you don't keep the model size in check.
Epsilon Epsilon
Yeah, the combinatorial explosion is brutal—if you treat every gate and weight as a binary decision, the number of possibilities grows faster than a micro‑organism on a petri dish. I usually start with a multi‑objective fitness: accuracy plus FLOPs plus latency, then prune aggressively by zeroing out entire sub‑layers that don't improve the composite score. Keeps the architecture lean and lets the algorithm converge before it turns into a bloated monster. What’s your go‑to method for measuring “size” in your pipelines?
Unsociable Unsociable
I usually just count parameters and multiply by the number of operations per forward pass. That gives a quick estimate of FLOPs, and I keep an eye on the model’s total MAC count. If it crosses a preset threshold, I prune or collapse the sub‑layer right away. Keeps the design bounded before the evaluation loop drags on.
Epsilon Epsilon
That’s a solid baseline—parameter count times ops per pass gives you a rough wall‑clock estimate. I sometimes augment that with a tiny profiler that samples the actual runtime on a target device; the hardware‑specific optimizations can skew the theoretical FLOPs. But keeping a hard MAC cap is the safest way to stop the search from spiraling into unmanageable size. Have you tried any dynamic sparsity tricks to prune after training instead?