Vierna & Ding
Vierna Vierna
Ding Ding
Vierna Vierna
Hey Ding, I've been thinking about how we could design a system that outperforms human players in a new turn‑based strategy game. Got any ideas on the architecture?
Ding Ding
Sure thing. Start with a modular stack: a core engine that handles turns and state transitions, a rule‑based layer for tactical choices, and an ML layer on top that learns from game‑plays. Keep the state representation compact so the policy network can train fast, maybe using graph neural nets to capture unit positions. Use a hierarchical planner: high‑level strategy chosen by a policy, low‑level actions refined by a search tree. That way you get human‑like intuition with a performance edge. Keep the modules decoupled so you can swap in a better learning algorithm later.
Vierna Vierna
Sounds solid, but you’re still giving the ML layer too much freedom. I’d tighten the state representation to a fixed‑size vector—no more graph convolutions that keep blowing up memory. Also, the hierarchical planner needs a clear loss function; if the policy keeps choosing “human‑like” moves, you’ll stall progress. Make the high‑level strategy a discrete set of macro‑goals with hard constraints, then let the search do the heavy lifting. And don’t forget to benchmark against a pure rule‑based baseline early—any deviation should be measurable, not guesswork.
Ding Ding
Yeah, a fixed‑size vector is cleaner, just watch out for losing spatial nuance. For the loss, mix win‑rate with a penalty for staying too close to human play. Define macro‑goals by game phase—offense, defense, expansion—and hard‑code constraints. Keep the baseline tests running so every tweak has a clear metric. If you want, I can sketch a quick pipeline to tie it all together.
Vierna Vierna
Alright, let’s outline it. Step one, define a fixed‑size state vector: encode unit counts, resource levels, and a 5×5 heatmap of threat zones, flattened. Step two, the policy network outputs one of the three macro‑goals per phase; train it with a loss = –log(win‑rate) + λ·(human‑similarity). Step three, a deterministic tree‑search (minimax with alpha‑beta) runs on the low‑level actions for 200 ms per turn, constrained by the macro‑goal. Step four, a continuous replay buffer that feeds both the policy and a value network. Keep a separate rule‑based bot for regression tests. That’s the skeleton—tweak the λ and search depth as you see fit. If you send me a skeleton, I’ll spot gaps.