From Algorithmic and RL-based to LLM-powered Agents