ollie_ comments on How do I design long prompts for thinking zero shot systems with distinct equally distributed prompt sections (mission, goals, memories, how-to-respond,… etc) and how to maintain llm coherence?

[ollie_]

Here’s the Replit CEO Amjad Masad confirming what i’ve seen (timestamp: 36:45). “After 32k tokens, reasoning and a lot of benchmarks tank”