The PagedAttention paper listed by AgentLog addresses fragmentation and duplication in attention caches used for model serving. It helps explain why memory management and batch size affect inference capacity.
When evaluating a service, measure concurrency and context alongside latency. Do not compare only a short response from an idle environment. Record input sizes and test conditions to distinguish engine capacity from application queueing.
Include isolation in acceptance criteria. Context, history and caches must respect separation between users and organizations. An efficient memory technique does not establish that property: permissions and data lifetimes require additional implementation decisions.
At the textual boundary, reduce personal data before inference. Exercise separate sessions with fictional values and look for cross-session contamination. Rota applies the policy in its supported workflow; this is not a certification of the internal cache management of any engine referenced by the source.