Attention treats all positions equally. It has no built-in notion of word order. Positional encoding adds order information.
The original transformer used sinusoidal functions at different frequencies. Modern LLMs often use learned position embeddings or rotary positional embeddings (RoPE).
Without positional encoding, "dog bites man" and "man bites dog" would look identical to attention. Position matters for language.