Here's what matters: you don't perform the swaps. Just calculate how many each movement needs. Moving from position to position takes swaps. Moving from position to position takes swaps.
I'll show you the edge case. If the tallest soldier starts behind the shortest soldier, they pass each other during swaps. This saves one second because they swap for free while crossing.
This teaches you to simulate operations mathematically, not step by step. You count what would happen without doing it. Faster to compute and easier to code.