The insight: you don't need to simulate the swaps. If the is at position and needs to reach , the number of moves is the Manhattan distance: .
Each step moves you one position closer in either rows or columns. Manhattan distance counts how many horizontal and vertical steps you need.
First, find where the is located by scanning the matrix with nested loops.