Merging both arrays takes . You need , which means binary search.
Think about the median differently. If the combined array has elements, you need to find the th element. This means partitioning both arrays so the left halves together contain exactly half the total elements.