Each SVI acts as the default gateway for its VLAN. Hosts in VLAN point their default gateway to the VLAN SVI IP address.
When a Layer switch receives a frame destined for another VLAN:
It examines the destination IP address
It consults its routing table
It rewrites the Layer header for the destination VLAN
It forwards the frame out the appropriate port
This all happens in hardware at wire speed. The first packet in a flow may be handled by the CPU to build the hardware entry. Subsequent packets are switched entirely in hardware.