A single point of failure (SPOF) is any component whose failure brings down the entire system. Finding them requires tracing every path data can take.
Common SPOFs include: the uplink to your ISP, core switches, firewalls, DNS servers, and authentication systems. Even power and cooling can be SPOFs if not redundant.
Draw your topology. For every component, ask: if this dies right now, does traffic still flow? If the answer is no, you found a SPOF. Fix it or accept the risk.