I don't quote know what book you are using, but here is a way to think about it. It clearly suffices to prove this for mappings since . So, let be continuous, we know that we can extend to . So, let . By continuity we know that is closed in . That said, since is an extension of and we know and so is disjoint from . Now, by Urysohn's lemma there exists some continuous map with and . So, define . From basic topology we know that is continuous and we know it's an extension for since for all . I'm pretty sure then you can checking where elements of and go under separately that . Make sense?