Launching from the poles would be about the same as launching from anywhere else on Earth. You would not get the advantage of launching eastward to use the Earth's rotation to add to the speed you need to get into orbit. And you would not be able to go into low-inclination orbits like the Hubble Space Telescope or even the International Space Station. But polar orbits would be easy! And polar orbits are very useful. Many Earth observation and low-altitude communications satellites are in polar orbits. The USA launches into polar orbits from Vandenberg Air Force Base in California, northwest of Los Angeles.
It really isn't possible to go straight up and keep going straight up very far, from anywhere on Earth. Even if you escape from Earth, you will still be in orbit around the Sun. No rocket can carry enough fuel to fly away from the Sun in anything like a straight line.
There isn't much within the Solar System straight up from the poles. All the known planets and the large majority of known asteroids in our Solar System are close to the ecliptic plane. That is because when the Sun and Solar System formed, the part of the dust cloud they formed from collapsed into a flat disk. However, some asteroids got thrown out of the plane of the disk when they passed close to large planets, so they are now in high-inclination orbits, where they might occasionally pass very high above the Earth's poles.