I think it helps to understand that spacetime itself is curved.

We like to think that the moon orbits the Earth by following a curved path around us that draws a circle... well, an ellipse... but this model isn't actually correct. The moon follows a straight line through curved spacetime, it is the Earth's gravity that curves spacetime. If the moon was constantly turning towards us, it would constantly lose momentum. It isn't losing momentum (aside from the tiny effects of tidal forces, but forget that), so it's going in a straight line. But we can see it isn't.

As we go away from the centre of the universe, we might be travelling in a straight line, but we're probably always travelling through curved space. So, an observer outside of the universe might not see us travelling in a straight line, even though we are. Like the moon is.