Retaining walls are installed with similar techniques as unit pavers. Walls could be built with precast (wall blocks) or natural stone. Walls will stay straight if drained well.
If water collects behind your wall it will freeze in the winter and push your wall. Each wall block is engineered to retain at a maximum certain height.
A city permit will be required if the wall is taller than one meter.