One of my favorite areas of Amsterdam in terms of architecture, is Silodam.

Built in 2002, the building looks like a bunch of containers stacked on top of each other. It faces the inner harbor, just a stones' throw away from Centraal Station.

Most tourists never make it over there, so it certainly qualifies as "off-the-beaten-path". To get there, I recommend going through Prinseneiland (also off the beaten path and beautiful). Here's a little google map that shows where Silodam begins. Remember its a residential building, so don't expect fancy pubs or restuarants, just enjoy the view.