The Scotsman, tipped as heavy favourite to triumph in the tournament Down Under following defending champion Novak Djokovic's own surprise exit earlier in the week, was undone in four sets by the world number 50.
Murray failed to find his rhythm throughout, dropping serve eight times in all on his way to a 7-5 5-7 6-2 6-4 loss in Melbourne - his earliest defeat in the competition since 2009.
Murray had fared well in the opening stages of the year's first major, heading into this latest test without having dropped a set in his previous three outings.
The five-time runner-up struggled from the outset against Zverev, however, with the German's attacking style causing his opponent all sorts of problems as he raced into a first-set lead.
Zverev, ranked outside the world's top 1,000 just two years ago, was pegged back in the second set as both players continued to struggle keeping hold of their serve.
Murray was visibly relieved to level things up at that point, but Zverev would hold throughout the remaining two sets as he cantered to a mammoth victory, breaking the Brit twice in the third set and once in the opening game of the fourth.