Goran Pandev had put North Macedonia ahead in the 2022 World Cup qualifier on the stroke of half-time.
Ilkay Gundogan equalised from the penalty spot early in the second half after Leroy Sane had been fouled.
But North Macedonia, ranked 65th in the world, snatched victory with five minutes remaining when Elmas struck from close range.
Chelsea forward Timo Werner missed a glorious chance to score for Joachim Low’s side before North Macedonia’s winner but somehow fired wide from close range.
‘We’ve never beaten a team with so many titles’
It was a first defeat in Group J for Germany and the first goals they had conceded in qualifying.
The loss also ended their 35-game unbeaten run in World Cup qualifying, stretching back to their 5-1 defeat by England in 2001.
“This is bitterly disappointing,” Low said.
“We were not fresh enough, we made mistakes. When we played quickly we were dangerous but we did not find the tools to be really threatening.
“Overall, it was disappointing to lose like that at home.”North Macedonia’s win was their second in Group J and means they are second with six points, three behind leaders Armenia with Germany third.
“These lads have made North Macedonia proud against the winners of four World Cup titles and three European championships,” North Macedonia coach Igor Angelovski said.
“We’ve never beaten a team with so many titles. Now we have to get some rest as what we have accomplished still hasn’t sunk in.”
Before kick-off, Germany’s players unfurled a banner supporting human rights.
Germany players also wore T-shirts to show support for Qatar migrant workers before their World Cup qualifying win over Iceland on Thursday.