Quirrel had seen Harry use /Diffendo/ on some trees, and later that the trees have been cut. He was unconscious (and in an extradimensional bag) when Harry had cut through the wall in Azkaban, and only saw a cut circle of wall. He may not have known that Harry had anything up his sleeve more complicated than a Cutting Charm; he certainly had no reason to believe that Harry could wordlessly transfigure the tip of his wand into well over a hundred feet of braided carbon nanotubes. Quirrelmort has never seen—only Dumbledore, Hermoine, and Professor McGonagall have.
But that’s really not the part that got him.
Quirrelmort had accepted the risk that Harry could have escaped, or killed everyone present, just as he accepted the possibility that the Unbreakable Vow wouldn’t have been enough to stop Harry from destroying the world. If he were absolutely certain, he’d not have bothered with backup plans. He did not care about the deaths of the present Death Eaters, and losing his own body was merely a minor setback. It’s Harry’s ability to instantly and permanently incapacitate without letting Quirrelmort’s spirit loose that made the threat serious. That’s a problem Dumbledore was relying on an ancient and frighteningly powerful artifact to implement, and Quirrelmort’s mode of thinking doesn’t exactly encourage thinking of these matters..
Is that what we’ve seen presented so far?
Dumbledore won during the Battle of the Three Armies. His assault on Azkaban would have gotten him killed (and more seriously, set back his efforts by years) for a stupid communication error, were Harry not willing to risk his own life and invent new magic to save the man. Hermoine outlasted several hours of the Defense Professor’s most aggressive psychological attacks possible, using fairly basic deontology. His ‘lesson plan’ with Ma-Ha-Su in Chapter 16 was bluntly stupid, even if Harry hadn’t used the easy way out. In Chapter 35, he fears that Harry has screwed over his plans because of voicing an obvious disagreement that Harry has repeatedly given privately before.
And that’s before we get to the stupidity that was enforced by canon : testing multiple novel spells (Horcruxes, however he ‘reformated’ the young Harry Potter) without sufficient and verified safeties, the highly fractious Death Eaters, the lackluster war with Dumbledore.
Quirrellmort is smart. He thinks ahead. But his fundamental philosophy is still very restricted. As much as he tries to claim otherwise, he’s running on distilled Command Push—we’ll note that no Death Eater gave him advice in this chapter, nor would we expect them to. His speech in Chapter 34 follows the same philosophy.
But more importantly, he underestimates risks. He’s a partially-formed rationalist, who has heard of Kolmogorov complexity but can’t quite understand why he should shut-up-and-multiply yet. He leaves Harry a wand because wanded Harry is only a threat because of that wand if he has a) wordless, b) motionless, c) wanded, d) magic that can instantly disable Death Eaters, e) can hit him at all and f) threatens an immortal. It’s understandable to not think Harry is a risk. A full-grown wizard in the same environment wouldn’t be a risk—Dumbledore or Mad-Eye Moody would have died, and died quickly. That’s not as unreasonable a mistake as you’d expect.