Calling something a "failure of imagination" comes across harsh. It suggests a cognitive failure that couldn't foresee something that never was.
The lack of well-known profound classical texts on the quality of imagination suggest to me that it's not a quality that's prioritized or aspired to. But anything new and valuable started off as an imagination in someone's mind. So it's safe to say all progress rest of imagination.
Yet we're not taught how to imagine. Is it even teachable, I wonder?
Imagination seems to be a root quality that doesn't rely other qualities. You don't need strenght to be imaginative; you don't need to be kind, maybe not even very curious. That makes cultivating imagination tricky.
I've been thinking about coming up with a framework that spurs imagination, like a kind of Hero's Journey for what never was. It didn't take long for me to doubt such an approach.
I'm rather convinced that deduction doesn't count as imagination. Suppose you were tasked to come with a fictional premise to an alien world and try to be imaginative about it. You do that by making the population single-gender and give everyone the ability of teleportation. The story proceed as logical progression of those premises. While the end result might come off as imaginative but what takes place in this fictional world is likely logical deduction on the premise.
A common advise towards better imagination is "read more". But all that does is exposing yourself to more existing ideas. By knowing more the chances of coming up with something to you would be higher. Does that necessarily mean you are capcable of conjuring up something entirely new on your own? I'm not sure.
This relates to the burden of knowledge. Is knowledge an asset or a burden to imagination? In other words does knowing more refrains your imagination to the shackle of what's possible? If so then someone completely ignorant should make the most imaginative person. That doesn't fly.
At this point I'm reminded of how drugs make you creative. The factor lies in the connections between knowledge, not in the amount of knowledge possessed. The more rigid the connections between concepts, the less imaginative we get.
Imagination happens when we connect concepts that have never been connected together before.
Now I'm interested in the process, not outcome. I wonder how the first human (maybe HG Well?) who came up with the idea of time travelling machine landed on it. I wish it's not entirely accidental. If there's a process to it (even if he was entirely unaware of it) can it be repeated?
The phrase "limited by your imagination" positively connotates that possiblity is vast. But imagination on an individual level isn't as vast as it sounds. Imagination is often the one thing that limits you.