It’s difficult to describe exactly what it is that Dav Rauch does. For one thing, he has a multitude of skills, such as writing and directing. But the projects he applies these skills to are particularly unusual.

For instance, his most significant work centres on designing the interaction between humans and computers in films set in the future, such as Iron Man and Avatar. That means imagining what technology looks like before it has been invented. Such as how should Tony Stark control an Iron Man suit? How should the AI parts of the story be represented? What does it look like when he interacts with it? These questions go into advancing narratives and building the character, rather than merely decorating them.

It’s something he’s dubbed, “FUI or FUX design” (future user interface and experience, respectively). “I see those acronyms bounced around a lot and they seem a little bit silly… which is what I like about them.” It’s obviously a rarefied field that isn’t shared by many people. But it’s one that merges his previous work in interaction, design and film. At a film production company called The Orphanage he worked on commercials, small independent films and big budget blockbusters. “Because it was a small company where you had to wear a lot of hats, I guess my common thread was coming from a design orientation, rather than a visual-effects one.” He did things as diverse as designing title sequences to applying the gritty filmic look to Grindhouse.

But where do you go after figuring out what the future might look like? For the past year and a half, Rauch has worked for IDEO, a design firm that helps businesses innovate and grow. Here he has applied to the real world his instincts for narrative and character that he honed working on films. He’s made trailers to promote Judaism to a younger generation (it ended up in front of Spielberg who loved it and found other backers), worked on an app designed for HIV-positive youth and designed concepts for brands. It will be his work with IDEO that he will focus on when he talks at Semi-Permenant next month.

“I think I flipped the equation. When I was working on film I was working on fantasy products and fantasy interactions between humans and machines,” he says. “And now I’m working on real machines and real things that people can interact with. But I bring a narrative sensibility.”

Throughout our phone call I hear the arrhythmic scratch of drawing or writing –suggesting his imagination is meandering around the peripheries of our conversation. But while exploration has its place, limitation is a more important consideration in creating for the real world: “Constraints are the most important ingredient to good design. And the constraints in the film world are really different. Film has allowed people to think about the future and articulate the future in a way that no other medium has allowed, except maybe for writing.”

Get our pick of the best news, features and events delivered twice a week

And the potential for designing the future is now far greater in the real world than it is on film: “There is so much exuberance and energy and so much creativity and money in the tech sector that the gap between what film can imagine and what the tech sector can produce has been closing.”

The future may have already been told in film, but bringing it to life makes for a more fascinating sequel.

Dav Rauch is speaking at Semi-Permanent which runs from May 22–23.