Julian Wiffen: How AI Is Changing Data Engineering Without the Hype

January 30, 2026

Julian Wiffen on How AI is Revolutionizing Data Engineering

In a world where everyone talks about AI, very few leaders explain it in a way that feels real and usable. Julian Wiffen, a senior leader in AI and data science at Matillion, does exactly that. He doesn’t sell dreams. He talks about what actually works inside real data teams—where deadlines are tight, data is messy, and “perfect” is not an option.

This is why Julian’s perspective matters: he shows how AI can help with the most common pain in data work—slow pipelines, messy inputs, repeated tasks, and unclear documentation.

From Chemistry to Smart Patterns

Julian’s journey into AI didn’t start with fancy tech titles. He studied chemistry, and in the 1990s he picked a research project in computational chemistry. He worked with early ideas like genetic algorithms—methods that try many options and keep improving until they find a good solution.

That early experience gave him a habit that still shows today:
keep experimenting, keep learning, and keep things practical.

The “Messy Prep Work” That Nobody Talks About

Early in his career, Julian worked in consulting and BI. His day-to-day work looked like what many data engineers still do today:

cleaning inconsistent data
fixing broken definitions
preparing data for reports
making sure results don’t confuse business users

It’s not glamorous, but it’s the foundation of everything. Julian’s point is simple:
If the base is weak, everything on top breaks.

He also did contracting work and noticed something important: sometimes solutions get delivered… but never become truly useful. That’s why he cares so much about adoption, not just “shipping.”

Cisco Taught Him a Surprising Lesson About Waste

At Cisco, Julian worked in large data-focused roles. One example he shared is powerful because it’s so easy to understand.

Cisco had a system where teams could order services like infrastructure. Julian found that when ordering was hard, people ordered extra “just in case.” But when ordering became clear and easy, people ordered only what they needed.

His simple supermarket example says it all:
If the store is nearby, you buy fresh as needed.
If the store is far, you stock up and waste more.

Good systems reduce waste.
And that’s exactly how he thinks about AI too.

Why He Joined Matillion

Julian moved to Matillion because he wanted speed. In big companies, teams can spend weeks discussing whether to experiment. In a faster environment, you can test and learn quickly.

His belief is direct:
Sometimes it’s better to try a small experiment today than wait for the perfect meeting next month.

At Matillion, his work focused on making AI useful in two clear ways:

AI inside data pipelines
AI inside the product that engineers use

Turning Unstructured Data into Simple Answers

Julian explains one of the biggest shifts in data engineering: unstructured data is now usable.

Before, companies ignored things like:

support tickets
customer reviews
documents and PDFs
voice notes
internal wikis

Because it was too hard to process at scale.

Now AI can help turn that messy input into clean fields that teams can measure. But Julian adds a key rule that makes AI more dependable:

Ask for tight answers, not open essays

Instead of asking, “What is this customer saying?”
Ask:

“Is there a defect mentioned? Yes/No”
“Is this a feature request? Yes/No”

Those Yes/No results can feed dashboards and workflows. Humans then review only what needs attention.

This is one of Julian’s strongest ideas:
AI becomes more useful when you control the output.

The Surprise: AI Could Work With Matillion’s YAML

Matillion pipelines are stored in a YAML format behind the scenes. The team didn’t expect AI to handle it well—but it did, because YAML is readable.

That discovery made a huge difference. It meant AI could do more than explain. It could help build and change pipelines.

This became part of the foundation for Maia, Matillion’s assistant inside the product.

Proof That Caught Everyone’s Attention

Julian shared results that are hard to ignore:

one pipeline took 10 hours to build manually
the same pipeline took 1 hour with Maia’s help
teams reported 5–10x productivity gains
one customer said they planned to hire five engineers but hired one because repetitive work dropped so much

This is the part leaders care about: not the story, the measurable outcome.

A Lesson Most Teams Learn the Hard Way: Your Docs Matter

Julian described a support automation workflow where AI drafted replies based on documentation. It worked well—but it also exposed a problem.

When the documentation was unclear, the AI’s answer became unclear too. In one case, the AI said “yes” to a connector capability, and even humans couldn’t agree just by reading the docs. The truth was “no,” but the docs were written in a confusing way.

Julian’s takeaway is simple and sharp:
If humans can’t understand your docs, AI won’t either.

Final Thoughts

Julian Wiffen’s message is not about fear or hype. It’s about practical change.

AI helps data engineering when it:

reduces repeated work
turns messy input into measurable fields
improves speed without breaking trust
pushes teams to write clearer documentation
helps modernize old systems into cleaner workflows

And most importantly, it becomes valuable when it becomes part of daily work—not a demo.

💡 Inspired by Julian’s approach to practical AI? Want more stories from data, AI, and business leaders through The Executive Outlook?

Search This Blog

The Executive Outlook