John Giles Interview pt.1

You could be forgiven for thinking life has just become a little complicated if you ever discover you have an elephant in the room… but what happens when you find one in your fridge?

A flippant but fun analogy maybe… but definitely a required one when it comes to trying to get a handle on the thoughts of John Giles, one of the most-respected authors on the subject of data modelling, who wrote the ‘Elephant in the Fridge’ in 2019 as a follow-up to ‘The Nimble Elephant.’

John has beaten a very unorthodox path to his current role as a consultant, who has helped a number of organisations Down Under get to grips with using Dan Linstedt’s Data Vault methodology to produce better data analytics and business intelligence solutions in an ever-changing commercial, and increasingly digital, age.

I caught up with John on a Zoom call to find out what drove him to write books about data modelling. John often joins the Meetups set up by the Data Vault User Group, which Datavault sponsors, when he should still be in his pyjamas… but I was able to call at a much more civilised time, after he had at least managed breakfast.

John is very much a disciple of Dan Linstedt’s Data Vault methodology – although it took a chance comment by a colleague back in 2013 to discover a method that maximised returns on his own views about data modelling.

He explained: “One day around 2013 or 2014, a colleague asked me: ‘What do you think about Data Vault John?’ And I had to be honest and say I had not heard about it.

“I bought Dan and Hans Hultgren’s books on the subject and received training from both of them. I quickly realised that with data warehousing – which had never really interested me before – instead of being end-of-month loads, I suddenly thought ‘Dan is onto something.’

“Within two years of discovering Dan’s thoughts about Data Vault, I was presenting at his conference in America. I jumped in at the deep end and didn’t muck around.”

He added: “I totally endorse Dan’s vision – he talks of an enterprise ontology and taxonomies being the basis for Data Vault, which is all about integration with history, even though that is an oversimplification.

“I loved what Dan was saying. It made heaps of sense and I started doing some real Data Vault projects.

“But I quickly saw that there were practitioners who weren’t following Dan’s advice from his books and teaching. I saw failure after failure when people just pushed data in – they were source-driven.

“Dan soundly condemns source-driven Data Vault, but some practitioners keep doing it.

“It was out of sheer frustration at observations that I took nine months off from paid-employment to write my book to offer people a fun – but sound – way of basing a Data Vault on the business’ view of their data.

That was John’s second book. But how did his first book come about seven years earlier?

He revealed: “I was working on a project when one Friday, the project manager came to me and said ‘Sorry John, we have run out of budget. We are shutting the project down with immediate effect.’

“He told me they did want me back – but in six months’ time, in the new financial year.

“I went home said to my wife. ‘I am passionate about agile, and I am passionate about patterns.’

“I was not going to sit at home for the next six months. I could have got another contract, but I decided to knuckle down and write this book, which had been on my mind.

“And you can guess what happened. I tied up with a publisher and a signed a contract to provide a manuscript in six months’ – only for the client to call and say ‘We found the money, can you come back on Monday?’

“Over the next horrendous six months, I worked three or four days a week, and was working the other days, evenings and weekends on the book.

“With ‘The Nimble Elephant,’ quality at speed is the theme – the speed that suits agile projects.

“The quality demanded by head office solution is the in the patterns identified by data modellers such as David Hay and Len Silverston.

“I have also published some lightweight patterns – so instead of reading some 2,000 pages from the published works of David and Len, let alone other authors, you can read a quick introduction in my second book, ‘The Elephant in the Fridge’.”

“So, neither of the books were part of a strategic desire to be an author. I am not a passionate author, but I am passionate about the topics I have written about.”

John added: “With ‘The Elephant in the Fridge,’ the topic is about having the data town plan to drive your Data Vault.

“The catalyst was I saw these data pushers pushing stuff into root source system data, and I thought there was a ‘better way.’

“Dan Linstedt knows the better way and endorses the better way. I wrote it to help people say ‘you don’t have to wait two years to get a perfect town plan.’ You can get a sufficient town plan in weeks.

But who do you get to build the data town plan that should underpin the Data Vault? “For starters, you get business people, often represented by business analysts,” John said.

“Unfortunately, I have encountered a number of business analysts who understand the business and its processes – but they struggle to understand data.

“The flip side is you have brilliant coders who can hack some Java code at lightning speed, but they have a personality-type that makes them uncomfortable engaging with senior managers – or they are just too busy, and don’t want to.

“But either way there is the possibility that you might get business people who don’t understand data, and developers who understand data, but don’t – or won’t – understand the business.

“That’s a challenge and I think remains a challenge because it relates to personality types and that’s where hopefully my publications will help reach those two groups.”

“I wrote my books to try and complement and compliment Dan and Hans’ books, nothing else.”

And to prove that point, John recounts a tale of working with a really good agile coach and mentor of his.

He said: “A project he was responsible for measured velocity – the number of points delivered per sprint.

“There was a downward trend as they were getting slower and slower, so he put the brakes on and asked, ‘What on earth is happening’?”

“The problem was in the first phase, physical tables were created with some coding that sat on top, leaving everyone happy.

“But with the second sprint, more tables were added with some tweaks to the original tables, which needed changes to the code.

“The pattern was set, and the same thing happened with a third sprint – new tables, more adjustments to the originals and resulting code changes, leading to less speed. They just got slower and slower.”

John added: “So the project manager paused the work and said he wanted to get a data person in who would look a few sprints ahead.

“Agile practitioners quite rightly don’t look to the two-year end-goal, perfect architecture, because we know there is no such thing. “But I would say, ‘Let’s have the town plan view to this sketch – this is roughly where we are going.’

“They brought in a data professional. They lost one sprint but then the delivery points went up the 13-fold. The data specialist stayed on the team and was constantly looking ahead.

“That is the theme of my first book – quality at speed. You want to be fast enough for the agile developers, and also have sufficient quality to keep the data head office happy.”

In part two, we talk to John Giles about how he first got to work in the world of computers and IT, let alone discover his passion for data modelling