Back to Insights & News

Reflecting on 35 Years in Data Management

Share this article:

The More Things Change, the More They Stay the Same, Same but Different

Early Days #dedupe #list

The year was 1989, and as a 19-year-old university dropout, with self-taught BASIC programming on a 32Kb home computer, I managed to talk my way into an account executive role at Southwark Computer Services (now Acxiom). On day one I didn’t actually know what a computer bureau was. But three months in following the departure of my ‘boss’,  I found myself with imposter syndrome at the wheel working on arguably at the time the world’s largest non-government data processing project—the National Consumer Database. Thankfully I was surrounded by some really smart people that more than made up for my lack of experience or knowledge. One lesson I learnt early was if you don’t know then watch, listen and learn.

The idea was to create a marketing list that was essentially a holistic view of each UK adult in terms of where they lived, their life stage, and socioeconomic standing, as there was demand from all sectors to target at scale through direct mail. We were bombarded with ammo boxes of magnetic tapes, data cartridges, and these newfangled ‘floppy disks’ containing a decade’s worth of electoral roll data, white pages listings, real property data, and share registers. All in different formats and varying degrees of quality. But all up close to a billion records. 

Behind the hermetically sealed doors of a warehouse near London Bridge, our $1 million IBM System 370 mainframe, the size of a small bus and with the equivalent computing power of a smart fridge, was being powered up by the men in white coats. As the lights of South London dimmed, months of coding was unleashed on the data and like a village green firework display it was a bit of a fizzle. It probably didn’t help that one of the white coats omitted to load one of the storage devices, and we discovered that Wales and the Southwest of England were missing from the output. What’s five days of reruns  between friends?

 What eventually came out was the first-ever truly national marketing database that could be used to truly target consumers based on their life stage and affluence. It became the foundation marketing list for pretty much every major advertiser in the UK including telcos, finance, political, charity, and automotive.

Early Career Lessons:

  1. Find the smartest person in the room, lose your ego and learn from them.
  2. Standardisation of Data Formats: Consistency in data formats simplifies integration. Find tools to make it easy.
  3. Data Hygiene: The importance of maintaining clean data cannot be overstated—it prevents costly errors. Go upstream and fix it at source.
  4. Deterministic Matching Is Key : If you’re going to resolve an identity from several sources to build a picture of a consumer, at least make sure it’s the same person.
  5. Economic Processing: As the saying goes, “Measure twice, cut once.”

New Beginnings Down Under #svc #datawarehouse

The turn of the millennium saw me relocating to Australia, where I joined the new kid on the block, Acxiom Australia, to spearhead the data sourcing and development of InfoBase. Marketing lists and dedupes became so last season, and instead, we learnt the new vocabulary of data warehousing, data integration, and the holy grail of a single view of the customer. Same, same but different. 

Drawing from the trusted recipe card of 1989, but mixing in several new elements like data co-ops and lifestyle surveys, we repeated the dedupe data integration process to create a marketing list data warehouse comprising the Australian adult population. But this time rather than a $1 million mainframe, we turned to a mid-range Unix server that looked suspiciously like a bar fridge but costing more like $50,000. As Mark Twain said, “It’s not the size of the dog in the fight, it’s the size of the fight in the dog,” and what used to take literally weeks could now be processed overnight at a fraction of the cost. 

Another reason for the speed gains was the use of Acxiom’s Abilitec Persistent ID, which enabled us to assign an ID to each individual data record and then use the Unix machines to sort, join, and merge massive data volumes at speed. Looking back, Acxiom’s Abilitec was 20 years ahead of its time. The market was sceptical of a cloud-based identifier then, but Abilitec laid the groundwork for what would evolve into what the industry terms Identity Graphs.

Advancements in Identity Resolution:

  • Improved Speed and Reduced Cost of Processing: Fact. You could now do things you just couldn’t dream of a decade before.
  • Persistent Identifiers: Hats off to Acxiom, Abilitec was a game-changer. It’s just that the market wasn’t ready for the game.
  • Improved Matching Algorithms: Since the 1970s, data matching and deduplication was 99% based around Names & Postal Addresses. For the first time with Abilitec, we introduced new portable identifiers such as change of address services, transferable landlines, emails, and mobile phone numbers. The result was more accurate and efficient data matching processes.

Going Global #IDR #IDV

Fast forward to 2024, I am currently leading WINR’s data products and managing identities on a global scale with over 4 billion profiles in 31 countries. Our operations are vastly different now; we utilise what is referred to as ‘serverless’ technology, and I couldn’t tell you what one of our AWS servers looks like, but I do know they’re cheap.

The last 18 months have seen an industry having an identity crisis. As we close our browser on third-party cookies, I feel the swing to MAIDs seems short termism, given both Apple and Google have them on the to be deprecated list. 

The industry’s over-reliance on cookies, MAIDs and IP addresses has taught us a critical lesson: these identifiers are not permanent. They are fleeting, sometimes with less longevity than a carton of milk. 

If we are to enhance identity resolution to support more effective onboarding, addressability, and measurement, we must embrace methods that may seem traditional but are undeniably effective. 

Consider this: how long have you had your mobile phone number? How long have your parents lived at their current address? These are the types of persistent identifiers that provide a stable foundation for building sophisticated identity graphs. 

The enduring nature of these traditional identifiers underscores their potential to support a more stable and reliable data infrastructure for identity resolution. As the digital landscape evolves, returning to these “old school” methods isn’t just a nod to nostalgia but a strategic move towards sustainability in digital identity practices. 

So, what have I learnt?

Who would have thought that an accidental omission of half of the UK would teach me the importance of quality assurance than any textbook could? Or that a server no bigger than a bar fridge could outperform a million-dollar mainframe, proving that sometimes the little guys do pack the mightiest punch (I’m only 5’10”).

Migrating from London’s drizzle to Sydney’s sun has been less about keeping up with technology and terminology and more about appreciating the constants. My BASIC programming days on a 32Kb computer plugged into my mum’s TV may have shown me the ropes of coding; but I soon realised I wasn’t very good at it helped me develop a ‘figure-it-out-no-matter-what’ mindset, just find smart people who can implement your ideas better than you can on your own.

Reflecting on my early days as a 19-year-old university dropout turned industry Account Exec, it’s clear that every tape-loaded, every QA report highlighted, every project completed, taught me not just how to manage data, but how to navigate an unpredictable industry with a bit of grace and a lot of humour.

As I now lead WINR’s global data initiatives, it’s these early lessons that keep me grounded. I’ve learned that while technology will continue to evolve, the true anchors—meticulousness, adaptability, and a good sense of humour—remain your best tools in the shed. And as for the future? Well, I look forward to more lessons, more laughs, and yes, even more unexpected detours. After all, isn’t that what keeps the journey interesting?