Chapter 7: The Power of Patterns
Grammar school had traditions, one of which was introduced on my first day. Pen and ink.
And the blotter.
Every student’s desk had an ink well.
You dipped your pen, wrote notes or calculations and then, without fail, you pressed the blotting paper to the page.
Write. Blot. Lift. Repeat.
The blotter safeguarded the smudge from your resting hand as it moved across the page to make a correction…if you were lucky. If you smudged, it meant a rewrite…no cut and paste unless you used scissors and sticky tape…but that looked bad! Sometimes you would get an ink bubble in the nib…or worse a hair…that popped or trailed ink across the page…oops.
But over time, something happened.
The blotter became a canvas for a mess of mirrored scribble or ruled lines...a bit like a famous painting in Canberra...The Jackson Pollack "Blue Poles".
What actually are patterns and why are they important?
A pattern is defined as a noun representing a set of instructions, a model or a design. Webster indicates it can be a 'repeated' design but not necessarily repeated...just repeatable.
DNA is a set of instructions and is important because of its ability self replicate.
Humans instinctively look for patterns to follow or copy...for all sorts of reasons. They are so fundamental to life that we cannot exist without them. They are addictive. They can be random. So with that segway...back to the blotter. It wasn’t art, but it wasn’t random either. It was something in between. Looking under a microscope at an ink-stained blotter, you see the fibres have randomly made avenues for the ink and the fibrous nature of the blotter shows a landscape with images of blue rivers on pink ground with windswept, leafless trees. Medical researchers now use ink stained slides to outline images of organisms. Years later, I realised that new patterns emerge when movement and combination happen over time. You don’t create them. You reveal them.
You can potentially see, hear, smell or feel a pattern. Early Lessons in Patterns - Data on Magnetic Tape The first time I merged two tapes, something magical happened. At first, it was just data — random rows of numbers, names, and transactions. But when I moved it and combined it, patterns began to appear. Names were sorted alphabetically. Payments were grouped into categories. Gaps became visible where data was missing. I didn’t create the pattern. I revealed it. This became one of the most enduring lessons of my career. If you want to see the pattern, move the data. Combine it. Watch what emerges. Patterns Are Revealed, Not Created This is one of the most important truths I know: Patterns aren’t created. They’re revealed. You don’t invent them. They’re already there, waiting to be seen. Think about it: When you shuffle a deck of cards, the pattern of suits and numbers isn’t “created.” It’s revealed by chance. When you analyse customer transactions, you don’t create buying behaviour. You see it. When an AI model “finds a pattern” in millions of transactions, it’s not creating anything. It’s revealing what was already present. This is the secret of all pattern recognition — from AI to human insight. You don’t create insight. You uncover it. This is why movement matters. If data stays still, you see nothing. But the moment you move it — merge it, combine it, sort it — the pattern becomes visible. The same is true for life itself. When life stays still, no patterns emerge. But when you move, travel, combine ideas, and interact with others, the patterns of life become clear. Movement reveals. Stagnation hides. Patterns in Design and Debugging In software design, patterns are everything. If you don’t see the pattern, you’ll never fix the bug. I spent hours debugging core dumps on systems like Honeywell and IBM. You’d get a hexadecimal dump — pages and pages of raw, meaningless hex numbers. But you learn to see the patterns. If you’ve done it enough, you know where to look. You know that “4C” means something specific (packed 4 decimal positive value) You know that a certain combination of numbers in a register means a pointer is broken. At first, it’s noise. But the more you look, the more you see the pattern. It's natural. Your brain remembers patterns and familiar patterns are faces, voices, smells... Patterns are easier to see if you’ve seen them before. Patterns in Instinct (Pre-Conscious Logic) When I spoke about instinct as pre-conscious logic. It arrives unscheduled, possibly during a conversation. A hesitation, wait, time stops for a millisecond often followed by an image, logical shapes with missing connectors. Breathing pauses...just for a second, at the recognition. And then you try to vocalise the context. This is pattern recognition in its purest form. My subconscious has seen the pattern before and it reacts faster than my conscious mind can. This is why instinct often feels like a sudden realisation. It’s not magic. It’s pattern recognition at a deeper level. The human brain is a pattern-detection machine. It’s why you see faces in clouds. It’s why you “just know” when something is off. But here’s the insight: If you want to improve your instinct, you have to move data. Expose your mind to more combinations, more ideas, more randomness. The more experience you have, the more patterns you’ll recognise. This is why the phrase “trust your gut” has a hidden truth. Your gut isn’t guessing. It’s seeing patterns your conscious mind hasn’t processed yet. Dark Data in Patterns There is another aspect of pattern recognition that people see every day but don’t realise. Data tells you a great deal…maybe it turns on virtual lights. That’s great, it tells you that something has happened. But what do the lights that haven’t been turned on tell you? There is another pattern there…a separate one…shaped by data that isn’t there! This pattern may tell you just as much as the ‘lights on’ pattern or sometimes more. It tells you what hasn’t happened. Another way of looking at dark data is when someone shows you a black and white drawing…what do you see? Firstly you see the shapes within the drawn lines…usually that’s distinct and has form. But take another look. This time pretend that the drawn lines are not lines but cuts in the paper. What shapes do you now see? You see the counter image…the remaining canvas. It is always there but it’s not immediately brought to the front of your consciousness…your brain is trained from birth to look at the drawn image. The blotting paper I mentioned before really points this out clearly. Black ink in the paper soaks in and leaves a very slight shadow which enhances the remaining shapes. Patterns in Credit Card Transactions (Combinations Happen at Once) There are moments when patterns are in volume and can crush you!. Switching over a bank’s credit card processing to a new platform is a monstrous exercise. Customers are always shopping, paying bills, booking hotels and want to use their credit card NOW! The old system has to be switched from authorising at the card issuer to authorising at the card scheme while the old system is turned off and the new system is turned on. During that time card transactions are held pending in a queue for issuer account update. There could be a hundred thousand or more transactions released to the issuer system when the floodgates open. The moment we “flip the switch,” the system has to process every possible transaction combination at the same time. Every permutation of payment types, card limits, merchant types, and system rules goes live simultaneously. Every pattern hits at once. This is why system switchover is one of the most dangerous moments in banking.You have to design for patterns you know and catch the ones you haven’t envisaged. The lesson I learned was this: Look for dark data. Design for the pattern you haven’t seen, because on the day of switchover, you’ll see them all. Closing Reflection (Revealing, Not Creating) Here’s the message I’d leave for anyone building systems, designing software, or trying to understand life itself: If you want to see the pattern, you have to move the data. It’s true for everything. If you want to see customer behaviour, resolve the data. If you want to see life’s patterns, move. Change your environment. Walk away and look back. If you want to see the flaws in a system, shake it. Stress it. Watch what breaks. If I could leave one lesson behind, it would be this: “Don’t try to create patterns. Move things. Combine them. The patterns are already there, waiting to be seen.”
Just like quantum theory of "superposition". We'll come to that later.
What actually are patterns and why are they important?
A pattern is defined as a noun representing a set of instructions, a model or a design. Webster indicates it can be a 'repeated' design but not necessarily repeated...just repeatable.
DNA is a set of instructions and is important because of its ability self replicate.
Humans instinctively look for patterns to follow or copy...for all sorts of reasons. They are so fundamental to life that we cannot exist without them. They are addictive. They can be random. So with that segway...back to the blotter. It wasn’t art, but it wasn’t random either. It was something in between. Looking under a microscope at an ink-stained blotter, you see the fibres have randomly made avenues for the ink and the fibrous nature of the blotter shows a landscape with images of blue rivers on pink ground with windswept, leafless trees. Medical researchers now use ink stained slides to outline images of organisms. Years later, I realised that new patterns emerge when movement and combination happen over time. You don’t create them. You reveal them.
You can potentially see, hear, smell or feel a pattern. Early Lessons in Patterns - Data on Magnetic Tape The first time I merged two tapes, something magical happened. At first, it was just data — random rows of numbers, names, and transactions. But when I moved it and combined it, patterns began to appear. Names were sorted alphabetically. Payments were grouped into categories. Gaps became visible where data was missing. I didn’t create the pattern. I revealed it. This became one of the most enduring lessons of my career. If you want to see the pattern, move the data. Combine it. Watch what emerges. Patterns Are Revealed, Not Created This is one of the most important truths I know: Patterns aren’t created. They’re revealed. You don’t invent them. They’re already there, waiting to be seen. Think about it: When you shuffle a deck of cards, the pattern of suits and numbers isn’t “created.” It’s revealed by chance. When you analyse customer transactions, you don’t create buying behaviour. You see it. When an AI model “finds a pattern” in millions of transactions, it’s not creating anything. It’s revealing what was already present. This is the secret of all pattern recognition — from AI to human insight. You don’t create insight. You uncover it. This is why movement matters. If data stays still, you see nothing. But the moment you move it — merge it, combine it, sort it — the pattern becomes visible. The same is true for life itself. When life stays still, no patterns emerge. But when you move, travel, combine ideas, and interact with others, the patterns of life become clear. Movement reveals. Stagnation hides. Patterns in Design and Debugging In software design, patterns are everything. If you don’t see the pattern, you’ll never fix the bug. I spent hours debugging core dumps on systems like Honeywell and IBM. You’d get a hexadecimal dump — pages and pages of raw, meaningless hex numbers. But you learn to see the patterns. If you’ve done it enough, you know where to look. You know that “4C” means something specific (packed 4 decimal positive value) You know that a certain combination of numbers in a register means a pointer is broken. At first, it’s noise. But the more you look, the more you see the pattern. It's natural. Your brain remembers patterns and familiar patterns are faces, voices, smells... Patterns are easier to see if you’ve seen them before. Patterns in Instinct (Pre-Conscious Logic) When I spoke about instinct as pre-conscious logic. It arrives unscheduled, possibly during a conversation. A hesitation, wait, time stops for a millisecond often followed by an image, logical shapes with missing connectors. Breathing pauses...just for a second, at the recognition. And then you try to vocalise the context. This is pattern recognition in its purest form. My subconscious has seen the pattern before and it reacts faster than my conscious mind can. This is why instinct often feels like a sudden realisation. It’s not magic. It’s pattern recognition at a deeper level. The human brain is a pattern-detection machine. It’s why you see faces in clouds. It’s why you “just know” when something is off. But here’s the insight: If you want to improve your instinct, you have to move data. Expose your mind to more combinations, more ideas, more randomness. The more experience you have, the more patterns you’ll recognise. This is why the phrase “trust your gut” has a hidden truth. Your gut isn’t guessing. It’s seeing patterns your conscious mind hasn’t processed yet. Dark Data in Patterns There is another aspect of pattern recognition that people see every day but don’t realise. Data tells you a great deal…maybe it turns on virtual lights. That’s great, it tells you that something has happened. But what do the lights that haven’t been turned on tell you? There is another pattern there…a separate one…shaped by data that isn’t there! This pattern may tell you just as much as the ‘lights on’ pattern or sometimes more. It tells you what hasn’t happened. Another way of looking at dark data is when someone shows you a black and white drawing…what do you see? Firstly you see the shapes within the drawn lines…usually that’s distinct and has form. But take another look. This time pretend that the drawn lines are not lines but cuts in the paper. What shapes do you now see? You see the counter image…the remaining canvas. It is always there but it’s not immediately brought to the front of your consciousness…your brain is trained from birth to look at the drawn image. The blotting paper I mentioned before really points this out clearly. Black ink in the paper soaks in and leaves a very slight shadow which enhances the remaining shapes. Patterns in Credit Card Transactions (Combinations Happen at Once) There are moments when patterns are in volume and can crush you!. Switching over a bank’s credit card processing to a new platform is a monstrous exercise. Customers are always shopping, paying bills, booking hotels and want to use their credit card NOW! The old system has to be switched from authorising at the card issuer to authorising at the card scheme while the old system is turned off and the new system is turned on. During that time card transactions are held pending in a queue for issuer account update. There could be a hundred thousand or more transactions released to the issuer system when the floodgates open. The moment we “flip the switch,” the system has to process every possible transaction combination at the same time. Every permutation of payment types, card limits, merchant types, and system rules goes live simultaneously. Every pattern hits at once. This is why system switchover is one of the most dangerous moments in banking.You have to design for patterns you know and catch the ones you haven’t envisaged. The lesson I learned was this: Look for dark data. Design for the pattern you haven’t seen, because on the day of switchover, you’ll see them all. Closing Reflection (Revealing, Not Creating) Here’s the message I’d leave for anyone building systems, designing software, or trying to understand life itself: If you want to see the pattern, you have to move the data. It’s true for everything. If you want to see customer behaviour, resolve the data. If you want to see life’s patterns, move. Change your environment. Walk away and look back. If you want to see the flaws in a system, shake it. Stress it. Watch what breaks. If I could leave one lesson behind, it would be this: “Don’t try to create patterns. Move things. Combine them. The patterns are already there, waiting to be seen.”
Just like quantum theory of "superposition". We'll come to that later.
Does Sport have patterns? Every sport does!
Competetive strategies seek out ways to win against the opposition by examining patterns of movement...and then crafting a stronger pattern. You'll be lucky to use that same winning pattern again...it may no longer be effective, thereby confirming the temporal nature of logic in sport.
Strategies in all forms of life are by their nature "for the time". Do you have an opinion on that? Let's hear it.
Competetive strategies seek out ways to win against the opposition by examining patterns of movement...and then crafting a stronger pattern. You'll be lucky to use that same winning pattern again...it may no longer be effective, thereby confirming the temporal nature of logic in sport.
Strategies in all forms of life are by their nature "for the time". Do you have an opinion on that? Let's hear it.
Customer segmentation is a favourite pattern of sales and marketing. They can see which groups of customers are buying which groups of products...in which region and at what time of year.
These patterns are supported by underlying data so that a click on the box will reveal who the customers are and what the products are in that group.
"Why does this customer only buy these products in one region but not another?" There could be many reasons including 'leakage' to 3rd parties, but this pattern shows them where to look.
These patterns are supported by underlying data so that a click on the box will reveal who the customers are and what the products are in that group.
"Why does this customer only buy these products in one region but not another?" There could be many reasons including 'leakage' to 3rd parties, but this pattern shows them where to look.