Sunday, October 16, 2016

You don't always need a justification for a test ...

I was just listening to a Neil Degrasse Tyson lecture whilst out in my garden stargazing, and he had this amazing story I knew needed to be added here.

As you can imagine - there's huge competition for time on the space-based Hubble telescope.  Not just anyone can use time on it - and typically you need to put forward a business case for it, something like,

  • We want to observe the following nebula, to look for any stars which might be in the process of forming
  • We want to look at this star over the following days to look for exoplanets
  • We want to get a better view of the following supernova remnant

Obviously the director in charge has to queue up these requests as best they can, but they're also allowed a small about of time, that they can allocate for any whimsical reason they want.

So the director tried something radical - they'd pick an area of the sky and stare at it for 10 days.  And for something even more daring, they're try to point away from the spiral of the Milky Way (our galaxy), away from any star clusters, or galactic clusters.

They'd try to find the most boring part of the sky, and just see what's there.

The area they looked at represents the amount of sky you can see through a narrow eyehole in a needle ... at arms length.

And what did they find?  Galaxies ... thousands of them ...

This went on to become known as the Hubble deep field, and for some is considered one of the most important discoveries in the lifetime of the telescope.  It confirmed and made real what we'd suspected about the sheer size of the cosmos in which we live in.  Every one of those points of light is a galaxy, comparable to our own of hundreds of billions of stars.  So many thousands of galaxies, all seen through the eye of a far away needle.

And this discovery wasn't one that you could have been able to justify the business case for.  It was whimsical, but NASA allowed it, because they know sometimes you find amazing things, just giving some time for curiosity.

It reminds me of something James Bach told me earlier in the year, you don't always need a justification for a test.  Indeed sometimes if you're about to try something and someone says "what problems are you expecting", it's easy to talk yourself out of it.  But it's important to give ourselves a little bit of time to try things out.

As long as it's not an unmanageable portion of your day, there's always time for a "I just wonder if..." experiment.

Now Playing: "The Universal", Blur

Saturday, October 8, 2016

Practice what you preach!

I think sometimes life has a weird symmetry - this week had an excellent example of this.

My son is in his first year at Victoria University in Wellington - he's done well, but not sure if he wants to do a second year without a considerable rethink.

He recently had an essay for Classics returned to him with some comments, for which he was allowed to make changes and resubmit.  He got a C-, but as he pointed out "it's still a pass", and wasn't so keen to redo it.

I had a chat with him - it's so rare to get useful feedback on something we've done.  Of course we all really want to be "done with" projects like essays or documents so we can just move on with our lives.  But applying those comments would help him think about what he's written, how to present a better argument, how to do it better.

Without feedback, we keep doing what we're currently doing, but it's unlikely to get any better.  We read through the comments together, he formed a plan of what to do, and resubmitted it yesterday.

During the week, one of our lead developers emailed me about my blog series looking at Java basics.  There were a few things I'd done which whilst not big things, would be considered bad practice in the team.  Would I be interested in the feedback?

My stomach turned - it might mean a lot of rewrites, and I'm working on my astronomy series right now.  And yet, although that series is meant only as an introduction to some key concepts, I wanted it to be as good as I can get it, without compromising the "sense of fun" I like to include in most of my writing.  [Although of course, this wasn't much "fun"]

So of course I wanted to know.  His comments were useful, because it made be think about some aspects I'd not thought about in detail,

  • Class names should be Capitalised, this makes it easier to tell them apart when declaring.
  • All code blocks should use braces "{...}" to contain statements, even if just one line.  I'd not used them in a few  DiceClass methods just to show that they're not mandatory (if you have a single line).  I changed all bar on, just to prove this, added a comment to the relevant blog, then never missed the braces afterwards
  • Some methods such as diceOnOrOver returned an integer, but should return a boolean.  I'd cheated there, because I knew I'd wanted to add the integer returned at a later date, but he was right, it should return true or false.  I know it's tempting to go "but I just need ...", but it can be a bit of a slippery slope.
  • I was using ArrayList, but I should declare them as a List not ArrayList, because ArrayList is a kind of List, and you want to keep the List interface.  StackOverflow discusses this here.

All told, with changes, retesting, committing to GitHub and blog modifications it took a few hours, but it did reinforce those points.  I don't think I'll ever forget to use a capital for a class name ever again now.

It's also a reminder to hold ourselves to the advice that we council others - and yes it's so rare to get good feedback, it's always useful to apply it so we learn from it and make the best essay/document/code we can.

Now playing: "River deep, mountain high", Ike & Tina Turner

Friday, September 30, 2016

Disposable automation ...

I had an interesting opportunity to play around with something useful today ...

We're currently doing a couple of sprints to change some look and feel to enhance our web page for mobile.  It's been a weird mix of interesting, and boring so far, and I want to share an approach I've taken.

Because our goal for our sprint is to tidy up the mobile experience, which involves a huge amount of changes, I'm being for once somewhat overwhelmed by builds being dropped into my test environment.

I came up with a checklist of things to cover which includes,

  • confirm all key fields are present
  • confirm error text when mandatory fields missing
  • confirm error text when junk input given for fields
  • confirm error text when duplicate entries (re-enter your password etc) don't match
  • do all this on multiple browsers
  • do all this for mobile and desktop settings

At the end of day one, we'd had 3 builds delivered.  I'd been really dutiful and methodical for the first build.  But by human nature my concentration was going on the huge checklist.

My concern was by end of sprint I might be getting a bit blase about it all.  Ironically it's the end of sprint the find details of the checklist matter the most.

The problem is our automation checks we can perform business flows, it doesn't check the fine details of what's on the page.  But that was kind of what I needed.

So I wrote my own using WebDriver, but here's the thing, it'd replicate all the above checks, but I acknowledged this was only to help us through the next few sprints whilst we put it through some UI changes.  Afterwards it would be just deleted and never used again, because it didn't make sense as ongoing regression.

It'd do all the checklist "ticks" for me, but leave the browser in a state for me to run my eye over for anything out of place, and allow me to do a little bit of playing.

Cool features it had would be,

  • Using multiple drivers for different browsers, so it'd check across browsers for me (mobile-proper would still be manual, but hey, it's freeing my time)
  • I would use the Selenium setSize command to mimic either mobile or desktop settings in my tests

I time-boxed myself to no more than 2 hours on this - I delivered what I wanted in 1.5 hours.  As usual I had a lot of ideas for enhancements, but reminded myself that would take me away from testing (which I was trying to save time on in the first place), and ultimately these pages being checked for content in this depth were not the kind of critical tests we'd need within our suite.

As expected, we had another 4 releases today, and it allowed me to keep better pace with them, whilst focusing my time on key changes for each task as delivered.

It was a good example of how you can knowingly break a lot of key rules of automation (I for instance use methods, but not the kind I break down to use on more detailed functional testing).  And yet keep to the fundamental one - let automation deal with anything which looks like an item to check, and leave more freedom to explore to the manual tester.

Now Playing: "History", The Verve

Thursday, September 22, 2016

Some personal thoughts on agile and letting go

I've been thinking a lot in this vein the last week, and really wanted to commit these thoughts down electronically.

I've spoken in the past about quite a traumatic event in my early 20s where I saw someone killed.  As you can imagine, it wasn't very nice, and there was a whole personal journey dealing with the post-traumatic stress of that.

The last few years however, I was very aware of something that lingered from that experience.  What is so troubling about an incident like that where you have flashbacks, is the sense of only witnessing, and being powerless.  It's had a knock on effect on my whole life where I always try and do something over nothing, because then I felt a sign of ownership in whatever outcome occured.

This is me on waterfall

In waterfall, that means I was a really proactive, guard-dog of testing and all aspects of quality (this really isn't the blog to have the argument about quality though guys).  Playing the superhero role whenever I could, I was always trying to be that guy who makes things happen, that guy who fixes it all.

In our agile transition at work, I've had some really great mentors.  First of all, don't get ideas that out of waterfall I don't care.  But I've learned to focus much more on those items which are expected of me, and doing the best I can about them, and learning to escalate up any problem which falls outside of that.  And to do that in such a way that it doesn't feel like I'm "telling tales" on other people - to escalate in a supportive way, but acknowledging I'm not always the right person to solve all the project's problems.

Sometimes to fix something properly, good intentions aren't enough, and you need the appropriate expertise, and I've learned to set aside my male pride and accept it's not always me.  [You should see my attempts at plumbing before I realised this]

But most of all, I've learned with fortnightly sprints, I don't have to be the attack terrier trying to keep everything on track.  We can and should be daring and try new things, even challenge "sacred" taboos every so often - the worst that could happen is we mess up a sprint, all of 2 weeks lost.  But just maybe we might also find a better way to do something which will help sprint-after-sprint.  Or learn a good reason for why we always should do X that no training course could have got across.

Far from not caring, it's allowed me to focus my energy so much more on areas I'm relied on much more, and actually enjoy what I'm doing.  But it's allowed me to accept that letting things play out isn't always a fraught case of life-and-death.  And to me, that's something more important than you can imagine.

Now Playing: "Sulphur", The House Of Love

TIME 3 - Time under duress

You're still with me?  Great.  Buckle up, because time is only going to get wierder.

Thank your uncle Albert for that...

Albert Einstein is of course famous for a couple of theories of Relativity which describe interactions between energy and matter, and particularly discuss how time changes with speed.  It also imposed a speed limit on the Universe.

And I mean an absolute speed limit.  Not like where we have a 55mph road, and you do 56mph and feel all ...

You can't go faster than the speed of light.  You don't get to play Spinal Tap, and do the speed of light ... plus 1.

And as my reader Ben pointed out, there's certainly no going .5 past the speed of light.  Also a parsec is a unit of measurement not time.  Your dad's been lying this whole time kid.

Let's try a thought experiment

Okay - we have the scenario below,

  • Star Destroyer - slow enough to be considered stationary
  • Millennium Falcon - moving away from the Star Destroyer at half the speed of light.  It is also firing it's forward laser cannons, the energy for which moves at the speed of light.

Let's look at it from the point of view of a couple of observers.

Darth Vader is on the Star Destroyer, and he can see the Millennium Falcon moving away at half the speed of light.  He can also see the laser beams which also are moving away from him at the speed of light.

The lasers go no faster for leaving an object which is already at half the speed of light.  You would think this would be a good case for .5 past the speed of light.  But no, the speed limit still applies for any observer.

Chewbacca is flying the Millennium Falcon.  I guess you can say he's flying it Hans-free.  Surely he sees the laser beams travelling slower?

No - he sees the laser also travelling away from him at the speed of light.  How can it be that two people at vastly different speeds are seeing those lasers travel at the same speed relative to their speed of motion?

The answer is they're observing time very much more differently.  For Chewbacca, time is going much slower than for Darth Vader.  That different is enough for the light to look like it's moving at the speed of light from his slowed-time framework.  This is called time dilation.

It's just a fancy theory isn't it?

All this of course sounds like a fanciful theory, with a bit of mathematical jiggery-pokery to make it work on paper.  Except science doesn't work like that (you're thinking of religion).

All my critical thinking senses are dubious.  As I've said to my office quite regularly, I'm a tester, I don't tend to overthink things when I can come up with an experiment to prove or disprove something.

Once Einstein had written his theory, it was up to physicists to look for proof ... and they found it!


We obviously don't have spacecraft capable of travelling quite so fast as the Millennium Falcon - but we have the next best thing ... the mighty muon.

Muons are an exotic subatomic particle.  It is pretty much a heavy electron - it has the same charge, half the spin, but 300 times the weight.

They're also highly unstable, with a mean life of 2.2 microseconds - that's 0.0000022 seconds - observed in the lab.  It decomposes into an electron and a couple of neutrinos.

They're found in nature - when cosmic rays (essentially high energy protons or helium nuclei) from outer space collide with the atmosphere they can create muons.  These muons are moving incredibly fast - 95 to 99% the speed of light.  But even at those speeds, the best they could travel in our atmosphere is about 600m.

So how are these particles, produced miles above us, able to be detected on the ground?  When we do the analysis, and apply relativity, this isn't surprising.  Moving at that speed, the particle has experienced much less time - and as we talked in part 1.  That means less change has occurred - so these particles reach the ground having yet to decay.

This brings us right back to part one.  We can't measure time, but we can measure change, which can give us an indication of time.

You can read more about other observations of time dilation here or can see the maths being explained a bit here.  It's not just a theory, we've managed (by thinking about it, and applying a little cunning) to do some neat experiments to observe it.

So what was the point of the "Time"series?

When I saw Jasper Fforde earlier this year, he talked about writing as a mental challenge.  To take an idea and work out how far you can explore and push things.  Start with something, an idea or definition, and keep pushing.

After seeing definitions like this ...

I wondered if I could take something that I knew, try to define it, explore the definition, try new things as I learned.

This thirst to think and play with ideas - I'm not sure if it's exclusively a critical thinking thing.  It's definitely a science thing.  But it's something that drives me a lot - why so much of this blog is about playing with ideas to understand them better.

This blog series was actually originally written several months ago - and part of the reason I've thought very hard about trying covering more science in my spin-off I've talked about here

Now Playing - "To Love Somebody", the amazing Nina Simone

Wednesday, September 21, 2016

TIME 2 - Time under test

Okay - so previously we took a little look at time and defining it.  I put forward that this was the best I could come up with ...

Mike Talks' Definition Of Time

Time - a phenomenon which allows change to occur.

That is to say, without time, a system will remain in stasis.

Today I want to explore some meaning to that definition.

Measuring time?

Well as I mentioned previously, we can't measure time, but we can measure change.  It seems fair to say that the more change you're observing, the more time has elapsed.

If I went to the bottom of my road and counted cars passing me as my "change" to monitor - is seems fair to say I'd expect it to take longer to count 1000 cars passing me than to count 10.  But to know for sure I'd need to be able to check it against something I could be certain changes consistently and regularly.

Some of you might say "well just check with a stopwatch" - to which I'd remind you from last time, all the stopwatch is doing is counting oscillations in a quartz crystal.  So how can you be sure that's correct?

Synchronizing watches

These days with smartphones which autocorrect the time regularly, we're losing touch with synchronizing our clocks.  Even the best watch in the world can be inaccurate and drift a little - typically the more accurate, the more expensive.

But even this requires us to synchronise our device with a reference clock which we believe to be accurate,
  • When I was young we used to have a series of radio pips on the radio to tell us when the hour was on.
  • We could ring up the speaking clock on the phone
  • Towns in England would use the chimes of a town/church clock or a time ball to make aware the turning of the hour

The faulty clocks conundrum

Consider this as a challenge - you're given two clocks and told that whilst one is accurate, the other isn't correctly keeping time.  How do you work out which of the clocks is faulty?

There isn't actually any way I can see to solve this puzzle - if you leave both clocks running, one will be fast, the other slow.  There's no way to tell which is the accurate and which is the faulty one without somehow having a third reference time which you know to be accurate.

It's possible that one is running so incredibly fast/slow that you can tell by watching that "that's not ticking correctly", but you're not really using the second clock.

By the way - I typically hate these kinds of puzzles. If someone knew one clock was accurate, why ever the hell didn't they label it as such?

How do you actually test a clock?

Actually this has made me want to look a lot into "'how did people test the accuracy of clocks"?

This took me to the story of 18th Century inventor John Harrison, who tried to create an accurate naval clock or chronometer for measuring longitude.  Typically a large clock of his period is designed to be accurate, but completely stationary.  Smaller pocket watches existed, but it wasn't unusual for them to lose as much as fifteen minutes in a single day.

The challenge in a chronometer was to create a device which could move on the rolling, salt heavy, humid conditions of the sea, but retain accuracy of a large stationary clock.

If on a long sea voyage you knew London time, and you measured the time of midday, when the sun's shadow is smallest and points due north (in the Northern Hemisphere anyway), then for every hours difference from midday London, you were 15 degrees of longitude from London.  Read here for more detail.

Through experimentation, he turned a lot of ideas on clocks on their heads.  Many believed clock accuracy was down to using slow moving, heavy pendulums, where John Harrison found smaller, faster moving parts worked better - indeed a longitude watch could keep better time than a clock, as was proved by his experiments.  [His chronometer was noticeably different from normal clocks/watches from it's rapid tick-tick-tick]

John Harrison typically spent over 31 years building and testing a series of devices.  One such test involved sending the H4 watch overseas to Jamaica with his son, where it was found to only lose a few seconds over the 6 week voyage.

What's somewhat frustrating looking through all the information there is on John Harrison - there's a lot documented on every revolutionary feature he incorporated into his device, and how they were made.  But only a few clues as to how they were tested - this was annoying because (a) I'm a tester and (b) the devices can only be as good as the tests used to measure them.

I had a go at trying to reverse engineer how this might have been tested, and came up with the following possibilities,

  • Checking noontime
  • Checking vs accurate land clock
  • Checking vs night sky

Checking noontime

One obvious method for checking a chronometer would be to synchronise the device to twelve noon using a sundial to measure when a shadow is smallest / Due North.

You wait for noon the next day, and see if it's showing 12 o'clock.  The problem with this is it's quite a shallow test - and noon isn't something you can measure "to the second".

You can probably increase the confidence in this by running it daily over months, to see how much it shifts.  This is probably why Harrison spent 31 years testing!

Fundamentally with this, you're using one source of change (the hands on your watch under test), and testing it against another source (the time it takes the Earth to rotate), looking for correlation.

Checking vs accurate clock

This helps to give you a gauge, but as said above, how can you tell which of the clocks were "most accurate".  That said, such clocks run night and day, and someone would notice if the clock was striking noon when the sun was just rising in the morning.

Like the method above, this is another form of correlation testing.

Finding longitude of a known location

There are some records that show John Harrison's son used the H4 watch in Jamaica to predict the longitudinal position, and found after the sea voyage it was correct within a few seconds.  However of course - if the 18th Century lacked any means to accurately get the position of Jamaica (the whole point of the chronometer), how do you know the H4 wasn't measuring it more accurately than had previously been possible?

I suspected that Jamaica's latitude and longitude might have been calculate using the night sky - but again, there's a frustrating lack of information just on the internet about this. 

But which was it?

Don't you hate not knowing?  Thankfully James Bach recommended a book Longitude by Dava Sobel which solved this in.

It seems they'd found an accurate way on land to test time by observing the motion of the moons of Jupiter.  This was originally put forward by Galileo, but the astronomer Cassini perfected this with a series of tables (little did they know, there was an error in the system that Cassini corrected for, which turned out to be due to the speed of light difference due to different relative ranges between the Earth and Jupiter).

But this method of taking time could only be done by a trained mathematician/astronomer on solid land.  Hence knowing Jamaica's position relatively accurately.  Obviously only a few people had the skills to do this calculation, and it was completely unsuitable for use on the rolling sea.

Read more about this method here.


I noticed that in 2015, a replica of the H4 was "tested"and found to keep time accurately.  You can read about it here.

Please notice my use of quotation marks there.  The replica was "tested" at the Greenwich observatory in a stationary location.  Part of the requirements of the chronometer for Harrison was that it to keep accurate time, whilst at sea.  So this isn't really an adequate test in my opinion.  [I'd have wanted to add in some rocking motion to the test]

Do you disagree?  Then comment below ...

Now playing:  "Make Me Smile", Steve Harley & Cockney Rebel

Tuesday, September 20, 2016

TIME 1 - Time under definition

Welcome!  This post is going to be a little philosophical, and challenge you a bit in reading.  The whole intent is to get you to just think a little bigger, and about something you know all about, but maybe it's too under your nose that you've never really thought about.

Half of critical thinking is thinking about something for which we ordinarily have an automatic response for ...

On Twitter I've been having discussions with a lot of other testers about how do you define things.  In many ways how we define things essentially shapes how we think about them.  An example of a bad kind of definition is one where you use the word you're trying to define within the defintion, an example came from an ISTQB slide "security testing: testing the security of a system".

GOOD NEWS - I'm not going to ask you to define testing.  Phew.

Instead -  take some time to think about how you would explain time to me.  Surely much simpler huh?  You constantly use it.  So describe it to me ... you can use the comment below if you'd like.

Okay?  How did we go?  It's actually pretty hard isn't it.

If your answer was something like "time is - hours, minutes, days, months, years?" or "as seen on a clock" - it's the start of a good answer, but a bit shallow.  Time is something we use a lot, we have a vague idea of it's rules, but describing it is really difficult.

Here's some definitions I came across ...


Time is the indefinite continued progression of existence and events that occur in apparently irreversible succession from the past through the present to the future. Time is a component quantity of various measurements used to sequence events, to compare the duration of events or the intervals between them, and to quantify rates of change of quantities in material reality or in the conscious experience. Time is often referred to as the fourth dimension, along with the three spatial dimensions.

Google tells me

the indefinite continued progress of existence and events in the past, present, and future regarded as a whole.

Webbsters dictionary has
the thing that is measured as seconds, minutes, hours, days, years, etc.

the measured or measurable period during which an action, process, or condition exists or continues

Some of those definitions help, some are probably right if you think about them.  As a physicist, I of course have a fascination with time, and so I've spent a few years thinking about it and how we use it.

My definition is fairly simple, elegant but it has consequences ...

Mike Talks' Definition Of Time

Time - a phenomenon which allows change to occur.

That is to say, without time, a system will remain in stasis.

It's pretty neat, and in many ways close to Webster's complex answer.  But as I said, there's consequences.

One of which for certain is that time is not directly measurable.  The other is that time may not actually exist ... certainly not as we think of it.

Okay - that's a big ask - so do check the date on this post, and confirm it's not April 1st.  I do have past form in trickery ... it isn't, and I'm serious.

By now, I'm hoping you're thinking about scrolling down to the bottom of the page, and typing something angrily into the comments.  Maybe something like "but I can measure time on my watch ...".

Thanks for bringing that up ... if you have a digital watch, then it's powered by a quartz crystal.  When this crystal is put under an electric charge - many would say that it vibrates 32,768 times a second.

And I'd have a problem with that.  Because we're not measuring and counting time - we're measuring and counting change.  In actual fact, when you measure 32,768 oscillations in a quartz crystal, you're at that point making a mark to say "I think one second has passed".

Let's revisit that shallow answer again ...

Remember we said a shallow answer to the question would be that "time is - hours, minutes, days, months, years?".

What is a day?  It's the length of time between consecutive midday (when the Sun casts the smallest shadow).  Or how long it takes for the Earth to rotate once.

What is a month?  Well, a lunar month is the time it takes between the Moon's relative phases.  Or how long it takes to orbit the Earth once.

What is a year?  The time it takes the Earth to orbit the Sun once.

Are you noticing that each of these is measuring a change, and giving the period a value which for you is a measure of the time that's?  Or in other words "this unit of time is when this much change has occurred".  Notice a pattern here?

Let's look at another consequence ...

If nothing is changing - time cannot be said to be occurring

If a system is in complete stasis, then you're not able to say that time is occurring.  This of course probably seems absurd to you.  You probably would say "I know it's passing", and apply logic by looking at something outside of the system to prove it - that "thing" outside of our system under observation is of course undergoing change (hands on the clock mayhap?).

Let's consider a hypothetical universe ... it's approximately the year one googolplex, and the Universe has ground to a halt.  It stopped expanding an untold time ago - the last star burned out billions of years before that.

The Universe is in a constant temperature of absolute zero, no planets orbit stars anymore, no galaxy moves.  There is no chemical reaction remaining, and even electrons inside atoms are stationary.  The Universe is exhausted of all energy, and remains unmoving.

How would you be able to tell that any time at all way passing?

You couldn't.  You'd have to introduce something to this universe/system like a watch, where change occurs.  But by itself, there seems to be no time passing, as there seems to be no change happening.

It's impossible to tell here if time exists or not.  Certainly without something to change, time becomes completely irrelevant.

[You might notice a certain familiarity here with the idea of the heat death of the Universe ... also it was used in the Doctor Who story Utopia]

That's quite enough today - next time we're going to play a bit more with this definition, and think about how we can test even more.

Now playing:  "The Logical Song", Supertramp