QA testing should do these 4 things:

  1. Make sure software works
  2. Make sure software does what it should
  3. Make sure software doesn’t do what it shouldn’t do
  4. Make sure software doesn’t break
    when you (the user) do something wrong, for example

  5. Make sure software delights the user

Most of the time, test automation is really only doing #1 — making sure that it works, by navigating around the application, and performing basic functions.

This is ok.

This is what automation is good at.

But you also need to do other things. Things that are harder. Things that are difficult for humans to figure out how to do, and even harder for computers, despite the hype around “AI”.

Sidebar: Artificial intelligence and artificial sweeteners

Artificial intelligence is like artificial sweetener. It tricks your brain without providing any of the benefits, like:

  • Energy (in the form of calories)
  • Pleasure (to the taste)
  • Tooth decay

Artificial sweeteners only simulate the pleasure from taste, which is really an anticipation of energy. It’s like the dopamine from a drug, or a video game that tricks your brain into thinking you’ve accomplished something.
Likewise, AI only simulates thinking, and it’s outputs give you a false confidence that someone has thought about the implications.
A large language model (LLM) like ChatGPT literally has no idea what it has written, whether it is bad or good, right or wrong, self contradictory or repetitive, or if it makes any sense at all.
The generative AI models don’t know how many fingers a human should have, whether a picture is a cate or a dog, or the thing it draws is representational at all, much less if it is possible to exist in any real space or follows any rules of logic or consistency.
The idea of leaving testing up to generative computer “AI” models is preposterous, given that testing is supposed to answer exactly these types of questions.

1. Does it work?

As I said, making sure something works is the easy part. Does an application start when I launch it? Can I login? Can I see a report when I click a button?

But does it work right?

B. Does it do what it should?

This is the area of functional testing. How can I tell software does what it should unless I know what it should do?

For that you need requirements.

Some requirements can be inferred. A tester can often figure out if software that is working, is doing the right thing using common sense and their knowledge about a topic.

Sidebar: Shopping cats and usability dark patterns

Wouldn’t it be nice if your shopping cart at the grocery store could scan the items as you take them off the shelf and tell you how much you’re going to spend?They discovered that you’re less likely to buy as much stuff if you realize how much you’re spending.

Look for this feature in online shopping carts as competition vanishes. When it’s hard to figure out your total, you can be pretty sure we are in an effective monopoly.

But some requirements are subtle. Does this item calculate price per item or per weight? What taxes are applied to which people and what products?

And some requirements require specialized knowledge. Domain knowledge. Knowledge about the business developing the software, or knowledge about the how and what software will be used for. Medical diagnostics, or aeronautic controls for example.

Sidebar: Agile considered as a helix of combinatorial complexity

If you have the requirements, that is, you can perhaps test them — assuming you understand them. But in this day and age of big a “Agile” top down bureaucracy and time filling meaningless ceremonies and complex processes and un-user-friendly tools, requirements are less clear than ever.
But I digress. Again.

If you have tests (and automation) that is going to make sure software does what it should, you’re going to need to

1. Know what it should do, and

2. Map your tests to those requirements

That is, assuming your tests (which are software), are actually doing what they should do.

Oh yeah, and you also need to

3. Know how to verify that those requirements are being met.

Because it’s virtually impossible for anyone to understand, much less enumerate, ***all requirements***, it stands to reason, you won’t be able to automate them all, or track what they are doing.

Combine this with the many numerous ways you can accomplish something:

- click a button
- click back
- no click it again
- hit refresh
- and so on

And you have a nearly infinite variety of ways that requirements can be met.

Not to mention the potential number of ways software can break along the way.

Actually, I am going to mention it. Because that is our next point:

4. Will it break?

Using the software will tell you if it works, and more than likely, as a tester you will discover novel and interesting 

(some may say “perverse”, or “diabolical”, but we’re not developers or project managers here.)

ways the software can break.

In fact, good testers relish in it. They love finding bugs. They love breaking things. They love seeing the smoke come out and the server room catch on fire.

Cloud data centers have really made this less fun, but there are additional benefits (*ahem* risks) to running your code over the network to servers halfway across the world controlled by someone else. And additional ways things can go wrong.

And they (I should say “we”, because I am myself a tester) get even more satisfaction, the more esoteric or bizarre ways they can trigger these bugs.

Perhaps nothing gives us more delight than hearing a developer scratch there head and say “It worked for me!” when we can clearly prove that’s not the case in all cases. Or for all definitions of “work”.

Breaking things is the delight of the tester, and while there are tools that can put stress on software with high loads and random inputs, nothing beats a dumb human for making dumb mistakes.

And finally, since I like to do things out of order (to see if something breaks) we need to see what else software can do (that it probably shouldn’t do):

# X. Bugs

Have you ever come across an unexpected behavior in software and been told,

“That’s not a bug, that’s a feature”

No, dear developer, that’s a bug, not a feature.
If it’s doing what it’s not supposed to, it’s not supposed to do it.
So it stands to reason that any undocumented feature should be considered a bug.
But as we pointed out earlier, not every feature requirement can be documented, and the effort probably isn’t even worth it, because, let’s be honest: no one will ever read comprehensive documentation, much less test it all

  • Every.
  • Single.
  • Time.
  • Something changes.

What’s the difference between a bug and a defect?

Some would have you say spelling is the only difference. I disagree. I think the pronunciation is also different.

A defect is when something you wanted in the system isn’t in it. Something is missing. A requirement isn’t met.

A bug (as Admiral Grace Hopper *allegedly* found out the hard way) is when something you didn’t want gets into the system.

Whether it’s a moth or Richard Pryor doesn’t matter. The point is, it’s not supposed to be there. But it is.

Sometimes this breaks the system (like in Admiral Hopper’s case) other times, it doesn’t break the system (as in Richard Pryor’s case).

It could be a security issue, but it doesn’t have to be. It could just live there happily, taking up bits, and burning cycles and nobody ever notices anything is wrong (except whoever pays the AWS bill).

Anyway, it shouldn’t be there if it isn’t intended to be there, even if it’s beneficial. If you discover it, and it turns out useful, you can document it, and then it becomes a documented feature.

No, adding a note to the bug report “Working as intended” does not count as documenting a feature.

But, it’s very hard to prove a negative. That is that it doesn’t have a feature it shouldn’t have.


So to reiterate, there are 4 things that testing (or Quality Assurance) should be accomplishing:

1. Making sure it works

Automation, or any random user, can see that this is the case. However, just because something works, doesn’t mean that it does what it’s supposed to, that it doens’t do what it shouldn’t, that it will keep working when it comes into contact with the enemy — I mean users.

Smoke tests fit well into this category, but it should go beyond just making sure it doesn’t burst into flames when you first turn it on.

B. Making sure it does what it’s supposed to

You need to know what it’s supposed to do to test this. Some things are obvious, but in some cases, requirements are needed. But comprehensive documentation is not practical.

This is often considered functional testing. Some of this can be automated, but due to many factors (including the reasons above), it’s not practical to automate everything.

 4. Making sure it doesn’t break

This can be harder to prove. But it’s important. Just because something is working at one point, doesn’t mean it always will be.

Load & Stress testing are a part of this. But so is “monkey testing” or “chaos testing” which as the names imply, are unguided.

Testers with their pernicious creativity and reasoning abilities can go beyond random behavior and deliberately try to break things.

The goal here is to make the system stable.

X. Making sure it doesn’t do what it’s not supposed to do

This is the hardest part, but the funnest part of testing. Often when something breaks, (e.g. a buffer overrun), it can also have unexpected behavior.

It can have serious security implications, but also may cause usability issues.

Which brings us to our bonus point:

# Bonus: Making sure it delights the user.

Something can work flawlessly, be perfectly secure, fulfill all requirements, and still be an unmitigated pain in the neck to use.

In actuality, trying to make something robust, reliable, secure, and complete ***usually*** ends up harming usability.

Add to this the simple principle that someone who created the system is ***probably*** going to understand the system better than someone who didn’t, means that they may make assumptions about how to use it that are either not valid, or obvious to the intended user.

Usability testing is an important part of testing and pretty much can’t be automated (although I’d be interested to hear ideas about how you think it could.)

Usability testing is also often neglected, or not done from the user perspective.

Anyway, that’s all I have to say about that, for now.

What are some Selenium WebDriver locator strategies?

Here is my answer to the question from Quora

What are some locator strategies that can be used in Selenium WebDriver?

Selenium WebDriver has several locator strategies — or methods for locating elements.

Whey you want to find an element, you need to locate it on the page. The way Selenium does this is by using Javascript to parse the HTML source content. In Javascript you can do the following:


WebDriver has corresponding locator strategies:


It also has additional methods for locating by XPATH, CSS Selector, and link text:


XPath and CSS selectors are ways to parse the HTML document and give more precise locators including a combination of element tag hierarchies, attributes, CSS classes, and relative position (parent, child, sibling). I won’t go into details, but these are powerful strategies for parsing a document and finding specific elements based on several criteria.

LinkText and PartialLinkText searches for anchor <a> tags that contain the given text for the locator.

By.linkText(“Click Here”)

WebDriver also has corresponding findElements() (plural) methods that can locate a list of matching elements. For instance, you can find all elements with tag name <div> or matching xpath //table/h1 (find all H1 tags within a table). By default, findElement() (singular) will return the first matching element.

Selenium 4 also introduced Relative Locators which can modify an existing locator with terms “above”, “below”, “rightOf”, “leftOf” or “near” (near meaning within 50 pixels). In practice, relative locators are often not reliable, because layout typically depends on a fixed screen size and layout. One use for relative locators is to check responsive layouts given a known screen size. For example, to make sure a button is below a div on mobile devices, but beside it on a full screen:

By mobileButton = RelativeLocator.with(“myButton”)).below(“myDiv”))
By desktopButton = RelativeLocator.with(“myButton”)).rightOf(“myDiv”))

Now, the next question is: Which locator strategy should I use — and why? is the most efficient locator, the most concise, and the least likely to change. But not every element has a unique id attribute. Use it when you can.

<button id=”login”>
driver.findElement(“login”)) is useful for form elements, and is also concise and specific.

<input name=”email”>

Tag and and class name are often useful for finding all elements that match that specific criteria:

driver.findElements(By.tagName(“a”)) ← this will find all links on the page
driver.findElements(By.className(“dark”)) ← this will find all elements with the “dark” class attribute.

XPATH is definitely the most versatile, but can be very ugly and easy to break

driver.findElement(By.xpath(“//table/*/div[\@class=’result’]/a[contains(text(), ‘edit’)]") ← find the edit link in the first table that contains a div element with class name “result”

But CSS selectors can do most of the same things as XPATH (except finding parent, child, sibling, or text nodes) and is often more readable.

driver.findElement(By.cssSelector(“table div.result > a”)) ← find the first link in the first table within a div with class name “result”.

Note: CSS selector cannot find elements by specific text.

As you can see, CSS (and XPATH) locators can incorporate the above strategies (tag name, class name, id) into one locator. Many people prefer to use one of these locator strategies exclusively for consistency.

However, an important “strategy” when using XPATH or CSS selectors is to not use complex selectors that depend on the document hierarchy. You should try to find a unique locator as specifically as possible, by id, name, or tag/class combination that will not be likely to change as the page layout changes.

If you cannot identify a single element definitively, you can look for the closest unique parent element. Using a relative XPATH or CSS selector (different from a relative locator like “above” or “below”) from that parent is a good strategy.

driver.findElement(By.cssSelector(“#uniqueId > div”)) ← find the first div child of an element with the uniqueId.

In CSS Selectors:

div#uniqueID ← search for a <div id=”uniqueId”> element with ID attribute
div.className ← search for a <div class”=myClass”> element with class attribute

Personally, I recommend that given the choice between XPATH and CSS selectors, to choose CSS when possible. Both for readability, and as a practical consideration — web developers know CSS selectors well, but usually do not use XPATH.

Finally, you can locate one element, and then search for other elements below it by performing two searches.

driver.findElement(By.xpath(“//table”)).findElement(By.cssSelector(“.results”)) ← find the first table, then find the element with className “results”. 

This does incur a slight performance penalty by making multiple WebDriver findElement calls. You should try to find elements with a single locator when possibly, but not at the expense of either readability (complex locators) or maintainability (likely to change). These often coincide.

In summary, you should try to find unique locators for elements that will not break as the page layout changes. Finding elements by ID is the most efficient. XPATH, and then CSS selectors are the most versatile, and you can often get whatever you want with one of these two. You should strive for simple locators that identify an element uniquely, but avoid complex hierarchical locator strategies when possible because they can lead to difficult to maintain code.

How to get started writing unit tests on a large complex codebase without existing tests

A little bit at a time.

Developing unit tests for a large scale application can be challenging. Especially if it was written without any tests, or with testing in mind.

Writing unit tests requires code to be modular — so you can test a unit in isolation. One of the biggest benefits of writing unit tests during development is not that your code will be well tested, but that it helps you think about writing it in a way that different parts of the application (units) can be tested without interacting with the whole application.

But if it hasn’t been written that way, it may require significant refactoring to be testable. Or it may require mocking, or creating fake services, or injected databases with dummy data. And this can make tests complex, brittle, and unreliable.

So a good way to start unit testing on a code base that is resistant to testing in isolation because it is complex & tightly coupled, is to add unit tests slowly, as you modify the codebase.

Find an area that is complex (and ideally has a bug that needs to be fixed). Once you identify the piece of code that you need to test (by looking at log files, stepping through it with a debugger, or adding “print” statements), work on recreating that manual verification process in a unit test.

Write a test that describes behavior you want to test (or defect you want to expose.)

Unit tests should not depend on a deployed application, external services, or need you to create many mocks or stubs. You do not want to do this, for the reasons mentioned above.

So if you can find a piece of logic, isolate it, and refactor the code so that you can test this in isolation.

You may need to mock 1 or 2 external interfaces, but don’t go overboard. Too many dependencies is a sign that your code is too tightly couples. If you find this is the case, you may want to start with unit tests in an area that is simpler to isolate. You can tackle the more complex areas of your codebase later — when you have more tests written, and refactoring it becomes less risky.

Another thing you do not want to do is create a bunch of simple unit tests for easy methods that have little or no logic — getters & setters, constructors, helper methods that just hide ugly APIs with verbose setup steps, etc. If you have code that is very unlikely to ever change or break, don’t write unit tests for it. Every test you write adds to your codebase, which means adding to maintenance cost. Don’t write unit tests just to increase numbers or code coverage. Make every test count.

One last thing to avoid is writing unit test that describe functionality or depend on the implementation. Tests with a lot of mocks or data setup or several assertions are probably because of this. By writing tests to the implementation, you may be verifying that it works as intended, but it will make it hard to refactor the codebase in the future. A lot of projects abandon unit testing when they find that the existing tests cause friction in refactoring, or worse, tests that end up getting ignored because they are not keeping up to date.

So to reiterate, the answer to getting started writing unit tests on a large existing codebase is to start slowly, adding tests as you work on specific areas of the codebase, refactoring the code as you work to make testing in isolation easier, and avoiding (at first) areas that are too difficult to break down into unit tests, and areas that are too simple to ever fail. Make ever test count, and make it tell you something useful, and don’t set up code coverage goals or write more tests just to increase numbers.

Here is the question on Quora that this answer addresses:

What is Selenium? Does it support multiple users?

Selenium is a programming tool used to automate the browser — simulating a real user by opening windows, going to URLs, clicking buttons, filling in forms, etc. Selenium is primarily used for test automation, but can also be used for other things.

Selenium is available as a library for most popular programming languages, including Java, JavaScript, TypeScript, C#, Python, Ruby, PHP, etc.

It is also referred to as Selenium WebDriver, because there were two different projects (Selenium & WebDriver) which did similar things and eventually merged. Selenium uses the WebDriver protocol (a W3C standard now) to communicate over the network via a REST API. This allows for remote automation,

There are other tools associated with Selenium, including Selenium Grid — which enables remote execution of automation and Selenium IDE — which allows you to record and play back automated steps without writing code, and has (limited) ability to to export from Selenium IDE to code that can run independent.

Selenium IDE does not support multiple users, but scripts can be exported and shared from one user to another.

Selenium Grid allows for parallel remote execution of Selenium scripts, which allows multiple people (or tests) to execute tests at the same time.

The concept of “supporting multiple users” does not really make sense in terms of Selenium as an open source development tool or coding library.

It would be like saying:
“Does Microsoft Word support multiple users?” or
“Does the Java programming language support multiple users?”

In the case of Microsoft Word, every user that has the program can use it, but collaboration is (primarily) done outside of the tool. With proprietary software like Microsoft Word, each user may need a license to run their own copy of the application, but Selenium is open source, so does not require the purchase of any license to use.

And as a programming library, any number of users can reference Selenium in their own code and execute it. Users can run multiple automated tests (or other scripts) at once — if they write their program to run in parallel.

But in order to maximize parallel execution (for single or multiple users) you need to have a remote Selenium grid. There is an open source grid that anyone can deploy, but there are also commercial services the host Selenium Grid with additional tools at a cost. These companies include Sauce Labs, BrowserStack, LambdaTest, and Applitools. Each company has their own policy about multiple users, and of the ones I mentioned, they all support multiple users in their own way.

This post is based on a question asked on Quora at:


You know how a search engine can help you with little things like converting units (like: meters cubed to fluid acres) and get other little answers to things like the weather or internet speed or language translations just by typing in your question.

AI prompts like ChatGPT have the potential to really enhance this capability (when it’s not going rogue on you (you know: turning racist, expressing creepy affection, threatening to destroy humanity).

Anyway, I’ve thought of another little tool I’d like to have built into my search prompt, or available for a writing tool like Grammarly, or whatever that app that helps suggest ways to make your writing more terse (Hemingway?).

So I’m announcing my startup and looking for funding… not really (unless you want to.)

It’s called “Circumnavigar”. Which is a Spanish term to that means (unsurprisingly — to go around, or circumvent.

You use it to describe how you’re trying to describe something but can’t quite think of the right word or phrase.

Like if you want to ask for ask for “honey” at a tienda and you don’t know much Spanish and your wife is at home in bed with morning sickness. So you say “azucar” and “dulce” and “amarillo” and make buzzing sounds and pantomime getting stung, and the lady furrows her eyebrows and then laughs at you and then walks away nervously and talks rapidly to her friend who then slaps her forehead and says “Ah! … Miel” and you shake your head because you think that means flour, but then when she hands you a jar you thank her profusely and quickly pay and walk away in embarrassment.

(That was before online translator apps and smart phones.)

But how many times are you writing (or talking) and can’t quite think of the right word — not even in translation, just in normal speech. You know you know the word, you just can’t think of it. (Or maybe that’s just me.)

For example, while writing this post, I couldn’t think of the word “potential” and I had written: “have the capacity to really enhance this capability”.

I knew that wasn’t quite right, besides “capacity” and “capability” sounding wrong (and redundant) together.

Or while writing another article, I had the word [basher] in brackets, because I couldn’t think of the right word “detractor” I had tried opponent, critic, complainer, basher.

So I want an app that can help me find the right word when I can’t think of it. Not necessarily just synonyms, but if I can’t think of the word at all, I can “circumnavigar” around it and the language processor can help me get it right.

Look for an upcoming article where I talk about bashing something… and then coming around to it.

Bashing React

Bashing React

I’ve been a long-time casual detractor of React.js. The occasional snide remark here & there, or long rant in the internet that nobody ever reads.

The gist of my complaint is that it’s overly complicated — while at the same time not doing enough to help developers.

Other frameworks (like Angular) are complex, but they aim to provide a more complete solution in a coherent form. To be effect (read: develop something beyond a toy app) With React, you need a whole lot of other things with React that you have to piece together. Redux, React Router, etc. While with other frameworks (like Svelte) both radically simplify and improve performance but don’t have the pretension of solving complex developer problems.

Recently, I decided to get professional about bashing React. Not more professional in my behavior, but I wanted to go pro as an anti-React troll.

I decided I needed to learn more about it. To, you know, criticise it more effectively.

So I turned to ChatGPT to start asking clarifying questions. When I diss something, I like to really get my facts straight — usually after flying off the handle and getting them all wrong.

Do I ever publicly apologise or issue a retraction / correction?


I used to joke that React feels like it was written by AI.

See, even a couple years ago, when we (normal people) talked about AI, we meant machine learning analyzed uncategorized data and neural network pieced together an algorithm that develops a hueristic for determining if something is, in fact, a picture of a kitten.

(Apparently AI couldn’t figure out how to identify street lights, or crosswalks, or busses — and outsourced the job to us.)

Anyway, the algorithms developed by AI algorithms are often surprising and usually inscrutable. But they worked (for the most part) with a few surprisingly strange exceptions. Which a good reason to use AI to inform, but not make decisions.

What I was saying is that React feels like it was developed organically, without a plan, by a large group of people working independently and it coalesced into something that may be useful for developing web applications, but it is bizarrely confusing to someone looking at it from the outside, and doesn’t have a coherent organizing structure. In other words, it was a cobbled together complex system that came out the unique boiler-room of Facebook’s needs and evolved with additional layers of complexity as it was adopted and adapted by a broader community.

Compared to something like Angular, which has a clear design (even if it was designed by committee and is too rigid and too heavyweight for most projects) that is logical, and it’s largely a matter of taste and (need for complexity and structure) that determines if you want to use it.

I recently watched the React.js documentary:

And was going to say that I feel like I was right, except that React came out of the mind of one person, Jordan Walke. Which is not quite true, because it didn’t take off until a core team of collaborators and contributors and evangelists pitched in. So it wasn’t really a gradual organic design from a large organization, but was a more deliberate design led by a single visionary to replace the organic jumble of tools used by that large organization, Facebook.

In a rare fit of humility, I decided (before watching the documentary) that there must be a reason so many thousands of developers find React so compelling, beyond the fact that it’s promoted by Facebook.

And coincidentally, I was experimenting with building a website out of components using Svelte.js, and feeling quite productive, despite not doing any real development, other than building the (mostly static) site out of composable compents.

The header contains the logo, menu, and login components.
The menu takes the topics (About, Blog, Contact, etc) as data properties.
The login component has two states (logged in, not logged in) and affects the menu (whether admin menu is displayed).
Each page contains the header, main, and footer sections and so forth.

Being able to compose a site from components really helps with structuring your code. But I still didn’t understand all the additional complexity.

Being happy with my progress using Svelte, I thought it would be worthwhile trying React for this simple scenario. And possibly even try to learn a bit about Next.js for server side rendering a static site. The main case I want is to be able to pull content from other sources (CMS, database, or git repo) and render them into a static site, but be able to update page and sections of pages by editing plain text (or markdown) files. In other words — like
a blog.

Perhaps someday, someone will invent a useful tool for this.

So I started exploring React as a way to structure HTML. I’ve long suspected that majority of React fans (and users) use it only for this. The claim of React without JSX was disingenuous, because JSX is the primary reason for using React.

People say that React is easy to learn because has a simple interface, but that’s nonsense. React has the most complex interface of any UI framework. Things like componentDidUpdate and componentWillUnmount that only make sense at the most low level for the most complex interactions — and details that almost nobody knew existed in JavaScript at all. I submit that it was actually the complex low level hooks (that React has, but nothing else did) that made it popular for complex sites like Facebook where lots of things are happing below the surface that users aren’t even aware of.

And people talk about the Virtual DOM and how it improves performance. While that may be true for extremely complex DOM interactions (hundreds or thousands of elements changing), but for most sites and applications, it doesn’t. In fact, for most cases, the virtual DOM is slower than just finding and replacing real DOM nodes in place. React’s DOM diffing strategy is actually very basic, and doesn’t (or didn’t) handle things like a <div> tag changing to a <span> tag and everything underneath it stays the same. Or (earlier) adding or removing elements from a list.

While the concept of a virtual DOM was a technical marvel, the implementation left a lot to be desired — and came at a big cost, both congnitively, and code complexity wise, as well and framework size and performance. “Lightweight” frameworks like Preact were create to replace the supposed lighweight React.

Which get’s to the real complexity. React is pretty complex. But it’s benefit comes from not needing to understand the complexity hidden inside — until you need it. Most frameworks can’t compete, not in simplicity, but in being able to adapt to the real complexity when needed.

Reacts “simplicity” comes at the additional cost of React not having a an answer to state management, routing, server side data, and a bunch of other things that are needed for creating a real web “application”. These are all either “extras” or “roll your own”. And you bring in things like Redux, React-Router, Apollo GraphQL, etc to handle these. Which then makes you wonder why you’re using React at all in the first place.

Most people are happy using JSX to compose UIs, and occasionally dip in to properties for passing data, and gradually let the React “expert” handle the complexity of state, lifecycle hooks, and the rest. And that’s probably the Facebook use case. Someone is working on the HTML and CSS for a small widget (like advertisments, or like buttons) and need to 1) make sure it works consistently in all sorts of displays and device. And someone else handles the lifecycle and state interaction.

That means using React, by itself, to compose UIs, is a perfectly acceptable solution.

Except JSX isn’t a good solution. It can’t handle looping, because it can’t handle statements, so you need to use map() it iterate over arrays, and inspecting arrays of object that have arrays can become a nightmare in React, which is actually pretty easy with a straightforward templating solution.

JSX was a nightmare to use for the first few years, because that meant setting up Babel transpilers, Grunt/Gulp tasks, and Webpack, and configuration. Thankfully, create-react-app (created by Dan Abramov) made this simplier, and now tools (like VS Code) can detect JSX and render it properly. Debugging was another major challenge that has improved.

But, I didn’t come here to criticize React, even though I’ve been doing that for a long time. I came here to praise it. Or rather, to explain that it may have it’s uses, and I want to better understand them. And understand why the decisions made to make React work the way it does were made. For that, I need to understand them better myself.

Earlier, I talked about how React felt like it was inscrutably designed by an AI system.

Part of the appeal of React (and many complex modern technologies) is their complexity. It’s that they are hard to learn, and it is a puzzle to figure them out, and you feel like an initiate into a secret society once you finally have obtained your hard-won knowledge. And people want to be like the esoteric experts, so they adopt their fashions.

What I’ve found is that, while I have been unable to understand React by going through tutorials and looking at sample code, part of that is failing to realize that you don’t need to understand it all.

Most uses of React don’t need to use Redux, and most well designed apps won’t use React-Router. And the less shared state that an application has, the better (usually). So a lot of the complexity associated with React applications (that are outside of React itself), are actually unneccesary for the majority of use cases (rendering content and layout on the screen).

What really clicked for me (after spending some time doing due diligence giving React a fair shake) came after reading this post by Dan Abramov (the creator of Redux) a core React team member.

Perhaps ironically (and perhaps not — depending on your opinion of Alanis Morriset or high school English teachers), the thing that really helped me to understand React, was that it was the AI system ChatGPT (which really just sythesizes existing content) that enabled me to understand (and appreciate) React better.

Maybe only an AI can explain a framework allegedly created by AI.

I’m going to be going through my interaction with ChatGPT as I learn, research, experiment, complain about, and use React. Look for more posts and videos. And if you’d like to study React with me, reach out.

What can I do to expand my skills beyond testing?

Someone asked about self-improvement after 10 years as a tester and wanting to expand their knowledge into software development.

I can sympathize with this attitude because I went through a similar mindset — which let to my eventual burnout and move to Fiji that I’ve written about previously.

Here is my response:

After 10 years as a tester you probably have pretty good testing skills.

Adding development skills can only increase your value as a tester because it will allow you to communicate better with developers and understand the architecture to find, anticipate, and troubleshoot bugs better.

And if you want to move into development (creative vs destructive role) your skills as a tester will help you and maybe influence other developers to think of testing first.

Other areas you could branch out into and expand your knowledge include operations/devops, understanding system architecture, project / product management, team leadership, or specialized domain knowledge (such as healthcare or machine learning) which can all benefit your work in testing or provide alternate career paths if you’re looking for change.

See the original post on LinkedIn here:

Who cares about this test?

Last week, I wrote about (one way) how to categorize tests.

And I promised a follow up about different ways to categorize tests. Or “how to slice the cake” as I put it.

This is not that follow up.

But after getting some feedback, I thought I’d expand a bit more on the “how” I categorizing tests.

So rather than talk about the different slices of cake, I’m going to talk about “why” I sliced it this way. If you get a smaller slice, it’s not (only) because I think you’re fat. And if you get a bigger pice, it’s not (just) because I like you more than your sister.

The title gives away the “how” — which was pointed out by a reader that I didn’t make very clear in my previous post. That is to say, I categories tests by “who cares about them”.

A developer test is obviously a test that a developer cares about. A user test is a test that a user cares about.

Maybe that’s why we let users do so much of our testing for us.

And so on.

In my current role, there are several gaps in “what” is being tested, and this leads to frustration with QA. The QA team, the QA process, the testers, the test automation framework, etc. It’s my job to help identify and solve this.

My first premise of testing is that you can’t test everything.
Which naturally leads to the second premise:
That there’s always room for improvement.

The question then, is where to improve? And that isn’t always obvious. Or different areas to improve may be obvious to different people. Because those people have different perspectives and different needs.

On a content focused website, it might sense that the content itself — and it’s presentation — are of primary importance. But it might be more or less important that the content can be found. Either by searching (or navigating) the site, or by search engine crawling. One perspective focuses on satisfying existing users of the site, the other perspective focusing on gaining new users. Which depends on your business model or stage.

But there are other concerns, and not just functional concerns. When you talk about non-functional concerns people talk about performance, or security, or usability. But what about operational stability? What about monitoring, error recovery, and risk.

One area I think that is mostly overlooked by QA are operational concerns.

How do the people who deploy and maintain and support your applications benefit from testing?

A quick answer is to have smoke tests.

By “smoke tests” I mean, in the technical electrical engineering sense of the term:

Does the smoke come out when you plug it in? [1]

  • When you deploy a change, how quickly can you detect that you haven’t broken something —
    or done (significant) harm to the system?
  • How can you tell that all parts of the system are working as expected?
  • How can you verify that your change is doing what it was intended to do?
  • Did anything else change and what was it impact?
  • Are there any unanticipated regressions?
  • Are all the parts of the system restored to normal capacity?
  • What (if any) impact did it have on users?

Not all of these points are smoke tests. Where do you draw the line?

It doesn’t have to be a line between “smoke tests” and “full testing”. You can have gradual stages. There may be things that you can report on in:

  • less than 10 seconds
  • under 5 minutes
  • about 1 hour
  • at lease a full day
  • several weeks or more

Again, these types of tests may be doing different things. And different people may care about them.

So I think testers should spend time thinking about the different people who might care about their tests, and plan accordingly. This also means budgeting time to work on tests that mean the most to the people who care about them. Because you can’t test everything.

A good exercise is to find out everyone who cares about the software product and find out what their concerns are. In QA, we often think about two groups of people:

  1. The users (as represented by the product owner)
  2. The developers (as represented by the developer sitting closest to you — or the one who gives you the most feedback).

    Not only should you ask them questions, but you should ask questions of other people including:
  3. Sales – they might care more about the experience of new users,
  4. Compliance & Legal – they might care more about accessibility, data retention, and privacy,
  5. Operations – they might care more about performance under load, security, ability to roll back features, and making sure all parts of the system are communicating.

I’m sure there are lots more people who are about the system.

Conversation on this top sparked one discussion about a real world scenario that probably never would have occurred to a tester focusing on functional requirement from a user perspective who doesn’t understand the architecture — and doesn’t know the deployment process.

The way to address all these concerns in testing is to communicate. And I can’t think of a better way to communicate than to have cross functional teams where all perspectives are represented — and everyone cares about testing, not just the testers, and that testing isn’t something you do just before “throwing it over the wall” to production.

[1] Everyone knows that smoke is the magical substance that makes electronics work. Because when the smoke comes out, it stops working.

How long does it take to learn Selenium and Python

This is another post that grew out of a question asked by someone else online.

Here is the original question (and my answer) on Quora:

There are several different things here, and they affect how long it will take you to learn Selenium with Python.

Let’s break it down:

  1. Learning Selenium
  2. Learning Python
  3. Learning programming
  4. Learning test automation

Your existing knowledge in each of these topics will affect how easy it is.

For example, if you’ve already used Selenium with another programming language, that means that (to some degree) you also know programming and test automation principles. So all you need to learn is Python.

But if you have some programming experience in JavaScript but have never done test automation (with or without Selenium) or used Python, you come with general programming knowledge (but using a different paradigm) there are more obstacles.

Alternately, you may have experience with test automation so you understand the goals, but have used a commercial low-code record and playback automation tool. This may actually be harder than starting from a programming background because it requires a paradigm shift in your strategy to test automation.

However, probably most people asking this question have some experience manually testing software, a basic knowledge of programming (either in Python, or some other language — but would not consider themselves expert), and want to know how long it would take them to become competent enough with Selenium and Python to either:

A. Get a job doing test automation with Selenium and Python or
B. Apply test automation with Selenium and Python in their current job
(which may be manual testing, or some other role).

So, I’ll try to answer this question.

Give yourself a few weeks to learn the basics of Python, general programming principles, and the “Pythonic” idioms. A good course on Udemy or book about Python is about what you need. Subtract if you already understand some of this, if you’re a fast learner, or have guidance (such as a mentor.)

But it’s not really about how quickly you can absorb knowledge, it’s practice to retain it, and having the time to make mistakes and experiment.

And then it will only take a week or two to pick up Selenium. It has a fairly straightforward API, and the concepts of automating finding buttons and clicking on them is fairly simple to understand. There are some obstacles that can trip you up though — things like provisioning and configuring a WebDriver, managing sessions, “getting” locators like XPath and CSS, data driven, using Pytest and fixtures, etc can trip you up and lead to hours (or days) of frustration — or you can establish bad habits that will bite you in the future.

But applying Selenium in a meaningful way to write useful test automation may take additional weeks, or months (or years) of practice. Again, this depends on your personal curiosity, cleverness, mentoring opportunities, and above all, practice and ability to apply it in the real world.

These challenges fall under writing good maintainable test automation, knowing what to test and how to approach it, and even little things like picking good locators, or naming functions and variables well.

If you were smart, and scrolled to the bottom, short answer is:

It should only take you a few weeks to pick up Python and Selenium, depending on where you are coming from experience-wise and how fast you learn. Having mentors and or guided learning can help you focus on what’s important and get past silly obstacles. But like anything, it will take a much longer time to master, and depend on real world experience applying the principles, making mistakes, and learning from them.

A reasonable time to go from basic programming knowledge and no test automation experience to competent enough to make an impact at a job (e.g. get hired as a junior test automation engineer) is a few months of steady learning and practice.

Skating on Thin Ice

The weather has been warming up in a Montana recently, leading to a slushy mess in our driveway.

But my daughter discovered something today when she went out with her mom to feed the animals…

We had a hard re-freeze last nice and the driveway is a sheet of ice. Very slippery, so she abandoned her chores and put on her ice skates.

Ice skating in the driveway