There are so many amazing testing resources on the web but they are all disorganized. What if there was a single place that used the information from these resources to create testing best practices for the world? Furthermore, each testing recommendation would be evaluated by a team of industry leaders to ensure that the practice makes sense. Testing Best Practices is that single source of truth for how to test all the relevant tech in the world.
These properties were originally proposed by Kent Beck
- Isolated — tests should return the same results regardless of the order in which they are run.
- Composable — if tests are isolated, then I can run 1 or 10 or 100 or 1,000,000 and get the same results.
- Fast — tests should run quickly.
- Inspiring — passing the tests should inspire confidence
- Writable — tests should be cheap to write relative to the cost of the code being tested.
- Readable — tests should be comprehensible for reader, invoking the motivation for writing this particular test.
- Behavioral — tests should be sensitive to changes in the behavior of the code under test. If the behavior changes, the test result should change.
- Structure-insensitive — tests should not change their result if the structure of the code changes.
- Automated — tests should run without human intervention.
- Specific — if a test fails, the cause of the failure should be obvious.
- Deterministic — if nothing changes, the test result shouldn’t change.
- Predictive — if the tests all pass, then the code under test should be suitable for production.
✅ Do: Only use retries when the cause of indeterminism is fully understood and is unavoidable
What is unavoidable indeterminism?
- Internet instability (or maybe we can mock this out)
- 3rd party Selenium Grid going down
- 3rd party service going down (or can we mock this out)
❌ Otherwise: We are simply putting a bandaid on instability and will pay for it later, either in the app or the build
© Credits & read-more
1. Jonathan Lipps, Appium creatorThe modern web application typically runs on some type of JavaScript framework such as Angular, React, or Vue. A guiding principle of testing such web applications is to think about how we can do so without an actual browser. In many cases, a combination of component, unit, and visual tests will be the most powerful in ensuring quality web applications.
✅ Do: Use a component test with a virtual DOM instead of an actual browser
❌ Otherwise: Extra dependencies will create more complications and work, with no added benefit:
✏ Code Examples
// App.js - the button being tested
function App() {
const [buttonColor, setButtonColor] = useState('red');
const newButtonColor = buttonColor === 'red' ? 'blue' : 'red';
return (
<div>
<button
style={{backgroundColor: buttonColor}}
onClick={() => setButtonColor(newButtonColor)}>
Change to {newButtonColor}
</button>
</div>
);
}Expected Behaviors
- Button starts in 'red' state
- Button starts with text that says "Change to blue"
- On click, button changes color to 'blue'
- On click, button changes text that says "Change to red"
import { render, screen, fireEvent } from '@testing-library/react';
import App from './App';
test('should start in red state', () => {
//render the component in a virtual DOM
render(<App />);
//find an element with a role of button and text
const button = screen.getByRole('button', {name: 'Change to blue'})
//expect the background color to be red
expect(button).toHaveStyle({backgroundColor: 'red'})
});
test('should change color to blue on click', () => {
//render the component in a virtual DOM
render(<App />);
//find an element with a role of button and text
const button = screen.getByRole('button', {name: 'Change to blue'})
//click button
fireEvent.click(button);
//expect the background color to be blue after click
expect(button).toHaveStyle({backgroundColor: 'blue'})
//check that text is changed to 'Changed to red'
expect(button.textContent).toBe('Change to red')
})© Credits & read-more
1. Nikolay Advolodkin - testing a button component✅ Do: Use a component test with a virtual DOM instead of an actual browser.
Avoid clicking a link as the clickability of <a> is a default browser behavior and not the behavior of our SUT (Software Under Test).
❌ Otherwise: Extra dependencies will create more complications and work, with no added benefit. Same problems as a button test.
✏ Code Examples
SUT
// App.js - the link being tested
function App() {
return (
<div className="App">
<header className="App-header">
<img src={logo} className="App-logo" alt="logo" />
<p>
Edit <code>src/App.js</code> and save to reload.
</p>
<a
className="App-link"
href="https://www.ultimateqa.com"
target="_blank"
rel="noopener noreferrer"
data-testid="learn-link"
>
Learn Testing with Mia
</a>
</header>
</div>
);
}Rendered View
Expected Behaviors
- Link goes to "https://www.ultimateqa.com"
- Link opens in new tab
test('link has correct url', () => {
//render our App component in a virtual DOM
render(<App />);
//get the link element by data-testid
const linkElement = screen.getByTestId('learn-link')
//assert that the href is correct
expect(linkElement.href).toContain('ultimateqa.com');
})
test('link opens in new tab', () => {
//render our App component in a virtual DOM
render(<App />);
const linkElement = screen.getByText('Learn Testing with Mia')
//Link should open a new tab
expect(linkElement.target).toBe('_blank')
})© Credits & read-more
1. Automation Best Practices workshopCAPTCHA is short for "Completely Automated Public Turing test to tell Computers and Humans Apart." It exists to prevent automation, so it's not even worth trying. Otherwise, it fails to be CAPTCHA, because it won’t be able to tell humans and computers apart (Selenium.dev).
Two primary strategies exist to get around CAPTCHA checks during testing. They are as follows:
- Disable CAPTCHAs in your test environment. This could be a simple configuration in the software under test. Or this can even be a URL parameter that the test sets. Example:
www.software.com/?disableCaptcha - Add a hook to allow tests to bypass the CAPTCHA.
(10 testing scenarios you should never automate with Selenium. TechBeacon.com)
https://techbeacon.com/app-dev-testing/10-testing-scenarios-you-should-never-automate-selenium
https://techbeacon.com/app-dev-testing/10-testing-scenarios-you-should-never-automate-selenium
https://techbeacon.com/app-dev-testing/10-testing-scenarios-you-should-never-automate-selenium
https://techbeacon.com/app-dev-testing/10-testing-scenarios-you-should-never-automate-selenium
https://techbeacon.com/app-dev-testing/10-testing-scenarios-you-should-never-automate-selenium
https://techbeacon.com/app-dev-testing/10-testing-scenarios-you-should-never-automate-selenium
https://techbeacon.com/app-dev-testing/10-testing-scenarios-you-should-never-automate-selenium https://github.com/saucelabs-training/automation-best-practices/blob/9cea9cb0af39de3adc91dd42e89d68a9a1a5df96/my-react-app/src/__tests__/Solution.test.js#L16
https://techbeacon.com/app-dev-testing/10-testing-scenarios-you-should-never-automate-selenium