Autumn 2026 review

content/full-cycle-ci.md

@@ -8,10 +8,10 @@
 
 ## Exercise a full-cycle collaborative workflow
 
-This exercise is a collaborative version of the [Automated testing exercise](./continuous-integration).
+This exercise is a collaborative version of the [Automated testing exercise](./remotely).
 
 ```{exercise} FullCI-1: Create and use a continuous integration workflow on GitHub or GitLab with pull requests and issues
-This is an expanded version of the {doc}`automated testing demonstration <continuous-integration>`.
+This is an expanded version of the {doc}`automated testing demonstration on the cloud <remotely>`.
 The exercise is performed in a collaborative circle within the exercise group
 (breakout room).
 
@@ -342,6 +342,8 @@ Your goal:
 
 
 ```{keypoints}
+- GitHub Actions (and other services, like GitLab CI/CD pipelines) can be used to run automatically the test suite when pushing.
+- Setting up is usually done with YAML files. Every platform has their own syntax, but there are templates one can use.
 - When fixing bugs or other problems reported in issues, use the issue
   autoclosing mechanism when you send the pull/merge request.
 ```

content/index.rst

@@ -1,21 +1,34 @@
 Automated testing - Preventing yourself and others from breaking your functioning code
 ======================================================================================
 
-Have you ever had some of these problems?:
-
-- You change B and C, and suddenly A doesn't work anymore.  Time
-  wasted trying to figure out what changed.
-- There was some simple problem, systematically testing could have
-  found it.
-- You get someone else's code and are afraid to touch it because who
-  knows what might break.  Plot twist: it's your own code!
-
-People have learned that some automatic way to check problems makes
-software development much easier.  This lesson will talk about the
-places it's useful for research code, and how easy it can be.
-We will discuss why testing often needs to be part of the
-software development cycle and how such a cycle can be implemented. We will
-see how automated testing works and practice designing and writing tests.
+In this lesson we discuss the basics of *automated* testing.
+
+We start discussing why automated testing is important.
+We then show how to set up automated testing in your projects
+in a few programming languages,
+so that you can run a test suite conveniently on your own computer.
+We will continue by showing how to have GitHub (or GitLab)
+run the test suite automatically
+(typically whenever someone pushes to the repository),
+and tell us when there was a problem.
+
+If time allows,
+we might do that in a collaborative fashion,
+running the test suite on a pull request
+to inform the code review process.
+
+Writing tests can be challenging sometimes,
+so we will discuss typical problems in test design,
+mentioning also *Test Driven Development*,
+and practice designing and writing tests.
+
+The goals of the module
+is to make the learners feel comfortable
+with setting up a test suite of automated tests,
+feel familiar with the automation options
+on Software forges (e.g. GitHub and GitLab),
+and make them aware of the typical challenges
+in writing automated tests.
 
 .. prereq::
 
@@ -43,7 +56,7 @@ see how automated testing works and practice designing and writing tests.
 
    15 min ; :doc:`motivation`
    25 min ; :doc:`locally`
-   30 min ; :doc:`continuous-integration`
+   30 min ; :doc:`remotely`
    30 min ; :doc:`test-design`
    5 min ; :doc:`conclusions`
 
@@ -54,7 +67,7 @@ see how automated testing works and practice designing and writing tests.
 
    motivation
    locally
-   continuous-integration
+   remotely
    test-design
    conclusions
    full-cycle-ci

content/locally.md

@@ -1,4 +1,4 @@
-# Testing locally
+# Automated testing on your computer 
 
 ```{questions}
 - How hard is it to set up a test suite for a first unit test?
@@ -9,7 +9,8 @@
 
 
 In this exercise we will make a simple function and use
-one of the language specific test frameworks to test it.
+one of the language specific test frameworks 
+to test it automatically.
 
 * This is easy to use by almost any project and doesn't rely on any
   other servers or services.  

content/motivation.md

@@ -5,8 +5,40 @@
 - Understand various benefits of testing
 ```
 
+Most scientists nowadays depend on software for research.
 
-## Untested software can be compared to uncalibrated detectors
+What can go wrong when research software has bugs?  Look no further:
+
+- [A Scientist's Nightmare: Software Problem Leads to Five Retractions](https://science.sciencemag.org/content/314/5807/1856.summary)
+- [Researchers find bug in Python script may have affected hundreds of studies](https://arstechnica.com/information-technology/2019/10/chemists-discover-cross-platform-python-scripts-not-so-cross-platform/)
+
+How can we avoid problems like these?
+
+## What are typical problems that *automated* tests can address?
+
+Have you ever had some of these problems?
+
+- You change B and C, and suddenly A doesn't work anymore.  Time
+  wasted trying to figure out what changed.
+- There was some simple problem, systematically testing could have
+  found it.
+  But testing manually takes too much time,
+  so nobody has ever done it
+  with the appropriate care.
+- You get someone else's code. 
+  You really need to change it 
+  but are afraid to touch it because who
+  knows what might break.
+  Plot twist: it's your own code!
+- You implement features to someone else's code and want to merge it,
+  but they are not sure your changes haven't broken anything 
+  and it's time consuming to test that.
+
+People have learned that some automatic way to check problems makes
+software development much easier.  This lesson will talk about the
+places it's useful for research code, and how easy it can be.
+
+## Untested software can be compared to uncalibrated measurement devices 
 
 *"Before relying on a new experimental device, an experimental scientist always
 establishes its accuracy. A new detector is calibrated when the scientist
@@ -17,18 +49,14 @@ calibration are compared against the expected response."*
 
 With testing, simulations and analysis using software *can* be held to the same standards as experimental measurement devices!
 
-What can go wrong when research software has bugs?  Look no further:
-
-- [A Scientist's Nightmare: Software Problem Leads to Five Retractions](https://science.sciencemag.org/content/314/5807/1856.summary)
-- [Researchers find bug in Python script may have affected hundreds of studies](https://arstechnica.com/information-technology/2019/10/chemists-discover-cross-platform-python-scripts-not-so-cross-platform/)
-
 ---
 
 ## Testing in a nutshell
 
-In software tests, expected results are compared with observed results
+In the most basic form of software tests, 
+expected results are compared with observed results
 in order to establish accuracy.  Why are we not comparing directly all
-digits with the expected result?:
+digits with the expected result?
 
 ````{tabs}
    ```{group-tab} Python
@@ -79,12 +107,34 @@ CORRECT
 
 ## What can tests help you do?
 
-**Preserving expected functionality**
-- Check old things when you add new ones
+```{list-table} Problems, Solutions and who is affected?
+:widths: 40 30 30
+* - Problem
+  - Solution
+  - Who is affected?
+* - Breaking old functionality  
+    when adding new features 
+  - End-to-End tests
+  - Developers
+* - Verify installation
+  - Smoke tests 
+  - Users
+* - Showing up-to-date example
+  - End-to-End tests
+  - Users
+* - Improve readability and names 
+  - Unit tests
+  - Developers
+* - Refactor and restructure
+  - All tests
+  - Developers
+* - Documentation out of date
+  - Executable notebooks  
+    and [nbval](https://github.com/computationalmodelling/nbval),  
+    End-to-End tests 
+  - Users
 
-**Help users of your code**
-- Verify it's installed correctly and works.
-- See examples of what it should do.
+```
 
 **Help other developers modify it**
 - Change things with confidence that nothing is breaking.
@@ -127,14 +177,18 @@ Use the collaborative notes to answer these questions:
 
 ## Testing vocabulary
 
-* Test functions one at a time - **Unit tests**
+* Test functions and methods one at a time - **Unit tests**
 
 * Test how parts work together - **Integration tests**
 
-* Test the whole thing running - **End-to-end tests**
+* Test the whole thing running, checking the output - **End-to-end tests**
   * For example, running on sample data.
 
+* Test that the whole thing runs in the simplest scenario possible - **Smoke Test**
+  * if this fails, no point in testing other things, usually.
+
 * Check results are the same as before - **Regression tests**
+  * Other names for the same thing: **Acceptance Tests**, **Golden-Master Tests**, **Characterization Tests**
 
 * Write test first (the output), then write code to make test pass -
   **Test-driven development**

content/continuous-integration.md → content/remotely.md

@@ -1,21 +1,24 @@
-# Automated testing
+# Automated testing remotely
 
 ```{questions}
 - How can we implement automatic testing each time we push changes to the repository?
 - Why is it good to autoclose issues with commit messages?
 ```
 
-## Continuous integration
 
 We will now learn to set up automatic tests using either GitHub Actions or
 GitLab CI - you can choose which one to use and instructions are provided for both.
 
+
+```{note} 
+
 This exercise can be run in "collaborative mode" by following instead the instructions
 in [Full-cycle collaborative workflow](./full-cycle-ci). In the collaborative version steps
 C-D below are performed by a collaborator.
+```
 
 
-```{challenge} Exercise CI-1: Create and use a continuous integration workflow on GitHub or GitLab
+```{challenge} Exercise CI-1: Create and use a workflow on GitHub or a pipeline on GitLab
 
 In this exercise, we will:
 
@@ -27,7 +30,7 @@ In this exercise, we will:
 - **F.** Create a test to increase the code coverage of our tests.
 ```
 
-### Prerequisites
+## Prerequisites
 
 If you are new to Git, you can find a step-by-step guide to
 setting up repositories and making commits in
@@ -36,9 +39,9 @@ If you are new to pull requests / merge requests, you can learn all about them
 in the [Collaborative Git lesson](https://coderefinery.github.io/git-collaborative/).
 
 
-### Step 1: Create a new repository on GitHub/GitLab OR fork from the example repo
+## Step 1: Create a new repository on GitHub/GitLab OR fork from the example repo
 
-#### Create a new repository
+### Create a new repository
 
 - Begin by creating a repository called (for example) *example-ci*.
 - **Before** you create the repository, select **"Initialize this repository
@@ -191,14 +194,14 @@ in the [Collaborative Git lesson](https://coderefinery.github.io/git-collaborati
 
 
 
-#### Fork and clone an existing example repository
+### Fork and clone an existing example repository
 
 - Fork the example repo. There are two options one for [Python](https://github.com/AaltoRSE/PyTestingExample) and one for [R](https://github.com/AaltoRSE/RTestingExample).
 - Clone your fork (`git clone git@github.com:<yourGitID>/<Py/R>TestingExample.git`).
 
 
 
-### Step 2: Run tests locally
+## Step 2: Run tests locally
 `````{tabs}
   ````{group-tab} Python
   You can now run your tests locally with
@@ -216,7 +219,7 @@ in the [Collaborative Git lesson](https://coderefinery.github.io/git-collaborati
 
 
 
-### Step 3: Enable automated testing
+## Step 3: Enable automated testing
 
 `````{tabs}
   ````{group-tab} GitHub-Python
@@ -439,7 +442,7 @@ in the [Collaborative Git lesson](https://coderefinery.github.io/git-collaborati
 `````
 
 
-### Step 4: Verify that tests have been automatically run
+## Step 4: Verify that tests have been automatically run
 
 `````{tabs}
    ````{group-tab} GitHub-Python
@@ -488,7 +491,7 @@ in the [Collaborative Git lesson](https://coderefinery.github.io/git-collaborati
 `````
 
 
-### Step 5: Add a test which reveals a problem
+## Step 5: Add a test which reveals a problem
 
 After you committed the workflow file, your GitHub/GitLab repository will be ahead of
 your local cloned repository.  Update your local cloned repository:
@@ -505,14 +508,14 @@ Verify that the test suite now fails on the "Actions" tab (GitHub)
 or the "CI/CD->Pipelines" tab (GitLab).
 
 
-### Step 6: Open an issue on GitHub/GitLab
+## Step 6: Open an issue on GitHub/GitLab
 
 Open a new issue in your repository about the broken test (click the
 "Issues" button on GitHub or GitLab and write a title for the issue).
 The plan is that we will fix the issue through a pull/merge request.
 
 
-### Step 7: Fix the broken test
+## Step 7: Fix the broken test
 
 Now fix the code **on a new branch**, you can call it `yourname/bugfix`.
 After you have fixed the code on the new branch, commit the following
@@ -531,7 +534,7 @@ you try to fix issue number 1).
 Then push to your repository.
 
 
-### Step 8: Open a pull request (GitHub)/ merge request (GitLab)
+## Step 8: Open a pull request (GitHub)/ merge request (GitLab)
 
 Go back to the repository on GitHub or GitLab and open a pull/merge
 request. **In a collaborative setting, you could request a code
@@ -544,7 +547,7 @@ can still add it to the pull/merge request: `my pull/merge request
 title, closes #1`.
 
 
-### Step 9: Accept the pull/merge request
+## Step 9: Accept the pull/merge request
 
 Observe how accepting the pull/merge request automatically closes the issue (provided
 the commit message or the pull/merge request contained the correct issue number).
@@ -555,14 +558,14 @@ See also:
 
 Discuss whether this is a useful feature. And if it is, why do you think is it useful?
 
-### Step 10: Increase your code coverage
+## Step 10: Increase your code coverage
 
 We are currently missing several functions in our tests. Write a test for the `multiply` function in a new branch and create a pull request.
 On Python you can directly observe the increase in code coverage.
 On R you can have a look at the action (`Actions -> last run of your action -> Select a job -> Test coverage`). If you compare this with the
 previous run, you should see an increase once the update is in.
 
-### Step 11 (optional): Repeat steps 5-9 for the `convert_fahrenheit_to_celsius` function:
+## Step 11 (optional): Repeat steps 5-9 for the `convert_fahrenheit_to_celsius` function:
 
 Repetition helps learning, so let's do the testing again for our `convert_fahrenheit_to_celsius` function.
 Uncomment the test for the `convert_fahrenheit_to_celsius` function and repeat steps 5 to 9 fixing the bug this test exposes.