4174 only read indats using the indat class by clmould · Pull Request #4523 · ukaea/PROCESS

clmould · 2026-08-13T16:21:25Z

Description

Moves validate_input to the InDat class and rename to check_obsolete_variables
parse_input_file no longer re-reads the file line by line and instead uses the data read in by InDat

Checklist

I confirm that I have completed the following checks:

My changes follow the PROCESS style guide
I have justified any large differences in the regression tests caused by this pull request in the comments.
I have added new tests where appropriate for the changes I have made.
If I have had to change any existing unit or integration tests, I have justified this change in the pull request comments.
If I have made documentation changes, I have checked they render correctly.
I have added documentation for my change, if appropriate.

codecov-commenter · 2026-09-02T13:08:16Z

Codecov Report

❌ Patch coverage is 73.24841% with 42 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.32%. Comparing base (e122edb) to head (10f890a).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
process/core/io/in_dat/base.py	61.38%	39 Missing ⚠️
process/core/input.py	93.61%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4523      +/-   ##
==========================================
+ Coverage   50.26%   50.32%   +0.06%     
==========================================
  Files         181      181              
  Lines       30632    30601      -31     
==========================================
+ Hits        15396    15399       +3     
+ Misses      15236    15202      -34

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

je-cook

First round, I think there will be a few subsequent reviews, its a lot to unpick!

je-cook

a few cleanup ones

timothy-nunn

A couple more in addition to James.

timothy-nunn · 2026-09-25T08:35:30Z

+        if variable_name == "bounds":
+            boundl = {}
+            boundu = {}
+            for index, bounds_dict in info.value.items():
+                try:
+                    idx = int(index)
+                except ValueError as e:
+                    raise ProcessValidationError(
+                        "The bounds need to be of the form boundl/u(index) = value"
+                    ) from e
+                if "l" in bounds_dict:
+                    boundl[idx] = bounds_dict["l"]
+                if "u" in bounds_dict:
+                    boundu[idx] = bounds_dict["u"]
+            # TODO could change them into arrays here?
+
+            info_boundl = info
+            info_boundl.value = boundl
+            info_boundl.name = "boundl"
+            variables = set_on_datastructure(
+                variable_name="boundl",
+                info=info_boundl,
+                data_structure_obj=data_structure_obj,
+                variables=variables,
+            )

-        # matches (variable name, array index, value)
-        # NOTE: array index is Fortran-based hence starts at 1.
-        line_match = re.match(
-            r"([a-zA-Z0-9_]+)(?:\(([0-9]+)\))?[ ]*=[ ]*([ +\-a-zA-Z0-9.,]+).*",
-            stripped_line,
-        )
+            info_boundu = info
+            info_boundu.value = boundu
+            info_boundu.name = "boundu"


Could this be an additional action, rather than an if statement in the parser?

Have discussed and decided, for now, to add a new function _handle_bounds - the additional actions are not performed until after the variable validating and cleaning, which means it would be tricky to move this to an additional action and still be able to clean and validate the variable. Can always change once we have decided on how to handle bounds in future

…bles

…undl(index) = value , and use f_nd_impurity_electrons instead of bound in a test for input array remove outdated comment and add TODO docstring to fill in after review

… update any associated mfiles

…d updating

je-cook

Looks good just the one structural comment. I think we can go further to collating some of these thing under one wrapper.
Move the init.py file into the core/data_structure folder I think. The contents of that file is 90% the check_process. I would rename the file to check_input.py. The other functions in that file can go onto the DataStructure object in data_structure/base.py except for get_git_summary which I would put in as a standalone function in the file. In my head the check_process function will eventually disappear as the validation moves onto the individual data structure objects. Right now we should just isolate it.

All of the functions in input.py can also move to the data structure. I would leave InputVariable and the associated dictionary where it is

timothy-nunn · 2026-10-29T10:42:25Z

Why is this (and examples/data/large_tokamak_varyrun_IN.DAT) changing?

because these are duplicate variables that threw an error as this PR raises an error now when there are duplicate vars

timothy-nunn · 2026-10-29T11:00:05Z

+        if variable_name == "bounds":
+            variables = _handle_bounds(info, data_structure_obj, variables)


Sorry I still don't like the custom handling of the bounds with if statements. I have thought of a way of doing this:

At the end of InDat.read_in_dat you should have access to self.data["bounds"]. You could process this dictionary into the respective boundl and boundu arrays and then you wouldn't need all of this custom stuff in the parser because the variables could be parsed essentially as they were before

…s before setting on data structure so as to not overwrite the bounds with unrealistic values

clmould linked an issue Aug 13, 2026 that may be closed by this pull request

Only read IN.DATs using the InDat class #4174

Open

1 of 2 tasks

clmould force-pushed the 4174-only-read-indats-using-the-indat-class branch from bc34539 to eda15dc Compare August 13, 2026 16:26

clmould mentioned this pull request Aug 14, 2026

Only read IN.DATs using the InDat class #4174

Open

1 of 2 tasks

clmould force-pushed the 4174-only-read-indats-using-the-indat-class branch 3 times, most recently from 72a627d to e8a494b Compare August 27, 2026 13:14

clmould force-pushed the 4174-only-read-indats-using-the-indat-class branch 6 times, most recently from 9816a42 to d4d920c Compare September 2, 2026 12:47

clmould force-pushed the 4174-only-read-indats-using-the-indat-class branch 2 times, most recently from 756ea41 to 698f4a5 Compare September 11, 2026 18:08

clmould force-pushed the 4174-only-read-indats-using-the-indat-class branch from fec93fb to bcd063a Compare September 21, 2026 12:27

clmould marked this pull request as ready for review September 22, 2026 14:06

clmould requested a review from a team as a code owner September 22, 2026 14:06

clmould assigned timothy-nunn and je-cook Sep 22, 2026

je-cook requested changes Sep 25, 2026

View reviewed changes

Comment thread process/core/io/in_dat/base.py Resolved

Comment thread process/core/io/in_dat/base.py Resolved

Comment thread process/core/io/in_dat/base.py Resolved

Comment thread process/core/input.py Resolved

je-cook reviewed Sep 25, 2026

View reviewed changes

timothy-nunn reviewed Sep 25, 2026

View reviewed changes

clmould force-pushed the 4174-only-read-indats-using-the-indat-class branch from c9765aa to 43c5333 Compare October 23, 2026 16:43

clmould marked this pull request as draft October 23, 2026 18:12

clmould force-pushed the 4174-only-read-indats-using-the-indat-class branch from cbd35eb to 3a44a01 Compare October 23, 2026 18:12

je-cook added the Input/Output Files Issues related to the input and output data files label Oct 26, 2026

clmould added 3 commits October 28, 2026 17:24

move validate_input to InDat class and rename to check_obsolete_varia…

459c0ec

…bles

Fix test_main.py failures from moving obsolete var check to InDat class

5fb7be4

check_process unused arg _inputs

eb6548f

clmould added 20 commits October 28, 2026 17:24

remove unused import

33d381c

WIP parse_input_file is working

4e2858f

fix failing tests - add error message if bounds aren't of the form bo…

7a32fe2

…undl(index) = value , and use f_nd_impurity_electrons instead of bound in a test for input array remove outdated comment and add TODO docstring to fill in after review

inform of duplicate variables and update these in our test files, and…

e958f57

… update any associated mfiles

remove refs to outdated vars impurity_radiation/imp_rad and vmec

82db0ff

add missing docstring in input.py

2480c8b

remove unused line number in in_dat/base.py

a56196a

tidy up variable_constraint_type_check, sort evals, reword error msg

5adee89

remove calls to unnecessary function

fe8388f

accidentally changed

1f3a20c

undo mistake

d89d0e9

inputs , not _inputs

ac5bacc

remove unnecessary code

b35da5d

check obsolete in read_in_dat, split into smaller fns for checking an…

9ba607f

…d updating

james comments

a1b4c12

create _handle_bounds fn

caf1a78

docstrign

7a64938

fix pre-commit failure

c8bf2a1

remove duplicate vars from example data files

ea735bd

improve comment

e2116ac

clmould force-pushed the 4174-only-read-indats-using-the-indat-class branch from 6788335 to e2116ac Compare October 28, 2026 17:24

clmould marked this pull request as ready for review October 28, 2026 17:34

clmould requested review from je-cook and timothy-nunn October 28, 2026 17:34

je-cook reviewed Oct 29, 2026

View reviewed changes

Comment thread process/core/init.py Resolved

timothy-nunn requested changes Oct 29, 2026

View reviewed changes

clmould mentioned this pull request Oct 29, 2026

Restructuring init.py & input.py #4719

Open

clmould added 3 commits October 29, 2026 17:39

split bounds into boundu and boundl arrays, perform additional action…

91d2c8c

…s before setting on data structure so as to not overwrite the bounds with unrealistic values

add back error message if bounds are in the wrong form

643f870

fix errors

10f890a

		if variable_name == "bounds":
		variables = _handle_bounds(info, data_structure_obj, variables)

Conversation

clmould commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

codecov-commenter commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

je-cook left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

je-cook left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

timothy-nunn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

timothy-nunn Sep 25, 2026

Choose a reason for hiding this comment

Uh oh!

clmould Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

je-cook left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

timothy-nunn Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

clmould Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

timothy-nunn Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

clmould commented Aug 13, 2026 •

edited

Loading

codecov-commenter commented Sep 2, 2026 •

edited

Loading

je-cook left a comment •

edited

Loading