feat: zero-copy views, part 2: oneofs, repeated fields and wrapper types

docs/codec-protobuf.md

@@ -285,30 +285,50 @@ own messages before turning it on; `BlockItemKindBench` in the integration tests
 ### Zero-Copy Views (`pbj.views`)
 
 The model gets a nested, reusable `View` class that reads fields in place from encoded bytes instead of parsing them
-into objects:
+into objects. [usage-guide.md](usage-guide.md#zero-copy-views) shows how to use one and the reuse pitfall.
 
 ```java
 final Order.View view = new Order.View();
 for (Bytes bytes : orders) {
-    view.wrap(bytes);          // or wrap(byte[], offset, length)
+    view.wrap(bytes);          // or wrap(bytes, strictMode, maxDepth, maxSize), as for parse()
     total += view.amount();    // numbers and enums are decoded once, by wrap
     if (view.hasCustomer()) {
         names.add(view.customer().name());   // a child view, re-wrapped with the parent
     }
+    for (var line = view.lines(); line.next(); ) {
+        count += line.value().quantity();
+    }
 }
 ```
 
 `wrap` indexes the message in one pass, with the same in-order fast path and the same checks of tags, wire types,
-lengths, bools and UTF-8 as parsing; a malformed message throws `ParseException`. Numbers are decoded into fields of
-the view, and strings, bytes and nested messages are recorded as offsets and lengths. String and bytes fields also have
-`xxxArray()`, `xxxOffset()`, `xxxLength()` and `xxxEquals(byte[])`, which read them without allocating; `xxx()` builds a
-`String` or `Bytes`. An enum field's `xxxProtoOrdinal()` returns the raw number. `toModel()` parses the wrapped bytes
-into a model object. Wrapping and reading numbers, enums, nested views and offsets allocate nothing.
-
-A view and its child views are reused, so a child view returned earlier reads whatever its parent was last wrapped
-over. The wrapped bytes must not change while a view reads them.
+lengths, bools and UTF-8 as parsing; a malformed message throws `ParseException`. `wrap(bytes, strictMode, maxDepth,
+maxSize)` applies the same limits and strict mode as `parse` with those arguments, and passes them down to nested
+views. In strict mode `wrap` also validates every nested message, repeated message element and wrapper type value, at
+any depth, so it fails fast on whatever `parseStrict` rejects. Otherwise they are validated when they are read, so a
+malformed one throws from its accessor or cursor rather than from `wrap`, and one that is never read is never
+validated: **a non-strict `wrap` is not a validation of untrusted input**; use strict mode or `parse` for that.
 
-A message gets a view only if every field is a singular scalar, string, bytes, enum or message, and every message
-field's type also gets a view. Messages with repeated fields, maps, oneofs or `google.protobuf` wrapper types get no
-view.
+What a view has for each kind of field:
 
+| Field | Accessors |
+|---|---|
+| Number, bool | `xxx()`, decoded by `wrap` |
+| Enum | `xxx()`, and `xxxProtoOrdinal()` for the raw number |
+| String, bytes | `xxx()` builds a `String` or `Bytes` (a view of the wrapped `Bytes`, or a copy when wrapping an array); `xxxLength()`, `xxxEquals(byte[])` and `xxxCopyTo(byte[], int)` read it without allocating |
+| Message | `xxx()` returns a child view, created once and re-wrapped with the parent |
+| `google.protobuf` wrapper type | `xxx()` returns the value, or null when absent |
+| Repeated | `xxxCount()`, and `xxx()` returns a reusable cursor: `next()` moves to each element, `value()` reads it (and `valueLength()`, `valueEquals(byte[])`, `valueCopyTo(byte[], int)` for strings and bytes) |
+| Oneof | `xxxKind()` returns which alternative is set; each alternative has the accessors of a singular field |
+
+Message and wrapper type fields also have `hasXxx()`. `toModel()` parses the wrapped bytes into a model object. Wrapping
+and reading numbers, enums, nested views, cursors and offsets allocate nothing.
+
+A view and its child views and cursors are reused, so a child view returned earlier reads whatever its parent was
+last wrapped over. The wrapped bytes must not change while a view reads them. Views never hand out the wrapped array:
+`Bytes` they return, and `toModel()`, share the wrapped `Bytes`, which is immutable, or copy from a wrapped array, so
+they stay the same when a caller reuses its array.
+
+A message gets a view only if it has no maps or repeated wrapper types, and every message field's type also gets a
+view (messages may contain each other). When a message asks for a view in its own or its file's comments and gets
+none, the build prints why. A message with a view that declares its own nested message or enum named `View` fails the build.

docs/usage-guide.md

@@ -373,6 +373,48 @@ HelloRequest msg = HelloRequest.PROTOBUF.parse(
 
 Strict mode throws `UnknownFieldException` on unrecognized fields — useful for validation.
 
+### Zero-Copy Views
+
+With `pbj.views`, a model gets a nested `View` class that reads fields in place from the encoded bytes, without
+parsing them into objects. A view is reusable: wrap it again for the next message.
+
+```java
+final Order.View view = new Order.View();
+for (Bytes bytes : orders) {
+    view.wrap(bytes);   // or wrap(bytes, strictMode, maxDepth, maxSize), as for parse()
+    total += view.amount();
+    for (var line = view.lines(); line.next(); ) {
+        count += line.value().quantity();
+    }
+}
+```
+
+`wrap` validates the message as `parse` does, with the same limits and strict mode. In strict mode it validates
+nested messages, repeated message elements and wrapper type values too, at any depth, before returning. Otherwise they
+are validated when they are read, so a malformed one throws from its accessor or cursor rather than from `wrap`, and
+one that is never read is never validated: to validate untrusted input, wrap it in strict mode or parse it. Messages with maps, or that contain a message without a view, get no view; the build
+says why for messages that ask for one in their comments.
+
+**Views and cursors are reused, so what they return changes under you.** A nested accessor returns the same child view
+every time, re-wrapped when the parent is, and a repeated field's cursor re-wraps one child view for each element.
+Keeping a reference and reading it later reads whatever the view was last wrapped over:
+
+```java
+// Wrong: every element of the list is the same child view, wrapped over the last line
+List<OrderLine.View> lines = new ArrayList<>();
+for (var line = view.lines(); line.next(); ) {
+    lines.add(line.value());
+}
+
+// Right: copy what you need, or parse the element into a model object
+List<OrderLine> lines = new ArrayList<>();
+for (var line = view.lines(); line.next(); ) {
+    lines.add(line.value().toModel());
+}
+```
+
+The wrapped bytes must not change while a view reads them.
+
 ### Streaming I/O
 
 Use `ReadableStreamingData` and `WritableStreamingData` for file or network I/O:

pbj-core/pbj-compiler/src/main/java/com/hedera/pbj/compiler/impl/LookupHelper.java

@@ -19,6 +19,7 @@
 import java.nio.file.FileSystems;
 import java.nio.file.Path;
 import java.util.ArrayDeque;
+import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.HashMap;
 import java.util.HashSet;
@@ -326,28 +327,144 @@ public boolean hasReverseWriter(String modelPackage, String completeClass) {
         return message != null && hasReverseWriter(messageDefinitions.get(message));
     }
 
-    /** Whether each message gets a view, by fully qualified proto name, computed on demand. */
-    private final Map<String, Boolean> views = new HashMap<>();
+    /** The messages that get a view, by fully qualified proto name, computed on first use. */
+    private Set<String> views;
+
+    /**
+     * Why each message that asks for a view in its own or its file's comments gets none, by fully qualified proto name.
+     * Messages that have {@code pbj.views} only from the project are left out, so turning everything on doesn't report
+     * every message with a map.
+     */
+    private final Map<String, String> viewDenials = new HashMap<>();
 
     /**
      * Whether a message gets a zero-copy view class: it has {@code pbj.views} (from its comments, its file's, or the
-     * project), every field is singular (repeated fields, maps and oneofs are not supported yet), and every message-typed
-     * field's type, if generated in this build, gets a view too.
+     * project), it has no maps or repeated {@code google.protobuf} wrapper types (not supported yet), and every message-typed
+     * field's type is generated in this build and gets a view too.
      *
      * @param message the message
      * @return true if a view is generated for it
      */
     public boolean hasView(MessageDefContext message) {
-        final String name = getFullyQualifiedProtoNameForContext(message);
-        final Boolean known = views.get(name);
-        if (known != null) {
-            return known;
+        return views().contains(getFullyQualifiedProtoNameForContext(message));
+    }
+
+    /**
+     * Why a message that has {@code pbj.views} gets no view.
+     *
+     * @param message the message
+     * @return the reason, or null if the message gets a view or does not ask for one
+     */
+    public String viewDenial(MessageDefContext message) {
+        views();
+        return viewDenials.get(getFullyQualifiedProtoNameForContext(message));
+    }
+
+    /**
+     * Compute which messages get views, as a greatest fixed point: start with every message that asks for a view and
+     * supports one on its own, then drop those with a message-typed field whose type was dropped, until nothing
+     * changes. Messages that contain each other, directly or through others, keep their views only if all of them can.
+     */
+    private Set<String> views() {
+        if (views != null) {
+            return views;
         }
-        // provisionally true, so messages that contain each other don't recurse forever
-        views.put(name, Boolean.TRUE);
-        final boolean result = option(name, message, PerformanceOptions::views) && viewSupports(name, message);
-        views.put(name, result);
-        return result;
+        final Set<String> generated = new HashSet<>(messagesByModelClass.values());
+        // each candidate's message-typed fields: field name and type, which must get views too
+        final Map<String, Map<String, String>> candidates = new HashMap<>();
+        for (Map.Entry<String, MessageDefContext> entry : messageDefinitions.entrySet()) {
+            final String name = entry.getKey();
+            if (generated.contains(name) && option(name, entry.getValue(), PerformanceOptions::views)) {
+                final Map<String, String> nested = viewNestedTypes(name, entry.getValue(), generated);
+                if (nested != null) {
+                    candidates.put(name, nested);
+                }
+            }
+        }
+        boolean changed = true;
+        while (changed) {
+            changed = false;
+            for (var it = candidates.entrySet().iterator(); it.hasNext(); ) {
+                final var candidate = it.next();
+                for (Map.Entry<String, String> field : candidate.getValue().entrySet()) {
+                    if (!candidates.containsKey(field.getValue())) {
+                        viewDenials.put(
+                                candidate.getKey(),
+                                "field '%s' has type %s, which gets no view"
+                                        .formatted(field.getKey(), field.getValue()));
+                        it.remove();
+                        changed = true;
+                        break;
+                    }
+                }
+            }
+        }
+        viewDenials
+                .keySet()
+                .removeIf(denied -> !PerformanceOptions.resolve(
+                        levels(denied, messageDefinitions.get(denied)).subList(0, 2),
+                        PerformanceOptions::views,
+                        true,
+                        false));
+        views = candidates.keySet();
+        return views;
+    }
+
+    /**
+     * The message-typed fields of a message that asks for a view, by field name, with their types' proto names; or
+     * null, with the reason recorded, if the message can't have a view whatever its nested types get.
+     */
+    private Map<String, String> viewNestedTypes(String name, MessageDefContext message, Set<String> generated) {
+        final Map<String, String> nested = new HashMap<>();
+        for (MessageElementContext element : message.messageBody().messageElement()) {
+            if (element.mapField() != null) {
+                viewDenials.put(
+                        name,
+                        "map field '%s' (maps are not supported by views)"
+                                .formatted(element.mapField().mapName().getText()));
+                return null;
+            }
+            final List<Map.Entry<String, Protobuf3Parser.Type_Context>> fields = new ArrayList<>();
+            if (element.oneof() != null) {
+                for (var alternative : element.oneof().oneofField()) {
+                    fields.add(Map.entry(alternative.fieldName().getText(), alternative.type_()));
+                }
+            }
+            final var field = element.field();
+            if (field != null && field.fieldName() != null) {
+                // repeated wrapper types are not supported
+                final var type = field.type_().messageType();
+                if (field.REPEATED() != null
+                        && type != null
+                        && WRAPPER_TYPES.contains(type.messageName().getText())) {
+                    viewDenials.put(
+                            name,
+                            "repeated wrapper type field '%s' (not supported by views)"
+                                    .formatted(field.fieldName().getText()));
+                    return null;
+                }
+                fields.add(Map.entry(field.fieldName().getText(), field.type_()));
+            }
+            for (Map.Entry<String, Protobuf3Parser.Type_Context> f : fields) {
+                final var type = f.getValue().messageType();
+                if (type == null || WRAPPER_TYPES.contains(type.messageName().getText())) {
+                    continue;
+                }
+                final String nestedName = getFullyQualifiedProtoName(messageFiles.get(name), type);
+                if (enumNames.contains(nestedName)) {
+                    continue;
+                }
+                if (!generated.contains(nestedName)) {
+                    viewDenials.put(
+                            name,
+                            "field '%s' has type %s, which is not generated in this build"
+                                    .formatted(f.getKey(), nestedName));
+                    return null;
+                }
+                nested.put(f.getKey(), nestedName);
+            }
+        }
+        return nested;
     }
 
     /**
@@ -362,7 +479,7 @@ public boolean hasView(String modelPackage, String completeClass) {
         return message != null && hasView(messageDefinitions.get(message));
     }
 
-    /** The {@code google.protobuf} wrapper types, which views do not support yet. */
+    /** The {@code google.protobuf} wrapper types, which views read as boxed values. */
     private static final Set<String> WRAPPER_TYPES = Set.of(
             "StringValue",
             "BoolValue",
@@ -374,52 +491,25 @@ public boolean hasView(String modelPackage, String completeClass) {
             "DoubleValue",
             "BytesValue");
 
-    private boolean viewSupports(String name, MessageDefContext message) {
-        for (MessageElementContext element : message.messageBody().messageElement()) {
-            if (element.oneof() != null || element.mapField() != null) {
-                return false;
-            }
-            final var field = element.field();
-            if (field == null || field.fieldName() == null) {
-                continue;
-            }
-            if (field.REPEATED() != null) {
-                return false;
-            }
-            final var type = field.type_().messageType();
-            if (type == null) {
-                continue;
-            }
-            if (WRAPPER_TYPES.contains(type.messageName().getText())) {
-                return false;
-            }
-            final String nestedName = getFullyQualifiedProtoName(messageFiles.get(name), type);
-            if (enumNames.contains(nestedName)) {
-                continue;
-            }
-            final MessageDefContext nested = messageDefinitions.get(nestedName);
-            if (nested == null || !messagesByModelClass.containsValue(nestedName) || !hasView(nested)) {
-                return false;
-            }
-        }
-        return true;
-    }
-
     /** Whether a message has {@code pbj.fast_parse}, from its comments, its file's, or the project. */
     private boolean fastParse(String name, MessageDefContext message) {
         return option(name, message, PerformanceOptions::fastParse);
     }
 
     /** Resolve a boolean fast code generation option for a message, from its comments, its file's, or the project. */
     private boolean option(String name, MessageDefContext message, Function<PerformanceOptions, Boolean> option) {
+        return PerformanceOptions.resolve(levels(name, message), option, true, false);
+    }
+
+    /** The options that apply to a message, in order of precedence: its comments, its file's, and the project's. */
+    private List<PerformanceOptions> levels(String name, MessageDefContext message) {
         final File file = messageFiles.get(name);
-        final List<PerformanceOptions> levels = List.of(
+        return List.of(
                 PerformanceOptions.fromComments(message.optionComment(), "message " + name),
                 file == null
                         ? PerformanceOptions.NONE
                         : fileOptions.getOrDefault(file.getAbsolutePath(), PerformanceOptions.NONE),
                 projectOptions);
-        return PerformanceOptions.resolve(levels, option, true, false);
     }
 
     /** Read a boolean PBJ message option from its special comments, defaulting to false. */