Repository navigation
Stream stdEnumerator IEnumVARIANT sources without caching - #186
Conversation
- Fetch Windows source values on demand and cache them for repeatable traversal - Return empty enumerators for non-positive First counts - Add Excel regression coverage and document the behavior
|
Relates to #25 also |
There was a problem hiding this comment.
Could we make this sequential, fetching items one at a time rather than loading the full collection at once? That would keep memory use down and let operations like counting process items as they arrive, instead of materializing everything in RAM first. A fully lazy enumerator is a long-term goal, but this approach would be a step in that direction.
| Select Case This.mode | ||
| Case FromIEnumVariant | ||
| #If Not Mac Then | ||
| Do While iCurrentIndex > This.Cache.Count And Not This.Exhausted |
There was a problem hiding this comment.
Do while loop here doesn't really make sense, and doesn't achieve the lazyness goals of the class. Sequential calls: the operation’s outer loop repeatedly calls NextItem; each NextItem should call COM’s Next once.
The better design is:
- Each operation obtains its own source cursor.
- Each
NextItemretrieves one value and passes it to the operation. - Counting retains only the current value and running total.
.First(100)retains its 100 returned results.- Repeating an operation obtains a fresh cursor and enumerates again.
sancarn
left a comment
There was a problem hiding this comment.
Please use enums rather than call indexes
| .Started = True | ||
| End If | ||
| Dim value As Variant, fetched As Long, hr As Long | ||
| hr = enumCall(ObjPtr(.Cursor), 3, 1&, VarPtr(value), VarPtr(fetched)) |
There was a problem hiding this comment.
Instead of calling a random index let's use an enum
|
Looks good 👍 |
CreateFromIEnumVariant currently reads the entire source twice during construction. Replace that eager snapshot with independent streaming traversals on Windows: each operation acquires its own IEnumVARIANT cursor and reads one item per Next call, without caching consumed source values.
Fixes #54.
Breaking changes
Windows construction previously captured an immediate snapshot. Operations now reread the live source, so mutations can affect later traversals and source errors occur during consumption. Callers needing stable replay, mutation isolation or eager error timing should capture immediately:
Set e = stdEnumerator.CreateFromArray(stdEnumerator.CreateFromIEnumVariant(o).AsArray())First with a non-positive count previously returned the entire source and now returns empty. Callers relying on First(0) or First(-1) to copy all values should use
stdEnumerator.CreateFromArray(e.AsArray()).Public signatures, argument names/defaults, keys and callback contracts are unchanged.
Validation
Windows 32-bit, legacy VBA and Mac runtime paths remain untested.