Performance Comparison of File Processing Methods (FreePascal)

This repository contains three demonstration programs written in FreePascal / Object Pascal designed to benchmark the performance of different file reading strategies for text-based data.

The goal is to measure and contrast the time efficiency of three common Object Pascal file I/O methods:

• TStringList.LoadFromFile: Loading the entire file into a dynamic array of strings (in-memory).
• Standard Pascal File I/O (AssignFile/ReadLn): Unbuffered, sequential line-by-line reading.
• TStreamReader: Optimized, buffered reading via the FCL/LCL stream classes.

Dataset Source

The tests use a substantial sample CSV file to ensure a consistent, real-world data source for benchmarking.

• Dataset Used: Organizations Dataset (large CSV file)
• Source URL: https://www.datablist.com/learn/csv/download-sample-csv-files#organizations-dataset

Before running the tests, please download the CSV file and place it in the designated input directory.

---

Programs Overview (FreePascal Implementation)

Program Name	FreePascal Method	Uses Unit(s)	Key Processing Logic
loadtest1	TStringList	Classes	Uses the built-in TStringList.LoadFromFile(FileName) method to load the entire file into memory as separate strings. Processing then iterates through the StringList.Strings[] array.
loadtest2	Standard I/O	System	Uses classic Pascal procedures: AssignFile, Reset, and a loop with while not EOF(F) do ReadLn(F, Line). This is typically the least performant due to unbuffered, line-by-line kernel calls.
loadtest3	TStreamReader	Classes (or SysUtils)	Creates a TStreamReader (usually wrapped around a TFileStream) and uses a loop with while not StreamReader.EndOfStream do StreamReader.ReadLine. This approach uses internal buffering for speed.

Note on Memory: While loadtest1 is often the easiest to code, it is the most memory-intensive and is prone to EOutOfMemory errors when processing very large files (several GBs) on 32-bit systems. loadtest3 manages memory more gracefully as it processes data in chunks.