Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ v0.3.1 (Unreleased)
* **`toStartOf*` date-time functions** via `EF.Functions`: `ToStartOfYear`, `ToStartOfQuarter`, `ToStartOfMonth`, `ToStartOfWeek` (with optional week `mode`), `ToStartOfDay`, `ToStartOfHour`, `ToStartOfMinute`, `ToStartOfSecond`, the fixed buckets `ToStartOfFiveMinutes` / `ToStartOfTenMinutes` / `ToStartOfFifteenMinutes`, and the general `ToStartOfInterval(source, value, unit)`. Each maps to the matching ClickHouse function and works in `GROUP BY`. Return types follow ClickHouse: the calendar buckets (`Year`/`Quarter`/`Month`/`Week`) return `Date`, the day/hour/minute buckets return `DateTime`, and `ToStartOfSecond` returns `DateTime64`. All accept `DateTime`/`DateTime64` columns, and the plain truncation functions also accept `DateOnly`; `ToStartOfInterval` requires a `DateTime`/`DateTime64` column on older ClickHouse, which rejects a `DateOnly` (Date/Date32) source with `Illegal type Date32 of 1st argument` for every unit; recent versions accept it. `ToStartOfInterval` takes a `ClickHouseInterval` unit (`Second`…`Year`) and emits `toStartOfInterval(source, toInterval<unit>(value))`; the unit must be a constant so it can be translated. The default `Date`/`DateTime` result types only span 1970–2149/2106, so ClickHouse narrows out-of-range values — enable `enable_extended_results_for_datetime_functions` (e.g. `set_enable_extended_results_for_datetime_functions=1` in the connection string) for range-preserving `Date32`/`DateTime64` results.

### Bug fixes
* **Code-only driver settings on `UseClickHouse(DbConnection)` and `UseClickHouse(DbDataSource)`** are no longer lost. When the provider added `join_use_nulls` on open, and when it made the master connection for `EnsureCreated`, `EnsureDeleted` and migrations, it rebuilt the driver settings from the connection string. That dropped every setting with no connection-string key, such as `SkipServerCertificateValidation`, `BearerToken`, `CustomHeaders`, `HttpClient` and `LoggerFactory`, so a server with a self-signed certificate failed with an untrusted-root error. The provider now copies the `ClickHouseClientSettings` object instead. ([#50](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/50))
* `ToStartOfWeek` now rejects row-dependent week modes during query translation, and `ToStartOfInterval` likewise rejects row-dependent interval sizes. ClickHouse requires these operands to be constant for the query; literals and captured query parameters remain supported.
* `Sum`/`SumAsync` over a `double` or `float` column no longer throws `InvalidCastException`. EF Core wraps a top-level aggregate so the empty case returns `0`, supplying that fallback as a boxed `Int32` carrying the `Float64`/`Float32` mapping; the literal generators now convert rather than unbox. The `Float32` read path also converts, since ClickHouse widens `sum(Float32)` to `Float64` (which the driver's `GetFloat()` refuses to downcast). ([#46](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/46))
* **SummingMergeTree with multiple sum columns**: `HasSummingMergeTreeEngine("A", "B")` now generates valid DDL (`SummingMergeTree((A, B))`). Previously it emitted a comma-separated argument list (`SummingMergeTree(A, B)`), which ClickHouse rejects with `NUMBER_OF_ARGUMENTS_DOESNT_MATCH`. Single-column and no-column usage are unaffected.
Expand Down
1 change: 1 addition & 0 deletions RELEASENOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ v0.3.1 (Unreleased)
* **`toStartOf*` date-time functions** are now translatable through `EF.Functions`, covering the full family: `ToStartOfYear`, `ToStartOfQuarter`, `ToStartOfMonth`, `ToStartOfWeek` (with an optional ClickHouse week `mode`), `ToStartOfDay`, `ToStartOfHour`, `ToStartOfMinute`, `ToStartOfSecond`, the fixed buckets `ToStartOfFiveMinutes` / `ToStartOfTenMinutes` / `ToStartOfFifteenMinutes`, and the general-purpose `ToStartOfInterval(source, value, unit)`. They compose in `GROUP BY` for time bucketing. Input and return types follow ClickHouse: the calendar buckets (`ToStartOfYear`/`Quarter`/`Month`/`Week`) return `Date`, `ToStartOfDay` and the hour/minute buckets return `DateTime`, and `ToStartOfSecond` returns `DateTime64`. All accept `DateTime`/`DateTime64` columns, and the plain truncation functions also accept `DateOnly` (Date/Date32); `ToStartOfInterval` is the exception — older ClickHouse rejects a `DateOnly` (Date/Date32) source with `Illegal type Date32 of 1st argument`, while recent versions accept it, so prefer a `DateTime`/`DateTime64` column for interval bucketing. `ToStartOfInterval` uses a `ClickHouseInterval` enum for the unit and is emitted as `toStartOfInterval(source, toInterval<unit>(value))`. Because the default result types (`Date`/`DateTime`) only span 1970–2149/2106, ClickHouse narrows out-of-range values (pre-1970 clamps to the epoch or wraps around); enable `enable_extended_results_for_datetime_functions` in your session (e.g. `set_enable_extended_results_for_datetime_functions=1` in the connection string) to get range-preserving `Date32`/`DateTime64` results.

### Bug fixes
* **Settings you set in code now apply when you pass a `ClickHouseConnection` or `ClickHouseDataSource` to `UseClickHouse`.** Settings that have no connection-string key, such as `SkipServerCertificateValidation`, `BearerToken`, `CustomHeaders` or a custom `HttpClient`, were lost when the provider opened the connection or created the database. For example, a server with a self-signed certificate failed with an untrusted-root error even with `SkipServerCertificateValidation = true`. They are now kept on every path, including `EnsureCreated`, `EnsureDeleted` and migrations. ([#50](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/50))
* `ToStartOfWeek` now rejects row-dependent week modes during query translation, and `ToStartOfInterval` likewise rejects row-dependent interval sizes. ClickHouse requires these operands to be constant for the query; literals and captured query parameters remain supported.
* Summing a `double` or `float` column (`.SumAsync(x => x.Value)`) no longer throws `InvalidCastException`. Two ClickHouse-specific mismatches were biting: EF Core hands the float literal generator a boxed `Int32` `0` as the empty-result fallback, and ClickHouse widens `sum(Float32)` to `Float64` so the driver couldn't read it back as a `float`. Both the literal generation and the `Float32` read path now convert instead of hard-casting. ([#46](https://github.com/ClickHouse/ClickHouse.EntityFrameworkCore/issues/46)) (Thanks to @HotTotem!)
* **SummingMergeTree with multiple sum columns** now produces valid DDL. Configuring more than one sum column (`HasSummingMergeTreeEngine("A", "B")`) previously emitted `SummingMergeTree(A, B)`, which ClickHouse rejects with `NUMBER_OF_ARGUMENTS_DOESNT_MATCH` — the engine takes a single optional parameter that must be a tuple of columns. Multiple columns are now wrapped in a tuple (`SummingMergeTree((A, B))`); single-column and no-column usage are unchanged.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,39 +45,45 @@ protected override DbConnection CreateDbConnection()
protected override bool SupportsAmbientTransactions => false;

/// <summary>
/// Ensures <c>join_use_nulls=1</c> is present in the underlying connection's connection
/// string before it's opened. Required because ClickHouse's default (0) makes LEFT JOIN
/// return column defaults rather than NULL, which breaks EF Core's null-based navigation
/// detection.
///
/// The ClickHouse.Driver HTTP protocol is stateless by default (<c>UseSession=False</c>):
/// a standalone <c>SET join_use_nulls = 1</c> statement does not persist to subsequent
/// queries. Instead the driver applies <c>set_*</c> connection-string parameters as URL
/// parameters on every query, so we must mutate the connection string itself.
///
/// Applies on all paths. For the connection-string path the setting is already baked in by
/// <see cref="ClickHouseDataSourceManager.EnsureDefaultSettings"/>; this override covers
/// the <see cref="DbConnection"/> and <see cref="DbDataSource"/> paths where EF hands us
/// connection objects we didn't construct. We only mutate when the connection is Closed
/// and the user has not explicitly configured <c>join_use_nulls</c>.
/// Applies <c>join_use_nulls=1</c> to a closed connection before it opens, unless the user set
/// <c>join_use_nulls</c> or disabled join null semantics. With the ClickHouse default (0), a
/// LEFT JOIN returns column defaults instead of NULL, and EF Core cannot detect a missing
/// navigation. The setting must be on the connection, because the driver sends it as a URL
/// parameter with each HTTP request; a <c>SET</c> statement does not persist between requests.
/// The connection-string path already has the setting from
/// <see cref="ClickHouseDataSourceManager.EnsureDefaultSettings"/>.
/// </summary>
public override bool Open(bool errorsExpected = false)
{
EnsureJoinUseNullsInConnectionString();
EnsureJoinUseNulls();
return base.Open(errorsExpected);
}

public override Task<bool> OpenAsync(CancellationToken cancellationToken, bool errorsExpected = false)
{
EnsureJoinUseNullsInConnectionString();
EnsureJoinUseNulls();
return base.OpenAsync(cancellationToken, errorsExpected);
}

private void EnsureJoinUseNullsInConnectionString()
private void EnsureJoinUseNulls()
{
if (_joinNullSemanticsDisabled || DbConnection.State != ConnectionState.Closed)
return;

// Assigning ConnectionString rebuilds the driver settings from the string, which loses
// settings that have no connection-string key, such as SkipServerCertificateValidation.
if (DbConnection is ClickHouseConnection clickHouseConnection)
{
var settings = clickHouseConnection.Settings;
if (settings.CustomSettings.ContainsKey(JoinUseNullsSetting))
return;

var updatedSettings = new ClickHouseClientSettings(settings);
updatedSettings.CustomSettings[JoinUseNullsSetting] = "1";
clickHouseConnection.Settings = updatedSettings;
return;
}

var cs = DbConnection.ConnectionString;
if (string.IsNullOrEmpty(cs)
|| cs.Contains("join_use_nulls", StringComparison.OrdinalIgnoreCase))
Expand All @@ -88,26 +94,54 @@ private void EnsureJoinUseNullsInConnectionString()
DbConnection.ConnectionString = ClickHouseDataSourceManager.EnsureDefaultSettings(cs);
}

private const string JoinUseNullsSetting = "join_use_nulls";

public IClickHouseRelationalConnection CreateMasterConnection()
{
var connectionStringBuilder = new ClickHouseConnectionStringBuilder(
_dataSource?.ConnectionString ?? ConnectionString)
{
Database = "default"
};
var optionsBuilder = new DbContextOptionsBuilder();

var masterConnectionString = connectionStringBuilder.ConnectionString;
if (TryGetClickHouseClientSettings(out var settings))
{
// Copy the settings object, not the connection string, to keep settings that have no
// connection-string key. The master connection owns the new connection and disposes it.
var masterSettings = new ClickHouseClientSettings(settings) { Database = "default" };
optionsBuilder.UseClickHouse(new ClickHouseConnection(masterSettings), contextOwnsConnection: true);
}
else
{
var masterConnectionString = new ClickHouseConnectionStringBuilder(
_dataSource?.ConnectionString ?? ConnectionString)
{
Database = "default"
}.ConnectionString;

var contextOptions = new DbContextOptionsBuilder()
.UseClickHouse(masterConnectionString)
.Options;
optionsBuilder.UseClickHouse(masterConnectionString);
}

return new ClickHouseRelationalConnection(
Dependencies with { ContextOptions = contextOptions },
Dependencies with { ContextOptions = optionsBuilder.Options },
dataSource: null,
_joinNullSemanticsDisabled);
}

private bool TryGetClickHouseClientSettings(out ClickHouseClientSettings settings)
{
if (_dataSource is ClickHouseDataSource clickHouseDataSource)
{
settings = clickHouseDataSource.Settings;
return true;
}

if (DbConnection is ClickHouseConnection clickHouseConnection)
{
settings = clickHouseConnection.Settings;
return true;
}

settings = null!;
return false;
}

public override IDbContextTransaction BeginTransaction(IsolationLevel isolationLevel)
=> new ClickHouseTransaction();

Expand Down
Loading
Loading