Blog | Giorgi Dalakishvili | Personal Website

DuckDB.NET 1.5.0: Simplified Scalar Functions, Table Functions, and NULL Handling

Tue, 10 Mar 2026 13:00:00 +0400

In the previous post, I covered the performance improvements in DuckDB.NET 1.5.0. This post covers the API side: a new high-level registration API for scalar and table user-defined functions, named parameter support for table functions, and explicit NULL handling for scalar UDFs.

Scalar Functions: Before and After

DuckDB.NET has supported scalar UDFs since version 1.0. The existing API gives you full control over vectors and row iteration - but even simple functions require boilerplate.

Here’s an is_prime function using the low-level API:

connection.RegisterScalarFunction<int, bool>("is_prime", (readers, writer, rowCount) =>
{
for (ulong index = 0; index < rowCount; index++)
{
var value = readers[0].GetValue<int>(index);
var prime = true;
for (int i = 2; i <= Math.Sqrt(value); i++)
{
if (value % i == 0)
{
prime = false;
break;
}
}
writer.WriteValue(prime, index);
}
});

You have to manually iterate over rows, read values from input vectors by index, and write results to the output vector. This works, but it’s a lot of ceremony for a function that just checks if a number is prime.

High-Level Scalar Functions

The new simplified API lets you register a scalar function with a plain Func<> delegate. The framework handles row iteration and vector access automatically:

connection.RegisterScalarFunction<int, bool>("is_prime", IsPrime);

That’s it. The library wraps your function, iterates over each row in the chunk, reads the input value, calls your function, and writes the result.

The API supports zero to four parameters, plus variable arguments:

// Zero parameters
connection.RegisterScalarFunction("the_answer", () => 42);
// Two parameters
connection.RegisterScalarFunction<long, long, long>("add", (a, b) => a + b);
// Three parameters
connection.RegisterScalarFunction<int, int, int, int>("clamp",
(value, min, max) => Math.Clamp(value, min, max));
// Variable arguments
connection.RegisterScalarFunction("sum_all", (long[] args) => args.Sum());

Variable argument functions receive all inputs as an array. DuckDB allows calling them with any number of arguments:

SELECT sum_all(1, 2, 3); -- 6
SELECT sum_all(10); -- 10
SELECT sum_all(); -- 0

Dynamic Type Support

The simplified API also supports object as an input type, which maps to DuckDB’s ANY type. This lets you write functions that accept any column type:

connection.RegisterScalarFunction<object, string>("net_type_name",
value => value.GetType().Name);

SELECT net_type_name(42); -- 'Int32'
SELECT net_type_name(42::BIGINT); -- 'Int64'
SELECT net_type_name('hello'); -- 'String'
SELECT net_type_name('2024-01-01'::DATE); -- 'DateOnly'

You can combine object with typed parameters. Here’s a .NET format function that accepts any formattable value:

connection.RegisterScalarFunction("format_net",
(object value, string format) => value is IFormattable f
? f.ToString(format, CultureInfo.InvariantCulture)
: value.ToString());

SELECT format_net(255, 'X'); -- 'FF'
SELECT format_net(0.15, 'P'); -- '15.00 %'
SELECT format_net('2024-11-06'::DATE, 'yyyy/MM/dd'); -- '2024/11/06'

NULL Handling for Scalar Functions

By default, DuckDB short-circuits NULLs automatically: if any input to a scalar function is NULL, the result is NULL and your function is never called. This is the correct behavior for most functions, but sometimes you need to handle NULLs explicitly - for example, to implement COALESCE-like logic or to return a default value.

With the high-level API, the opt-in is the type signature itself. Use a nullable parameter type and DuckDB.NET automatically detects it at registration time and configures the function to receive NULLs:

connection.RegisterScalarFunction<int?, string>("describe_val",
x => x.HasValue ? x.Value.ToString() : "nothing");

SELECT describe_val(42); -- '42'
SELECT describe_val(NULL::INT); -- 'nothing'

This works for reference types too - string? (with nullable annotations enabled) opts into null handling, while string keeps the default NULL propagation:

connection.RegisterScalarFunction<string?, string>("echo_or_default",
s => s ?? "was_null");

SELECT echo_or_default(NULL::VARCHAR); -- 'was_null'
SELECT echo_or_default('hello'); -- 'hello'

You can mix nullable and non-nullable parameters. If any parameter is nullable, the function receives NULLs - but non-nullable parameters will throw a clear error if they get one:

connection.RegisterScalarFunction<int?, int, string>("coalesce_add",
(a, b) => a.HasValue ? (a.Value + b).ToString() : b.ToString());

SELECT coalesce_add(NULL::INT, 5); -- '5'
SELECT coalesce_add(10, 5); -- '15'

NULL handling also works with the low-level vector API. In scalar function callbacks, GetValue<T> returns null for NULL rows when T is a nullable type (int?, string, etc.) instead of throwing, so you can write:

connection.RegisterScalarFunction<string, string>("echo_nullable", (readers, writer, rowCount) =>
{
for (ulong i = 0; i < rowCount; i++)
{
var value = readers[0].GetValue<string>(i);
writer.WriteValue(value ?? "was_null", i);
}
}, new() { HandlesNulls = true });

Table Functions: Before and After

The low-level table function API requires you to define columns, return a TableFunction with data, and provide a mapper callback that writes each row to output vectors:

connection.RegisterTableFunction<int>("employees", parameters =>
{
var count = parameters[0].GetValue<int>();
var employees = Enumerable.Range(1, count)
.Select(i => new Employee(i, $"Employee{i}", 50000 + i * 100));
return new TableFunction(
new List<ColumnInfo> { new("id", typeof(int)), new("name", typeof(string)) },
employees);
},
(item, writers, rowIndex) =>
{
var employee = (Employee)item!;
writers[0].WriteValue(employee.Id, rowIndex);
writers[1].WriteValue(employee.Name, rowIndex);
});

The column definitions and mapper are separate, so they can easily get out of sync - add a column to the schema but forget the mapper, or reorder them differently.

High-Level Table Functions

The new API uses a projection expression to define both the columns and the mapper in one place:

connection.RegisterTableFunction("employees",
(int count) => GetEmployees(count),
e => new { e.Id, e.Name });

The first argument is the function name. The second is a data function that receives the SQL parameters and returns an IEnumerable<T>. The third is a projection expression that defines which properties to expose as columns - column names and types are extracted automatically from the expression.

The projection supports anonymous types, object initializers, computed columns, and single properties:

// Computed columns
connection.RegisterTableFunction("ext_computed",
(int count) => GetEmployees(count),
e => new { FullName = "Dr. " + e.Name, DoubleSalary = e.Salary * 2 });
// Object initializer
connection.RegisterTableFunction("ext_init",
(int count) => GetEmployees(count),
e => new EmployeeDto { Id = e.Id, Name = e.Name });

Like the scalar API, table functions support zero to four parameters:

// No parameters
connection.RegisterTableFunction("all_employees",
() => employees.AsEnumerable(),
e => new { e.Id, e.Name });
// Four parameters
connection.RegisterTableFunction("ext_four",
(int start, int count, string prefix, double multiplier) =>
Enumerable.Range(start, count).Select(i => new Employee(i, $"{prefix}{i}", i * multiplier)),
e => new { e.Id, e.Name, e.Salary });

Async data sources work too - just call ToBlockingEnumerable():

connection.RegisterTableFunction("ext_async",
(int count) => FetchEmployeesAsync(count).ToBlockingEnumerable(),
e => new { e.Id, e.Name });

Named Parameters for Table Functions

DuckDB supports named parameters for table functions. DuckDB.NET 1.5.0 exposes this with the [Named] attribute:

connection.RegisterTableFunction("employees",
(int count, [Named] string? prefix) =>
GetEmployees(count).Select(e => e with { Name = (prefix ?? "") + e.Name }),
e => new { e.Id, e.Name });

In SQL, positional parameters come first, and named parameters use the = syntax:

SELECT * FROM employees(3, prefix = 'Dr. ');
-- (1, 'Dr. Employee1'), (2, 'Dr. Employee2'), (3, 'Dr. Employee3')

-- Named parameters are optional - omit them and they're NULL
SELECT * FROM employees(3);
-- (1, 'Employee1'), (2, 'Employee2'), (3, 'Employee3')

You can have multiple named parameters:

connection.RegisterTableFunction("employees",
(int count, [Named] string? prefix, [Named] double? multiplier) =>
GetEmployees(count).Select(e => e with
{
Name = (prefix ?? "") + e.Name,
Salary = e.Salary * (multiplier ?? 1)
}),
e => new { e.Id, e.Name, e.Salary });

-- Provide both
SELECT * FROM employees(2, prefix = 'X', multiplier = 2.0);
-- Provide only one
SELECT * FROM employees(2, prefix = 'Y');

By default, the SQL parameter name matches the C# parameter name. You can override it with a custom name:

connection.RegisterTableFunction("employees",
(int count, [Named("max_rows")] int? limit) => GetEmployees(limit ?? count),
e => new { e.Id, e.Name });

SELECT * FROM employees(10, max_rows = 2);

Named parameters should typically be nullable types (string?, int?) since they’re optional in SQL. If a named parameter is non-nullable and the caller omits it, DuckDB.NET throws a clear error:

Table function 'employees' named parameter 'limit' is NULL, but parameter type 'Int32' is non-nullable.

Summary

DuckDB.NET 1.5.0 adds four improvements to user-defined functions:

High-level scalar functions: Register with a plain Func<> instead of manually iterating vectors. Supports 0–4 parameters, variable arguments, and dynamic object type.
High-level table functions: Define columns and mapping with a single projection expression. Supports 0–4 parameters, computed columns, and async data sources.
NULL handling for scalar functions: Use nullable parameter types (int?, string?) and DuckDB.NET automatically configures the function to receive NULLs instead of short-circuiting.
Named parameters for table functions: The [Named] attribute maps C# parameters to DuckDB’s = syntax for optional, named arguments.

The low-level vector-based APIs remain available for cases where you need direct control over chunk processing.

DuckDB.NET 1.5.0 Performance: Up to 40% Faster Writes and 22% Fewer Allocations

Tue, 03 Mar 2026 05:00:00 +0400

What’s New in DuckDB.NET 1.5.0

DuckDB.NET 1.5.0 focuses on performance. The codebase has been optimized across multiple layers - from the low-level native interop ( LibraryImport migration, SuppressGCTransition) to the ADO.NET provider (reader reuse, appender boxing elimination) and type converters (decimal rewrite, BigNum conversion). The results show up across every major code path: reading, writing, and type conversion.

Note that the current pre-release (1.5.0-alpha) still uses the DuckDB 1.4.4 native library under the hood - the performance gains come entirely from improvements on the .NET side. The stable DuckDB.NET 1.5.0 release will ship with DuckDB 1.5.0 once it becomes available.

What Changed

LibraryImport Migration

All P/Invoke declarations have been migrated from [DllImport] to the source-generated [LibraryImport]. The runtime no longer needs to generate marshalling stubs at JIT time - the source generator produces them at compile time, eliminating stub overhead on every native call. As part of this migration, string returns now use custom marshallers (DuckDBOwnedStringMarshaller and DuckDBCallerOwnedStringMarshaller) that correctly and transparently handle ownership semantics - whether DuckDB or the caller is responsible for freeing the memory.

SuppressGCTransition on Fast Native Calls

Many DuckDB C API calls are trivially fast - retrieving a vector data pointer, checking validity, getting chunk size. These calls complete in nanoseconds, but the .NET runtime’s GC transition (cooperative → preemptive → cooperative) adds measurable overhead on each call. Adding [SuppressGCTransition] to these methods skips the transition entirely. This is only safe for native functions that execute in under a microsecond, perform no blocking syscalls or I/O, don’t call back into the runtime, don’t throw exceptions, and don’t manipulate locks. The attribute was applied to every DuckDB C API method that meets these criteria - primarily vector data and validity pointer access, chunk size queries, and similar lightweight operations.

AggressiveInlining on Hot-Path Methods

[MethodImpl(MethodImplOptions.AggressiveInlining)] was added to frequently called methods in the reader and writer paths - including IsValid(), GetFieldData<T>(), AppendValueInternal<T>(), and type conversion helpers. For example, IsValid() and GetFieldData<T>() are called for every column of every row. At 100,000 rows and 20 columns, that’s 2 million calls per query. Inlining these small methods directly into the read/write loops removes call overhead.

Appender Boxing Elimination

In 1.4.4, all AppendValue() overloads funneled into a single generic AppendValueInternal<T>(T? value) without a struct constraint. When T is a value type, the nullable wrapper was boxed on every call. In 1.5.0, this is split into struct-constrained and class-constrained overloads - the struct path unwraps Nullable<T> directly via HasValue/Value, eliminating the boxing. With 8 columns per row, that’s 6 fewer heap allocations per row for a typical mixed-type schema.

Decimal Conversion Rewrite

The decimal reader was rewritten for all four internal storage paths:

SmallInt/Integer/BigInt paths: Replaced decimal.Divide(raw, powerOfTen) with the direct constructor new decimal(Math.Abs(raw), 0, 0, raw < 0, scale). This constructs the decimal from its binary components without any arithmetic.
HugeInt path: Uses BigInteger.DivRem with pre-computed BigInteger[] powers of ten instead of repeated intermediate BigInteger arithmetic.
Pre-computed lookup tables: Static decimal[] and BigInteger[] arrays for powers of ten (scales 0–28 and 0–38 respectively) are computed once at startup and reused for all conversions.

Vector Reader Reuse Across Chunks

DuckDB returns data in chunks of up to 2,048 rows. In 1.4.4, new VectorDataReader instances were allocated for each chunk. In 1.5.0, readers implement Reset(IntPtr vector) which updates data and validity pointers in place, reusing the same reader objects across all chunks in a result set. Composite readers (struct, list, map, decimal) override Reset to also update their nested child readers.

BIGNUM O(n) Conversion

The BIGNUM (previously VarInt) to BigInteger conversion was rewritten from an O(n²) digit-by-digit algorithm to a direct O(n) construction using BigInteger's byte-span constructor. For positive values, the raw bytes are passed directly. For negative values, a byte complement is needed - small payloads (≤128 bytes) use stackalloc for this, larger ones rent from ArrayPool<byte>. The result: 93% faster reads and 98-99% fewer allocations at 10K–100K rows.

Benchmarks

All benchmarks compare DuckDB.NET.Data.Full 1.4.4 (stable, from NuGet.org) against 1.5.0-alpha.35 (pre-release, from GitHub) using BenchmarkDotNet. Each version is compiled and run independently using WithMsBuildArguments to swap the package version at build time, so both versions run against the same benchmark code with no shared state.

Reader: 17-22% Faster

The reader benchmark creates a table with a mix of column types (INT, VARCHAR, DOUBLE, BOOLEAN, DECIMAL, BIGINT, TIMESTAMP) and reads 100,000 rows using typed getters (GetInt32, GetString, GetDecimal, etc.).

Method	Job	ColumnCount	Mean	Ratio	Allocated	Alloc Ratio
ReadAllColumns	alpha-1.5.0	5	20.45 ms	-22%	3.82 MB	-1%
ReadAllColumns	stable-1.4.4	5	26.12 ms	baseline	3.85 MB

ReadAllColumns	alpha-1.5.0	10	44.15 ms	-17%	7.63 MB	-1%
ReadAllColumns	stable-1.4.4	10	53.60 ms	baseline	7.68 MB

ReadAllColumns	alpha-1.5.0	20	89.56 ms	-17%	11.45 MB	-1%
ReadAllColumns	stable-1.4.4	20	107.50 ms	baseline	11.56 MB

Allocations are nearly identical because both versions allocate the same string objects for VARCHAR columns. The speedup comes from LibraryImport, SuppressGCTransition, and AggressiveInlining working together on the hot read path.

Appender: 20-40% Faster, 22% Less Memory

The appender benchmark creates rows with 8 columns (INT, VARCHAR, DOUBLE, BOOLEAN, DECIMAL, TIMESTAMP, BIGINT, VARCHAR) using the CreateRow().AppendValue(...).EndRow() API.

Method	Job	RowCount	Mean	Ratio	Allocated	Alloc Ratio
AppendRows	alpha-1.5.0	10,000	19.58 ms	-20%	8.85 MB	-22%
AppendRows	stable-1.4.4	10,000	24.41 ms	baseline	11.36 MB

AppendRows	alpha-1.5.0	100,000	86.90 ms	-41%	89.20 MB	-22%
AppendRows	stable-1.4.4	100,000	147.46 ms	baseline	114.38 MB

AppendRows	alpha-1.5.0	1,000,000	803.55 ms	-40%	899.61 MB	-22%
AppendRows	stable-1.4.4	1,000,000	1,349.50 ms	baseline	1151.38 MB

The consistent 22% allocation reduction is the boxing elimination at work. At 1 million rows with 8 columns, that’s millions of heap allocations removed. The 40% speed gain comes from boxing elimination, reduced GC pressure from fewer allocations, and AggressiveInlining on the write path.

Summary

Area	Speedup	Memory Reduction
Reader (mixed types)	17-22%	~1%
Appender (8 columns)	20-40%	22%

These improvements require no code changes - just update the DuckDB.NET.Data package to 1.5.0 and the optimizations apply automatically.

In the next post, I’ll cover the API improvements in 1.5.0 - including a simplified API for scalar and table user-defined functions.

PostgreSQL range types and Entity Framework Core

Thu, 21 Dec 2023 01:01:02 +0400

This blog post is part of The C# Advent Calendar, a series of 50 posts about C#. Be sure to check out the rest of the blog posts in the calendar!

Ranges in Relational Databases

When modeling scheduling applications, such as an application for managing calendar events or hotel bookings, it is needed to store the start date(time) and end date(time) of the events. It is also common to store price history or market data with an associated validity period. Finally, if you are building a system where users can define price based on the order quantity (If you buy less than five items, the unit price is $100, between 5 and 10 unit price is $90, etc), you need to store the starting and ending quantities.

The typical solution for such scenarios is to store the range in two separate columns (such as StartDate and EndDate) and use a query like this to lookup for the events that occur at a particular time: SELECT * FROM events WHERE @lookupTime BETWEEN StartDate AND EndDate. While this works in simple cases, there is no easy way to make sure that no two events overlap, and more advanced scenarios can quickly become complex to support.

In this article, I will show how to use PostgreSQL range types to represent ranges, how it simplifies querying and can guarantee non-overlapping periods in the database, and how to use them with Entity Framework Core.

Ranges in PostgreSQL

Range types in PostgreSQL represent a range of values for their underlying type. Postgres comes with several built-in range types: int4range and int8range represent ranges of int and bigint, numrange - ranges of numerics, tstzrange, daterange, and tsrange represent ranges of timestamptz, date, and time types. You can use these types just like any other:

CREATE TABLE IF NOT EXISTS public."Meetings"
(
"Id" integer NOT NULL GENERATED BY DEFAULT AS IDENTITY ( INCREMENT 1 START 1 MINVALUE 1 MAXVALUE 2147483647 CACHE 1 ),
"RoomName" character varying(20) NOT NULL,
"Title" character varying(50) NOT NULL,
"Time" tstzrange NOT NULL,
CONSTRAINT "PK_Meetings" PRIMARY KEY ("Id")
)

In this example, the Time column represents the start time and end time of the corresponding meeting. Next, let’s insert a couple of records:

INSERT INTO public."Meetings"("RoomName", "Title", "Time")
VALUES ('Winterfell', 'PostgreSQL Demo', '[2023-12-23 11:30, 2023-12-23 12:30)');
INSERT INTO public."Meetings"("RoomName", "Title", "Time")
VALUES ('Jobria', 'Npgsql Demo', '[2023-12-23 13:00, 2023-12-23 13:30]');
INSERT INTO public."Meetings"("RoomName", "Title", "Time")
VALUES ('Casterly Rock', 'Upgrade PostgreSQL', '[2023-12-23 14:00, 2023-12-23 15:30]');

Range values are created by specifying the start and end bounds, and they follow mathematical notation for interval: A square bracket means inclusive bound, and parentheses means exclusive bound.

Range Operators in PostgreSQL

The real power of PostgreSQL ranges comes from range operators. The most commonly used operators are:

&& - Overlap operator, tests if two ranges overlap.
@> and <@ - Containment operators test if the first range contains the second one or vice versa.
-|- - Checks if two ranges are adjacent.
* - Computes intersection of two ranges.

Let’s see some real-life examples:

Find all meetings that occur in the specified time range:

Select "RoomName", "Title", "Time" from "Meetings" where
"Time" <@ '[2023-12-23 13:00, 2023-12-23 15:00]'::tstzrange
| "RoomName" | "Title" | "Time" |
|------------|---------------|-----------------------------------------------------------|
| "Jobria" | "Npgsql Demo" | "[""2023-12-23 13:00:00+04"",""2023-12-23 13:30:00+04""]" |
| | | |

Find all meetings that overlap with the specified time range:

Select "RoomName", "Title", "Time" from "Meetings" where
"Time" && '[2023-12-23 13:00, 2023-12-23 15:00]'::tstzrange
| "RoomName" | "Title" | "Time" |
|-----------------|----------------------|-----------------------------------------------------------|
| "Jobria" | "Npgsql Demo" | "[""2023-12-23 13:00:00+04"",""2023-12-23 13:30:00+04""]" |
| "Casterly Rock" | "Upgrade PostgreSQL" | "[""2023-12-23 14:00:00+04"",""2023-12-23 15:30:00+04""]" |
| | | |

Find all busy times in the specified time range:

Select "RoomName", "Title", "Time",
"Time" *'[2023-12-23 13:00, 2023-12-23 15:00]'::tstzrange as BusyTime
from "Meetings" where "Time" && '[2023-12-23 13:00, 2023-12-23 15:00]'::tstzrange
| "RoomName" | "Title" | "Time" | "BusyTime" |
|-----------------|----------------------|-----------------------------------------------------------|-----------------------------------------------------------|
| "Jobria" | "Npgsql Demo" | "[""2023-12-23 13:00:00+04"",""2023-12-23 13:30:00+04""]" | "[""2023-12-23 13:00:00+04"",""2023-12-23 13:30:00+04""]" |
| "Casterly Rock" | "Upgrade PostgreSQL" | "[""2023-12-23 14:00:00+04"",""2023-12-23 15:30:00+04""]" | "[""2023-12-23 14:00:00+04"",""2023-12-23 15:00:00+04""]" |

Ranges Exclusion Constraints

Finally, let’s make sure that no two events are happening at the same time in the same room. First, you need to load btree_gist extension by running the following command: CREATE EXTENSION btree_gist;. Next, create an Exclusion Constraint that guarantees that there are no overlapping periods for any given room:

Alter Table "Meetings"
Add Constraint "NoOverlap"
EXCLUDE USING gist ("RoomName" WITH =, "Time" WITH &&)

This Exclude constraint checks that no two rows with the same RoomName have Time values that overlap. With this constraint in place, if you try to insert an overlapping record, the insert will fail:

INSERT INTO public."Meetings"("RoomName", "Title", "Time")
VALUES ('Winterfell', 'Overlapping Event', '[2023-12-23 12:00, 2023-12-23 12:30]');

ERROR: Key ("RoomName", "Time")=(Winterfell, ["2023-12-23 12:00:00+04","2023-12-23 12:30:00+04"])
conflicts with existing key ("RoomName", "Time")=(Winterfell, ["2023-12-23 11:30:00+04","2023-12-23 12:30:00+04")).
conflicting key value violates exclusion constraint "NoOverlap"
ERROR: conflicting key value violates exclusion constraint "NoOverlap"
SQL state: 23P01
Detail: Key ("RoomName", "Time")=(Winterfell, ["2023-12-23 12:00:00+04","2023-12-23 12:30:00+04"])
conflicts with existing key ("RoomName", "Time")=(Winterfell, ["2023-12-23 11:30:00+04","2023-12-23 12:30:00+04")).

Now that we have covered how range types work in PostgreSQL, let’s see how to use them from Entity Framework Core.

Entity Framework Core and PostgreSQL Range Types.

Entity Framework Core provider for PostgreSQL supports mapping ranges and performing operations on ranges in LINQ queries. To represent the PostgreSQL range in C# I will use NpgsqlRange<T> type provided by Npgsql:

class MeetingRoom
{
public int Id { get; set; }
public string Name { get; set; };
public int MaxCapacity { get; set; }
}
class Meeting
{
public int Id { get; set; }
public string Title { get; set; }
public MeetingRoom Room { get; set; }
public NpgsqlRange<DateTime> Time { get; set; }
}

The exclusion constraint can’t be specified using the Entity Framework ModelBuilder api, so I’ll use raw SQL inside a migration to create it:

public partial class ExcludeConstraint : Migration
{
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql(@"Alter Table ""Meetings""
 Add Constraint ""NoOverlap""
 EXCLUDE USING gist (""RoomId"" WITH =, ""Time"" WITH &&)");
}
}

Next, let’s insert some rooms and events and see how to perform queries on ranges with Entity Framework Core.

demoContext.Meetings.Add(new Meeting
{
Title = "PostgreSQL Demo",
Room = winterfellMeetingRoom,
Time = new NpgsqlRange<DateTime>(new DateTime(2023, 12, 23, 11, 30, 0, DateTimeKind.Utc),
new DateTime(2023, 12, 23, 12, 30, 0, DateTimeKind.Utc))
});
demoContext.Meetings.Add(new Meeting
{
Title = "Npgsql Demo",
Room = jobriaMeetingRoom,
Time = new NpgsqlRange<DateTime>(new DateTime(2023, 12, 23, 13, 00, 0, DateTimeKind.Utc),
new DateTime(2023, 12, 23, 13, 30, 0, DateTimeKind.Utc))
});
demoContext.Meetings.Add(new Meeting
{
Title = "Upgrade PostgreSQL",
Room = casterlyRockMeetingRoom,
Time = new NpgsqlRange<DateTime>(new DateTime(2023, 12, 23, 14, 00, 0, DateTimeKind.Utc),
new DateTime(2023, 12, 23, 15, 30, 0, DateTimeKind.Utc))
});
await demoContext.SaveChangesAsync();

As you can see inserting range values is quite straightforward with Entity Framework Core. Performing operations on ranges is simple as well:

var targetRange = new NpgsqlRange<DateTime>(new DateTime(2023, 12, 23, 13, 00, 0, DateTimeKind.Utc),
new DateTime(2023, 12, 23, 15, 00, 0, DateTimeKind.Utc));
//Find all meetings that occur in the specified time range
var meetings = await demoContext.Meetings.Where(m => m.Time.ContainedBy(targetRange)).ToListAsync();
//Find all meetings that overlap with the specified time range
meetings = await demoContext.Meetings.Where(m => m.Time.Overlaps(targetRange)).ToListAsync();
//Find all busy times in the specified time range
var meetingsWithBusyTimes = await demoContext.Meetings.Where(m => m.Time.Overlaps(targetRange))
.Select(m => new
{
Meeting = m,
BusyTime = m.Time.Intersect(targetRange)
}).ToListAsync();

Entity Framework Core and Npgsql translate these operations to corresponding PostgreSQL range operations so they get evaluated on the database server.

Finally, if you try to insert a new meeting that overlaps with an existing one, Entity Framework Core will throw DbUpdateException:

try
{
demoContext.Meetings.Add(new Meeting
{
Title = "Conflicting meeting",
Room = casterlyRockMeetingRoom,
Time = new NpgsqlRange<DateTime>(new DateTime(2023, 12, 23, 15, 00, 0, DateTimeKind.Utc),
new DateTime(2023, 12, 23, 16, 00, 0, DateTimeKind.Utc))
});
await demoContext.SaveChangesAsync();
}
catch (DbUpdateException ex)
{
var npgsqlException = ex.GetBaseException() as NpgsqlException;
if (npgsqlException?.SqlState == PostgresErrorCodes.ExclusionViolation)
{
Console.WriteLine("This meeting overlaps with another meeting in the same room");
}
}

Conclusion

In this article I showed how PostgreSQL range types can simplify representing and performing operations on ranges. Apart from range types PostgreSQL has many other interesting features not found in other relational databases and I recommend every developer to check it out.

The sample code from this post is available on GitHub: https://github.com/Giorgi/PostgresRangeTypes

How Cloudflare Broke My Build and How I Fixed It

Mon, 15 Aug 2022 12:00:52 +0400

Mysterious Build Failure

About a month ago, my open-source project EntityFramework.Exceptions failed to build on AppVeyor, the continuous integration service that I have been using for several years. I hadn’t made any changes that could have caused the build to fail, so I set out to investigate the cause of the failure.

It turned out that the build failed because it failed to upload source code coverage data to coveralls.io. The error message was very minimal:

BadRequest - <html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<hr><center>nginx</center>
</body>
</html>

I checked coveralls status page, but there wasn’t any ongoing incident, so I reached out to coveralls support. Coveralls support responded that they had recently migrated their services to different servers and were now using Cloudflare as a service provider but these changes should have been transparent to the end users. To ensure this wasn’t a DNS issue, I added ping -n 1 coveralls.io to the appVeyor.yml file. The resolved IP address was correct, so the issue wasn’t related to DNS.

Coveralls support tried to check the cause of the Bad Request error in their logs but surprisingly my attempts to upload the coverage data didn’t show up. This meant that the Bad Request didn’t originate from coveralls.io and was coming from Cloudflare. Why was Cloudflare suddenly rejecting my attempts to upload coverage that was working flawlessly for the last couple of years?

Finding the Root Cause of the Issue

Now that I knew that the failure was Cloudflare’s fault, I asked the coveralls.io team to get in touch with Cloudflare support to find out the cause of the error response. I downloaded coveralls.net (Coveralls uploader for .Net Code coverage) to my local machine, attempted to submit the coverage data, and captured the request with Fiddler. I saved the HTTP request as a har file and sent it to coveralls.io support, hoping Cloudflare would help solve the issue. Unfortunately, Cloudflare was not helpful at all.

My next step, as suggested by James from coveralls.io, was to submit the coverage file with curl and see if that worked. I ran the following command: curl https://coveralls.io/api/v1/jobs --form "json_file=@uploadedCoverage.json" and to my surprise, the coverage file was successfully received by coveralls.io. Now that I was able to submit coverage data with curl, all I had to do, was to compare the HTTP requests issued by curl and coveralls.net to find the difference between them that was causing the failure.

The HTTP request looked like this (request bodies omitted):

coveralls.net HTTP request:

POST https://coveralls.io/api/v1/jobs HTTP/1.1
Host: coveralls.io
Content-Type: multipart/form-data; boundary="e5d8bb1a-4ab2-4bb8-9d8b-152c5b704cfa"
Content-Length: 3093
--e5d8bb1a-4ab2-4bb8-9d8b-152c5b704cfa
Content-Type: text/plain; charset=utf-8
Content-Disposition: form-data; name=json_file; filename=coverage.json; filename*=utf-8''coverage.json

curl HTTP request:

POST https://coveralls.io/api/v1/jobs HTTP/1.1
Host: coveralls.io
User-Agent: curl/7.83.1
Accept: */*
Content-Length: 3080
Content-Type: multipart/form-data; boundary=------------------------c8eebbf0920b1cc0
--------------------------c8eebbf0920b1cc0
Content-Disposition: form-data; name="json_file"; filename="uploadedCoverage.json"
Content-Type: application/octet-stream

There are several differences between these requests, but can you guess what was causing Cloudflare to fail? It turned out that Cloudflare returned Bad Request because of double quotes around the boundary directive value in the Content-Type header!

Content-Type boundary with double quotes is perfectly valid, so why does Cloudflare treat it as an error? Unfortunately, I don’t have an answer to this question. The pull request to handle quotes was merged more than two years ago, so either Cloudflare uses an older version of nginx upload module, or Cloudflare uses another module that cannot handle double quotes around the boundary directive.

Fixing Coveralls Uploader

Now that I knew what was causing the failure, it was easy to fix the coveralls.net:

var boundary = formData.Headers.ContentType?.Parameters.FirstOrDefault(o => o.Name == "boundary");
if (boundary != null)
{
boundary.Value = boundary.Value?.Replace("\"", string.Empty);
}

I submitted a pull request to the coveralls.net repo, and as soon as it was merged, my build was green again!

Introducing GraphQLinq - Strongly Typed GraphQL Queries with LINQ to GraphQL.

Tue, 27 Apr 2021 10:10:19 +0400

Consuming a GraphQL api in C# is straightforward with either using HttpClient directly or using a client library such as GraphQL.Client but both suffer from the same problems: The GraphQL queries do not go through any compile-time checking, and any mistake in the query isn’t discovered until you execute the query at runtime. For example, to query SpaceX GraphQL API for all missions where the manufacturer is Orbital ATK you need to run the following query:

{
missions (find: {manufacturer: "Orbital ATK"}) {
id
name
description
twitter
website
wikipedia
manufacturers
}
}

.Net solved the same problem for SQL with LINQ: LINQ queries are first-class language objects and are strongly typed. This way, there are no hardcoded queries in the project, and the LINQ queries are type-checked by the compiler.

In this article, I will show how GraphQLinq - a LINQ to GraphQL library solves this problem.

Getting Started with GraphQLinq

GraphQLinq consists of two parts: GraphQLinq.Scaffolding for generating strongly typed classes from the GrapQL endpoint and client library, GraphQLinq.Client, for writing and executing LINQ to GraphQL queries.

GraphQLinq.Scaffolding is a .NET tool, so run the following command to install it:

dotnet tool install --global --version 1.0.0-beta GraphQLinq.Scaffolding

This will install the tool globally and make it available for all projects.

Next, navigate to your project folder and run the graphql-scaffold tool to scaffold the client-side code:

graphqlinq-scaffold https://api.spacex.land/graphql -o SpaceX -n SpaceX

This will generate the code in the SpaceX folder (specified by the o option) and use SpaceX as a namespace too.

Scaffolding GraphQL client with GraphQLinq.Scaffolding

Now, install the GraphQLinq.Client package in your project to write some LINQ to GraphQL queries:

dotnet add package GraphQLinq.Client --version 1.0.0-beta

Running GraphQL Queries with LINQ

The scaffolding tool generates QueryContext class that serves as an entry point for writing and running LINQ queries. For example, you can write the above graphQL query like this:

var spaceXContext = new QueryContext();
var missionsQuery = spaceXContext.Missions(new MissionsFind { Manufacturer = "Orbital ATK" }, null, null)
.Include(mission => mission.Manufacturers);
var missions = missionsQuery.ToList();

This LINQ query will generate a graphQL query that includes all primitive properties of the Mission class and also includes the list of manufacturers in the query.

Here are a couple of more LINQ to GraphQL queries:

Query all Primitive Properties of a Type

Quering for all primitive properties is very straightforward:

var spaceXContext = new QueryContext();
var company = spaceXContext.Company().ToItem();

Query Specific Properties

var companySummary = spaceXContext.Company().Select(c => new CompanySummary
{
Ceo = c.Ceo,
Name = c.Name,
Headquarters = c.Headquarters
}).ToItem();
/// Alternatively, use an anonymous object

var companySummaryAnonymous = spaceXContext.Company().Select(c => new { c.Ceo, c.Name, c.Headquarters }).ToItem();

var companyWithHeadquartersAndLinks = spaceXContext.Company()
.Include(info => info.Headquarters)
.Include(info => info.Links).ToItem();

View Generated Query

To view the generated GraphQL query, call ToString on the LINQ query itself. You will get the GraphQL query together with the variables bound to the query.

var missionsQuery = spaceXContext.Missions(new MissionsFind { Manufacturer = "Orbital ATK" }, null, null)
.Include(mission => mission.Manufacturers);
var query = missionsQuery.Query;
var fullQuery = missionsQuery.ToString();

After executing the above code snippet query will be equal to:

query ($find: MissionsFind) { result: missions (find: $find) {
description
id
name
twitter
website
wikipedia
manufacturers
}}

and fullQuery will be:

{"query":"query ($find: MissionsFind) { result: missions (find: $find) {
 description
 id
 name
 twitter
 website
 wikipedia
 manufacturers
 }}","variables":{"find":{"manufacturer":"Orbital ATK"}}}

Roadmap

The library is in the early stages and does not support all possible GraphQL schemas and queries. Check open issues for a list of proposed features and known issues.

If you find the library useful, don’t forget to ⭐ the GitHub repository. If you have any questions or suggestions, you are welcome to submit an issue or send a PR.

Updating LINQPad.QueryPlanVisualizer for LINQPad6 And Entity Framework Core

Mon, 01 Mar 2021 15:15:19 +0400

LINQPad.QueryPlanVisualizer is a custom visualizer for LINQPad that shows a database query plan inside LINQPad. It also shows missing indexes for the query that you can create directly from LINQPad.

Since then, a new major version, LINQPad 6, was released that targets .NET Core 3 and .NET 5 and uses Entity Framework Core (as well as LINQ-to-SQL) for running Linq queries.

To support these changes, I have just released a new version of LINQPad.QueryPlanVisualizer that is compatible with LINQPad 6, supports Entity Framework Core, and, last but not least, adds support for viewing PostgreSQL query plans in LINQPad. It also can share SQL Server query plan to Brent Ozar’s Paste The Plan and share PostgreSQL query plan explain.dalibo.com.

Using LINQPad.QueryPlanVisualizer

To use LINQPad.QueryPlanVisualizer in your queries, either reference it from NuGet or download the latest release from GitHub and copy QueryPlanVisualizer.LinqPad6.dll to My Documents\LINQPad Plugins\NetCore3 folder.

To view the query plan call DumpPlan method on any IQueryable instance:

Posts.OrderBy(post => post.PostDate.Year > 2020).OrderByDescending(post => post.BlogId).DumpPlan();

LINQPad showing SQL Server Query Execution Plan

If you run the same query on an PostgreSQL database, you will see a similar output:

LINQPad showing PostgreSQL Query Execution Plan

Internals of LINQPad.QueryPlanVisualizer

As the visualizer supports multiple ORMs (EF Core and LINQ-to-SQL) and different databases (SQL Server and PostgreSQL), the project uses Bridge Pattern to avoid creating separate classes for all possible combinations. The main class hierarchies are OrmHelper, which is responsible for extracting DbCommand from IQueryable for a specific ORM; DatabaseProvider is responsible for getting query plan for a specific database; and PlanProcessor that converts raw plan to Html. The OrmHelper also contains a factory method for creating an instance for the ORM that the current query uses:

public static OrmHelper Create<T>(IQueryable<T> queryable, object dataContext)
{
if (dataContext is DbContext dbContext)
{
var efCoreHelper = new EFCoreHelper(dbContext.Database.ProviderName);
efCoreHelper.Initialize(queryable);
return efCoreHelper;
}
var queryType = queryable.GetType();
var dataQueryType = queryType.Assembly.GetType("System.Data.Linq.DataQuery`1");
var tableQueryType = queryType.Assembly.GetType("System.Data.Linq.Table`1");
var queryGenericType = queryType.GetGenericTypeDefinition();
if (queryGenericType == dataQueryType || queryGenericType == tableQueryType)
{
var contextField = queryType.GetField("context", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.GetField);
var context = contextField?.GetValue(queryable);
if (context != null)
{
var linqToSqlHelper = new LinqToSqlHelper(context);
linqToSqlHelper.Initialize(queryable);
return linqToSqlHelper;
}
}
return null;
}

The dataContext object is the data context that LIQNPad exposes via Util.CurrentDataContext.

The constructor of EFCoreHelper sets DatabaseProvider and PlanProcessor based on the target database:

public EFCoreHelper(string provider) : base(CreateParameters(provider))
{
}
public static (DatabaseProvider provider, PlanProcessor planConvertor) CreateParameters(string provider)
{
return provider switch
{
"Microsoft.EntityFrameworkCore.SqlServer" => (new SqlServerDatabaseProvider(), new SqlServerPlanProcessor()),
"Npgsql.EntityFrameworkCore.PostgreSQL" => (new PostgresDatabaseProvider(), new PostgresPlanProcessor()),
_ => (null, null)
};
}

The LinqToSqlHelper always passes SqlServerDatabaseProvider and SqlServerPlanProcessor to the base class as it only supports SQL Server.

Extracting and Formatting Query Plan from SQL Server

To extract query plan for SQL Server, the SqlServerDatabaseProvider class executes SET STATISTICS XML ON command before running the actual query and captures the plan returned by the database:

protected override string ExtractPlanInternal(DbCommand command)
{
using var setStatisticsCommand = command.Connection.CreateCommand();
setStatisticsCommand.CommandText = "SET STATISTICS XML ON";
setStatisticsCommand.ExecuteNonQuery();
using var reader = command.ExecuteReader();
while (reader.NextResult())
{
if (reader.GetName(0) == "Microsoft SQL Server 2005 XML Showplan")
{
reader.Read();
return reader.GetString(0);
}
}
return null;
}

Next, SqlServerPlanProcessor uses html-query-plan to display the plan as Html.

Extracting Query Plan from PostgreSQL

For PostgreSQL, the PostgresPlanProcessor class prepends EXPLAIN (ANALYZE, COSTS, VERBOSE, BUFFERS) to the query text to get the plan from a database:

protected override string ExtractPlanInternal(DbCommand command)
{
command.CommandText = "EXPLAIN (ANALYZE, COSTS, VERBOSE, BUFFERS) " + command.CommandText;
using var reader = command.ExecuteReader();
var plan = string.Join(Environment.NewLine, reader.Cast<IDataRecord>().Select(r => r.GetString(0)));
return plan;
}

The PostgreSQL query plan is then displayed directly as plain text.

Conclusion

LINQPad is a handy tool for prototyping and testing Linq queries. LINQPad.QueryPlanVisualizer takes it further by showing query plan and adding other helpful features. If you have questions or suggestions please leave a comment, create an issue and star the repository.

EntityFramework.Exceptions 3.1.1 - Support for Entity Framework Core 3 and Improved API

Thu, 12 Mar 2020 07:35:16 +0000

In the previous article I introduced EntityFramework.Exceptions, a library which simplifies handling exceptions in Entity Framework Core but the library had one important limitation: In order to use it you had to inherit your custom DbContext from ExceptionProcessorContextBase class. This means that if you wanted to use some other base class for your DbContext you were out of luck. The latest version of the library solves this issue by replacing one of the internal services used by entity framework core with a custom implementation and also adds support for Entity Framework Core 3.1.1

The service that needs to be replaced is IStateManager and is used by the ChangeTracker. The custom implementation of IStateManager interface inherits from the built in StateManager class and overrides SaveChanges and SaveChangesAsync methods. Let’s see how it works:

public abstract class ExceptionProcessorStateManager<T> : StateManager where T : DbException
{
private static readonly Dictionary<DatabaseError, Func<DbUpdateException, Exception>> ExceptionMapping = new Dictionary<DatabaseError, Func<DbUpdateException, Exception>>
{
{DatabaseError.MaxLength, exception => new MaxLengthExceededException("Maximum length exceeded", exception.InnerException) },
{DatabaseError.UniqueConstraint, exception => new UniqueConstraintException("Unique constraint violation", exception.InnerException) },
{DatabaseError.CannotInsertNull, exception => new CannotInsertNullException("Cannot insert null", exception.InnerException) },
{DatabaseError.NumericOverflow, exception => new NumericOverflowException("Numeric overflow", exception.InnerException) },
{DatabaseError.ReferenceConstraint, exception => new ReferenceConstraintException("Reference constraint violation", exception.InnerException) }
};
protected ExceptionProcessorStateManager(StateManagerDependencies dependencies) : base(dependencies)
{
}
public override int SaveChanges(bool acceptAllChangesOnSuccess)
{
try
{
return base.SaveChanges(acceptAllChangesOnSuccess);
}
catch (DbUpdateException originalException)
{
var exception = GetException(originalException);
if (exception != null)
{
throw exception;
}
throw;
}
}
public override async Task<int> SaveChangesAsync(bool acceptAllChangesOnSuccess, CancellationToken cancellationToken = new CancellationToken())
{
try
{
var result = await base.SaveChangesAsync(acceptAllChangesOnSuccess, cancellationToken);
return result;
}
catch (DbUpdateException originalException)
{
var exception = GetException(originalException);
if (exception != null)
{
throw exception;
}
throw;
}
}
private Exception GetException(DbUpdateException ex)
{
if (ex.GetBaseException() is T dbException && GetDatabaseError(dbException) is DatabaseError error && ExceptionMapping.TryGetValue(error, out var ctor))
{
return ctor(ex);
}
return null;
}
protected abstract DatabaseError? GetDatabaseError(T dbException);
}

The abstract ExceptionProcessorStateManager class catches any database exception thrown during SaveChanges call and tries to translate it into one of the supported exception instances. If it succeeds it throws the new exception and if it doesn’t it simply rethrows the original exception. The GetDatabaseError is overriden in database specific projects and returns DatabaseError based on the specific DbException that was thrown:

class SqlServerExceptionProcessorStateManager: ExceptionProcessorStateManager<SqlException>
{
public SqlServerExceptionProcessorStateManager(StateManagerDependencies dependencies) : base(dependencies)
{
}
private const int ReferenceConstraint = 547;
private const int CannotInsertNull = 515;
private const int CannotInsertDuplicateKeyUniqueIndex = 2601;
private const int CannotInsertDuplicateKeyUniqueConstraint = 2627;
private const int ArithmeticOverflow = 8115;
private const int StringOrBinaryDataWouldBeTruncated = 8152;
protected override DatabaseError? GetDatabaseError(SqlException dbException)
{
switch (dbException.Number)
{
case ReferenceConstraint:
return DatabaseError.ReferenceConstraint;
case CannotInsertNull:
return DatabaseError.CannotInsertNull;
case CannotInsertDuplicateKeyUniqueIndex:
case CannotInsertDuplicateKeyUniqueConstraint:
return DatabaseError.UniqueConstraint;
case ArithmeticOverflow:
return DatabaseError.NumericOverflow;
case StringOrBinaryDataWouldBeTruncated:
return DatabaseError.MaxLength;
default:
return null;
}
}
}

In order to actually replace IStateManager with the custom implementation you need to install either SQL Server, PostgreSQL or MySQL nuget package and call UseExceptionProcessor method of the ExceptionProcessorExtensions from the database specific package:

class DemoContext : DbContext, IDemoContext
{
public DbSet<Product> Products { get; set; }
public DbSet<ProductSale> ProductSale { get; set; }
protected override void OnModelCreating(ModelBuilder builder)
{
builder.Entity<Product>().Property(b => b.Price).HasColumnType("decimal(5,2)").IsRequired();
builder.Entity<Product>().Property(b => b.Name).IsRequired().HasMaxLength(15);
builder.Entity<Product>().HasIndex(u => u.Name).IsUnique();
}
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlServer(@"Data Source=(localdb)\ProjectsV13;Initial Catalog=Test;Integrated Security=True;Connect Timeout=30;")
.UseExceptionProcessor();
}
}

The UseExceptionProcessor method is very simple and all it does is a call to DbContextOptionsBuilder.ReplaceService<TService,TImplementation> method:

public static class ExceptionProcessorExtensions
{
public static DbContextOptionsBuilder UseExceptionProcessor(this DbContextOptionsBuilder self)
{
self.ReplaceService<IStateManager, SqlServerExceptionProcessorStateManager>();
return self;
}
public static DbContextOptionsBuilder<TContext> UseExceptionProcessor<TContext>(this DbContextOptionsBuilder<TContext> self) where TContext : DbContext
{
self.ReplaceService<IStateManager, SqlServerExceptionProcessorStateManager>();
return self;
}
}

Once you have done it you will start getting database specific exceptions instead of DbUpdateException:

using (var demoContext = new DemoContext())
{
demoContext.Products.Add(new Product
{
Name = "Moon Lamp",
Price = 1
});
demoContext.Products.Add(new Product
{
Name = "Moon Lamp",
Price = 10
});
try
{
demoContext.SaveChanges();
}
catch (UniqueConstraintException e)
{
//Unique index was violated. Show corresponding error message to user.
 }
}

The full source code of the library is available on GitHub: EntityFramework.Exceptions If you have questions or suggestions feel free to leave a comment, create an issue and star the repository.

Introducing EntityFramework.Exceptions

Tue, 11 Dec 2018 07:35:16 +0000

When using Entity Framework Core for data access all database exceptions are wrapped in DbUpdateException. If you need to know whether the exception was caused by a unique constraint, value being too long or value missing for a required column you need to dig into the concrete DbException subclass instance and check the error number to determine the exact cause.

EntityFramework.Exceptions simplifies this by handling all the database specific details and throwing different exceptions for different cases. All you have to do is

inherit your DbContext from ExceptionProcessorContext and handle the exception(s) (such as UniqueConstraintException, CannotInsertNullException, MaxLengthExceededException, NumericOverflowException) you need.

The Problem with Entity Framework Exceptions

Let’s say we have a Product table which has Name column with a unique index and Price column. Entity Framework context will look like this:

class DemoContext : DbContext
{
public DbSet<Product> Products { get; set; }
protected override void OnModelCreating(ModelBuilder builder)
{
builder.Entity<Product>().Property(b => b.Price).HasColumnType("decimal(5,2)").IsRequired();
builder.Entity<Product>().Property(b => b.Name).IsRequired().HasMaxLength(15);
builder.Entity<Product>().HasIndex(u => u.Name).IsUnique();
}
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlServer(@"Data Source=localhost;Initial Catalog=Test;Integrated Security=True;Connect Timeout=30;");
}
}

If we try to insert two records with the same name we will get DbUpdateException. As we have already mentioned DbUpdateException is thrown every time when saving changes to the database fails. In order to check that the exception was caused by the unique index we need to get the specific database DbException subclass instance and check the error number. In case of SQL Server we can do it like this:

using (var demoContext = new DemoContext())
{
demoContext.Products.Add(new Product
{
Name = "Moon Lamp",
Price = 1
});
demoContext.Products.Add(new Product
{
Name = "Moon Lamp",
Price = 10
});
try
{
demoContext.SaveChanges();
}
catch (DbUpdateException e)
{
var sqlException = e.GetBaseException() as SqlException;
//2601 is error number of unique index violation
 if (sqlException != null && sqlException.Number == 2601)
{
//Unique index was violated. Show corresponding error message to user.
 }
}
}

While this works it has several disadvantages. First of all it is repetitive to write this code every time we try to save changes to database. Secondly, there are other database errors to handle such as trying to insert null in non-null column or trying to insert longer value than the column allows. Finally, the error numbers are different for different database servers.

To avoid these issues I have created a library EntityFramework.Exceptions which handles database specific errors and throws different exceptions for different database errors.

EntityFramework.Exceptions – Easy Way to Handle Exceptions

With EntityFramework.Exceptions we can rewrite the above code like this:

using (var demoContext = new DemoContext())
{
demoContext.Products.Add(new Product
{
Name = "Moon Lamp",
Price = 1
});
demoContext.Products.Add(new Product
{
Name = "Moon Lamp",
Price = 10
});
try
{
demoContext.SaveChanges();
}
catch (UniqueConstraintException e)
{
//Unique index was violated. Show corresponding error message to user.
 }
}

As you can see we no longer have to deal with database specific exception and error numbers. Instead UniqueConstraintException is thrown when either a unique index or unique constraint is violated. As a result our code is cleaner, shorter and easier to understand. What’s more the library provides other exceptions such as CannotInsertNullException, MaxLengthExceededException, NumericOverflowException. For example if we try insert a product with Name longer than 15 characters we will get MaxLengthExceededException exception:

using (var demoContext = new DemoContext())
{
demoContext.Products.Add(new Product
{
Name = "Moon Lamp Change 3 Colors",
Price = 1
});
try
{
demoContext.SaveChanges();
}
catch (MaxLengthExceededException e)
{
//Max length of Name column exceeded. Show corresponding error message to user.
 }
}

Apart from this EntityFrameworkCore.Exceptions supports other database servers such as PostgreSQL and MySQL so if you ever switch to different database server your exception handling code will stay the same.

To get started with EntityFramework.Exceptions all you need to do is install either SQL Server, PostgreSQL or MySQL nuget package and inherit your DbContext from ExceptionProcessorContext:

class DemoContext : ExceptionProcessorContext
{
public DbSet<Product> Products { get; set; }
}

How Does EntityFramework.Exceptions Work ?

The implementation is pretty straightforward. There is an ExceptionProcessorContextBase class in EntityFramework.Exceptions.Common project which inherits from DbContext, overrides SaveChanges and handles any exception that occurs. It gets the database specific exception instance and asks derived classes to tell which exception it should throw for the DbException that occurred. For more details please check the GitHub repository. If you find the library useful don’t forget to Star the repository. If you have any questions or suggestions you are welcome to submit an issue or send a PR.

Introducing LINQPad.QueryPlanVisualizer

Tue, 15 Mar 2016 07:00:58 +0000

If you are a heavy LINQPad user you have probably wished to be able to see query execution plan details inside LINQPad. Currently the only way to view query execution plan is to switch to SQL tab, click Analyze SQL button and open the query in SQL Server Management Studio. I got tired of clicking all these buttons and decided to write custom visualizer which solves the issue. With LINQPad.QueryPlanVisualizer you can now view query execution plan and missing indexes as well as create those missing indexes without leaving LINQPad.

LINQPad.QueryPlanVisualizer

With LINQPad.QueryPlanVisualizer you can:

View query execution plan
View missing indexes for query
Create missing indexes
Open plan in SQL Server Management Studio or other default app
Save plan to xml file

LINQPad showing Query Execution Plan

Missing Index

How it works

The visualizer executes SET STATISTICS XML ON command before executing the actual query to retrieve the actual execution plan from database. After that it converts the xml plan to html and uses html-query-plan library to display the resulting html. Missing index details (if any) are extracted from the execution plan xml too.

For more details and installation guide visit the project at https://github.com/Giorgi/LINQPad.QueryPlanVisualizer

Faking num lock, caps lock and scroll lock leds

Thu, 10 May 2012 12:21:55 +0000

Introduction

Some time ago I came across a post where the author wanted to turn on and off num lock, cap lock and scroll lock led light without actually toggling their state. This sounded challenging and impossible but after some googling I found a piece of code in C for achieving exactly what I was looking for. In this post I show how it works and how to port it to C#

Toggling Keyboard Indicators from C#

The C version is available at rohitab.com As you can see from the code toggling the keyboard indicators consists of three steps:

Define MS-DOS device name for the target keyboard.
Get handle of the target keyboard.
Send command to the handle for changing keyboard indicators.

The first two steps are the pretty straightforward and easy to interop from C#. In order to define an MS-DOS name we need to call DefineDosDevice method pass the target keyboard path, desired MS-DOS device name and DDD_RAW_TARGET_PATH flag. The C# code looks like this:

[DllImport("kernel32.dll")]
static extern bool DefineDosDevice(uint dwFlags, string lpDeviceName, string lpTargetPath);
private const int DddRawTargetPath = 0x00000001;

We can call it like this:

DefineDosDevice(DddRawTargetPath, "keyboard", "\\Device\\KeyboardClass0");

Once we have defined Dos device name we can now use CreateFile to get handle to it. After that we use DeviceIoControl method to send IOCTL_KEYBOARD_SET_INDICATORS control code which indicates that we want to set keyboard indicator status. IOCTL_KEYBOARD_SET_INDICATORS control code is defined in Ntddkbd.h using CTL_CODE macro. In C# we can define it like this:

private const uint FileAnyAccess = 0;
private const uint MethodBuffered = 0;
private const uint FileDeviceKeyboard = 0x0000000b;
private static uint IOCTL_KEYBOARD_SET_INDICATORS =
ControlCode(FileDeviceKeyboard, 0x0002, MethodBuffered, FileAnyAccess);
static uint ControlCode(uint deviceType, uint function, uint method, uint access)
{
return ((deviceType) << 16) | ((access) << 14) | ((function) << 2) | (method);
}

The information about which indicators should be on and which should be off is specified by a KEYBOARD_INDICATOR_PARAMETERS structure.

The final code looks like this:

private static uint IOCTL_KEYBOARD_SET_INDICATORS = ControlCode(FileDeviceKeyboard, 0x0002, MethodBuffered, FileAnyAccess);
private const uint FileAnyAccess = 0;
private const uint MethodBuffered = 0;
private const uint FileDeviceKeyboard = 0x0000000b;
private const int DddRawTargetPath = 0x00000001;
public static void ToggleLights(Locks locks)
{
DefineDosDevice(DddRawTargetPath, "keyboard", "\\Device\\KeyboardClass0");
var indicators = new KeyboardIndicatorParameters();
using (var hKeybd = CreateFile("\\\\.\\keyboard", FileAccess.Write, 0, IntPtr.Zero, FileMode.Open, 0, IntPtr.Zero))
{
indicators.UnitId = 0;
indicators.LedFlags = locks;
var size = Marshal.SizeOf(typeof(KeyboardIndicatorParameters));
uint bytesReturned;
DeviceIoControl(hKeybd, IOCTL_KEYBOARD_SET_INDICATORS, ref indicators, (uint)size,
IntPtr.Zero, 0, out bytesReturned, IntPtr.Zero);
}
}
static uint ControlCode(uint deviceType, uint function, uint method, uint access)
{
return ((deviceType) << 16) | ((access) << 14) | ((function) << 2) | (method);
}
struct KeyboardIndicatorParameters
{
public ushort UnitId;
public Locks LedFlags;
}

[Flags]
public enum Locks : ushort
{
None = 0,
KeyboardScrollLockOn = 1,
KeyboardNumLockOn = 2,
KeyboardCapsLockOn = 4
}

Conclusion

Even though toggling keyboard indicators may not be very useful porting it to C# was interesting and fun. The only real application that I was able to find is an addon for Miranda IM which uses it as a notification mechanism.

Building Expression Evaluator with Expression Trees in C# – Improving the Api

Wed, 18 Apr 2012 09:03:09 +0000

This post is part of a series about building a simple mathematical expression evaluator. For previous posts and full source code see Table of Contents.

Introduction

In previous part we added support for variables but it has a big disadvantage: values of the variables are bound by position and not by name. This means that you should pass values for the variables in the same order as they appear in the expression. For example the following test will fail:

public void Can_Process_Multiple_Variables()
{
var a = 6;
var b = 4.32m;
var c = 24.15m;
Assert.That(engine.Evaluate("(c+b)*a", a, b, c), Is.EqualTo((c + b) * a));
}

On the other hand binding by name has following advantages: If you change the expression you do not have to worry about keeping order of the variables you pass in sync. So in this post we are going to change expression evaluator to bind parameters by name. We will also add a new overload for better performance when we need to evaluate same expression with different parameter values.

Binding Variables by Name

The easiest way to pass variables with their name is to pass a dictionary of variables with variable name acting as a key. While this will work it is not the most elegant and readable solution. A better solution will be to pass an anonymous object with properties pointing to the variable values. So our first task is to make the following test pass:

public void Can_Process_Multiple_Variables()
{
var a = 6;
var b = 4.32m;
var c = 24.15m;
Assert.That(engine.Evaluate("(c+b)*a", new { a, b, c }), Is.EqualTo((c + b) * a));
}

Passing Parameters with Anonymous Objects

The first thing we need to do when an object containing variables is passed is to enumerate its properties and build a dictionary of variable name and value. Once we have the dictionary we pass it to the private Enumerate method which we implemented in previous post. The code is pretty straightforward:

public decimal Evaluate(string expression, object argument = null)
{
if (argument == null)
{
return Evaluate(expression, new Dictionary<string, decimal>());
}
var argumentType = argument.GetType();
var properties = argumentType.GetProperties(BindingFlags.Instance | BindingFlags.Public)
.Where(p => p.CanRead && IsNumeric(p.PropertyType));
var arguments = properties.ToDictionary(property => property.Name,
property => Convert.ToDecimal(property.GetValue(argument, null)));
return Evaluate(expression, arguments);
}
private bool IsNumeric(Type type)
{
switch (Type.GetTypeCode(type))
{
case TypeCode.SByte:
case TypeCode.Byte:
case TypeCode.Int16:
case TypeCode.UInt16:
case TypeCode.Int32:
case TypeCode.UInt32:
case TypeCode.Int64:
case TypeCode.UInt64:
case TypeCode.Single:
case TypeCode.Double:
case TypeCode.Decimal:
return true;
}
return false;
}

The Enumerate method has not changed much. The most important change is that it needs to reorder the variables in the same order as they appear in the expression.

private decimal Evaluate(string expression, Dictionary<string, decimal> arguments)
{
if (string.IsNullOrWhiteSpace(expression))
{
return 0;
}
if (arguments == null)
{
arguments = new Dictionary<string, decimal>();
}
var arrayParameter = Expression.Parameter(typeof(decimal[]), "args");
parameters.Clear();
operatorStack.Clear();
expressionStack.Clear();
using (var reader = new StringReader(expression))
{
int peek;
while ((peek = reader.Peek()) > -1)
{
var next = (char)peek;
if (char.IsDigit(next))
{
expressionStack.Push(ReadOperand(reader));
continue;
}
if (char.IsLetter(next))
{
expressionStack.Push(ReadParameter(reader, arrayParameter));
continue;
}
if (Operation.IsDefined(next))
{
var currentOperation = ReadOperation(reader);
EvaluateWhile(() => operatorStack.Count > 0 && operatorStack.Peek() != '(' &&
currentOperation.Precedence <= ((Operation)operatorStack.Peek()).Precedence);
operatorStack.Push(next);
continue;
}
if (next == '(')
{
reader.Read();
operatorStack.Push('(');
continue;
}
if (next == ')')
{
reader.Read();
EvaluateWhile(() => operatorStack.Count > 0 && operatorStack.Peek() != '(');
operatorStack.Pop();
continue;
}
if (next == ' ')
{
reader.Read();
}
else
{
throw new ArgumentException(string.Format("Encountered invalid character {0}", next),
"expression");
}
}
}
EvaluateWhile(() => operatorStack.Count > 0);
var lambda = Expression.Lambda<Func<decimal[], decimal>>(expressionStack.Pop(), arrayParameter);
var compiled = lambda.Compile();
var values = parameters.Select(parameter => arguments[parameter]).ToArray();
return compiled(values);
}

This is all we need to add support for binding arguments by name to our expression evaluator. We will now add another overload for passing parameters to the Evaluate method.

Passing Parameters with Named Arguments

Passing arguments by anonymous objects works but we can do better: We can use named arguments together with dynamic keyword to allow specifying arguments in the following way:

[Test]
public void Can_Pass_Named_Variables()
{
dynamic dynamicEngine = new ExpressionEvaluator();
var a = 6;
var b = 4.5m;
var c = 2.6m;
Assert.That(dynamicEngine.Evaluate("(c+b)*a", a: 6, b: 4.5, c: 2.6),
Is.EqualTo((c + b) * a));
}

As it is based on features introduced in .Net 4 this overload will only be available for users using .Net 4 or upper.

To implement this feature we need to inherit ExpressionEvaluator class from DynamicObject and override TryInvokeMember method. This method will get invoked whenever a method is called which should be resolved at runtime. Parameter names as well as values are passed to this method. This allows us to build a dictionary of arguments and pass them to the Enumerate method showed in this post. This is how we do it:

public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result)
{
if ("Evaluate" != binder.Name)
{
return base.TryInvokeMember(binder, args, out result);
}
if (!(args[0] is string))
{
throw new ArgumentException("No expression specified for parsing");
}
//args will contain expression and arguments,
 //ArgumentNames will contain only argument names
 if (args.Length != binder.CallInfo.ArgumentNames.Count + 1)
{
throw new ArgumentException("Argument names missing.");
}
var arguments = new Dictionary<string, decimal>();
for (int i = 0; i < binder.CallInfo.ArgumentNames.Count; i++)
{
if (IsNumeric(args[i + 1].GetType()))
{
arguments.Add(binder.CallInfo.ArgumentNames[i], Convert.ToDecimal(args[i + 1]));
}
}
result = Evaluate((string)args[0], arguments);
return true;
}

Now we can specify parameter names in a more elegant way without having to construct an anonymous object.

Improving Expression Evaluator Performance

Let’s say you have an expression which you want to evaluate multiple times but with different parameters. Currently the Evaluate method will parse and compile the expression tree for every evaluation even thought the expression is same every time. As the compilation of expression trees is an expensive operation it will be better if the library allowed to compile the expression only once and execute it multiple times with different values of variables.

To support this scenario we will add a new method Compile which takes an expression and returns a delegate which you can execute multiple times. Here is an example of what we want to do:

[Test]
public void Can_Invoke_Expression_Multiple_Times()
{
var a = 6m;
var b = 3.9m;
var c = 4.9m;
var compiled = engine.Compile("(a+b)/(a+c)");
Assert.That(compiled(new { a, b, c }), Is.EqualTo((a + b) / (a + c)));
a = 5.4m;
b = -2.4m;
c = 7.5m;
Assert.That(compiled(new { a, b, c }), Is.EqualTo((a + b) / (a + c)));
}

The Compile method is the same as the Evaluate method with the only difference being that it returns the compiled delegate instead of invoking it. This means we can extract a new method and use it from both methods. I will not show the full source code here as it is does not add any new parsing logic and the code is available at github.

Conclusion

In this post we improved expression evaluator and added support for binding variables by name. We also added a new overload for specifying variables as named arguments. Finally we added a new method returning compiled delegate to avoid multiple compilations.

In the next post we will add support for unary negation. Do not miss it

Using Reactive Extensions for Streaming Data from Database

Mon, 02 Apr 2012 22:04:03 +0000

You have probably heard about Reactive Extensions, a library from Microsoft that greatly simplifies working with asynchronous data streams and allows to query them with Linq operators. There are many different scenarios where using rx results in a much more simple and flexible code. This post demonstrates how to use reactive extensions for loading data from database asynchronously in chunks.

The Problem

Recently I had to load data from sqlite database in grid view but the query was taking long time as there were hundreds of thousands of rows and the query was doing like %search term% filtering. The customer wanted to run the query in background and load data in chunks as they became available similar to sql server management studio. This way the grid would show first 200 available rows in the beginning and then when the next 200 rows were loaded they would be added to the grid and so on until the query completes. Here are some screenshots that demonstrate the desired functionality:

Solution without Reactive Extensions

The most straightforward way to solve the problem is to call the method that reads the data in background thread and send the data to main thread when there are enough rows. Here is how to do it in WinForms application using backgroundworker:

internal class CompanyDal
{
public IEnumerable<Company> GetCompanies(string title)
{
using (var connection = new SQLiteConnection(connectionString))
{
using (var cmd = connection.CreateCommand())
{
cmd.CommandText =
"select company, category, state, phone from maindata where company like @name";
cmd.Parameters.AddWithValue("@name", String.Format("%{0}%", title));
connection.Open();
var reader = cmd.ExecuteReader();
using (reader)
{
while (reader.Read())
{
var company = new Company
{
Title = reader.GetString(0),
Category = reader.GetString(1),
State = reader.GetString(2),
Phone = reader.GetString(3)
};
yield return company;
}
}
}
}
}
}

private void WorkerDoWork(object sender, DoWorkEventArgs e)
{
var data = new List<Company>();
foreach (var company in companyDal.GetCompanies(e.Argument.ToString()))
{
data.Add(company);
if (data.Count % BufferSize == 0)
{
worker.ReportProgress(0, data.ToArray());
data.Clear();
}
}
worker.ReportProgress(0, data.ToArray());
}
private void WorkerProgressChanged(object sender, ProgressChangedEventArgs e)
{
var data = e.UserState as IEnumerable<Company>;
if (data != null)
{
var index = Math.Max(companyGridView.FirstDisplayedScrollingRowIndex, 0);
companies.AddRange(data);
companyBindingSource.ResetBindings(false);
progressLabel.Text = string.Format("Loaded {0} rows", companies.Count);
companyGridView.FirstDisplayedScrollingRowIndex = index;
}
}

The solution works but it can easily become complicated if you want to have more control when the data is loaded or want to be able to stop loading data. For example if you want to load next chunk every 2 seconds or load data when user clicks load button then most part of the code will be dealing with thread synchronization and it will be difficult to understand what is the code really doing. Let’s see how reactive extensions can help.

Reactive Extensions – Simplifying the Solution

To solve the problem with rx first need to convert the source collection which in our case is matching companies to observable stream. We can do it with ToObservable method. We want the query to run on background thread so we pass a thread pool scheduler as a parameter. Now we need to tell the stream that we want to get data in chunks. We do it by calling Observable.Buffer method and pass desired chunk size. As we want the observer to provide notifications on the gui thread we call ObserveOn method and pass SynchronizationContext.Current. Finally we subscribe to the observable to actually process the data:

private void SearchButtonClick(object sender, EventArgs e)
{
companies.Clear();
companyBindingSource.ResetBindings(false);
stopButton.Enabled = true;
searchButton.Enabled = false;
progressLabel.Text = string.Format("Searching for {0} ...", searchBox.Text);
companyDal.GetCompanies(searchBox.Text).ToObservable(Scheduler.ThreadPool).
Buffer(BufferSize).ObserveOn(SynchronizationContext.Current).
Subscribe(loadedData =>
{
var index = Math.Max(companyGridView.FirstDisplayedScrollingRowIndex, 0);
companies.AddRange(loadedData);
companyBindingSource.ResetBindings(false);
progressLabel.Text = string.Format("Loaded {0} rows", companies.Count);
companyGridView.FirstDisplayedScrollingRowIndex = index;
},
exception => { progressLabel.Text = exception.Message; },
() => { progressLabel.Text = "Finished loading data"; });
}

We now have basic solution implemented with just a few lines of codes. The important points are that there is practically no threading code, no need to implement partial loading manually and as we will see the above code is easy to extend.

Extending Reactive Extensions Solution

First of all we want to toggle search and stop button state when the search is completed. We could do that in onError and onCompleted callbacks but that would result in duplicated code. Instead we can use Observable.Finally method which executes in both cases. As we want the Finally callback to execute on the gui thread too we append it to ObserveOn method call.

Secondly we want the query to stop when stop button is clicked. The most straightforward way is to store Subscribe method return value in a field and dispose subscription when stop button is clicked but as you have probably guessed there is ‘reactive’ way to do the same. In the form constructor we create an observable from the stop button click event by using Observable.FromEventPattern method. We then combine it with our data stream by TakeUntil method which tells the data stream to return records until the click event observable produces a value:

public MainForm()
{
InitializeComponent();
companyBindingSource.DataSource = companies;
stopClicked = Observable.FromEventPattern<EventArgs>(stopButton, "Click");
}
private void SearchButtonClick(object sender, EventArgs e)
{
companies.Clear();
companyBindingSource.ResetBindings(false);
stopButton.Enabled = true;
searchButton.Enabled = false;
progressLabel.Text = string.Format("Searching for {0} ...", searchBox.Text);
companyDal.GetCompanies(searchBox.Text).ToObservable(Scheduler.ThreadPool).
Buffer(BufferSize).TakeUntil(stopClicked).
ObserveOn(SynchronizationContext.Current).
Finally(() =>
{
searchButton.Enabled = true;
stopButton.Enabled = false;
}).
Subscribe(loadedData =>
{
var index = Math.Max(companyGridView.FirstDisplayedScrollingRowIndex, 0);
companies.AddRange(loadedData);
companyBindingSource.ResetBindings(false);
progressLabel.Text = string.Format("Loaded {0} rows", companies.Count);
companyGridView.FirstDisplayedScrollingRowIndex = index;
},
exception => { progressLabel.Text = exception.Message; },
() => { progressLabel.Text = "Finished loading data"; });
}

Finally let’s have a look at how easily we can change buffering logic. The buffer method has many overloads so if we want to buffer data based on time spans we can simply use the overload that takes a timespan instance as a parameter. We can also combine buffer size with timespan so that buffer is created when any of the conditions are met. To show a little but more useful example imagine that we want to load next chunk of data when user clicks load button. We can achieve it by using buffer overload which takes another observable indicating when to create next buffer. As with stop button we first create an observable from load button click event and then we combine our observables like this:

private void SearchButtonClick(object sender, EventArgs e)
{
companies.Clear();
companyBindingSource.ResetBindings(false);
searchButton.Enabled = false;
loadButton.Enabled = stopButton.Enabled = true;
progressLabel.Text = string.Format("Searching for {0} ...", searchBox.Text);
companyDal.GetCompanies(searchBox.Text).ToObservable(Scheduler.ThreadPool).
Buffer(() => loadClicked).TakeUntil(stopClicked).
ObserveOn(SynchronizationContext.Current).
Finally(() =>
{
searchButton.Enabled = true;
loadButton.Enabled = stopButton.Enabled = false;
}).
Subscribe(loadedData =>
{
var index = Math.Max(companyGridView.FirstDisplayedScrollingRowIndex, 0);
companies.AddRange(loadedData);
companyBindingSource.ResetBindings(false);
progressLabel.Text = string.Format("Loaded {0} rows", companies.Count);
companyGridView.FirstDisplayedScrollingRowIndex = index;
},
exception => { progressLabel.Text = exception.Message; },
() => { progressLabel.Text = "Finished loading data"; });
}

Conclusion

As we can see reactive extensions enables us to solve complex problems very easily. It makes working with asynchronous streams simple and allow us to concentrate on business logic instead of managing threads manually. For more information about rx and its internals I highly recommend Channel 9 videos.

Have you used rx to solve real life problems? Did you it help you implement complex solutions in a concise way? Share your experience and thoughts about it in comments.

Building Expression Evaluator with Expression Trees in C# – Part 3

Wed, 21 Mar 2012 08:14:52 +0000

Introduction

In part two of this series we built an expression evaluator capable of parsing expressions with parentheses. In this part we are going to add support for expression with variables.

Adding support for single variable to expression evaluator

The simplest case is to make the following test pass:

[Test]
public void Can_Process_Simple_Variables()
{
decimal a = 2.6m;
Assert.That(engine.Evaluate("a", a), Is.EqualTo(a));
}

First of all we need to add a second parameter to Evaluate method. To keep the existing tests green we will also specify a default value for it. Running the test causes Evaluate method to throw an exception as it does not recognize variables. To avoid exception during parsing the expression when we encounter a letter we will read the variable name and as variables are similar to numeric operands store them on expression stack. To represent the variables in expression tree we are going to use Expression.Parameter Method which is a factory method for creating instances of ParameterExpression class. The method for parsing variables looks like this:

private ParameterExpression ReadParameter(TextReader reader)
{
var operand = string.Empty;
int peek;
while ((peek = reader.Peek()) > -1)
{
var next = (char)peek;
if (char.IsLetter(next))
{
reader.Read();
operand += next;
}
else
{
break;
}
}
var parameter = Expression.Parameter(typeof(decimal), operand);
return parameter;
}

This change solves the issue of parsing variables but running the test causes an InvalidOperationException as we are not passing any parameters to Expression.Lambda method. To fix this issue we need to keep track of all variables that we encounter while parsing the expression and pass them to Expression.Lambda method. As we need to pass the value of the parameter to the compiled expression tree we also have to change type of the delegate.

The Evaluate method now looks like this:

public decimal Evaluate(string expression, decimal variable1 = 0)
{
if (string.IsNullOrWhiteSpace(expression))
{
return 0;
}
operatorStack.Clear();
expressionStack.Clear();
using (var reader = new StringReader(expression))
{
int peek;
while ((peek = reader.Peek()) > -1)
{
var next = (char)peek;
if (char.IsDigit(next))
{
expressionStack.Push(ReadOperand(reader));
continue;
}
if (char.IsLetter(next))
{
var parameterExpression = ReadParameter(reader);
parameters.Add(parameterExpression);
expressionStack.Push(parameterExpression);
continue;
}
if (Operation.IsDefined(next))
{
var currentOperation = ReadOperation(reader);
EvaluateWhile(() => operatorStack.Count > 0 && operatorStack.Peek() != '(' &&
currentOperation.Precedence <= ((Operation)operatorStack.Peek()).Precedence);
operatorStack.Push(next);
continue;
}
if (next == '(')
{
reader.Read();
operatorStack.Push('(');
continue;
}
if (next == ')')
{
reader.Read();
EvaluateWhile(() => operatorStack.Count > 0 && operatorStack.Peek() != '(');
operatorStack.Pop();
continue;
}
if (next != ' ')
{
throw new ArgumentException(string.Format("Encountered invalid character {0}", next), "expression");
}
}
}
EvaluateWhile(() => operatorStack.Count > 0);
var compiled = Expression.Lambda<Func<decimal, decimal>>(expressionStack.Pop(), parameters).Compile();
return compiled(variable1);
}

The new test now passes but we have successfully broken previous test when we do not pass any variable. To make all tests pass again we will need to build the delegate based on number of parameters and pass corresponding variable if there is one. The simplest solution is to build the appropriate delegate based on number of parameters like this:

if (parameters.Count > 0)
{
var compiled = Expression.Lambda<Func<decimal, decimal>>(expressionStack.Pop(), parameters).Compile();
return compiled(variable1);
}
else
{
var compiled = Expression.Lambda<Func<decimal>>(expressionStack.Pop()).Compile();
return compiled();
}

This works but it only supports one parameter and this approach cannot be used when we do not know how many variables will be in the expression. Also, the Func delegate supports only 16 parameters and it would be very ugly to use it.

Adding support for multiple variables

A better solution is to build and expression tree that accepts an array of variables. This way when there is no parameter we will simply pass an empty array and when there are several parameters we will simply pass them. The delegate signature will be the same in all situations so we will not need any if else statements to handle different scenarios.

To help us with the implementation of new approach we are going to add these tests to our test suite:

[Test]
public void Can_Process_Simple_Variables()
{
decimal a = 2.6m;
decimal b = 5.7m;
Assert.That(engine.Evaluate("a", a), Is.EqualTo(a));
Assert.That(engine.Evaluate("a+a", a), Is.EqualTo(a + a));
Assert.That(engine.Evaluate("a+b", a, b), Is.EqualTo(a + b));
}

[Test]
public void Can_Process_Multiple_Variables()
{
var a = 6;
var b = 4.32m;
var c = 24.15m;
Assert.That(engine.Evaluate("(((9-a/2)*2-b)/2-a-1)/(2+c/(2+4))", a, b, c),
Is.EqualTo((((9 - a / 2) * 2 - b) / 2 - a - 1) / (2 + c / (2 + 4))));
}

The expression tree that we are going to build will have one array parameter. To access individual members of this array in the expression we will use Expression.ArrayIndex Method We will also change the Evaluate method to accept variable number of parameters by params keyword. Let’s look at the modified code:

public decimal Evaluate(string expression, params decimal[] arguments)
{
if (string.IsNullOrWhiteSpace(expression))
{
return 0;
}
operatorStack.Clear();
expressionStack.Clear();
var arrayParameter = Expression.Parameter(typeof(decimal[]), "args");
using (var reader = new StringReader(expression))
{
int peek;
while ((peek = reader.Peek()) > -1)
{
var next = (char)peek;
if (char.IsDigit(next))
{
expressionStack.Push(ReadOperand(reader));
continue;
}
if (char.IsLetter(next))
{
var parameterExpression = ReadParameter(reader, arrayParameter);
expressionStack.Push(parameterExpression);
continue;
}
if (Operation.IsDefined(next))
{
var currentOperation = ReadOperation(reader);
EvaluateWhile(() => operatorStack.Count > 0 && operatorStack.Peek() != '(' &&
currentOperation.Precedence <= ((Operation)operatorStack.Peek()).Precedence);
operatorStack.Push(next);
continue;
}
if (next == '(')
{
reader.Read();
operatorStack.Push('(');
continue;
}
if (next == ')')
{
reader.Read();
EvaluateWhile(() => operatorStack.Count > 0 && operatorStack.Peek() != '(');
operatorStack.Pop();
continue;
}
if (next != ' ')
{
throw new ArgumentException(string.Format("Encountered invalid character {0}", next),
"expression");
}
}
}
EvaluateWhile(() => operatorStack.Count > 0);
var compiled = Expression.Lambda<Func<decimal[], decimal>>(expressionStack.Pop(), arrayParameter).Compile();
return compiled(arguments);
}

You will notice that there are several changes from the previous version: There is only one parameter in the expression tree we are building, we are passing this parameter to the ReadParameter method and we are generating the same type of delegate for all cases.

The only remaining change is need in ReadParameter method so that it returns correct item from the parameter array when we encounter a variable. We also need to make sure that we return the same item when the same variable is used again the expression. For this we will need to keep track of the variables and return corresponding item from the ReadParameter method. Here is how it works:

private Expression ReadParameter(TextReader reader, Expression arrayParameter)
{
var parameter = string.Empty;
int peek;
while ((peek = reader.Peek()) > -1)
{
var next = (char)peek;
if (char.IsLetter(next))
{
reader.Read();
parameter += next;
}
else
{
break;
}
}
if (!parameters.Contains(parameter))
{
parameters.Add(parameter);
}
return Expression.ArrayIndex(arrayParameter, Expression.Constant(parameters.IndexOf(parameter)));
}

As you can see whenever we encounter a variable we add it to parameter collection if it is not already present and then build an array index expression using corresponding index.

Using expression tree visualizer we can see how the expression tree that we build using the expression evaluator for last test case looks like:

Conclusion

Our new implementation of expression evaluator now supports expressions that contain numbers as well as variables. In the next post we will add several overloads to make passing parameters easier and add a new overload for better performance when we need to evaluate same expression with different parameter values. Full source code will be made available on github in couple of days.

Building Expression Evaluator with Expression Trees in C# – Part 2

Tue, 28 Feb 2012 07:41:10 +0000

Introduction

In previous post I showed how to build a simple expression evaluator using expression trees. Although it works fine it has some drawbacks:

It does not support parentheses.
It does not support variables.
In some cases the result can be wrong as the parser is not using left to right parsing.
It is compiling delegates for every expression which results in slower execution.

To solve these issues we will use a different algorithm and deal with all points except variable support.

Implementing expression evaluator using shunting yard algorithm

The algorithm that we are going to implement is called Shunting yard algorithm and works like this:

If the next token is number add it to operand stack.
If it is an operation token and operator stack is empty push it to operator stack.
If it is an operation token and operator stack is not empty and current operation has higher precedence push it to operator stack. If not pop the operator from stack and evaluate it with arguments from the operand stack and jump to step 2.
When we reach the end evaluate all operations on the stack

Translating this into code is pretty straightforward and results in following expression evaluator:

public decimal Evaluate(string expression)
{
if (string.IsNullOrWhiteSpace(expression))
{
return 0;
}
var expressionStack = new Stack<Expression>();
var operatorStack = new Stack<char>();
using (var reader = new StringReader(expression))
{
int peek;
while ((peek = reader.Peek()) > -1)
{
var next = (char)peek;
if (char.IsDigit(next))
{
expressionStack.Push(ReadOperand(reader));
continue;
}
if (Operation.IsDefined(next))
{
var currentOperation = ReadOperation(reader);
while (true)
{
if (operatorStack.Count == 0)
{
operatorStack.Push(next);
break;
}
var lastOperition = operatorStack.Peek();
if (currentOperation.Precedence > ((Operation)lastOperition).Precedence)
{
operatorStack.Push(next);
break;
}
var right = expressionStack.Pop();
var left = expressionStack.Pop();
expressionStack.Push(((Operation)operatorStack.Pop()).Apply(left, right));
}
continue;
}
if (next != ' ')
{
throw new ArgumentException("Invalid character encountered", "expression");
}
}
}
while (operatorStack.Count > 0)
{
var right = expressionStack.Pop();
var left = expressionStack.Pop();
expressionStack.Push(((Operation)operatorStack.Pop()).Apply(left, right));
}
var compiled = Expression.Lambda<Func<decimal>>(expressionStack.Pop()).Compile();
return compiled();
}

internal sealed class Operation
{
private readonly int precedence;
private readonly string name;
private readonly Func<Expression, Expression, Expression> operation;
public static readonly Operation Addition = new Operation(1, Expression.Add, "Addition");
public static readonly Operation Subtraction = new Operation(1, Expression.Subtract, "Subtraction");
public static readonly Operation Multiplication = new Operation(2, Expression.Multiply, "Multiplication");
public static readonly Operation Division = new Operation(2, Expression.Divide, "Division");
private static readonly Dictionary<char, Operation> Operations = new Dictionary<char, Operation>
{
{ '+', Addition },
{ '-', Subtraction },
{ '*', Multiplication},
{ '/', Division }
};
private Operation(int precedence, Func<Expression, Expression, Expression> operation, string name)
{
this.precedence = precedence;
this.operation = operation;
this.name = name;
}
public int Precedence
{
get { return precedence; }
}
public static explicit operator Operation(char operation)
{
Operation result;
if (Operations.TryGetValue(operation, out result))
{
return result;
}
else
{
throw new InvalidCastException();
}
}
public Expression Apply(Expression left, Expression right)
{
return operation(left, right);
}
public static bool IsDefined(char operation)
{
return Operations.ContainsKey(operation);
}
}

This code passes all the tests we have in previous post and solves issues three and four. Let’s refactor it and extract common code in a separate method:

private void EvaluateWhile(Func<bool> condition)
{
while (condition())
{
var right = expressionStack.Pop();
var left = expressionStack.Pop();
expressionStack.Push(((Operation)operatorStack.Pop()).Apply(left, right));
}
}

The evaluate method now looks like this:

public decimal Evaluate(string expression)
{
if (string.IsNullOrWhiteSpace(expression))
{
return 0;
}
operatorStack.Clear();
expressionStack.Clear();
using (var reader = new StringReader(expression))
{
int peek;
while ((peek = reader.Peek()) > -1)
{
var next = (char)peek;
if (char.IsDigit(next))
{
expressionStack.Push(ReadOperand(reader));
continue;
}
if (Operation.IsDefined(next))
{
var currentOperation = ReadOperation(reader);
EvaluateWhile(() => operatorStack.Count > 0 &&
currentOperation.Precedence <= ((Operation)operatorStack.Peek()).Precedence);
operatorStack.Push(next);
continue;
}
if (next != ' ')
{
throw new ArgumentException(string.Format("Encountered invalid character {0}", next), "expression");
}
}
}
EvaluateWhile(() => operatorStack.Count > 0);
var compiled = Expression.Lambda<Func<decimal>>(expressionStack.Pop()).Compile();
return compiled();
}

At this point we are ready to add support for parentheses. Our goal is to make the following test pass:

[Test]
public void Supports_Parentheses()
{
Assert.That(engine.Evaluate("2*(5+3)"), Is.EqualTo(2 * (5 + 3)));
Assert.That(engine.Evaluate("(5+3)*2"), Is.EqualTo((5 + 3) * 2));
Assert.That(engine.Evaluate("(5+3)*5-2"), Is.EqualTo((5 + 3) * 5 - 2));
Assert.That(engine.Evaluate("(5+3)*(5-2)"), Is.EqualTo((5 + 3) * (5 - 2)));
Assert.That(engine.Evaluate("((5+3)*3-(8-2)/2)/2"), Is.EqualTo(((5 + 3) * 3 - (8 - 2) / 2) / 2m));
Assert.That(engine.Evaluate("(4*(3+5)-4-8/2-(6-4)/2)*((2+4)*4-(8-5)/3)-5"),
Is.EqualTo((4 * (3 + 5) - 4 - 8 / 2 - (6 - 4) / 2) * ((2 + 4) * 4 - (8 - 5) / 3) - 5));
Assert.That(engine.Evaluate("(((9-6/2)*2-4)/2-6-1)/(2+24/(2+4))"),
Is.EqualTo((((9 - 6 / 2) * 2 - 4) / 2m - 6 - 1) / (2 + 24 / (2 + 4))));
}

We only need to add the following changes:

If the token is left parentheses push it to operator stack.
If the token is right parentheses evaluate all operators in operator stack until we reach left parentheses

The first point is really straightforward and for the second one we can use the EvaluateWhile method we introduced above. After adding this code our method looks like this:

public decimal Evaluate(string expression)
{
if (string.IsNullOrWhiteSpace(expression))
{
return 0;
}
operatorStack.Clear();
expressionStack.Clear();
using (var reader = new StringReader(expression))
{
int peek;
while ((peek = reader.Peek()) > -1)
{
var next = (char)peek;
if (char.IsDigit(next))
{
expressionStack.Push(ReadOperand(reader));
continue;
}
if (Operation.IsDefined(next))
{
var currentOperation = ReadOperation(reader);
EvaluateWhile(() => operatorStack.Count > 0 && operatorStack.Peek() != '(' &&
currentOperation.Precedence <= ((Operation)operatorStack.Peek()).Precedence);
operatorStack.Push(next);
continue;
}
if (next == '(')
{
reader.Read();
operatorStack.Push('(');
continue;
}
if (next == ')')
{
reader.Read();
EvaluateWhile(() => operatorStack.Count > 0 && operatorStack.Peek() != '(');
operatorStack.Pop();
continue;
}
if (next != ' ')
{
throw new ArgumentException(string.Format("Encountered invalid character {0}", next), "expression");
}
}
}
EvaluateWhile(() => operatorStack.Count > 0);
var compiled = Expression.Lambda<Func<decimal>>(expressionStack.Pop()).Compile();
return compiled();
}

Conclusion

Modifying our expression evaluator using shunting yard algorithm was pretty easy and has several advantages compared to previous implementation. In the next post we will add support for parsing expressions with variables.

Building Expression Evaluator with Expression Trees in C# – Part 1

Thu, 23 Feb 2012 13:00:15 +0000

Introduction

This is first part of a series about writing expression evaluator in C#. Even though there are several of them available already I decided to write my own as this one is based on expression trees and is very simple. The evaluator supports numeric expressions as well as expression with parameters.

In this part we will build a simple expression evaluator which will support expressions like this: “2+2″, “2.7+3.2″, “1.7+2.9+14.24+6.58″, “84+15+4-4*3*9+24+4-54/3-5-7+47″, “25/3+1.34*2.56+1.49+2.36/1.48″, etc. It will however not be able to parse expression with parentheses or with variables. These features will be implemented in the coming posts.

Getting Started

As the evaluator is based on expression trees it you should have a basic understanding of how they work. I will not cover them here as there are many good articles available out there.

The first thing we need to do is to make the following to tests pass:

[TestFixture]
class ExpressionEvaluatorTests
{
private BasicExpressionEvaluator evaluator;
private Random generator;

 [TestFixtureSetUp]
public void SetUp()
{
evaluator = new BasicExpressionEvaluator();
generator = new Random();
}

 [Test]
public void Empty_String_Is_Zero()
{
Assert.That(evaluator.Evaluate(" "), Is.EqualTo(0));
}

 [Test]
public void Decimal_Is_Treated_As_Decimal()
{
var left = generator.Next(1, 100);
var right = generator.Next(1, 100);
decimal input = left + right / 100m;
Assert.That(evaluator.Evaluate(input.ToString()), Is.EqualTo(input));
}
}

The tests are very simple and the necessary code to make them pass is quite straightforward too:

public class BasicExpressionEvaluator
{
public decimal Evaluate(string expression)
{
decimal value = 0;
decimal.TryParse(expression, out value);
return value;
}
}

Let’s add support for adding to decimal numbers. Here is the test we need to pass:

[Test]
public void Can_Add_Two_Decimal_Numbers()
{
Assert.That(evaluator.Evaluate("2.7+3.2"), Is.EqualTo(2.7m + 3.2m));
}

Implementation is again quite easy. All we have to do it parse the operands and construct corresponding Expression by calling Expression.Add Method.

public decimal Evaluate(string expression)
{
if (expression.Contains('+'))
{
var parts = expression.Split('+');
Expression sum = Expression.Add(
Expression.Constant(decimal.Parse(parts[0])),
Expression.Constant(decimal.Parse(parts[1])));
var lambda = Expression.Lambda<Func<decimal>>(sum);
var compiled = lambda.Compile();
return compiled();
}
decimal value = 0;
decimal.TryParse(expression, out value);
return value;
}

We can also extend it and implement adding multiple numbers:

public decimal Evaluate(string expression)
{
if (expression.Contains('+'))
{
var parts = expression.Split('+');
Expression sum = Expression.Constant(decimal.Parse(parts[0]));
for (int i = 1; i < parts.Length; i++)
{
sum = Expression.Add(sum, Expression.Constant(decimal.Parse(parts[i])));
}
var lambda = Expression.Lambda<Func<decimal>>(sum);
var compiled = lambda.Compile();
return compiled();
}
decimal value = 0;
decimal.TryParse(expression, out value);
return value;
}

We can take advantage of Enumerable.Aggregate method and refactor the above snippet like this:

public decimal Evaluate(string expression)
{
if (expression.Contains('+'))
{
var parts = expression.Split('+');
var sum = parts.Aggregate(Expression.Constant(0m) as Expression,
(current, next) =>
Expression.Add(current, Expression.Constant(decimal.Parse(next))));
var lambda = Expression.Lambda<Func<decimal>>(sum);
var compiled = lambda.Compile();
return compiled();
}
decimal value = 0;
decimal.TryParse(expression, out value);
return value;
}

Next step is to support subtraction. First we will process expressions like “5-2″ and “15.2-2.3-4.8-0.58″. As with additions we will split the string and subtract the terms:

public decimal Evaluate(string expression)
{
if (expression.Contains('+'))
{
var parts = expression.Split('+');
var sum = parts.Aggregate(Expression.Constant(0m) as Expression,
(current, next) => Expression.Add(current, Expression.Constant(decimal.Parse(next))));
var lambda = Expression.Lambda<Func<decimal>>(sum);
var compiled = lambda.Compile();
return compiled();
}
if (expression.Contains('-'))
{
var parts = expression.Split('-');
Expression sum = Expression.Constant(decimal.Parse(parts[0]));
for (int i = 1; i < parts.Length; i++)
{
sum = Expression.Subtract(sum, Expression.Constant(decimal.Parse(parts[i])));
}
var lambda = Expression.Lambda<Func<decimal>>(sum);
var compiled = lambda.Compile();
return compiled();
}
decimal value = 0;
decimal.TryParse(expression, out value);
return value;
}

We can refactor it again by using the Aggregate method again. However with subtraction we cannot start subtracting from zero as we did with addition. Instead we have to start subtracting from the first term:

public decimal Evaluate(string expression)
{
if (expression.Contains('+'))
{
var parts = expression.Split('+');
var sum = parts.Aggregate(Expression.Constant(0m) as Expression,
(current, next) => Expression.Add(current, Expression.Constant(decimal.Parse(next))));
var lambda = Expression.Lambda<Func<decimal>>(sum);
var compiled = lambda.Compile();
return compiled();
}
if (expression.Contains('-'))
{
var parts = expression.Split('-');
Expression sum = Expression.Constant(decimal.Parse(parts[0]));
sum = parts.Skip(1).Aggregate(sum, (current, next) =>
Expression.Subtract(current, Expression.Constant(decimal.Parse(next))));
var lambda = Expression.Lambda<Func<decimal>>(sum);
var compiled = lambda.Compile();
return compiled();
}
decimal value = 0;
decimal.TryParse(expression, out value);
return value;
}

Now, let’s mix additions and subtractions in one expression. We want the following test to pass:

[Test]
public void Can_Add_And_Subtract_Multiple_Numbers()
{
Assert.That(evaluator.Evaluate("15+8-4-2+7"),
Is.EqualTo(15 + 8 - 4 - 2 + 7));
Assert.That(evaluator.Evaluate("17.89-2.47+7.16"),
Is.EqualTo(17.89m - 2.47m + 7.16m));
}

Currently the test fails with FormatException being thrown as we are trying to parse strings. To avoid the exception and pass the test instead of parsing the terms we will recursively evaluate the term and parse the result. This is how it looks:

public decimal Evaluate(string expression)
{
if (expression.Contains('+'))
{
var parts = expression.Split('+');
var sum = parts.Aggregate(Expression.Constant(0m) as Expression,
(current, next) =>
Expression.Add(current,
Expression.Constant(Evaluate(next))));
var lambda = Expression.Lambda<Func<decimal>>(sum);
var compiled = lambda.Compile();
return compiled();
}
if (expression.Contains('-'))
{
var parts = expression.Split('-');
Expression sum = Expression.Constant(decimal.Parse(parts[0]));
sum = parts.Skip(1).Aggregate(sum, (current, next) =>
Expression.Subtract(current, Expression.Constant(decimal.Parse(next))));
var lambda = Expression.Lambda<Func<decimal>>(sum);
var compiled = lambda.Compile();
return compiled();
}
decimal value = 0;
decimal.TryParse(expression, out value);
return value;
}

Notice that the only change is in the part of the code which processes additions.

Adding multiplication and division

Now we are ready to add support for multiplication and division. Evaluating expressions which contain only multiplication or division is easy so I will jump to expression which contain all operations. As we did with addition and subtraction we need to recursively evaluate the operands before adding or subtracting them. Also, as multiplication and division have higher priority than addition and subtraction we first have to split the input string by ‘+’ and ‘-‘ and than split the result by ‘*’ and ‘/’. This way we will first perform higher priority operations first and than move up the stack to perform operations with less priority. For example splitting 2+5*3 by ‘+’ will result in 2 and 5*3 which we will split by ‘*’ and as a result calculate 5*3 before performing addition. With this in mind this is how the corresponding test looks like:

public decimal Evaluate(string expression)
{
if (expression.Contains('+'))
{
var parts = expression.Split('+');
Expression result = Expression.Constant(Evaluate(parts[0]));
result = parts.Skip(1).Aggregate(result, (current, next) => Expression.Add(current, Expression.Constant(Evaluate(next))));
var lambda = Expression.Lambda<Func<decimal>>(result);
var compiled = lambda.Compile();
return compiled();
}
if (expression.Contains('-'))
{
var parts = expression.Split('-');
Expression result = Expression.Constant(Evaluate(parts[0]));
result = parts.Skip(1).Aggregate(result, (current, next) => Expression.Subtract(current, Expression.Constant(Evaluate(next))));
var lambda = Expression.Lambda<Func<decimal>>(result);
var compiled = lambda.Compile();
return compiled();
}
if (expression.Contains('*'))
{
var parts = expression.Split('*');
Expression result = Expression.Constant(Evaluate(parts[0]));
result = parts.Skip(1).Aggregate(result, (current, next) => Expression.Multiply(current, Expression.Constant(Evaluate(next))));
var lambda = Expression.Lambda<Func<decimal>>(result);
var compiled = lambda.Compile();
return compiled();
}
if (expression.Contains('/'))
{
var parts = expression.Split('/');
Expression result = Expression.Constant(Evaluate(parts[0]));
result = parts.Skip(1).Aggregate(result, (current, next) => Expression.Divide(current, Expression.Constant(Evaluate(next))));
var lambda = Expression.Lambda<Func<decimal>>(result);
var compiled = lambda.Compile();
return compiled();
}
decimal value = 0;
decimal.TryParse(expression, out value);
return value;
}

You can notice that all if block have very similar code: The first term is evaluated and other terms are aggregated based on the current operation. We can take advantage of this and refactor the above code in one small snippet. First, we will introduce a dictionary which maps operation to corresponding expression tree building method. Using it we can replace all if statements with one foreach loop:

private Dictionary<char, Func<Expression, Expression, Expression>> operations =
new Dictionary<char, Func<Expression, Expression, Expression>>
{
{ '+', (current, next) => Expression.Add(current, next) },
{ '-', (current, next) => Expression.Subtract(current, next) },
{ '*', (current, next) => Expression.Multiply(current, next) },
{ '/', (current, next) => Expression.Divide(current, next) }
};
public decimal Evaluate(string expression)
{
foreach (var operation in operations)
{
if (expression.Contains(operation.Key))
{
var parts = expression.Split(operation.Key);
Expression result = Expression.Constant(Evaluate(parts[0]));
result = parts.Skip(1).Aggregate(result,
(current, next) =>
operation.Value(current, Expression.Constant(Evaluate(next))));
var lambda = Expression.Lambda<Func<decimal>>(result);
var compiled = lambda.Compile();
return compiled();
}
}
decimal value = 0;
decimal.TryParse(expression, out value);
return value;
}

The above function finds the first operation in the given expression and starts recursively evaluating the terms. This in turn will find other operations and evaluate them again recursively until all the operations are performed.

Using the above method you can evaluate any expression as long as it does not contain variables or parentheses. This is also confirmed by the following test results:

Conclusion

As you can see we can build a very simple expression evaluator very easily using very little lines of code. In the following posts we will add support for expression with parentheses and expressions containing variables. Stay tuned!

Building Expression Evaluator with Expression Trees in C# – Table of Contents

Tue, 21 Feb 2012 17:43:45 +0000

This is table of contents for Building Expression Evaluator with Expression Trees in C# series. We are going to build a simple mathematical expression evaluator in C# using expression trees. The library supports simple expressions such as 2.5+5.9, 17.89-2.47+7.16, 5/2/2+1.5*3+4.58, expressions with parentheses (((9-6/2)*2-4)/2-6-1)/(2+24/(2+4)) and expressions with variables:

var a = 6;
var b = 4.32m;
var c = 24.15m;
Assert.That(engine.Evaluate("(((9-a/2)*2-b)/2-a-1)/(2+c/(2+4))", a, b, c),
Is.EqualTo((((9 - a / 2) * 2 - b) / 2 - a - 1) / (2 + c / (2 + 4))));

At the end of the series full source code ~~will be available at github~~ and the library will be published to NuGet.

Source code is available at github: Math-Expression-Evaluator

Dynamic ViewState in ASP.Net WebForms – Part 2

Tue, 31 Jan 2012 08:26:35 +0000

In previous post I showed an approach combining DynamicObject with ViewState for accessing ViewState members using dynamic properties instead of string indexes. However that approach had some disadvantages:

It is tied some a concrete base page class.
It does not have all the properties/methods of StateBag class
Mixing ViewState and ViewBag can be confusing for new developers.

In order to overcome these issues I made several changes to the DynamicViewState class.

First of all I changed the constructor to accept a StateBag instance instead of Page class so that the DynamicViewState class does not need to be nested inside base page class. Secondly I added all the interfaces which are implemented by StateBag. This makes it fully compatible with StateBag class so we can simply rename the new property in base page class from ViewBag to ViewState. This way you can leave the old code that uses ViewState and use the new approach for all new code. Here is a sample snippet:

public partial class SamplePage : BasePage
{
protected void Page_Load(object sender, EventArgs e)
{
if (IsPostBack)
{
ViewState.loadCount++;
loadLabel.Text = string.Format("Load count: {0}", ViewState.loadCount);
}
else
{
ViewState["loadCount"] = 0;
loadLabel.Text = "Load count: 0";
}
}
}

Getting started with DynamicViewState

If you would like to use DynamicViewState in your code the simplest way for getting it is using Nuget package available at Dynamic ViewState Nuget package. Once you add the package to your project simply change the base class of your pages to BasePage and that is all you need. Source code is available at github.

Performance Benchmarks

In a comment to previous post Jalpesh Vadgama asked about performance impact. I ran benchmark to compare dynamic implementation with ordinary ViewState usage. As expected dynamic implementation is slower but the difference is quite small:

Average milliseconds with dynamic: 0.0003
Average milliseconds without dynamic: 0.00016

As you can see dynamic access is twice slower compared to ordinary ViewState usage.

Dynamic ViewState in ASP.Net WebForms

Tue, 17 Jan 2012 09:19:01 +0000

If you have worked with ASP.Net MVC 3 you are probably aware of ViewBag property which allows you to pass data from Controller to View by using dynamic properties instead of hard coding strings. While the properties still are not strongly typed and compiler does not check them it can provide an advantage. For example you can avoid casting as it returns dynamic objects. This post will show how to use the same approach in ASP.Net Web Forms for ViewState property giving us a dynamic viewstate class.

In order to access the properties which do not exist at compile time the class has to inherit from DynamicObject. This will allow us to override methods which are called when we get or set dynamic properties. In these methods we will use the ViewState of the current page to store and extract values.

To start with we need let’s create the DynamicViewState class and base page class which will contain the ViewBag property:

public class DynamicViewState : DynamicObject
{
public override bool TrySetMember(SetMemberBinder binder, object value)
{
return base.TrySetMember(binder, value);
}
public override bool TryGetMember(GetMemberBinder binder, out object result)
{
return base.TryGetMember(binder, out result);
}
}
public class BasePage : Page
{
private readonly dynamic viewBag;
protected dynamic ViewBag
{
get { return viewBag; }
}
}

The TryGetMember and TrySetMember methods are invokes whenever we get or set properties of ViewBag instance. This means that we should have access to ViewState of the current page so let’s pass the page to the constructor of DynamicViewState:

public class DynamicViewState : DynamicObject
{
private readonly BasePage basePage;
public DynamicViewState(BasePage basePage)
{
this.basePage = basePage;
}
public override bool TrySetMember(SetMemberBinder binder, object value)
{
return base.TrySetMember(binder, value);
}
public override bool TryGetMember(GetMemberBinder binder, out object result)
{
return base.TryGetMember(binder, out result);
}
}
public class BasePage : Page
{
private readonly dynamic viewBag;
public BasePage()
{
viewBag = new DynamicViewState(this);
}
protected dynamic ViewBag
{
get { return viewBag; }
}
}

We now have the page but as ViewState is protected property we cannot access it from our methods. We could add methods for storing and retrieving items from the ViewState but that would allow all other classes to manipulate the page’s ViewState. Instead a better option is to make the DynamicViewState class nested. This way we will be able to access the ViewState property without exposing it to the outside world. The implementation of the DynamicViewState now looks like this:

public class BasePage : Page
{
private class DynamicViewState : DynamicObject
{
private readonly BasePage basePage;
public DynamicViewState(BasePage basePage)
{
this.basePage = basePage;
}
public override bool TrySetMember(SetMemberBinder binder, object value)
{
basePage.ViewState[binder.Name] = value;
return true;
}
public override bool TryGetMember(GetMemberBinder binder, out object result)
{
result = basePage.ViewState[binder.Name];
return true;
}
}
private readonly dynamic viewBag;
public BasePage()
{
viewBag = new DynamicViewState(this);
}
protected dynamic ViewBag
{
get { return viewBag; }
}
}

Sample usage looks like this:

public partial class SamplePage : BasePage
{
protected void Page_Load(object sender, EventArgs e)
{
if (IsPostBack)
{
ViewBag.loadCount++;
loadLabel.Text = string.Format("Load count: {0}", ViewState["loadCount"]);
}
else
{
ViewBag.loadCount = 0;
loadLabel.Text = "Load count: 0";
}
}
}

As you can see we can mix ViewBag and ViewState together without any issues as ViewBag is simply a façade over ViewState.

If we also override TrySetIndex and TryGetIndex methods we can access ViewBag properties by using indexes exactly in the same way as ViewState. This is how final implementation look like:

public class BasePage : Page
{
private class DynamicViewState : DynamicObject
{
private readonly BasePage basePage;
public DynamicViewState(BasePage basePage)
{
this.basePage = basePage;
}
public override bool TrySetMember(SetMemberBinder binder, object value)
{
basePage.ViewState[binder.Name] = value;
return true;
}
public override bool TryGetMember(GetMemberBinder binder, out object result)
{
result = basePage.ViewState[binder.Name];
return true;
}
public override bool TrySetIndex(SetIndexBinder binder, object[] indexes, object value)
{
if (indexes.Length == 1)
{
basePage.ViewState[indexes[0].ToString()] = value;
return true;
}
return base.TrySetIndex(binder, indexes, value);
}
public override bool TryGetIndex(GetIndexBinder binder, object[] indexes, out object result)
{
if (indexes.Length == 1)
{
result = basePage.ViewState[indexes[0].ToString()];
return true;
}
return base.TryGetIndex(binder, indexes, out result);
}
}
private readonly dynamic viewBag;
public BasePage()
{
viewBag = new DynamicViewState(this);
}
protected dynamic ViewBag
{
get { return viewBag; }
}
}

Apart from dynamic viewstate the above approach can be also used for providing similar implementations for Session, Cache and similar objects.

Full source code with a demo application is available at Dynamic ViewState in ASP.Net WebForms

AppDomain.AssemblyResolve Event Tips

Wed, 04 Jan 2012 09:21:51 +0000

AppDomain.AssemblyResolve event can be used to load referenced assemblies at runtime. This can be useful in multiple scenarios: you can use it to load a library from a different location or load a library based on bitness or load a library which is embedded in the executable file. In all cases there are some tips and tricks which can help you to easily implement the event handler and avoid exceptions.

Tip 1: Use AssemblyName class to parse the name of the assembly

When the AssemblyResolve event is fired you can use the Name property of ResolveEventArgs class to find the name of the requested assembly. However, it will also include version, culture and public key token of the assembly. Instead of manipulating the string to extract the name part it is much more easier to construct a new instance of AssemblyName class and use the Name property which returns only the name of the library.

private Assembly ResolveAssembly(object sender, ResolveEventArgs args)
{
//args.Name looks like this:
 //MyLibrary, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
 //Don't parse it yourself
 string name = args.Name.Split(',')[0];
//Use the AssemblyName class to get the name
 name = new AssemblyName(args.Name).Name;
}

Tip 2: AssemblyResolve firing multiple times.

AssemblyResolve event can fire multiple times for the same assembly. Instead of loading it again you should return previously loaded assembly. To keep track of the assemblies you have loaded you can use a dictionary or you can get the loaded assemblies by calling AppDomain.GetAssemblies method. Make sure that you use appropriate locking mechanism to prevent race conditions.

Tip 3: AssemblyResolve and FileNotFoundException.

Even if you are loading the assembly in the AssemblyResolve event handler you might still get a FileNotFoundException if you are not careful. For example the following code will generate the exception (mainclass is defined in the library we are loading) :

static void Main(string[] args)
{
AppDomain.CurrentDomain.AssemblyResolve += ResolveAssembly;
var mainClass = new MainClass();
mainClass.Print();
}
static Assembly ResolveAssembly(object sender, ResolveEventArgs args)
{
return Assembly.LoadFile(@"path to the library");
}

The exception is occurring because jit tries to resolve all assemblies as soon as it hits the Main method. At that point the AssemblyResolve event does not have any subscribers so the event is not raised at all. On way to solve the problem is to move the code which needs the assembly to a separate method:

static void Main(string[] args)
{
AppDomain.CurrentDomain.AssemblyResolve += ResolveAssembly;
}
static void Print()
{
var mainClass = new MainClass();
mainClass.Print();
}
static Assembly ResolveAssembly(object sender, ResolveEventArgs args)
{
return Assembly.LoadFile(@"path to the library");
}

Another solution is to subscribe to the event before the Main method is hit. For example, you can change the Program class to a static class and put the assignment in static constructor.

static class Program
{
    static Program()
    {
        AppDomain.CurrentDomain.AssemblyResolve += ResolveAssembly;
    }
    static void Main(string[] args)
    {
        var mainClass = new MainClass();
     mainClass.Print();
    }
    static Assembly ResolveAssembly(object sender, ResolveEventArgs args)
    {
        return Assembly.LoadFile(@"path to the library");
    }
}

Silverlight slideshow with remote control and windows taskbar support

Wed, 28 Dec 2011 10:04:11 +0000

Source code for this article is available at Silverlight Remote Control Source Code

Native Extensions For Microsoft Silverlight is a collection of COM automation based runtime libraries and Silverlight libraries which allow elevated trusted out of browser applications to utilize windows platform feature. For example you can build applications which use Sensor API, Speech API, react to remote control, integrate with windows 7 taskbar or listen to windows messages sent to the host window. In this post I will build a simple slideshow application to demonstrate some of the capabilities of the library.

The application allows users to choose images from their computer and control the slideshow with remote control as well as with taskbar buttons. Building the slideshow itself is very easy so I will briefly explain how it works: When the user selects pictures a timer is started. When the timer tick event is fired it fades out current image, switches to the next image and fades it in. The user is also able to stop the slideshow and manually switch between the images. The buttons can be also triggered with remote control and taskbar buttons.

Responding to remote control buttons

According to remote control documentation to respond to remote control you need to listen to various windows messages. For example, let’s implement pause/resume when APPCOMMAND_MEDIA_PAUSE or APPCOMMAND_MEDIA_PLAY commands are received. To interpret these commands we need to catch WM_APPCOMMAND message.

public MainPage()
{
InitializeComponent();
WindowMessageInterceptor.Current.WindowMessage += MessageReceived;
WindowMessageInterceptor.Current.AddMessageIntercept(NativeMethods.WM_APPCOMMAND);
}

As you have probably guessed MessageReceived is a method which will be called when the host window receives WM_APPCOMMAND message. The handler will also receive message details such as wParam and lParam. To get the command which triggered the message we need to process lParam by calling GET_APPCOMMAND_LPARAM Macro. The macro is defined in Winuser.h as

#define GET_APPCOMMAND_LPARAM(lParam) ((short)(HIWORD(lParam) & ~FAPPCOMMAND_MASK))

HIWORD is another macro defined in Windef.h as

#define HIWORD(l) ((WORD)((((DWORD_PTR)(l)) >> 16) & 0xffff))

The corresponding code in C# looks like this

public static int GET_APPCOMMAND_LPARAM(int lParam)
{
return (short)(HIWORD(lParam) & ~FAPPCOMMAND_MASK);
}
private static int HIWORD(int lParam)
{
return ((lParam >> 16) & 0xffff);
}

We are now ready to implement the MessageReceived handler:

private void MessageReceived(object sender, WindowMessageEventArgs e)
{
if (e.Message == NativeMethods.WM_APPCOMMAND)
{
int cmd = NativeMethods.GET_APPCOMMAND_LPARAM(e.lParam);
switch (cmd)
{
case NativeMethods.APPCOMMAND_MEDIA_PLAY:
ResumeSlideShow();
break;
case NativeMethods.APPCOMMAND_MEDIA_PAUSE:
PauseSlideShow();
break;
}
}
}

Let’s add support for navigating between the images using left and right buttons on the controller. These buttons don’t generate WM_APPCOMMAND message. Instead they trigger WM_KEYDOWN message which Silverlight supports out of the box. This means that to respond to these buttons we need to simply respond to KeyDown event:

private void SlideShowKeyDown(object sender, KeyEventArgs e)
{
if (e.Key == Key.Left)
{
PreviousImage();
}
if (e.Key == Key.Right)
{
NextImage();
}
}

Adding support for taskbar buttons

The first step in adding support for taskbar buttons is to add the actual buttons. First we create the buttons and than instruct the taskbar to show them:

private void CreateTaskbarButtons()
{
for (int i = 0; i < 4; i++)
{
var thumbbarButton = TaskbarButton.Current.CreateThumbbarButton((uint)(i + 1));
thumbbarButton.Flags = THUMBBUTTONFLAGS.THBF_ENABLED;
thumbbarButton.ImageDataType = ButtonImageDataType.PNG;
var resourceInfo = Application.GetResourceStream(new Uri(string.Format("images/{0}.png", i), UriKind.Relative));
using (var br = new BinaryReader(resourceInfo.Stream))
{
thumbbarButton.Image = br.ReadBytes((int)resourceInfo.Stream.Length);
}
taskbarButtons.Add(thumbbarButton);
}
taskbarButtons[0].Tooltip = "First Image";
taskbarButtons[1].Tooltip = "Previous Image";
taskbarButtons[2].Tooltip = "Next Image";
taskbarButtons[3].Tooltip = "Last Image";
TaskbarButton.Current.ShowThumbbarButtons();
}

You have probably notices that the code snippet is creating buttons but there are no event handlers to respond when they are clicked. Instead we need to use WindowMessageInterceptor class again and subscribe to CommandMessage event. The event is raised in response to WM_COMMAND message which is generated when a taskbar button is clicked. Apart from subscribing to event we need to call AddCommandMessageIntercept method and specify in which control notification code we are interested it. This way the event will be raised only for that particular notification code. In case of taskbar buttons the notification code is THBN_CLICKED.

WindowMessageInterceptor.Current.CommandMessage += CommandMessage;
WindowMessageInterceptor.Current.AddCommandMessageIntercept(NativeMethods.THBN_CLICKED);

When the event handler is called it will receive an instance of CommandMessageEventArgs class specifying notification code and control identifier. The control identifier corresponds to the ButtonID we passed to CreateThumbbarButton method when we created the buttons. This way we can identify which button caused the event to fire.

private void CommandMessage(object sender, CommandMessageEventArgs e)
{
if (e.NotifyCode != NativeMethods.THBN_CLICKED)
{
return;
}
switch (e.ControlID)
{
case 1:
FirstImage();
break;
case 2:
PreviousImage();
break;
case 3:
NextImage();
break;
case 4:
LastImage();
break;
}
}

How to Get Elevated Process Path in .Net

Thu, 22 Dec 2011 08:46:23 +0000

As you know retrieving process path in .Net is quite straightforward. For example if you have process instance someProcess you can get the process path by accessing someProcess.MainModule.FileName However for elevated processes the previous code snippet throws Win32Exception.

As you have probably guessed the reason is related to the bugs I mentioned in my previous post. In this case too the Process class is using PROCESS_QUERY_INFORMATION access right to open the desired process. The difference from the previous case is that in order to get the main module and process path the code really needs PROCESS_QUERY_INFORMATION so we cannot simply change it with PROCESS_QUERY_LIMITED_INFORMATION.

Fortunately Microsoft has introduced a new function for getting process path QueryFullProcessImageName which does work with PROCESS_QUERY_LIMITED_INFORMATION access right. To call the function we need to open the process ourselves and pass the handle.

private static string GetExecutablePath(Process Process)
{
//If running on Vista or later use the new function
 if (Environment.OSVersion.Version.Major >= 6)
{
return GetExecutablePathAboveVista(Process.Id);
}
return Process.MainModule.FileName;
}
private static string GetExecutablePathAboveVista(int ProcessId)
{
var buffer = new StringBuilder(1024);
IntPtr hprocess = OpenProcess(ProcessAccessFlags.PROCESS_QUERY_LIMITED_INFORMATION,
false, ProcessId);
if (hprocess != IntPtr.Zero)
{
try
{
int size = buffer.Capacity;
if (QueryFullProcessImageName(hprocess, 0, buffer, out size))
{
return buffer.ToString();
}
}
finally
{
CloseHandle(hprocess);
}
}
throw new Win32Exception(Marshal.GetLastWin32Error());
}

[DllImport("kernel32.dll")]
private static extern bool QueryFullProcessImageName(IntPtr hprocess, int dwFlags,
StringBuilder lpExeName, out int size);
[DllImport("kernel32.dll")]
private static extern IntPtr OpenProcess(ProcessAccessFlags dwDesiredAccess,
bool bInheritHandle, int dwProcessId);

[DllImport("kernel32.dll", SetLastError = true)]
private static extern bool CloseHandle(IntPtr hHandle);

You can also turn the above code snippet into an extension method too. If you want Microsoft to add a new property for easily getting the process path vote for this suggestion at the connect website: Add Path property to Process class so that it works with elavated processes too.

Bugs in System.Diagnostics.Process Class

Tue, 20 Dec 2011 09:00:48 +0000

While working on an application which makes use of Process class I found several bugs which occur in special conditions. More specifically accessing StartTime or HasExited properties causes a Win32Exception with NativeErrorCode equal to five. This indicates that the exception is caused by not having enough rights to retrieve the required information.

The exception occurs under the following conditions:

The source application should not be elevated
- The process instance should represent an elevated process
- The instance should not be a result of Process.Start call

The snippet below demonstrates these points:

//The following code will not throw an exception unless explorer is running elevated
var explorer = Process.GetProcessesByName("explorer").First();
Console.WriteLine(explorer.StartTime);
Console.WriteLine(explorer.HasExited);
var startInfo = new ProcessStartInfo("notepad") { Verb = "runas" };
var notepad = Process.Start(startInfo);
//Even though notepad is an elevated process no exception will be thrown
Console.WriteLine(notepad.StartTime);
Console.WriteLine(notepad.HasExited);
var notepad2 = Process.GetProcessById(notepad.Id);
//Exception is thrown.
try
{
Console.WriteLine(notepad2.StartTime);
}
catch(Win32Exception e)
{
Console.WriteLine(e.ToString());
}
try
{
Console.WriteLine(notepad2.HasExited);
}
catch(Win32Exception e)
{
Console.WriteLine(e.ToString());
}

By looking at the stacktrace we get when accessing StartTime property we can see that Process.StartTime calls Process.GetProcessTimes which calls Process.GetProcessHandle which in turn calls ProcessManager.OpenProcess. This is where the exception is occurring. Using Reflector we can see that ProcessManager.OpenProcess invokes native function OpenProcess and passes the desired access. In our case the access that is passed is equal to 0x400 which corresponds to PROCESS_QUERY_INFORMATION access right. However, starting from Vista there is a new access right called PROCESS_QUERY_LIMITED_INFORMATION which is enough for calling GetProcessTimes function. Let’s see what we get when we call GetProcessTimes with the new access right:

var startinfo = new ProcessStartInfo("notepad") { Verb = "runas" };
var process = Process.Start(startinfo);
var process2 = Process.GetProcessById(process.Id);
long create, exit, kernel, user;
var handle = NativeMethods.OpenProcess(0x1000, false, process2.Id);
NativeMethods.GetProcessTimes(handle, out create, out exit, out kernel, out user);
NativeMethods.CloseHandle(handle);
Console.WriteLine(DateTime.FromFileTime(create));

If you run this snippet you will see that process start time is printed and no exception occurs. This solves the problem with StartTime property.

Printing the stacktrace of HasExited property shows that the exception occurs at exactly the same place. In this case the requested access is 0x100400 which is combination of PROCESS_QUERY_INFORMATION and SYNCHRONIZE. As you have probably guessed it is again enough to use PROCESS_QUERY_LIMITED_INFORMATION instead of PROCESS_QUERY_INFORMATION. I will not show the full code here as it is quite straightforward.

So, if you receive access denied exception when querying StartTime or HasExited make sure that the target process is not an elevated process. If it is you will have to manually retrieve the information you need.

Blog | Giorgi Dalakishvili | Personal Website

DuckDB.NET 1.5.0: Simplified Scalar Functions, Table Functions, and NULL Handling

Scalar Functions: Before and After

High-Level Scalar Functions

Dynamic Type Support

NULL Handling for Scalar Functions

Table Functions: Before and After

High-Level Table Functions

Named Parameters for Table Functions

Summary

DuckDB.NET 1.5.0 Performance: Up to 40% Faster Writes and 22% Fewer Allocations

What’s New in DuckDB.NET 1.5.0

What Changed

LibraryImport Migration

SuppressGCTransition on Fast Native Calls

AggressiveInlining on Hot-Path Methods

Appender Boxing Elimination

Decimal Conversion Rewrite

Vector Reader Reuse Across Chunks

BIGNUM O(n) Conversion

Benchmarks

Reader: 17-22% Faster

Appender: 20-40% Faster, 22% Less Memory

Summary

PostgreSQL range types and Entity Framework Core

Ranges in Relational Databases

Ranges in PostgreSQL

Range Operators in PostgreSQL

Ranges Exclusion Constraints

Entity Framework Core and PostgreSQL Range Types.

Conclusion

How Cloudflare Broke My Build and How I Fixed It

Mysterious Build Failure

Finding the Root Cause of the Issue

Fixing Coveralls Uploader

Introducing GraphQLinq - Strongly Typed GraphQL Queries with LINQ to GraphQL.

Getting Started with GraphQLinq

Running GraphQL Queries with LINQ

Query all Primitive Properties of a Type

Query Specific Properties

Include Navigation Properties

View Generated Query

Roadmap

Updating LINQPad.QueryPlanVisualizer for LINQPad6 And Entity Framework Core

Using LINQPad.QueryPlanVisualizer

Internals of LINQPad.QueryPlanVisualizer

Extracting and Formatting Query Plan from SQL Server

Extracting Query Plan from PostgreSQL

Conclusion

EntityFramework.Exceptions 3.1.1 - Support for Entity Framework Core 3 and Improved API

Introducing EntityFramework.Exceptions

The Problem with Entity Framework Exceptions

EntityFramework.Exceptions – Easy Way to Handle Exceptions

How Does EntityFramework.Exceptions Work ?

Introducing LINQPad.QueryPlanVisualizer

LINQPad.QueryPlanVisualizer

How it works

Faking num lock, caps lock and scroll lock leds

Introduction

Toggling Keyboard Indicators from C#

Conclusion

Building Expression Evaluator with Expression Trees in C# – Improving the Api

Introduction

Binding Variables by Name

Passing Parameters with Anonymous Objects

Passing Parameters with Named Arguments

Improving Expression Evaluator Performance

Conclusion

Using Reactive Extensions for Streaming Data from Database

The Problem

Solution without Reactive Extensions

Reactive Extensions – Simplifying the Solution

Extending Reactive Extensions Solution

Conclusion

Building Expression Evaluator with Expression Trees in C# – Part 3

Introduction

Adding support for single variable to expression evaluator

Adding support for multiple variables

Conclusion

Building Expression Evaluator with Expression Trees in C# – Part 2