P/Invoke and LibraryImport for .NET

.NET P/Invoke

Calling native code from .NET is powerful but unforgiving. Incorrect signatures, garbled strings, and leaked or freed memory are the most common sources of bugs — all can manifest as intermittent crashes, silent data corruption, or access violations far from the actual defect.

This skill covers both DllImport (available since .NET Framework 1.0) and LibraryImport (source-generated, .NET 7+). When targeting .NET Framework, always use DllImport. When targeting .NET 7+, prefer LibraryImport for new code. When native AOT is a requirement, LibraryImport is the only option.

When to Use This Skill

• Writing a new [DllImport] or [LibraryImport] declaration from a C/C++ header
• Reviewing P/Invoke signatures for correctness (type sizes, calling conventions, string encoding)
• Wrapping an entire C library for use from .NET
• Debugging AccessViolationException, DllNotFoundException, or silent data corruption at the native boundary
• Migrating DllImport declarations to LibraryImport for AOT/trimming compatibility
• Diagnosing memory leaks or heap corruption involving native handles or buffers

Stop Signals

• Single function? Map the signature (Steps 1-3), handle strings/memory only if relevant, skip tooling and migration sections.
• Don't migrate existing DllImport to LibraryImport unless the user asks or AOT/trimming is an explicit requirement.
• Don't recommend CsWin32 unless the target is specifically Win32 APIs.
• Don't generate callbacks (Step 8) unless the native API requires function pointers.
• Review request? Use the validation checklist — don't rewrite working code.

Inputs

Agent behavior: When documentation and native headers diverge, always trust the header. Online documentation (including official Win32 API docs) frequently omits or simplifies details about types, calling conventions, and struct layout that are critical for correct P/Invoke signatures.

Workflow

Step 1: Choose DllImport or LibraryImport

Step 2: Map Native Types to .NET Types

The most dangerous mappings — these cause the majority of bugs:

For the complete type mapping table, struct layout, and blittable type rules, see references/type-mapping.md.

❌ NEVER use int or long for C long — it's 32-bit on Windows, 64-bit on Unix. Always use CLong. ❌ NEVER use ulong for size_t — causes stack corruption on 32-bit. Use nuint or UIntPtr. ❌ NEVER use bool without MarshalAs — the default marshal size is wrong.

Step 3: Write the Declaration

Given a C header:

int32_t process_records(const Record* records, size_t count, uint32_t* out_processed);

DllImport:

[DllImport("mylib")]
private static extern int ProcessRecords(
    [In] Record[] records, UIntPtr count, out uint outProcessed);

LibraryImport:

[LibraryImport("mylib")]
internal static partial int ProcessRecords(
    [In] Record[] records, nuint count, out uint outProcessed);

Calling conventions only need to be specified when targeting Windows x86 (32-bit), where Cdecl and StdCall differ. On x64, ARM, and ARM64, there is a single calling convention and the attribute is unnecessary.

Agent behavior: If you detect that Windows x86 is a target — through project properties (e.g., <PlatformTarget>x86</PlatformTarget>), runtime identifiers (e.g., win-x86), build scripts, comments, or developer instructions — flag this to the developer and recommend explicit calling conventions on all P/Invoke declarations.

// DllImport (x86 targets)
[DllImport("mylib", CallingConvention = CallingConvention.Cdecl)]

// LibraryImport (x86 targets)
[LibraryImport("mylib")]
[UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])]

If the managed method name differs from the native export name, specify EntryPoint to avoid EntryPointNotFoundException:

// DllImport
[DllImport("mylib", EntryPoint = "process_records")]
private static extern int ProcessRecords(
    [In] Record[] records, UIntPtr count, out uint outProcessed);

// LibraryImport
[LibraryImport("mylib", EntryPoint = "process_records")]
internal static partial int ProcessRecords(
    [In] Record[] records, nuint count, out uint outProcessed);

Step 4: Handle Strings Correctly

1. Know what encoding the native function expects. There is no safe default.
2. Windows APIs: Always call the W (UTF-16) variant. The A variant needs a specific reason and explicit ANSI encoding.
3. Cross-platform C libraries: Usually expect UTF-8.
4. Specify encoding explicitly. Never rely on CharSet.Auto.
5. Never introduce StringBuilder for output buffers.

❌ NEVER rely on CharSet.Auto or omit string encoding — there is no safe default.

// DllImport — Windows API (UTF-16)
[DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
private static extern int GetModuleFileNameW(
    IntPtr hModule, [Out] char[] filename, int size);

// DllImport — Cross-platform C library (UTF-8)
[DllImport("mylib")]
private static extern int SetName(
    [MarshalAs(UnmanagedType.LPUTF8Str)] string name);

// LibraryImport — UTF-16
[LibraryImport("kernel32", StringMarshalling = StringMarshalling.Utf16,
    SetLastPInvokeError = true)]
internal static partial int GetModuleFileNameW(
    IntPtr hModule, [Out] char[] filename, int size);

// LibraryImport — UTF-8
[LibraryImport("mylib", StringMarshalling = StringMarshalling.Utf8)]
internal static partial int SetName(string name);

String lifetime warning: Marshalled strings are freed after the call returns. If native code stores the pointer (instead of copying), the lifetime must be manually managed. On Windows or .NET Framework, CoTaskMemAlloc/CoTaskMemFree is the first choice for cross-boundary ownership; on non-Windows targets, use NativeMemory APIs. The library may have its own allocator that must be used instead.

Step 5: Establish Memory Ownership

When memory crosses the boundary, exactly one side must own it — and both sides must agree.

❌ NEVER free with a mismatched allocator — Marshal.FreeHGlobal on malloc'd memory is heap corruption.

Model 1 — Caller allocates, caller frees (safest):

[LibraryImport("mylib")]
private static partial int GetName(
    Span<byte> buffer, nuint bufferSize, out nuint actualSize);

public static string GetName()
{
    Span<byte> buffer = stackalloc byte[256];
    int result = GetName(buffer, (nuint)buffer.Length, out nuint actualSize);
    if (result != 0) throw new InvalidOperationException($"Failed: {result}");
    return Encoding.UTF8.GetString(buffer[..(int)actualSize]);
}

Model 2 — Callee allocates, caller frees (common in Win32):

[LibraryImport("mylib")]
private static partial IntPtr GetVersion();
[LibraryImport("mylib")]
private static partial void FreeString(IntPtr s);

public static string GetVersion()
{
    IntPtr ptr = GetVersion();
    try { return Marshal.PtrToStringUTF8(ptr) ?? throw new InvalidOperationException(); }
    finally { FreeString(ptr); } // Must use the library's own free function
}

Critical rule: Always free with the matching allocator. Never use Marshal.FreeHGlobal or Marshal.FreeCoTaskMem on malloc'd memory.

Model 3 — Handle-based (callee allocates, callee frees): Use SafeHandle (see Step 6).

Pinning managed objects — when native code stores the pointer or runs asynchronously:

// Synchronous: use fixed
public static unsafe void ProcessSync(byte[] data)
{
    fixed (byte* ptr = data) { ProcessData(ptr, (nuint)data.Length); }
}

// Asynchronous: use GCHandle
var gcHandle = GCHandle.Alloc(data, GCHandleType.Pinned);
// Must keep pinned until native processing completes, then call gcHandle.Free()