diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index b41ecd24..c75d8865 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -49,4 +49,7 @@ /plugins/dotnet/skills/dotnet-aot-compat/ @agocke @dotnet/appmodel /tests/dotnet/dotnet-aot-compat/ @agocke @dotnet/appmodel +/plugins/dotnet/skills/migrating-newtonsoft-to-system-text-json/ @mrsharm +/tests/dotnet/migrating-newtonsoft-to-system-text-json/ @mrsharm + /plugins/dotnet/agents/optimizing-dotnet-performance.agent.md @dotnet/appmodel diff --git a/plugins/dotnet/skills/migrating-newtonsoft-to-system-text-json/SKILL.md b/plugins/dotnet/skills/migrating-newtonsoft-to-system-text-json/SKILL.md new file mode 100644 index 00000000..1391b31e --- /dev/null +++ b/plugins/dotnet/skills/migrating-newtonsoft-to-system-text-json/SKILL.md @@ -0,0 +1,249 @@ +```skill +--- +name: migrating-newtonsoft-to-system-text-json +description: Migrate from Newtonsoft.Json to System.Text.Json, handling behavioral differences, custom converters, and common breaking changes. Use when converting a project from Newtonsoft.Json (Json.NET) to the built-in System.Text.Json serializer. +--- + +# Migrating from Newtonsoft.Json to System.Text.Json + +## When to Use + +- Migrating an existing project from Newtonsoft.Json to System.Text.Json +- Removing the Newtonsoft.Json dependency for performance or AOT compatibility +- Fixing serialization differences after switching to System.Text.Json + +## When Not to Use + +- The project requires Newtonsoft.Json features that System.Text.Json cannot support (extremely rare edge cases like `$ref/$id` with deep graphs) +- The user is already using System.Text.Json and just needs help with it +- The user explicitly wants to keep Newtonsoft.Json + +## Inputs + +| Input | Required | Description | +|-------|----------|-------------| +| Code using Newtonsoft.Json | Yes | Models, serialization calls, custom converters | +| .NET version | No | Determines which System.Text.Json features are available | + +## Workflow + +### Step 1: Understand the critical behavioral differences + +**System.Text.Json is NOT a drop-in replacement.** These behaviors differ by default: + +| Behavior | Newtonsoft.Json | System.Text.Json | Impact | +|----------|----------------|-------------------|--------| +| **Property naming** | PascalCase by default (as declared) | **PascalCase by default** | Same ✓ (unless you used a custom ContractResolver) | +| **Missing properties** | Ignored silently | Ignored silently | Same ✓ | +| **Extra JSON properties** | Ignored by default | Ignored by default (can opt-in to throw in .NET 8+) | Same ✓ (stricter behavior available via options) | +| **Trailing commas** | Allowed | **Rejected by default** | Parse errors on valid-looking JSON | +| **Comments in JSON** | Allowed | **Rejected by default** | Config files break | +| **Number in string** (`"123"`) | Coerced automatically | **Throws by default** | Deserialization breaks! | +| **Enum serialization** | Numeric by default | Numeric by default | Same ✓, but converter syntax differs | +| **null → non-nullable value type** | Sets to default(T) | Sets to default(T) | Same ✓ (null becomes default(T)) | +| **Case sensitivity** | Case-insensitive | **Case-sensitive by default** | Property matching breaks | +| **Max depth** | 64 | 64 | Same ✓ | +| **Circular references** | `$ref/$id` with PreserveReferencesHandling | `ReferenceHandler.Preserve` (.NET 5+) | API differs | + +### Step 2: Configure System.Text.Json to match Newtonsoft.Json behavior + +```csharp +// In Program.cs (ASP.NET Core) — configure globally +builder.Services.ConfigureHttpJsonOptions(options => +{ + ConfigureJsonOptions(options.SerializerOptions); +}); + +// Also configure for controllers if using MVC +builder.Services.AddControllers() + .AddJsonOptions(options => + { + ConfigureJsonOptions(options.JsonSerializerOptions); + }); + +static void ConfigureJsonOptions(JsonSerializerOptions options) +{ + // Match Newtonsoft.Json default behavior: + options.PropertyNamingPolicy = JsonNamingPolicy.CamelCase; // Newtonsoft default + options.PropertyNameCaseInsensitive = true; // Newtonsoft default + options.NumberHandling = JsonNumberHandling.AllowReadingFromString; // Newtonsoft coerces + options.ReadCommentHandling = JsonCommentHandling.Skip; // Newtonsoft allows + options.AllowTrailingCommas = true; // Newtonsoft allows + options.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull; // Common Newtonsoft setting + + // Enum string serialization (replaces StringEnumConverter) + options.Converters.Add(new JsonStringEnumConverter(JsonNamingPolicy.CamelCase)); + + // Handle circular references (replaces PreserveReferencesHandling) + options.ReferenceHandler = ReferenceHandler.IgnoreCycles; // or Preserve for $ref/$id +} +``` + +### Step 3: Replace attribute mappings + +| Newtonsoft.Json Attribute | System.Text.Json Equivalent | +|--------------------------|----------------------------| +| `[JsonProperty("name")]` | `[JsonPropertyName("name")]` | +| `[JsonIgnore]` | `[JsonIgnore]` (same name, different namespace!) | +| `[JsonProperty(Required = Required.Always)]` | `[JsonRequired]` (.NET 7+) | +| `[JsonProperty(NullValueHandling = NullValueHandling.Ignore)]` | `[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]` | +| `[JsonProperty(DefaultValueHandling = DefaultValueHandling.Ignore)]` | `[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]` | +| `[JsonConverter(typeof(MyConverter))]` | `[JsonConverter(typeof(MyConverter))]` (different base class!) | +| `[JsonConstructor]` | `[JsonConstructor]` (same name, different namespace) | +| `[JsonExtensionData]` | `[JsonExtensionData]` + must be `Dictionary` (NOT `JToken`) | + +**Regex for finding Newtonsoft attributes:** +```bash +# Find all files using Newtonsoft attributes +grep -rn "using Newtonsoft.Json" --include="*.cs" +grep -rn "\[JsonProperty\|JsonConverter\|JsonIgnore\|JsonConstructor" --include="*.cs" +``` + +### Step 4: Convert custom JsonConverters + +**Newtonsoft converter pattern:** +```csharp +// OLD: Newtonsoft.Json +public class UnixDateTimeConverter : Newtonsoft.Json.JsonConverter +{ + public override DateTime ReadJson(JsonReader reader, Type objectType, + DateTime existingValue, bool hasExistingValue, JsonSerializer serializer) + { + var timestamp = (long)reader.Value!; + return DateTimeOffset.FromUnixTimeSeconds(timestamp).DateTime; + } + + public override void WriteJson(JsonWriter writer, DateTime value, + JsonSerializer serializer) + { + var timestamp = new DateTimeOffset(value).ToUnixTimeSeconds(); + writer.WriteValue(timestamp); + } +} +``` + +**System.Text.Json converter pattern:** +```csharp +// NEW: System.Text.Json +public class UnixDateTimeConverter : System.Text.Json.Serialization.JsonConverter +{ + public override DateTime Read(ref Utf8JsonReader reader, Type typeToConvert, + JsonSerializerOptions options) + { + var timestamp = reader.GetInt64(); // Note: strongly typed reader methods + return DateTimeOffset.FromUnixTimeSeconds(timestamp).DateTime; + } + + public override void Write(Utf8JsonWriter writer, DateTime value, + JsonSerializerOptions options) + { + var timestamp = new DateTimeOffset(value).ToUnixTimeSeconds(); + writer.WriteNumberValue(timestamp); + } +} +``` + +**Key differences in converter API:** +- Reader is `ref Utf8JsonReader` (struct, passed by ref) — NOT a class +- Writer is `Utf8JsonWriter` — write methods are `WriteStringValue`, `WriteNumberValue`, `WriteBooleanValue` (typed) +- No `serializer` parameter — use `options` and call `JsonSerializer.Serialize/Deserialize` for nested objects +- For polymorphic deserialization: use `JsonTypeInfo` and `[JsonDerivedType]` (.NET 7+) instead of custom type handling + +### Step 5: Replace JToken/JObject/JArray with JsonDocument/JsonElement + +| Newtonsoft.Json | System.Text.Json | Notes | +|----------------|-------------------|-------| +| `JToken.Parse(json)` | `JsonDocument.Parse(json)` | **JsonDocument is IDisposable!** Must wrap in `using` | +| `JObject obj = ...` | `JsonElement obj = doc.RootElement` | JsonElement is a struct (no allocation) | +| `obj["key"]` | `obj.GetProperty("key")` | Throws if missing; use `TryGetProperty` for safe access | +| `obj["key"]?.Value()` | `obj.GetProperty("key").GetInt32()` | Type-specific getters | +| `obj.Add("key", value)` | **Not possible** — JsonElement is read-only | Use `JsonNode` (System.Text.Json.Nodes) for mutable DOM | + +**For mutable DOM operations, use JsonNode (NOT JsonDocument):** +```csharp +// Mutable DOM — replaces JObject/JArray mutation patterns +var node = JsonNode.Parse(json)!; +node["newProperty"] = "value"; // Add/set properties +node["nested"] = new JsonObject // Create nested objects +{ + ["key"] = 42 +}; +var result = node.ToJsonString(); // Serialize back +``` + +### Step 6: Handle polymorphic serialization + +**Newtonsoft.Json (uses $type discriminator):** +```csharp +var settings = new JsonSerializerSettings +{ + TypeNameHandling = TypeNameHandling.Auto // SECURITY RISK! +}; +``` + +**System.Text.Json (.NET 7+ — type discriminators):** +```csharp +[JsonDerivedType(typeof(CreditCardPayment), typeDiscriminator: "credit")] +[JsonDerivedType(typeof(BankTransferPayment), typeDiscriminator: "bank")] +public abstract class Payment +{ + public decimal Amount { get; set; } +} + +public class CreditCardPayment : Payment +{ + public string CardNumber { get; set; } = ""; +} + +// Serializes as: {"$type":"credit","amount":99.99,"cardNumber":"..."} +// Note: System.Text.Json uses "$type" by default (configurable) +``` + +### Step 7: Update package references + +```xml + + + + + + + +``` + +**Update using statements:** +```csharp +// Remove: +using Newtonsoft.Json; +using Newtonsoft.Json.Linq; +using Newtonsoft.Json.Serialization; +using Newtonsoft.Json.Converters; + +// Add: +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Text.Json.Nodes; // For JsonNode (mutable DOM) +``` + +## Validation + +- [ ] All `using Newtonsoft.Json` references removed +- [ ] All `[JsonProperty]` replaced with `[JsonPropertyName]` +- [ ] Custom converters use `System.Text.Json.Serialization.JsonConverter` base +- [ ] `JObject`/`JToken` replaced with `JsonDocument` (read-only) or `JsonNode` (mutable) +- [ ] API responses match previous JSON format (property casing, null handling) +- [ ] Deserialization handles edge cases: trailing commas, comments, numbers-as-strings +- [ ] No `TypeNameHandling` equivalent (security improvement) +- [ ] `JsonDocument` usages wrapped in `using` statements + +## Common Pitfalls + +| Pitfall | Solution | +|---------|----------| +| Forgetting `PropertyNameCaseInsensitive = true` | Deserialization silently returns default values for all properties | +| `JsonDocument` not disposed | Memory leak — always `using var doc = JsonDocument.Parse(...)` | +| Using `JsonElement` after `JsonDocument` is disposed | JsonElement is invalid after dispose; clone with `element.Clone()` if needed | +| `[JsonIgnore]` from wrong namespace | Both Newtonsoft and System.Text.Json have `[JsonIgnore]` — wrong `using` = attribute ignored | +| Custom converter reading past the current token | System.Text.Json reader is strict — must read exactly the right tokens | +| `JsonExtensionData` with `Dictionary` | Must be `Dictionary` — not `object` or `JToken` | +``` diff --git a/tests/dotnet/migrating-newtonsoft-to-system-text-json/eval.yaml b/tests/dotnet/migrating-newtonsoft-to-system-text-json/eval.yaml new file mode 100644 index 00000000..338b1db4 --- /dev/null +++ b/tests/dotnet/migrating-newtonsoft-to-system-text-json/eval.yaml @@ -0,0 +1,49 @@ +scenarios: + - name: "Migrate model with Newtonsoft.Json attributes to System.Text.Json" + prompt: | + I'm migrating our ASP.NET Core 8 project from Newtonsoft.Json to System.Text.Json. Here's a model class that uses Newtonsoft attributes and a custom converter. Convert this to System.Text.Json: + + ```csharp + using Newtonsoft.Json; + using Newtonsoft.Json.Converters; + using Newtonsoft.Json.Linq; + + public class Order + { + [JsonProperty("order_id")] + public int Id { get; set; } + + [JsonProperty(Required = Required.Always)] + public string CustomerName { get; set; } + + [JsonProperty(NullValueHandling = NullValueHandling.Ignore)] + public string? Notes { get; set; } + + [JsonConverter(typeof(StringEnumConverter))] + public OrderStatus Status { get; set; } + + [JsonExtensionData] + public Dictionary? AdditionalData { get; set; } + } + ``` + + Also show me how to configure the JSON options globally to match Newtonsoft.Json's default behavior (case insensitivity, trailing commas, number-from-string coercion). + assertions: + - type: "output_contains" + value: "JsonPropertyName" + - type: "output_matches" + pattern: "(JsonIgnore.*WhenWritingNull|JsonIgnoreCondition)" + - type: "output_matches" + pattern: "(PropertyNameCaseInsensitive|CamelCase|PropertyNamingPolicy)" + - type: "output_matches" + pattern: "(JsonElement|JsonNode)" + rubric: + - "Replaced [JsonProperty(\"order_id\")] with [JsonPropertyName(\"order_id\")]" + - "Replaced NullValueHandling.Ignore with [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]" + - "Replaced [JsonConverter(typeof(StringEnumConverter))] with System.Text.Json equivalent (JsonStringEnumConverter)" + - "Changed [JsonExtensionData] Dictionary value type from JToken to JsonElement (critical difference!)" + - "Configured PropertyNameCaseInsensitive = true to match Newtonsoft default case-insensitive behavior" + - "Configured AllowTrailingCommas = true and NumberHandling = AllowReadingFromString for Newtonsoft compatibility" + - "Warned about behavioral differences (default PascalCase casing in STJ vs camelCase in Newtonsoft, strict parsing)" + expect_tools: ["bash"] + timeout: 120