MSVC debugger natvis file

[R2RT]

I've recently started using xsimd and for sake of my sanity I came up with .nativs file (MSVC Debugger format file), which flattens xsimd batches in view/preview.

Views for batch, batch_bool, batch_constant and batch<complex> should simply work. I've checked them for avx and avx512bw.

I guess I will still tweak it, but might as well share it already.

If you are open into adding .nativs to repo, I can prepare PR later.

batch int

batch complex

batch constant

batch bool

xsimd.natvis

<?xml version="1.0" encoding="utf-8"?>

<AutoVisualizer xmlns="http://schemas.microsoft.com/vstudio/debugger/natvis/2010">

  <Type Name="xsimd::batch&lt;*, *&gt;">
    <DisplayString>batch [{size}] ({($T1*)&amp;data,[size]na})</DisplayString>
    <Expand>
      <Item Name="[data]" ExcludeView="simple">&amp;data,!</Item>
      <Item Name="[size]" ExcludeView="simple">size</Item>
      <ArrayItems>
        <Size>size</Size>
        <ValuePointer>($T1*)&amp;data</ValuePointer>
      </ArrayItems>
    </Expand>
  </Type>

  <Type Name="xsimd::batch_constant&lt;*&gt;">
    <DisplayString>batch_constant [{size}]</DisplayString>
    <Expand>
      <Item Name="[size]">size</Item>
      <Item Name="[0]" Condition="size &gt; 0" Optional="true">$T3</Item>
      <Item Name="[1]" Condition="size &gt; 1" Optional="true">$T4</Item>
      <Item Name="[2]" Condition="size &gt; 2" Optional="true">$T5</Item>
      <Item Name="[3]" Condition="size &gt; 3" Optional="true">$T6</Item>
      <Item Name="[4]" Condition="size &gt; 4" Optional="true">$T7</Item>
      <Item Name="[5]" Condition="size &gt; 5" Optional="true">$T8</Item>
      <Item Name="[6]" Condition="size &gt; 6" Optional="true">$T9</Item>
      <Item Name="[7]" Condition="size &gt; 7" Optional="true">$T10</Item>
      <Item Name="[8]" Condition="size &gt; 8" Optional="true">$T11</Item>
      <Item Name="[9]" Condition="size &gt; 9" Optional="true">$T12</Item>
      <Item Name="[10]" Condition="size &gt; 10" Optional="true">$T13</Item>
      <Item Name="[11]" Condition="size &gt; 11" Optional="true">$T14</Item>
      <Item Name="[12]" Condition="size &gt; 12" Optional="true">$T15</Item>
      <Item Name="[13]" Condition="size &gt; 13" Optional="true">$T16</Item>
      <Item Name="[14]" Condition="size &gt; 14" Optional="true">$T17</Item>
      <Item Name="[15]" Condition="size &gt; 15" Optional="true">$T18</Item>
      <Item Name="[16]" Condition="size &gt; 16" Optional="true">$T19</Item>
      <Item Name="[17]" Condition="size &gt; 17" Optional="true">$T20</Item>
      <Item Name="[18]" Condition="size &gt; 18" Optional="true">$T21</Item>
      <Item Name="[19]" Condition="size &gt; 19" Optional="true">$T22</Item>
      <Item Name="[20]" Condition="size &gt; 20" Optional="true">$T23</Item>
      <Item Name="[21]" Condition="size &gt; 21" Optional="true">$T24</Item>
      <Item Name="[22]" Condition="size &gt; 22" Optional="true">$T25</Item>
      <Item Name="[23]" Condition="size &gt; 23" Optional="true">$T26</Item>
      <Item Name="[24]" Condition="size &gt; 24" Optional="true">$T27</Item>
      <Item Name="[25]" Condition="size &gt; 25" Optional="true">$T28</Item>
      <Item Name="[26]" Condition="size &gt; 26" Optional="true">$T29</Item>
      <Item Name="[27]" Condition="size &gt; 27" Optional="true">$T30</Item>
      <Item Name="[28]" Condition="size &gt; 28" Optional="true">$T31</Item>
      <Item Name="[29]" Condition="size &gt; 29" Optional="true">$T32</Item>
      <Item Name="[30]" Condition="size &gt; 30" Optional="true">$T33</Item>
      <Item Name="[31]" Condition="size &gt; 31" Optional="true">$T34</Item>
      <Item Name="[32]" Condition="size &gt; 32" Optional="true">$T35</Item>
      <Item Name="[33]" Condition="size &gt; 33" Optional="true">$T36</Item>
      <Item Name="[34]" Condition="size &gt; 34" Optional="true">$T37</Item>
      <Item Name="[35]" Condition="size &gt; 35" Optional="true">$T38</Item>
      <Item Name="[36]" Condition="size &gt; 36" Optional="true">$T39</Item>
      <Item Name="[37]" Condition="size &gt; 37" Optional="true">$T40</Item>
      <Item Name="[38]" Condition="size &gt; 38" Optional="true">$T41</Item>
      <Item Name="[39]" Condition="size &gt; 39" Optional="true">$T42</Item>
      <Item Name="[40]" Condition="size &gt; 40" Optional="true">$T43</Item>
      <Item Name="[41]" Condition="size &gt; 41" Optional="true">$T44</Item>
      <Item Name="[42]" Condition="size &gt; 42" Optional="true">$T45</Item>
      <Item Name="[43]" Condition="size &gt; 43" Optional="true">$T46</Item>
      <Item Name="[44]" Condition="size &gt; 44" Optional="true">$T47</Item>
      <Item Name="[45]" Condition="size &gt; 45" Optional="true">$T48</Item>
      <Item Name="[46]" Condition="size &gt; 46" Optional="true">$T49</Item>
      <Item Name="[47]" Condition="size &gt; 47" Optional="true">$T50</Item>
      <Item Name="[48]" Condition="size &gt; 48" Optional="true">$T51</Item>
      <Item Name="[49]" Condition="size &gt; 49" Optional="true">$T52</Item>
      <Item Name="[50]" Condition="size &gt; 50" Optional="true">$T53</Item>
      <Item Name="[51]" Condition="size &gt; 51" Optional="true">$T54</Item>
      <Item Name="[52]" Condition="size &gt; 52" Optional="true">$T55</Item>
      <Item Name="[53]" Condition="size &gt; 53" Optional="true">$T56</Item>
      <Item Name="[54]" Condition="size &gt; 54" Optional="true">$T57</Item>
      <Item Name="[55]" Condition="size &gt; 55" Optional="true">$T58</Item>
      <Item Name="[56]" Condition="size &gt; 56" Optional="true">$T59</Item>
      <Item Name="[57]" Condition="size &gt; 57" Optional="true">$T60</Item>
      <Item Name="[58]" Condition="size &gt; 58" Optional="true">$T61</Item>
      <Item Name="[59]" Condition="size &gt; 59" Optional="true">$T62</Item>
      <Item Name="[60]" Condition="size &gt; 60" Optional="true">$T63</Item>
      <Item Name="[61]" Condition="size &gt; 61" Optional="true">$T64</Item>
      <Item Name="[62]" Condition="size &gt; 62" Optional="true">$T65</Item>
      <Item Name="[63]" Condition="size &gt; 63" Optional="true">$T66</Item>
    </Expand>
  </Type>

  <Type Name="xsimd::batch_bool&lt;*, *&gt;">
    <!-- Handle n-elemnent mask, i.e. reduce full sized bit mask to 1 boolean per batch element -->
    <DisplayString Optional="true" Condition="sizeof(data) &gt; 8 &amp;&amp; size == 1">[1]-element mask ({(($T1*)&amp;data)[0] ? 1 : 0})</DisplayString>
    <DisplayString Optional="true" Condition="sizeof(data) &gt; 8 &amp;&amp; size == 2">[2]-element mask ({(($T1*)&amp;data)[0] ? 1 : 0}{(($T1*)&amp;data)[1] ? 1 : 0})</DisplayString>
    <DisplayString Optional="true" Condition="sizeof(data) &gt; 8 &amp;&amp; size == 4">[4]-element mask ({(($T1*)&amp;data)[0] ? 1 : 0}{(($T1*)&amp;data)[1] ? 1 : 0}{(($T1*)&amp;data)[2] ? 1 : 0}{(($T1*)&amp;data)[3] ? 1 : 0})</DisplayString>
    <DisplayString Optional="true" Condition="sizeof(data) &gt; 8 &amp;&amp; size == 8">[8]-element mask ({(($T1*)&amp;data)[0] ? 1 : 0}{(($T1*)&amp;data)[1] ? 1 : 0}{(($T1*)&amp;data)[2] ? 1 : 0}{(($T1*)&amp;data)[3] ? 1 : 0}{(($T1*)&amp;data)[4] ? 1 : 0}{(($T1*)&amp;data)[5] ? 1 : 0}{(($T1*)&amp;data)[6] ? 1 : 0}{(($T1*)&amp;data)[7] ? 1 : 0})</DisplayString>
    <DisplayString Optional="true" Condition="sizeof(data) &gt; 8 &amp;&amp; size == 16">[16]-element mask ({(($T1*)&amp;data)[0] ? 1 : 0}{(($T1*)&amp;data)[1] ? 1 : 0}{(($T1*)&amp;data)[2] ? 1 : 0}{(($T1*)&amp;data)[3] ? 1 : 0}{(($T1*)&amp;data)[4] ? 1 : 0}{(($T1*)&amp;data)[5] ? 1 : 0}{(($T1*)&amp;data)[6] ? 1 : 0}{(($T1*)&amp;data)[7] ? 1 : 0} {(($T1*)&amp;data)[8] ? 1 : 0}{(($T1*)&amp;data)[9] ? 1 : 0}{(($T1*)&amp;data)[10] ? 1 : 0}{(($T1*)&amp;data)[11] ? 1 : 0}{(($T1*)&amp;data)[12] ? 1 : 0}{(($T1*)&amp;data)[13] ? 1 : 0}{(($T1*)&amp;data)[14] ? 1 : 0}{(($T1*)&amp;data)[15] ? 1 : 0})</DisplayString>
    <DisplayString Optional="true" Condition="sizeof(data) &gt; 8 &amp;&amp; size == 32">[32]-element mask ({(($T1*)&amp;data)[0] ? 1 : 0}{(($T1*)&amp;data)[1] ? 1 : 0}{(($T1*)&amp;data)[2] ? 1 : 0}{(($T1*)&amp;data)[3] ? 1 : 0}{(($T1*)&amp;data)[4] ? 1 : 0}{(($T1*)&amp;data)[5] ? 1 : 0}{(($T1*)&amp;data)[6] ? 1 : 0}{(($T1*)&amp;data)[7] ? 1 : 0} {(($T1*)&amp;data)[8] ? 1 : 0}{(($T1*)&amp;data)[9] ? 1 : 0}{(($T1*)&amp;data)[10] ? 1 : 0}{(($T1*)&amp;data)[11] ? 1 : 0}{(($T1*)&amp;data)[12] ? 1 : 0}{(($T1*)&amp;data)[13] ? 1 : 0}{(($T1*)&amp;data)[14] ? 1 : 0}{(($T1*)&amp;data)[15] ? 1 : 0} {(($T1*)&amp;data)[16] ? 1 : 0}{(($T1*)&amp;data)[17] ? 1 : 0}{(($T1*)&amp;data)[18] ? 1 : 0}{(($T1*)&amp;data)[19] ? 1 : 0}{(($T1*)&amp;data)[20] ? 1 : 0}{(($T1*)&amp;data)[21] ? 1 : 0}{(($T1*)&amp;data)[22] ? 1 : 0}{(($T1*)&amp;data)[23] ? 1 : 0} {(($T1*)&amp;data)[24] ? 1 : 0}{(($T1*)&amp;data)[25] ? 1 : 0}{(($T1*)&amp;data)[26] ? 1 : 0}{(($T1*)&amp;data)[27] ? 1 : 0}{(($T1*)&amp;data)[28] ? 1 : 0}{(($T1*)&amp;data)[29] ? 1 : 0}{(($T1*)&amp;data)[30] ? 1 : 0}{(($T1*)&amp;data)[31] ? 1 : 0})</DisplayString>

    <!-- Fallback for element-mask -->
    <DisplayString Optional="true" Condition="sizeof(data) &gt; 8">[{size}]-element mask</DisplayString>

    <!-- Handle bit mask, e.g. __mmask64 used by avx512 -->
    <DisplayString>[{sizeof(data) * 8}]-bit mask ({data, bb})</DisplayString>

    <Expand>
      <!-- Handle n-elemnent mask, i.e. reduce full sized bit mask to 1 boolean per batch element -->
      <IndexListItems Condition="sizeof(data) &gt; 8" Optional="true">
        <Size>size</Size>
        <ValueNode>(($T1*)&amp;data)[$i] != 0</ValueNode>
      </IndexListItems>

      <!-- Handle bit mask, e.g. __mmask64 used by avx512 -->
      <Item Condition="sizeof(data) &lt;= 8" Optional="true" Name="[Mask]">data,bb</Item>
      <IndexListItems Condition="sizeof(data) &lt;= 8" Optional="true">
        <Size>sizeof(data) * 8</Size>
        <ValueNode>(data &amp; (1 &lt;&lt; $i)) != 0</ValueNode>
      </IndexListItems>
    </Expand>
  </Type>

  <Type Name="xsimd::batch&lt;std::complex&lt;*&gt;, *&gt;">
    <DisplayString>batch_complex [{size}]</DisplayString>
    <Expand>
      <Item Name="[real]">m_real</Item>
      <Item Name="[imag]">m_imag</Item>
      <Item Name="[size]" ExcludeView="simple">size</Item>
      <Synthetic Name="[0]" Condition="size &gt; 0">
        <DisplayString Condition="m_imag[0] &lt; 0">{m_real[0]}-i*{-m_imag[0]}</DisplayString>
        <DisplayString>{m_real[0]}+i*{m_imag[0]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[1]" Condition="size &gt; 1">
        <DisplayString Condition="m_imag[1] &lt; 0">{m_real[1]}-i*{-m_imag[1]}</DisplayString>
        <DisplayString>{m_real[1]}+i*{m_imag[1]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[2]" Condition="size &gt; 2">
        <DisplayString Condition="m_imag[2] &lt; 0">{m_real[2]}-i*{-m_imag[2]}</DisplayString>
        <DisplayString>{m_real[2]}+i*{m_imag[2]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[3]" Condition="size &gt; 3">
        <DisplayString Condition="m_imag[3] &lt; 0">{m_real[3]}-i*{-m_imag[3]}</DisplayString>
        <DisplayString>{m_real[3]}+i*{m_imag[3]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[4]" Condition="size &gt; 4">
        <DisplayString Condition="m_imag[4] &lt; 0">{m_real[4]}-i*{-m_imag[4]}</DisplayString>
        <DisplayString>{m_real[4]}+i*{m_imag[4]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[5]" Condition="size &gt; 5">
        <DisplayString Condition="m_imag[5] &lt; 0">{m_real[5]}-i*{-m_imag[5]}</DisplayString>
        <DisplayString>{m_real[5]}+i*{m_imag[5]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[6]" Condition="size &gt; 6">
        <DisplayString Condition="m_imag[6] &lt; 0">{m_real[6]}-i*{-m_imag[6]}</DisplayString>
        <DisplayString>{m_real[6]}+i*{m_imag[6]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[7]" Condition="size &gt; 7">
        <DisplayString Condition="m_imag[7] &lt; 0">{m_real[7]}-i*{-m_imag[7]}</DisplayString>
        <DisplayString>{m_real[7]}+i*{m_imag[7]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[8]" Condition="size &gt; 8">
        <DisplayString Condition="m_imag[8] &lt; 0">{m_real[8]}-i*{-m_imag[8]}</DisplayString>
        <DisplayString>{m_real[8]}+i*{m_imag[8]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[9]" Condition="size &gt; 9">
        <DisplayString Condition="m_imag[9] &lt; 0">{m_real[9]}-i*{-m_imag[9]}</DisplayString>
        <DisplayString>{m_real[9]}+i*{m_imag[9]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[10]" Condition="size &gt; 10">
        <DisplayString Condition="m_imag[10] &lt; 0">{m_real[10]}-i*{-m_imag[10]}</DisplayString>
        <DisplayString>{m_real[10]}+i*{m_imag[10]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[11]" Condition="size &gt; 11">
        <DisplayString Condition="m_imag[11] &lt; 0">{m_real[11]}-i*{-m_imag[11]}</DisplayString>
        <DisplayString>{m_real[11]}+i*{m_imag[11]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[12]" Condition="size &gt; 12">
        <DisplayString Condition="m_imag[12] &lt; 0">{m_real[12]}-i*{-m_imag[12]}</DisplayString>
        <DisplayString>{m_real[12]}+i*{m_imag[12]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[13]" Condition="size &gt; 13">
        <DisplayString Condition="m_imag[13] &lt; 0">{m_real[13]}-i*{-m_imag[13]}</DisplayString>
        <DisplayString>{m_real[13]}+i*{m_imag[13]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[14]" Condition="size &gt; 14">
        <DisplayString Condition="m_imag[14] &lt; 0">{m_real[14]}-i*{-m_imag[14]}</DisplayString>
        <DisplayString>{m_real[14]}+i*{m_imag[14]}</DisplayString>
      </Synthetic>
      <Synthetic Name="[15]" Condition="size &gt; 15">
        <DisplayString Condition="m_imag[15] &lt; 0">{m_real[15]}-i*{-m_imag[15]}</DisplayString>
        <DisplayString>{m_real[15]}+i*{m_imag[15]}</DisplayString>
      </Synthetic>
    </Expand>
  </Type>

</AutoVisualizer>

[serge-sans-paille]

I'm not familiar at all with this format. Can you provide some input on it, and provide information about how to keep it in sync if the source, if it makes sense?

[R2RT]

Sure, although I am not certain how doable is it.

.natvis file are used by Visual Studio to prettify watched variables when debugging the executable. By default each inheritance and composition level creates a foldout.

E.g. for std::array<float, 4> raw view is kinda simple:

std::array<float, 4> has internal float _Elems[4], so you can expand it and see values.

Still, MSVC provides built-in view for it so elements are exposed on top-level foldout:

For xsimd types situation is quite heavier [1]. E.g. batch contains data, which is built-in __m128 union, which on debugger level - has all potential values expandable.

So there are 2 problems: user have to expand few foldouts to see the data. And he also have to be aware what data is stored in the register (union).

For working examples, see in MR description.

About .natvis format: I found it very tricky to get anything going. Technically, you can specify any operation that is valid in the MSVC C++ debugger, that is not much: arithmetics, pointer operations and pointer casting [2]. It can access private fields.
It also supports pattern matching for sake of templates. No function calls, no constructors, no temporary variables with pretty layout. Whatever binary buffor data is available, we can only reinterpret it. Everything being XML does not help.

This is .natvis reference page: https://learn.microsoft.com/en-us/visualstudio/debugger/create-custom-views-of-native-objects?view=vs-2022

E.g. in case <Type Name="xsimd::batch&lt;*, *&gt;"> debugger loads given view if expanded type name matches xsimd::batch<*,*> pattern. $T1 is and $T2 are special tokens matching template arguments.

Later it has defined DisplayString and Expand, which specify how to show inline/expanded view of variable.
It works under assumption that each xsimd::batch has 2 (private) fields: size and data. Given the $T1 is 1st template argument, we tell debugger to show array-view pointing at data with size elements.
And this is the only binding contract for this view. So in order to keep it in sync with codebase, if size or data goes missing, .natvis has to be updated. (Or template changes)

batch_constant is hack that expands template variables as array view. Not too pretty implementation, but works.

batch_bool is hacky one, since as far as I found, there are 2 possible underlying layouts of its internals.
It contains either batch<T> filled with 0/1 in each element or built-in binary mask (__mmask64). So the implementation has 2 variants, depending on the sizeof(data).

batch<complex> is another hack, since in debugger I cannot change memory layout. So values in m_real and m_imag have to be merged into single string. Building a string is possible only in DisplayString tag, so no Array/ListView can be used.

I know all of it is compiler-specific and based on xsimd internals, but it is how it works for .natvis.
On the other I don't think you should worry too much about keeping the internals with sync to .natvis file. If it does not work, debugger will simply show raw, folded version and that's it. No errors, not logs (unless manually enabled).
It is nice-to-have feature, that is common in other libs (Eigen, OpenCV).
I am not aware of any way to do automatic validation of .natvis over codebase.

[1] Same for majority of STL containers
[2] Much more is possible in managed languages, what seems to have influenced .natvis spec.

[serge-sans-paille]

Oh, there's a similar approach in gdb, with python bindings to prettify data structures, that would indeed be nice to have. In gdb, it's pretty straight forward to write a small validation script that starts gdb, breakpoints, dump the content of some variable and we can assert that the output is as pretty as expected. Would that be possible with msvc debugger?