Bug with unsafe (fixed buffer) structs

.NET obfuscator and automated error reporting

Moderators: Luke Jefferson, melvyn.harbour, dom.smith

Bug with unsafe (fixed buffer) structs

Postby ApocDev » Wed Dec 28, 2011 9:51 am

For performance reasons, we have a few structures that use fixed-buffer arrays to align padding, and provide quicker array access (via pointer usage) instead of the typical [MarshalAs] approach.

Example struct:

Code: Select all
[StructLayout(LayoutKind.Sequential, Pack=1)]
unsafe struct Example
{
        public IntPtr InternalEntries;
        public uint MaxEntries;
        private fixed byte padding8[8];
        public uint NumEntries;
        private fixed byte padding14[4];
        public IntPtr InternalEntriesSecond;
        private fixed byte padding1c [4];
}


Code: Select all
    public static class MarshalCache<T>
    {
        public static int Size;

        static MarshalCache()
        {
            if (typeof(T).IsEnum)
            {
                var underlying = typeof(T).GetEnumUnderlyingType();
                Size = Marshal.SizeOf(underlying);
            }
            else Size = Marshal.SizeOf(typeof(T));
        }
    }


The above class is simply a cache for Marshal.SizeOf (its fairly slow on complex types, so we just pseudo-cache it with the static-generic trick)

In a console application:
Code: Select all
Console.WriteLine(MarshalCache<Example>.Size)

The above will print "32" when built from VS. (Assuming x86 mode due to IntPtr changing between 4/8 bytes)

However, if you run SA over the same console application, with literally zero options enabled, it will print "19". This is causing a ton of bugs in our application where we've done optimizations to reduce CPU usage.
ApocDev
 
Posts: 10
Joined: Thu Sep 30, 2010 10:43 pm

Postby Brian Donahue » Thu Dec 29, 2011 11:12 am

Thanks - I will turn this over to the development team as I really don't understand this very well -- you have an enum, which would normally be a long or int, but instead you are making an enum out of struct, and the struct is packed. Then you ask marshal about the size of the struct and it does not report the correct length. Is that the gist of it?

It would probably be important to let us know which protection options you are using so we can try to isolate which one may be interfering with this.
Brian Donahue
 
Posts: 6670
Joined: Mon Aug 23, 2004 10:48 am

Re:

Postby ApocDev » Thu Dec 29, 2011 7:39 pm

Brian Donahue wrote:Thanks - I will turn this over to the development team as I really don't understand this very well -- you have an enum, which would normally be a long or int, but instead you are making an enum out of struct, and the struct is packed. Then you ask marshal about the size of the struct and it does not report the correct length. Is that the gist of it?

It would probably be important to let us know which protection options you are using so we can try to isolate which one may be interfering with this.


Not quite. The issue is with structure sizes. (The enum sanity check is due to Marshal.SizeOf not being able to determine the size of an enum, so we grab the underlying type (enum Foo: uint, will cause the underlying type to be uint, [4 bytes]))

The bug is with structs with fixed buffers. (public fixed byte Padding[50];)

This happens with zero protection options turned on. (Not even strong-name signing, or error reporting. No obfuscation, no pruning, no string encryption, etc)
ApocDev
 
Posts: 10
Joined: Thu Sep 30, 2010 10:43 pm

Postby Brian Donahue » Fri Dec 30, 2011 10:00 am

The problem happens when the object is not an enum, then.
Code: Select all
Marshal.SizeOf(typeof(T));
Brian Donahue
 
Posts: 6670
Joined: Mon Aug 23, 2004 10:48 am

Postby Brian Donahue » Tue Jan 03, 2012 3:29 pm

I tried to get a reproduction, but both before and after SmartAssembly, my example prints out "32".
Code: Select all
using System;
using System.Collections.Generic;
using System.Text;
using System.Runtime.InteropServices;

namespace ConsoleApplication1
{
 class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine(MarshalCache<UnsafeStuff.Example>.Size);
            Console.ReadLine();
        }
    }
 public class UnsafeStuff
    {
        [StructLayout(LayoutKind.Sequential, Pack = 1)]
        public unsafe struct Example
        {
            public IntPtr InternalEntries;
            public uint MaxEntries;
            private fixed byte padding8[8];
            public uint NumEntries;
            private fixed byte padding14[4];
            public IntPtr InternalEntriesSecond;
            private fixed byte padding1c[4];
        }
    }

class MarshalCache<T>
    {
        public static int Size;
        static MarshalCache()
        {
            if (typeof(T).IsEnum)
            {
                 Size = 9999; // cut this bit as it's not relevant
            }
            else Size = Marshal.SizeOf(typeof(T));
        }
    }

}
Brian Donahue
 
Posts: 6670
Joined: Mon Aug 23, 2004 10:48 am


Return to SmartAssembly 6

Who is online

Users browsing this forum: No registered users and 1 guest