Output char buffer is too small

Compares and synchronizes SQL database content.

Moderators: Chris Auckland, David Atkinson, richardjm, david connell

Output char buffer is too small

Postby Seany » Thu Feb 24, 2011 11:22 pm

Hi all,

I am getting the following error when I try to generate a data compare script from one DB to another.

The message is:
The output char buffer is too small to contain the decoded characters, encoding 'Unicode' fallback 'System.Text.DecoderReplacementFallback'.
Parameter name: chars


Thanks in advance
Seany
 
Posts: 2
Joined: Thu Feb 24, 2011 11:18 pm
Location: Ireland

Postby james.billings » Mon Feb 28, 2011 2:46 pm

Thanks for your post.

I've looked through our historic data here, and this error message has only cropped up once before. Unfortunately, the customer on that occasion never responded to our request for more detail so we never discovered it.

Thinking at the time was that is potentially a problem in the .net framework, but could you give a little more information:

- What exact version of SQL Data Compare are you using?
- Does this occur when working with two actual databases, or are you using script folders?
- Does the process succeed if you let SQL Data Compare perform the update (as opposed to generating a script)?
- Do your database(s) use any unusual collation?
- What do you have set in Tools > Application Options under the Synchronization Scripts Encoding option? Do any other options here work?

If you want to contact us directly to work through this, please mail us directly at support@red-gate.com, quoting F0045000 in the subject line.
james.billings
 
Posts: 1144
Joined: Wed Jun 16, 2010 11:10 am
Location: My desk.

Postby Seany » Mon Feb 28, 2011 3:36 pm

Thanks for the reply James.

To answer some of your queries:
    v8.0.2.5
    Both databases are in the same instance.
    Yes, SQL Compare does synchronise the DB's as opposed to when it tries to generate the script(s).
    It is a client's DB that has been live since ~SQL2000 and is currently running in a SQL 2008 environment. There are no unusual collations. There are ~1.5million rows.
    Current encoding is set to UTF16. I will try some of the others in the mean time.
Seany
 
Posts: 2
Joined: Thu Feb 24, 2011 11:18 pm
Location: Ireland

Postby james.billings » Mon Feb 28, 2011 3:43 pm

OK - it may also be worth turning up the logging to "verbose" to see if it gives anything more specific.

To do this, right-click the main SQL Data Compare window titlebar, and select Min Logging Level > Verbose.
Restart the app and generate your error, then check the log by right clicking once more on the titlebar and picking "Locate Log Files", then opening the most recent one.
james.billings
 
Posts: 1144
Joined: Wed Jun 16, 2010 11:10 am
Location: My desk.

Postby james.billings » Mon Feb 28, 2011 7:24 pm

Also - are you mapping together any differing datatypes that may be incompatible?
james.billings
 
Posts: 1144
Joined: Wed Jun 16, 2010 11:10 am
Location: My desk.

Postby Bastiaan Molsbeck » Fri Nov 18, 2011 9:12 am

Hi Seany,

I'm experiencing the same error.
I'm using version 8.0.2.1 of the RedGate.SQLDataCompare.Engine.dll.

Did you ever solved your problem?

Kind regards,
Bastiaan.
Bastiaan Molsbeck
 
Posts: 48
Joined: Fri Mar 26, 2010 11:12 am
Location: The Netherlands

Postby Brian Donahue » Fri Nov 18, 2011 12:01 pm

Hi Bastiaan,
If you are using the SDK, can you get a stack trace from the debugger when this error occurs? It's difficult to isolate the code that throws this without some direction.
Brian Donahue
 
Posts: 6670
Joined: Mon Aug 23, 2004 10:48 am

Postby Bastiaan Molsbeck » Fri Nov 18, 2011 12:18 pm

Yes, the full stack trace is:

System.ArgumentException: The output char buffer is too small to contain the decoded characters, encoding 'Unicode' fallback 'System.Text.DecoderReplacementFallback'.
Parameter name: chars
at System.Text.Encoding.ThrowCharsOverflow()
at System.Text.Encoding.ThrowCharsOverflow(DecoderNLS decoder, Boolean nothingDecoded)
at System.Text.UnicodeEncoding.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, DecoderNLS baseDecoder)
at System.Text.DecoderNLS.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, Boolean flush)
at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex, Boolean flush)
at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex)
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.a(FieldType , BinaryReader , Boolean )
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.b(FieldType , BinaryReader , Boolean )
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.a(Object[] , RowType& , BinaryReader , FieldPairs , List`1[] , Boolean , Byte[]& )
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.a(BinaryReader , Int32 , Int32 , Int64 , FieldPairs , List`1[] , Boolean )
at RedGate.SQLDataCompare.Engine.ResultsStore.StoreEnumerator.MoveNext()
at bC.a(aF , TableDifference , Reader , RowType , Boolean , ProgressPercentageTracker , SelectionDelegate , Boolean )
at RedGate.SQLDataCompare.Engine.SqlProvider.b(ComparisonSession , SelectionDelegate , Boolean )
at RedGate.SQLDataCompare.Engine.SqlProvider.GetMigrationSQL(ComparisonSession session, SelectionDelegate select, Boolean runOnTwo)
at RedGate.SQLDataCompare.Engine.SqlProvider.GetMigrationSQL(ComparisonSession session, Boolean runOnTwo)
Bastiaan Molsbeck
 
Posts: 48
Joined: Fri Mar 26, 2010 11:12 am
Location: The Netherlands

Postby Brian Donahue » Fri Nov 18, 2011 3:53 pm

I'm not an expert on text encoding but I think there may be corrupt data in the field...
Brian Donahue
 
Posts: 6670
Joined: Mon Aug 23, 2004 10:48 am

Postby Bastiaan Molsbeck » Tue May 29, 2012 2:03 pm

No, the field does not contain corrupt data.
Do you have any other suggestions?

Edit: when using version 9.0 of the SQL Data Compare Engine the same error occurs, but with a slightly different stacktrace:

System.ArgumentException: The output char buffer is too small to contain the decoded characters, encoding 'Unicode' fallback 'System.Text.DecoderReplacementFallback'.
Parameter name: chars
at System.Text.Encoding.ThrowCharsOverflow()
at System.Text.Encoding.ThrowCharsOverflow(DecoderNLS decoder, Boolean nothingDecoded)
at System.Text.UnicodeEncoding.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, DecoderNLS baseDecoder)
at System.Text.DecoderNLS.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, Boolean flush)
at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex, Boolean flush)
at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex)
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.#tN(FieldType type, BinaryReader reader, Boolean summary)
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.#ceL(FieldType fieldType, BinaryReader reader, Boolean summary)
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.#beL(BinaryReader reader, Boolean summary)
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.#deL(Object[] values, RowType& rowType, BinaryReader reader, FieldPairs fieldPairs, List`1[] types, Boolean summary, Byte[]& rowDifferences)
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.GetRow(BinaryReader reader, Int32 valuesCount, Int32 currentIndex, Int64 currentOffset, FieldPairs fields, List`1[] types, Boolean summary)
at RedGate.SQLDataCompare.Engine.ResultsStore.StoreEnumerator.MoveNext()
at #8ZK.#n2K.#QeL(#x1K writer, TableDifference summary, Reader rs, RowType resultsStoreType, Boolean runOnTwo, ProgressPercentageTracker progress, SelectionDelegate select, Boolean showComments)
at RedGate.SQLDataCompare.Engine.SqlProvider.#DbL(ComparisonSession session, SelectionDelegate select, Boolean runOnTwo)
at RedGate.SQLDataCompare.Engine.SqlProvider.GetMigrationSQL(ComparisonSession session, SelectionDelegate select, Boolean runOnTwo)
at RedGate.SQLDataCompare.Engine.SqlProvider.GetMigrationSQL(ComparisonSession session, Boolean runOnTwo)
Bastiaan Molsbeck
 
Posts: 48
Joined: Fri Mar 26, 2010 11:12 am
Location: The Netherlands

Postby Bastiaan Molsbeck » Tue May 29, 2012 4:23 pm

But when using version 10.0 of the SQL Data Compare Engine, the error does NOT occur! The migration SQL script is generated successfully.

Problem solved!
Bastiaan Molsbeck
 
Posts: 48
Joined: Fri Mar 26, 2010 11:12 am
Location: The Netherlands


Return to SQL Data Compare Previous Versions

Who is online

Users browsing this forum: No registered users and 0 guests