Need to generate an XML column that is 1 GB for testing

The simplest way to populate your database with test data.

Moderators: David Atkinson, Anu Deshpande, Lionel

Need to generate an XML column that is 1 GB for testing

Postby COMTIBoy » Fri May 28, 2010 6:06 pm

I'm evaluating the SQL Data Generator and was curious to know if there were any settings to control the size of data generated for the column?. I'm using the XML generator and don't see anywhere where to specify the size of the size. BY default it seems to generate 1000 rows and two columns. The XML contains string with random text. I'd like to be able to generate the string to be 1 GB in size.

Any tips greatly appreciated.
COMTIBoy
 
Posts: 2
Joined: Fri May 28, 2010 6:00 pm

Postby Brian Donahue » Mon May 31, 2010 2:39 pm

Hello,

It is possible to customize one of the generic generators and specify a minimum and maximum size for the data. In my tests, however, taking it up to 1GB tended to run SQL Data Generator out of memory, so it may not be practical to do. But if you want to try...
  • Open program files\\Red Gate\\sql data generator 1\\userexample\\config\\XmlGenerator.xml in Notepad.
  • Modify it to add the "MinLength" and "MaxLength" properties as below.
  • Save it as Program Files\\Red Gate\\SQL Data Generator 1\\Config\\BigXmlGenerator.xml
  • When you re-launch SDG, your XML columns will have an "Example" category with BigXmlGenerator in it.

Here is the code for BigXmlGenerator:
Code: Select all
<?xml version="1.0" encoding="iso-8859-1"?>
<generators>
   <generator
      type="RedGate.SQLDataGenerator.Generators.Others.XMLGenerator"
      name="BigXMLGenerator"
      description="XMLGenerator up to 1GB"
      category="Example"
      >

      <property name="Unique">true</property>
      <property name="MinLength">500000000</property>
      <property name="MaxLength">1000000000</property>

      <matches field="Col_1" score="1" minlen="5"/>

      <type sqlType="xml"/>
   </generator>
</generators>
Brian Donahue
 
Posts: 6669
Joined: Mon Aug 23, 2004 10:48 am

Postby COMTIBoy » Tue Jun 01, 2010 6:53 pm

Thanks for the tips.

Yes, you were right , in that it didn't work (got the oom ) .
I tried 500MB and 100MB and that still seemed to fail.
Oh well :(
COMTIBoy
 
Posts: 2
Joined: Fri May 28, 2010 6:00 pm

Postby Brian Donahue » Wed Jun 02, 2010 9:31 am

The only thing I can think of would be to create a custom generator in C#, but I don't know how you'd get more capacity for more XML data. The built-in generator above uses a StringBuilder that tries to allocate the 1GB up-front, which is usually as much memory as a 32-bit .NET application can expect to use.

Here is the page regarding custom generators: http://www.red-gate.com/supportcenter/C ... c52077.htm
Brian Donahue
 
Posts: 6669
Joined: Mon Aug 23, 2004 10:48 am


Return to SQL Data Generator 1

Who is online

Users browsing this forum: No registered users and 0 guests