16. Comparing databases

16. Comparing databases
Prev		Next

16.1. Compare two database schemas - WbSchemaDiff
16.2. Compare data across databases - WbDataDiff

There are two SQL Workbench/J specific commands that can compare either the structure of two databases or the data contained in them.

16.1. Compare two database schemas - WbSchemaDiff

WbSchemaDiff analyzes two schemas (or a list of tables) and outputs the differences between those schemas as an XML file. The XML file describes the changes that need to be applied to the target schema to have the same structure as the reference schema, e.g. modify column definitions, remove or add tables, remove or add indexes.

The output is intended to be transformed using XSLT (e.g. with the WbXSLT Command). Sample XSLT transformations can be found on the SQL Workbench/J homepage

The command supports the following parameters:

Parameter	Description
-referenceProfile	The name of the connection profile for the reference connection. If this is not specified, then the current connection is used.
-referenceGroup	If the name of your reference profile is not unique across all profiles, you will need to specify the group in which the profile is located with this parameter.
-targetProfile	The name of the connection profile for the target connection (the one that needs to be migrated). If this is not specified, then the current connection is used. If you use the current connection for reference and target, then you should prefix the table names with schema/user or use the `-referenceschema` and `-targetschema` parameters.
-targetGroup	If the name of your target profile is not unique across all profiles, you will need to specify the group in which the profile is located with this parameter.
-file	The filename of the output file. If this is not supplied the output will be written to the message area
-referenceTables	A (comma separated) list of tables that are the reference tables, to be checked.
-targetTables	A (comma separated) list of tables in the target connection to be compared to the source tables. The tables are "matched" by their position in the list. The first table in the `-referenceTables` parameter is compared to the first table in the `-targetTables` parameter, and so on. Using this parameter you can compare tables that do not have the same name. If you omit this parameter, then all tables from the target connection with the same names as those listed in `-referenceTables` are compared. If you omit both parameters, then all tables that the user can access are retrieved from the source connection and compared to the tables with the same name in the target connection.
-referenceSchema	Compare all tables from the specified schema (user)
-targetSchema	A schema in the target connection to be compared to the tables from the reference schema.
-excludeTables	A comma separated list of tables that should not be compared. If tables from several schemas are compared (using `-referenceTables=schema_one., schema_two.`) then the listed tables must be qualified with a schema, e.g. `-excludeTables=schema_one.foobar, schema_two.fubar`
-encoding	The encoding to be used for the XML file. The default is UTF-8
-includePrimaryKeys	Select whether primary key constraint definitions should be compared as well. The default is `true`. Valid values are `true` or `false`.
-includeForeignKeys	Select whether foreign key constraint definitions should be compared as well. The default is `true`. Valid values are `true` or `false`.
-includeTableGrants	Select whether table grants should be compared as well. The default is `false`.
-includeTriggers	Select whether table triggers are compared as well. The default value is `true`.
-includeConstraints	Select whether table and column (check) constraints should be compared as well. SQL Workbench/J compares the constraint definition (SQL) as stored in the database. The default is to compare table constraints (`true`) Valid values are `true` or `false`.
-useConstraintNames	When including check constraints this parameter controls whether constraints should be matched by name, or only by their expression. If comparing by names the diff output will contain elements for constraint modification otherwise only drop and add entries will be available. The default is to compare by names(`true`) Valid values are `true` or `false`.
-includeViews	Select whether views should also be compared. When comparing views, the source as it is stored in the DBMS is compared. This comparison is case-sensitiv, which means `SELECT * FROM foo;` will be reported as a difference to `select * from foo;` even if they are logically the same. A comparison across different DBMS will also not work properly! The default is `true` Valid values are `true` or `false`.
-includeProcedures	Select whether stored procedures should also be compared. When comparing procedures the source as it is stored in the DBMS is compared. This comparison is case-sensitiv. A comparison across different DBMS will also not work! The default is `false` Valid values are `true` or `false`.
-includeIndex	Select whether indexes should be compared as well. The default is to not compare index definitions. Valid values are `true` or `false`.
-includeSequences	Select whether sequences should be compared as well. The default is to not compare sequences. Valid values are `true`, `false`.
-useJdbcTypes	Define whether to compare the DBMS specific data types, or the JDBC data type returned by the driver. When comparing tables from two different DBMS it is recommended to use `-useJdbcType=true` as this will make the comparison a bit more DBMS-independent. When comparing e.g. Oracle vs. PostgreSQL a column defined as `VARCHAR2(100)` in Oracle would be reported as beeing different to a `VARCHAR(100)` column in PostgreSQL which is not really true As both drivers ropert the column as java.sql.Types.VARCHAR, they would be considered as identical when using `-useJdbcType=true`. Valid values are `true` or `false`.
-additionalTypes	Select additional object types that are not compared by default (using the `-includeXXX` parameters) such as Oracle `TYPE` definitions. Those objects are compared on source code level (like procedures) rather than on attribute level. Valid values are object type names as shown in the "Type" dropdown in the DbExplorer.
-includeExtendedOptions	Include additional information for tables like tablespace definition or partition defintion for Oracle. Valid values are `true` or `false`. The default is `false`
-stylesheet	Define the filename of a XSLT transformation that is to be applied to the generated XML file.
-xsltOutput	The name of the generated output file when applying the XSLT transformation.

16.2. Compare data across databases - WbDataDiff

The WbDataDiff command can be used to generate SQL scripts that update a target database such that the data is identical to a reference database. This is similar to the WbSchemaDiff but compares the actual data in the tables rather than the table structure.

For each table the command will create up to three script files, depending on the needed statements to migrate the data. One file for UPDATE statements, one file for INSERT statements and one file for DELETE statements (if -includeDelete=true is specified)


	As this command needs to read every row from the reference and the target table, processing large tables can take quite some time, especially if `DELETE` statements should also be generated.

WbDataDiff requires that all involved tables have a primary key defined. If a table does not have a primary key, WbDataDiff will stop the processing.

To improve performance (a bit), the rows are retrieved in chunks from the target table by dynamically constructing a WHERE clause for the rows that were retrieved from the reference table. The chunk size can be controlled using the property workbench.sql.sync.chunksize The chunk size defaults to 25. This is a conservative setting to avoid problems with long SQL statements when processing tables that have a PK with multiple columns. If you know that your primary keys consist only of a single column and the values won't be too long, you can increase the chunk size, possibly increasing the performace when generating the SQL statements. As most DBMS have a limit on the length of a single SQL statement, be careful when setting the chunksize too high. The same chunk size is applied when generating DELETE statements by the WbCopy command, when syncDelete mode is enabled.

The command supports the following parameters:

Parameter	Description
-referenceProfile	The name of the connection profile for the reference connection. If this is not specified, then the current connection is used.
-referenceGroup	If the name of your reference profile is not unique across all profiles, you will need to specify the group in which the profile is located with this parameter. If the profile's name is unique you can omit this parameter
-targetProfile	The name of the connection profile for the target connection (the one that needs to be migrated). If this is not specified, then the current connection is used. If you use the current connection for reference and target, then you should prefix the table names with schema/user or use the `-referenceschema` and `-targetschema` parameters.
-targetGroup	If the name of your target profile is not unique across all profiles, you will need to specify the group in which the profile is located with this parameter.
-file	The filename of the main script file. The command creates two scripts per table. One script named `update_<tablename>.sql` that contains all needed `UPDATE` or `INSERT` statements. The second script is named `delete_<tablename>.sql` and will contain all `DELETE` statements for the target table. The main script merely calls (using WbInclude) the generated scripts for each table. You can enable writing a single file that includes all statements for all tables by using the parameter `-singleFile=true`
-singleFile	If this parameter's value is `true`, then only one single file containing all statements will be written.
-referenceTables	A (comma separated) list of tables that are the reference tables, to be checked. You can specify the table with wildcards, e.g. `-referenceTables=P%` to compare all tables that start with the letter `P`.
-targetTables	A (comma separated) list of tables in the target connection to be compared to the source tables. The tables are "matched" by their position in the list. The first table in the `-referenceTables` parameter is compared to the first table in the `-targetTables` parameter, and so on. Using this parameter you can compare tables that do not have the same name. If you omit this parameter, then all tables from the target connection with the same names as those listed in `-referenceTables` are compared. If you omit both parameters, then all tables that the user can access are retrieved from the source connection and compared to the tables with the same name in the target connection.
-referenceSchema	Compare all tables from the specified schema (user)
-targetSchema	A schema in the target connection to be compared to the tables from the reference schema.
-excludeTables	A comma separated list of tables that should not be compared. If tables from several schemas are compared (using `-referenceTables=schema_one., schema_two.`) then the listed tables must be qualified with a schema, e.g. `-excludeTables=schema_one.foobar, schema_two.fubar`
-checkDependencies	Valid values are `true`, `false`. Sorts the generated scripts in order to respect foreign key dependencies for deleting and inserting rows. The default is `true`.
-includeDelete	Valid values are `true`, `false`. Generates `DELETE` statements for rows that are present in the target table, but not in the reference table. The default is `false`. The default is `false`.
-type	Valid values are `sql`, `xml` Defines the type of the generated files.
-encoding	The encoding to be used for the SQL scripts. The default depends on your operating system. It will be displayed when you run `WbDataDiff` without any parameters. You can overwrite the platform default with the property `workbench.encoding` in the file `workbench.settings` XML files are always stored in UTF-8
-sqlDateLiterals	Valid values: `jdbc`, `ansi`, `dbms`, `default` Controls the format in which the values of DATE, TIME and TIMESTAMP columns are written into the generated SQL statements. For a detailed description of the possible values, please refer to the WbExport command.
-ignoreColumns	With this parameter you can define a list of column names that should not be considered when comparing data. You can e.g. exclude columns that store the last access time of a row, or the last update time if that should not be taken into account when checking for changes. They will however be part of generated `INSERT` statements.
-alternateKey	With this parameter alternate keys can be defined for the tables that are compared. The parameter can be repeated multiple times to set the keys for multiple tables in the following format: `-alternateKey='table_1=column_1,column_2'` Note that each value has to be enclosed in either single or double quotes to mask the equals sign embedded in the parameter value. Once an alternate (primary) key has been defined, the primary key columns defined on the tables are ignored. By default the real PK columns will however be included in `INSERT` statement that are generated. To avoid this, set the parameter `-excludeRealPK` to true.
-excludeRealPK	Valid values are `true`, `false`. This parameter controls the usage of the real PK columns in case alternate PK columns are defined. If set to `true` the real PK columns are excluded from generated `INSERT` statements (assuming that a new PK value will be generated during inserting the rows). Note that this parameter will enable/disable the use of the real PK columns for all tables for which alternate key columns were defined. This parameter has no effect if no alternate keys were specified using the `-alternateKey` option.
-showProgress	Valid values: true, false, <numeric value> Control the update frequence in the statusbar (when running in GUI mode). The default is every 10th row is reported. To disable the display of the progress specifiy a value of 0 (zero) or the value `false`. `true` will set the progress interval to 1 (one).

WbDataDiff Examples

Compare all tables between two connections, and write the output to the file migrate_staging.sql, but do not generate DELETE statements.

WbDataDiff -referenceProfile="Production"
           -targetProfile="Staging"
           -file=migrate_staging.sql
           -includeDelete=false

Compare a list of matching tables between two databases and write the output to the file migrate_staging.sql including DELETE statements.

WbDataDiff -referenceProfile="Production"
           -targetProfile="Staging"
           -referenceTables=person,address,person_address
           -file=migrate_staging.sql
           -includeDelete=true

Compare three tables that are differently named in the target database and ignore all columns (regardless in which table they appear) that are named LAST_ACCESS or LAST_UPDATE

WbDataDiff -referenceProfile="Production"
           -targetProfile="Staging"
           -referenceTables=person,address,person_address
           -targetTables=t_person,t_address,t_person_address
           -ignoreColumns=last_access,last_update
           -file=migrate_staging.sql
           -includeDelete=true

Prev		Next
15. Copy data across databases	Home	17. Other SQL Workbench/J specific commands