Filter

Databases

There are several ways to specify which databases to process. The most simple method is to provide a list of database(s) to process.

The first is to set the database(s) to process explicitly.

-db <databases>|--database <databases> 'databases' is a comma separated list of target databases to process.

databases:
- db_name
- db_name2

The second method is to use a Regular Expression to match the database(s) to process.

-dbRegEx <regex>|--database-regex <regex> 'regex' is a valid "Regular Expression" used to filter which databases to process from the target cluster.

Using the -dbRegEx on the CLI will populate the databases section of the config file with the regular expression matches.

databases:
- db_name
- db_name2

The third method is the most flexible and complete option. Use Warehouse Plans to define the which databases to process. This method also allows you to define individual database location values.

-wps,--warehouse-plans <db=ext-dir:mngd-dir[,db=ext-dir:mngd-dir]...>

translator:
  warehousePlans:
    db1:
      source: "PLAN"
      externalDirectory: "/finance/external"
      managedDirectory: "/finance/managed"
    db2:
        source: "PLAN"
        externalDirectory: "/marketing/external"
        managedDirectory: "/marketing/managed"

Database Skip Properties (via RegEx)

A user managed list of properties that will be filter OUT from the migration. For example: If you don't want to migrate a DBPROPERTY like repl.incl.test=hello_world, then add repl\.incl.* to this list.

-dbsp,--database-skip-properties <properties> Comma separated list of database properties (regex) to skip during the migration process. This will prevent the property from being set on the target cluster.

filter:
  dbPropertySkipList:
    - "repl\\.inc.*"

Tables

When nothing is specified, all tables in the processed databases are included. To limit the tables processed, you can use the following options.

By Regular Expression

-tf <regex> 'regex' used to match tables in the database to process. process.

-tef <regex> 'regex' used to match tables that would be 'EXCLUDED' in the database to process. process.

filter:
  tblRegEx: "test.*"

filter:
  tblExcludeRegEx: "tmp.*"

By Limits

-tfp <partition-count> 'partition-count' would be a limit of the number of partitions in a table that would be processed. Tables with more partitions than the limit would be excluded. A value of -1 would include all tables.

-tfs <size MB> 'size MB' would be a limit in size for tables that would be processed. Tables greater in size would be excluded. A value of -1 would include all tables.

A value of -1 would include all tables.

A value of -1 would include all tables.

filter:
  tblSizeLimit: -1
  tblPartitionLimit: -1

Last modified: 14 April 2025