When attempting to load data in a batch manner instead of replication, it would be great to see additional options for added flexibility. These include:
- When reloading, is to add a function to do a logical delete from the target table (instead of drop & recreate, truncate, or do nothing), to only delete a portion of the data set. Becomes important if the target table has Row Level Security applied, where certain operations like table drops cant be performed, or if only wanting to "reload" a delta, and a partial targeted delete would help capture all updates.
- On full reloads with a filter on source table, to pull the filter value based on a "Max" value from the column in the target table.
- When scheduling starts, stops, and reloads, ability to do hourly scheduling as well.
