Labels

Apache Hadoop (3) ASP.NET (2) AWS S3 (2) Batch Script (3) BigQuery (21) BlobStorage (1) C# (3) Cloudera (1) Command (2) Data Model (3) Data Science (1) Django (1) Docker (1) ETL (7) Google Cloud (5) GPG (2) Hadoop (2) Hive (3) Luigi (1) MDX (21) Mongo (3) MYSQL (3) Pandas (1) Pentaho Data Integration (5) PentahoAdmin (13) Polybase (1) Postgres (1) PPS 2007 (2) Python (13) R Program (1) Redshift (3) SQL 2016 (2) SQL Error Fix (18) SQL Performance (1) SQL2012 (7) SQOOP (1) SSAS (20) SSH (1) SSIS (42) SSRS (17) T-SQL (75) Talend (3) Vagrant (1) Virtual Machine (2) WinSCP (1)

Monday, July 2, 2012

Configure Change Data Capture Parameters in SQL Server

After enabling CDC in SQL Server (see: http://mahadevanrv.blogspot.in/2011/05/change-data-capture-in-sql-server.html). We can modify the retention period and the number of transactions that to be handled in Change Data Capture table.

Before configureing one should understand the basic terms in CDC Configuration:

 
  • Polling interval – (Default 5 seconds) the amount of time to wait between log scans
  • Maxtrans – (Default 500) the number of transactions to grab with each scan
  • Maxscans – (Default 10) the number of scans performed between each polling interval
  • Retention – (Default 72 hours, 4320 mins, 3 days). The period for which the new/updated/deleted data have to be retrieved and displayed in CDC tables. 


  • Execute the below query to get the CDC configured values:

    SELECT * from msdb.dbo.cdc_jobs


    Execute the below query to change capture instances:

    EXEC sys.sp_cdc_change_job @job_type = 'capture'
           ,@maxtrans = 501
           ,@maxscans = 10
           ,@continuous = 1
           ,@pollinginterval = 5


    Execute the below query to change retention period:

    EXEC sys.sp_cdc_change_job @job_type = 'cleanup'
                              ,@retention = 4320 -- Number of minutes to retain (72 hours)
                              ,@threshold = 5000


    Using this method we can use CDC hold the required period of historical data, i.e., for last 1 month, last 1 year or last 10 days, etc.





    1 comment:

    1. you mentiond above that @threshold = 5000. what does that mean and what is default value.

      What difference it makes if you change from default to something different number like 5000.

      Thanks

      ReplyDelete