Labels

Apache Hadoop (3) ASP.NET (2) AWS S3 (2) Batch Script (3) BigQuery (21) BlobStorage (1) C# (3) Cloudera (1) Command (2) Data Model (3) Data Science (1) Django (1) Docker (1) ETL (7) Google Cloud (5) GPG (2) Hadoop (2) Hive (3) Luigi (1) MDX (21) Mongo (3) MYSQL (3) Pandas (1) Pentaho Data Integration (5) PentahoAdmin (13) Polybase (1) Postgres (1) PPS 2007 (2) Python (13) R Program (1) Redshift (3) SQL 2016 (2) SQL Error Fix (18) SQL Performance (1) SQL2012 (7) SQOOP (1) SSAS (20) SSH (1) SSIS (42) SSRS (17) T-SQL (75) Talend (3) Vagrant (1) Virtual Machine (2) WinSCP (1)

Wednesday, November 9, 2011

Incremental Data load in Pentaho using Insert/Update control

Incremental data loading can be achieved in pentaho using Inser/Update controls. The following steps helps one to understand the control:

1. Create a new transformation, containing source and target data connections.
2. Add a "Table Input" configure the connection properties and input columns. In below example two columns EmployeeID and EmployeeName are selected from Employee table.
3. Add a Insert/Update control join by hop.
4. Configure the insert update control as shown below (Steps 1, 2, & 3).
5. My target table is DimEmployee, and look up column in EmployeeID (Steps 4 &5).
6. Click on Get Update fields (Step 5). Map Input and Output columns.
7. Set update Y or N for columns (Y for allow updates, N for fixed attribute).
8. Click OK.


3 comments:

  1. Hi Mahadev

    I am developing reports using pentaho report designer(prd),i am using 5 subreports in one report.
    My intention is to use same datasource (JNDI) connection from main report to all the sub reports instead of using
    connections for each and every report.

    is it possible,if possible can u pls let me know the process.

    Also please explain me how to set the pool property in reports.

    Thanks in advance,
    Karthikeya

    ReplyDelete
  2. Keep posting..Thanks for sharing your knowledge.I am new to pentaho.
    I have question regarding "How to migrate code of pdi from one server to another? with all parametes or if any file is passed as input?" request you to please explain in detail so as to improve my knowledge.

    ReplyDelete
  3. HI Mahadev I only want INSERTs..I do not want update. then do I need to select fields in 'Update fields:' section and set 'Update' to N ?

    ReplyDelete