DOC

kettle_API

By Phyllis Watkins,2014-07-08 13:59
6 views 0
kettle_API

Kattle API 实战

    前言,

为什么要用KettleKETTLE JAVA API(

    Kettle是什么(kettle,是一个开源ETL工具。kettle提供了基于java的图形化界面!使用很方便!kettleETL工具集合也比较多!常用的ETL工具都包含了。

为什么使用KETTLE JAVA API,就像kettle文档所说,KETTLE JAVA API : Program your own

    Kettle transformationkettle提供了基于 JAVA的脚步编写功能!可以灵活地自定义ETL程!使自行定制、批量处理等成为可能!这才是一个程序员需要做的工作!而不仅是象使用word一样操作 kettle用户界面。

KETTLE JAVA API 实战操作记录,

    一、 搭建环境 :http://www.kettle.be网站下载kettle的源码包!加压缩!例如解压缩到d:\kettle目录

    二、 打开eclipse!新建一个项目!要使用jdk1.5.0!因为kettle的要使用System.getenv(),只有在jdk1.5.0才被支持。提起getenv?,!好像有一段几起几落的记录!曾一度被抛弃!现在又被jdk1.5支持了.

    三、 建一个class : TransBuilder.java,可以把d:\kettle\ extra\TransBuilder.java的内容原样拷贝到你的TransBuilder.java里。

    四、 根据需要编辑源码。并需要对原程序进行如下修改!在头部增加,

    import org.eclipse.swt.dnd.Transfer;

    //这个包被遗漏了!原始位置kettle根目录\libswt\win32\swt.jar

    //add by chq(www.chq.name) on 2006.07.20

?后来发现!不必加这个引用!因为编译时不需要,

    五、 编译准备!在eclipse中增加jar包!主要包括?主要依据extra\TransBuilder.bat,,

\lib\kettle.jar

    \libext\CacheDB.jar

    \libext\SQLBaseJDBC.jar

    \libext\activation.jar

    \libext\db2jcc.jar

    \libext\db2jcc_license_c.jar

    \libext\edtftpj-1.4.5.jar

    \libext\firebirdsql-full.jar

    \libext\firebirdsql.jar \libext\gis-shape.jar

    \libext\hsqldb.jar

    \libext\ifxjdbc.jar

    \libext\javadbf.jar

    \libext\jconn2.jar

    \libext\js.jar

    \libext\jt400.jar

    \libext\jtds-1.1.jar

    \libext\jxl.jar

    \libext\ktable.jar

    \libext\log4j-1.2.8.jar \libext\mail.jar

    \libext\mysql-connector-java-3.1.7-bin.jar

    \libext\ojdbc14.jar

    \libext\orai18n.jar

    \libext\pg74.215.jdbc3.jar \libext\edbc.jar

(注意 :下面这个包被遗漏了!要加上。原始位置kettle根目录\libswt\win32\swt.jar)

    \libswt\win32\swt.jar

六、 编译成功后!准备运行

为使程序不必登陆就可以运行!需要设置环境署文件,kettle.properties!位置在用户目录里!

    一般在 \Documents and Settings\用户\.kettle\,主要内容如下,

    KETTLE_REPOSITORY=kettle@m80

KETTLE_USER=admin

    KETTLE_PASSWORD=passwd

七、 好了!现在可以运行一下了!看看数据是不是已经拷贝到目标表了。

以下是运行时的控制台信息输出,

    下面是自动生成的Transformation :

以下为修改后的程序源码,

--------------------------------------------------------------------------------

package name.chq.test;

import java.io.DataOutputStream;

import java.io.File;

import java.io.FileOutputStream;

import be.ibridge.kettle.core.Const;

import be.ibridge.kettle.core.LogWriter;

import be.ibridge.kettle.core.NotePadMeta;

    import be.ibridge.kettle.core.database.Database;

    import be.ibridge.kettle.core.database.DatabaseMeta;

    import be.ibridge.kettle.core.exception.KettleException;

    import be.ibridge.kettle.core.util.EnvUtil;

import be.ibridge.kettle.trans.StepLoader;

import be.ibridge.kettle.trans.Trans;

import be.ibridge.kettle.trans.TransHopMeta;

import be.ibridge.kettle.trans.TransMeta;

import be.ibridge.kettle.trans.step.StepMeta;

    import be.ibridge.kettle.trans.step.StepMetaInterface;

    import be.ibridge.kettle.trans.step.selectvalues.SelectValuesMeta;

    import be.ibridge.kettle.trans.step.tableinput.TableInputMeta;

    import be.ibridge.kettle.trans.step.tableoutput.TableOutputMeta;

//这个包被遗漏了!原始位置kettle根目录\libswt\win32\swt.jar

    //add by chq([link=http://www.chq.name]www.chq.name[/link]) on 2006.07.20

//import org.eclipse.swt.dnd.Transfer;

/**

     * Class created to demonstrate the creation of transformations on-the-fly.

     *

     * @author Matt

     *

     */

public class TransBuilder

{

     public static final String[] databasesXML = {

     "" +

     "" +

     "target" +

     "192.168.17.35" +

     "ORACLE" +

     "Native" +

     "test1" +

     "1521" +

     "testuser" +

     "pwd" +

     "" +

     "" +

     "" +

     "" +

    "EXTRA_OPTION_MYSQL.defaultFetchSize500e>" +

    "EXTRA_OPTION_MYSQL.useCursorFetchtrue>" +

    "PORT_NUMBER1521" +

     "" +

     "" ,

     "" +

     "" +

     "source" +

     "192.168.16.12" +

     "ORACLE" +

     "Native" +

     "test2" +

     "1521" +

     "testuser" +

     "pwd2" +

     "" +

     "" +

     "" +

     "" +

    "EXTRA_OPTION_MYSQL.defaultFetchSize500e>" +

    "EXTRA_OPTION_MYSQL.useCursorFetchtrue>" +

    "PORT_NUMBER1521" +

     "" +

     ""

     };

     /**

     * Creates a new Transformation using input parameters such as the tablename to read from.

     * @param transformationName The name of the transformation

     * @param sourceDatabaseName The name of the database to read from

     * @param sourceTableName The name of the table to read from

     * @param sourceFields The field names we want to read from the source table

     * @param targetDatabaseName The name of the target database

     * @param targetTableName The name of the target table we want to write to

     * @param targetFields The names of the fields in the target table (same number of fields as sourceFields)

     * @return A new transformation

     * @throws KettleException In the rare case something goes wrong

     */

     public static final TransMeta buildCopyTable(

     String transformationName,String sourceDatabaseName, String sourceTableName,

     String[] sourceFields, String targetDatabaseName, String targetTableName,

     String[] targetFields)

     throws KettleException

     {

     LogWriter log = LogWriter.getInstance();

     EnvUtil.environmentInit();

     try

     {

     //

     // Create a new transformation...

     //

     TransMeta transMeta = new TransMeta();

     transMeta.setName(transformationName);

     // Add the database connections

     for (int i=0;i

     {

     DatabaseMeta databaseMeta = new DatabaseMeta(databasesXML[i]);

     transMeta.addDatabase(databaseMeta);

Report this document

For any questions or suggestions please email
cust-service@docsford.com