Tech Notes

[...] At the heart of the Cloudera Kite Morphlines is the configuration file that contains all of your commands that you want to execute as a part of your ETL process. [...]

[...] Cloudera Kite Morphlines In the last post we have seen the internals of a configuration file also known as morphline. In this post we are going to explore the actual code that does all [...]

[...] instead of directly interacting with the Java API , you are doing it with the help of a configuration file. This makes the development of complex ETL processes on HDFS much faster and less [...]

[...] : INSERT INTO “TABLE_PARAMS” If you had read my previous post (Anatomy of a configuration file) where i described the flow of the application on which I was working . In the last command [...]

[...] used above belong to their respective owners. Related articles across the web Anatomy of a Command Builder with Example – Cloudera Kite Morphlines Anatomy of a Configuration File with Example- [...]

[...] the detail of these 2 methods : getNames() : – This is the place where you give name to your Command Builder so that it can be used in the configuration file. Command name and command builders are [...]

[...] for the schema file location. Thats all for now. Related articles across the web Anatomy of a Command Builder with Example – Cloudera Kite Morphlines Cloudera Kite Morphlines Getting Started Example [...]

[...] in the previous command. As per the design of my application I decided that i will store my avro schema file in the local file system instead of saving it on HDFS. Everything was working fine as [...]

[...] here. Now lets see a custom command builder which I have written to generate an Avro Schema File on the fly on the basis of input java object. Below is the complete code for this command [...]

[...] to the next command(generateAvroSchemaFile) of morphline. This command will generate the avro schema file on the fly by reading the Java object and place this schema file on the HDFS(Why HDFS ?? [...]

[...] to favorites • Email • RSS The post Single Field Indexes MongoDB Java Example appeared first on Tech Notes. [...]

[...] Add to favorites • Email • RSS The post Compound Indexes MongoDB Java Example appeared first on Tech Notes. [...]

[...] java.lang.IllegalArgumentException: Wrong FS: expected: file:/// – Hadoop appeared first on Tech Notes. [...]

[...] • Email • RSS The post Replace unicode characters from a Java String appeared first on Tech Notes. [...]

[...] At the heart of the Cloudera Kite Morphlines is the configuration file that contains all of your commands that you want to execute as [...]

[...] Cloudera Kite Morphlines In the last post we have seen the internals of a configuration file also known as [...]

[...] While evaluating the Cloudera Kite Morphlines , I came across this exception while reading the data from the table. java.io. [...]

[...] are most welcomed.Disclaimer : All the logos and images used above belong to their respective owners. Related articles across the web java.lang.IncompatibleClassChangeError: Found [...]

[...] examples can be downloaded from here. Disclaimer : All the images belong to their respective owners. Share and Enjoy • Facebook • Twitter • Delicious • LinkedIn • StumbleUpon • Add to [...]

[...] are most welcomed. Disclaimer : All the logos and images used above belong to their respective owners. Share and Enjoy • Facebook • Twitter • Delicious • LinkedIn • StumbleUpon • Add to [...]

[...] are most welcomed. Disclaimer : All the logos and images used above belong to their respective owners. Share and Enjoy • Facebook • Twitter • Delicious • LinkedIn • StumbleUpon • Add to [...]

[...] . loadDataIntoHive : This is the custom command that I have written to load the data into the Hive table in Parquet Format. Here to load the data into the Hive table , i have used the Kite Data Hive [...]

[...] file , reading JSON data , validate the record on the basis of some input rules ,load data into Hive table etc , possibilities are countless. As Kite Morphlines API is evolving you will find the [...]

[...] , as this avro schema file will be used in the next command when I load this data into the hive table in Parquet Format. This is one of simple use case of command builder , we can utilize the [...]

[...] in next post.). Now control will be transferred to last command (loadDataInHive)to create a Hive table using the avro schema , generated by last command and will load the data using Cloudera Kite [...]

[...] . It doesn’t make a difference whether you are using the in built command (bundled with Cloudera Kite Morphlines SDK) or writing your own custom command , basic structure and semantics of all the [...]

[...] basis of some input rules ,load data into Hive table etc , possibilities are countless. As Kite Morphlines API is evolving you will find the commands for most of the use cases of ETL processing in [...]

[...] : After Fix : It was bad on my side as I was not using the latest stable release for Cloudera Kite Morphlines i.e 0.14.0. As soon as I updated the version to 0.14.0, it downloaded the version 1.4.1 [...]