I’m Experienced, result-oriented, resourceful and problem-solving Data engineer with 4 years of diverse experience in Information Technology field, includes Development and Implementation of various applications in Big data and Data Warehousing and reporting solutions.
• Involved in data modelling for end to end ETL pipeline using various Big Data components
• Extensive experience in data processing using Spark RDD, SQL and DataFrame and writing complex Spark UDF.
• Deep knowledge on Spark internals, DAG and Performance Optimization techniques.
• Implemented various projects using different Hadoop technology stacks like HDFS, Spark,Hive, Sqoop, Kafka,Hbase,Elasticsearch.
TOOLS AND TECHNOLOGY WORKED ON:
• Big Data: Spark, Kafka, Hive, HDFS, Sqoop.
• Languages: Python , Scala, Core Java, SQL , PL/SQL, HiveQL, Shell
• Databases: Oracle, MySql, MariaDB, HBase,Cassandra
• Cloud(AWS): EC2,S3,Lambda,RDS,RedShift,EMR,Glue,DynamoDB,Athena,Kinesis,Elastic Search,Kibana
• Development Tools: Intellij Idea, Eclipse,SBT, Databricks, Zeppelin, SQL Workbench, WinSCP, Putty, Jupyter Notebook, Oracle Forms, Reports 10g, XML Publisher, SQL Plus, SQL Developer, PL/SQL Developer, Toad, Jenkin, Git, Grafana,Jira,Confluent