Looking for a Hadoop Spark / Redshift Developer

Hi, My name is Benjamin. I'm an expert with over 14 years of experience. I have worked primarily with Spark for ETL. The data sources were Amazon S3 and REST APIs, formats being Parquet, JSON or CSV. 1 -> The case was to take transaction data and glean BI views 2 -> Yes, this should be easy once we have the data, with group by and sums 3 -> Data transformation happens on the cluster 4 -> Did a process automation for Machine Learning, apart from a few ETLs 5 -> I have experience loading data into RedShift for visualization. Would love to work with you on this. Look forward to hearing from you. Regards,

I have extensive experience working on Hadoop ecosystem: Hive, Spark, Sqoop, HBase, Redshift, Oozie, Storm, Impala, Kylin etc. Also, MongoDB, Cassandra, Spark SQL, Spark ML lib, Spark Streaming Q1. Please provide one (small / medium ) use case of your ETL work in detail. 1. Web Scrappers -> Kafka -> Elasticsearch -> Kibana [ 10^7 logs in 1PB data per day] 2. Twitter -> Python Producers -> Pyspark EMR -> Elasticsearch + Redshift -> Tableau [1GB per day] 3. Radio API -> Kinesis -> Hive Map Reduce -> MySQL [2 GB per day] 4. Web API + Mobile API -> Kafka -> Python Consumers -> Teradata -> Power BI [4 GB per day] less Q2. If there are some X number of customers and they made some purchases. Can you write an SQL to find out the TOP 5 customers who made the most purchases. SELECT top 5 custid, COUNT(distinct orderid) AS 'Purchases' FROM orders GROUP BY custid ORDER BY 2 DESC Q3. What did you use Map Reduce for? What did you use PIG for ? What did you use Hive for ? where does the data gets transformed? / While performing transformations where is the data? Map Reduce is a framework, it was used in the use case #3 above. I have not worked on PIG but understand how it is different from Hive. I have used Hive in use case #3 above. During Map Reduce the data is mapped (read) from a storage (usually HDFS or S3 bucket) and Reduced (transformed, aggregated etc.) using Hive Storage. On the other hand, if we do the same in Spark, it happens in memory.

Hi, Its quite unfortunate that we cannot answer the questions you have raised in an attachment and post as freelancer.com never allows a bidder to attach a file until the client responses once at least. We have highly skilled developer on Hadoop and Spark. If you want I can patch you to talk so that you may judge him the technical skills. I will discuss with you the required commercials. Agree? Please response and make a schedule. Let’s discuss, Reasons to choose us: ****************** 1. We are a Govt, registered company named Eclipse Technoconsulting Global Pvt. Ltd 2. We are the proud company to get Global quality appraisal as CMMi level 3. 3. We are ISO 9001:2008 Certified company. Regards, Mit,

Hi, I have experience in spark/hadoop and machine learning. for more information ping me. I will provide all details.

Hi, I’m a Web Designer/Developer from the UK. My name is Mike. Your project description sounds interesting to me and I do have skills & experience that are required to complete this project. Let's have a quick chat when you're online.

We are a Team of Data Scientists having healthy experience into Big Data technologies like Hadoop,MapReduce and Data Analytics like R,HBase etc. The Team has qualified engineers having expertise in solving complex problems.

I have strong technical and data analytical skills. Have 2 yrs of experience in Data analytics and data engineering. Hard working.

I am Big Data expert with over 14 years of experience. Among that over 5 years in Big Data Technologies. I have worked all the mentioned technologies, except RedShift.

Hi plz discuss with me I am here to work on hourly basis plz I am waiting ur reply I will discuss with u point by point I am not known that aboutique first point but understood restate of plz message to discuss with u

Dear Client, I am an expert data analytics having 5 year of experience. I have been worked with big companies like Careem, OLACAB, TCS as well before as in my past experience. I have excellent command on R language, Hadoop, Bigdata tools etc. Thanks Prateek

A proposal has not yet been provided