Systems Analyst - Thiruvananthapuram | Zoek India | 20ada2dfcf6d42e2845cce2bd5d9b709
Thiruvananthapuram, Kerala, India
Permanent (Full time)
Job Description Responsibility
•Designing data pipeline on Google cloud.
•Will be working on microservices to develop ETL
•Distributed data processing using Spark
•Working with event based / streaming technologies to ingest and process data
•Expertise in Java programming minimum of 4 years of experience in Java
•Hands on experience of building data piplines using Azure Data Factory and Apache Spark (preferably Databricks).
•Experience using frameworks on Apache Spark and associated design and development patterns
•Hands on experience designing and delivering solutions using the Azure Data Analytics platform
including Azure Storage; Azure SQL Data Warehouse; Azure Data Lake; Azure Cosmos DB; Azure Stream Analytics – cloud storage ; Data mining Patterns (cloud Datalab; cloud data prep or any mining stratergies); Data marts (cloud Dataprep); stream processing (cloud pub/sub; cloud dataflow)
•Hands on experience on datalake building.
•Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB; Cassandra; Neo4J)
•Good SQL Knowledge
•Experience working in a Dev/Ops environment -> Knowledge on Apache Airflow or Apache Beam or any tools related on automating data processing pipelines