Technical Lead Big Data

Zoek Pin Inamgaon, Pune Division, Maharashtra

₹ 14,00,000 - 20,00,000 PA.

Permanent (Full time)

Posted 44 days ago

Roles and Responsibilities Create and maintain optimal data pipeline architectureAssemble large, complex data sets that meet functional / non-functional business the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Open Source and AWS big data technologiesBuild analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure with data and analytics experts to strive for greater functionality in our data Candidate Profile We are looking for a candidate with 4+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative 2 years of Team leading working SQL knowledge and experience working with relational databases as well as working familiarity with a variety of building and optimizing big data pipelines, architectures and performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for interacting with customers and various analytical skills related to working with unstructured processes supporting data transformation, data structures, metadata, dependency and workload knowledge of message queuing, stream processing, and highly scalable big data project management and organizational supporting and working with cross-functional teams in a dynamic should also have experience using the following software/tools:Big data technologies: Hadoop, Spark, Kafka, SQL and NoSQL databases, including Postgres and pipeline and workflow management tools: Airflow, NiFi services: AWS - EMR, RDS, Redshift, Glue. Azure - Databricks, Data Factory. GCP - Dataproc, Pub/SubStream-processing systems: Storm, Spark Streaming, Flink -oriented, functional or scripting languages: Python, Java, Scala, and Benefits

Report job View Company Page
Apply on Hirer's Site
Apply on Hirer's Site
Similar Jobs
Loading...