Experience Level: 8–10 Years
We are seeking a highly experienced Senior dBT Engineer to design, build, and optimize our cloud-based data architecture. In this role, you will be responsible for building highly scalable data pipelines, optimizing data warehouse performance, and ensuring the highest standards of data quality. The ideal candidate will have 8 to 10 years of deep technical experience in databases, data warehousing, and modern cloud data stacks (AWS, dbt, PySpark).
Data Modeling & Transformation (dbt): Design and maintain clean, scalable data models. Write custom macros and implement automated testing to ensure data integrity before it moves upstream.
Big Data Processing (PySpark): Write highly efficient, production-grade PySpark code to process massive datasets, clean messy data, and prepare it for enterprise storage.
Data Warehousing (AWS Redshift): Architect, build, and maintain Redshift table structures. Actively monitor and tune database performance to ensure blazing-fast query execution.
Cloud Data Infrastructure: Leverage core AWS services to build end-to-end data solutions, utilizing S3 (file storage), AWS Glue (data cataloging/ETL), Lambda (automation scripts), and Step Functions (workflow orchestration).
Pipeline Development: Develop robust pre-integration processes to extract raw data from various APIs and third-party sources into staging areas.
Data Quality & CI/CD: Build automated systems to catch errors and bad data early in the pipeline. Manage code deployments, version control, and collaboration using Git.
Experience: 8–10 years of proven, hands-on experience working with enterprise databases and data warehousing technologies.
Data Warehousing Fundamentals: Deep theoretical and practical knowledge of key data warehousing concepts, including Facts, Dimensions, Slowly Changing Dimensions (SCD1, SCD2), Data Loads, and Transformations.
Python & SQL Mastery:
Must be a highly proficient Python coder capable of using libraries like pandas and boto3 to build scripts and connect to complex APIs.
Ability to write, debug, and optimize advanced SQL queries with ease.
Cloud Expertise: Solid, hands-on experience operating within the AWS ecosystem (Redshift, S3, Glue, Lambda, Step Functions).
Big Data Tools: Strong background in Apache Spark/PySpark for large-scale data transformations.
Modern Data Stack: Practical experience with dbt (data build tool) for data transformation and testing.