Key Skills:
Strong AWS knowledge in terms of designing new architecture and providing optimized solution for existing one.
In-depth knowledge with respect to Snowflake and its architecture.
provide solutioning and drive the implementation of DOPS features which includes - building IAC infra, Git Lab re-structuring, Git lab Upgrades monitoring etc, Should have the vision on data strategy and able to deliver the same. Should be able to lead the design and implementation of data management processes, including data sourcing, integration, and transformation.
Able to manage and lead a team of data professionals, providing guidance, mentoring and foster a collaborative and innovative team culture focused on continuous improvement.
To evaluate and recommend data-related technologies, tools, and platforms. Collaborate with IT teams to ensure seamless integration of data solutions. should have experience in Implementing and enforce data security protocols and ensure compliance with relevant regulations.
Responsibilities:
Work with stakeholders to understand needs for data structure, availability, scalability, and accessibility.
Develop tools to improve data flows between internal/external systems and the data lake/warehouse.
Build robust and reproducible data ingest pipelines to collect, clean, harmonize, merge, and consolidate data sources.
Understanding existing data applications and infrastructure architecture
Build and support new data feeds for various Data Management layers and Data Lakes Evaluate business needs and requirements
Support migration of existing data transformation jobs in Oracle, and MS-SQL to Snowflake.
Lead the migration of the existing data transformation jobs in Oracle, Hive, Impala etc. into Spark, Python on Glue etc.
Able to document the processes and steps.
Develop and maintain datasets.
Improve data quality and efficiency.
Lead Business requirements and deliver accordingly.