In addition to creating and maintaining an optimal pipeline architecture, key responsibilities for a Senior Data Engineer include:
ï‚·Developing high-quality, high-performance, scalable applications using best practices and the latest generation of technologies
ï‚·Building required infrastructure for optimal extraction, transformation and loading of data from various data sources using AWS and SQL technologies
ï‚·Assembling large, complex sets of data that meet non-functional and functional business requirements.
ï‚·Identifying, designing, and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes
ï‚·Building analytical tools/dashboards to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency
ï‚·Creating and supporting internal APIs and products that enable workflows.
ï‚·Working with stakeholders including data, design, product and executive teams and assisting them with data-related technical issues
ï‚·Following Agile best practices effectively and actively participate in all scrum ceremonies
ï‚·Being part of a culture of continuous learning and improvement of both the team and codebase
ï‚·Responding to and resolving production issues
Desired Skills
ï‚·Ability to build and optimize data sets, data pipelines and architecturesÂ
ï‚·Ability to perform root cause analysis on external and internal processes and data to identify opportunities for improvement
ï‚·Excellent analytic skills associated with working on unstructured datasetsÂ
ï‚·Ability to build processes that support data transformation, workload management, data structures, dependency and metadata
ï‚·Experience with auto scaling, performance testing, and capacity planning
ï‚·Experience owning infrastructure in production, as well as designing and creating build/deploy & monitoring systems
ï‚·Exceptional analytical and problem-solving skills