Senior Data Engineer
Primary Responsibilities
- Design, build and launch extremely efficient & reliable data pipelines to move data to our Data Warehouse/Data Mart.
- Own end-to-end data quality for the data pipelines you build
- Develop ETL routines to populate databases from multiple disparate data sources and create aggregates
- Create and run data migrations across different servers and different databases including Enterprise CRM and ERP applications.
- Perform complex data transformations, create/update stored procedures/functions, and optimize existing stored procedures/functions using indexing, temp tables, views, logic changes, etc.
- Design/develop new systems and tools to enable stakeholders to consume and understand data faster
- Data cleansing and manipulation using your expert SQL & programming skills
- Troubleshoot data issues and present solutions to the issues
- Prepare activity and progress reports regarding database & data health and status
- Design and improve agile development processes as it applies to data and data structure design
- Design, code and automate data quality checks, metrics, standards and guidelines
- Work across multiple teams in high visibility roles and own the solution end-to-end
Requirements
- BS or MS in Computer Science, Information Management, or related field
- 5+ years of experience as a Data Engineer.
- Candidate must have a deep understanding of logical and physical data modeling for OLTP and OLAP systems.
- Ability to translate a logical data model into a relational or non-relational solution as appropriate
- Familiar with multiple relational platforms, recent MSSQL Server experience is required.
- Hands-on expertise in database development using views, T-SQL, MSSQL and/or SQL scripts and SSIS packages and transformations. Experience building and troubleshooting SSAS cubes.
- Fluent in using tools like SQL Server Management Studio or similar.
- Recent experience in SQL tuning, indexing, partitioning, data access patterns and scaling strategies
- Programming/Scripting experience in Windows (C#, PowerShell) as well as Unix/Linux environments (Python, Bash)
- Experience in NoSQL/Big Data technologies (Couchbase or MongoDB)
- Excellent analytical problem solving and decision-making skills
- Experience working with large complex sets of data in a high-availability environment
- Experience with agile methodology process and development practices
Plus to have
- Payment or e-commerce industry experience
- Experience working with Informatica Intelligent Cloud Services (IICS)
- Experience in Business Intelligence tools and technologies
- Experience with Snowflake – Snowpipe, SnowSQL, Snowflake Procedures
- Experience in building out BI solutions in Looker