Data Engineer
United Kingdom /
Engineering – Engineering /
Full Time
/ Remote
About the role
Smartwyre is seeking to expand our data reporting capabilities by improving our ability to store, process, compute, and rapidly retrieve many different views of our existing datasets. To that end, we are seeking a Data Engineer to help build out this approach to rapid data reporting at scale.
The Data Engineer will work closely with established team members and become familiar with our business, the domain, and our existing data systems. The focus will be to build new data management systems and improve our existing ones.
Your Responsibilities
- Building and managing data warehouses or data lakes
- Develop and maintain data architecture and data models
- Ensure data quality and consistency across all data sources
- Implement and manage data security protocols and access controls
- Collaborate with prdouct owners, analysts, and other stakeholders to understand their data needs, build supporting systems, and provide structured data for reports
- Monitor and troubleshoot data pipelines and infrastructure
- Evaluate and implement new data tools and technologies to improve the efficiency and scalability of data infrastructure
- Ensure compliance with data security, governance, and privacy regulations
- Develop and maintain documentation related to data infrastructure and data systems
- Guide the design and work with the cloud team to construct and maintain data infrastructure
- Develop and implement ETL pipelines to move and transform data from various sources to internal data structures
Our Ideal Candidate
You will be someone who has experience building well-designed, performant and effective applications in support of complex business rules/flows and be interested in taking a hands on approach to defining and implementing such a solution.
We need you to have the following technical skills:
C#/.NET
Partitioning and sharding using PostgreSql
Working with Microsoft Azure data storage and processing technologies (e.g., Data Factory, Data Bricks, Data Lake, etc.)
Querying and ingesting data from third-party APIs
Writing complex SQL queries and optimizing database performance in both relational and non-relational databases
You will have a strong background in relational database design, optimization, and development and building out scalable, distributed, production systems
Having communication skills and the ability to work collaboratively with cross-functional teams, including data scientists, analysts, and other stakeholders is really important to us as this role supports all teams across Smartwyre.
You will be a trouble shooter, someone who really enjoys solving problems in complex data systems
