Data Engineer

Demeter Logistics is a Louisville-based startup technology platform company supporting international commodities markets. Demeter is seeking an experienced Data Engineer to develop complex packages supporting both inbound and outbound data interfaces. Inbound data will be sourced from public data sources, partner systems, exchanges, third-party systems, and client interactions with the platform. Outbound data will be provided through file transfers, APIs, web services and other to-be-determined methods. 

Both inbound and outbound data interfaces will be automated and monitored, with robust exception handling capabilities. All data integration processes and target data stores will be cloud-based and should be developed using agile/DevOps practices. 

The ideal candidate will have experience with metadata tools. Knowledge of enterprise data models is a must. 

This is an exciting and rare opportunity to build a cloud-native data platform from scratch, unencumbered by legacy systems and architectures. The successful candidate will have hands-on experience with the latest industry best practices, demonstrate a leadership mindset, possess solid operational experience, and be able to creatively solve tough technical challenges.

Cybersecurity is a crucial aspect of the data platform that must be fully integrated into the entire platform development framework. 

Data Engineer Reports to: Chief Technology Officer

·      Build, maintain and support production and non-production data environments within AWS

·      Be a productive member of a world-class data engineering team

·      Evaluate and recommend best cloud-based data integration tools, including AWS Data Migration, Glue and other third-party systems

·      Support ongoing development, change control, and production maintenance

·      Support business and client needs with robust ETL processes 

·      Participate in the development of a comprehensive data de-identification process, which may include ML-based synthetic data creation

·      Collaborate with DevOps and data science teams to support metadata management and data quality

·      Provide data engineering expertise to standardize, normalize and automate creation of data stores to support data science teams

·      Work within a robust cybersecurity framework

·      Build effective monitoring, alerts, and metrics for production data services and applications

·      Degree in Computer Science, Information Systems, or related field, or other relevant training

·      Ability to thrive in a fast-paced, creative environment

·      Strong knowledge of DevOps platform tooling

·      Experience with various data modeling techniques, including 3NF/OLTP and de-normalized OLAP methods

·      Experience with a range of data integration/ETL tools and processes

·      Working knowledge of data quality practices and tools; and,

·      Experience with both traditional databases and NoSQL data stores.