|
Job Description
Role: Senior Data Engineer Location: Remote Duration: 1 year Rate: $60.89/hour W2
Duties: - Implement data pipelines using best practices for ETL / ELT, data management, and data governance. Analyze and process complex data sources in a fast-paced environment.
- Perform data modeling against large data sets for peak efficiency.
- Identify, design, and implement process improvement solutions that automate manual processes and leverage standard frameworks and methodologies.
- Understand and incorporate data quality principles that ensure optimal reliability, impact, and user experience.
- Partner across teams to support cross-platform operations.
- Create and document functional and technical specifications.
- Drive exploration of new features, versions, and related technologies, and provide recommendations to enhance our offerings.
- Mentor junior engineers within the team.
Education: - Bachelor's degree in Computer Science, Information Technology or related field; OR equivalent 5+ years of experience. 5+ years of hands-on experience programming in SQL. 3+ years of experience building and maintaining automated data pipelines and data assets using batch and/or streaming processes.
Languages: - English (Read, Write, Speak)
Scope of Work: - Prepare and configure the Azure Microsoft Fabric instance to support integration with Cogito Cloud
- Build and deploy required data infrastructure by July 2026 to align with Cogito Cloud readiness
- Design and implement scalable data pipelines, transformations, and platform components within Fabric
- Support data validation, lineage, and certification processes to ensure trusted data delivery
- Partner with stakeholders to ensure successful integration, testing, and readiness for go-live
Core Requirements: - 5+ years of experience in data engineering / data platform development
- Hands-on experience with Microsoft Fabric (required), including:
- Lakehouse architecture
- Data Factory (Fabric pipelines)
- Dataflows Gen2
- Warehousing / SQL endpoints
- Strong experience with Azure data services (ADLS, Synapse, Azure SQL)
Technical Skills: - Proficiency in:
- Python and/or PySpark
- SQL (advanced)
- Experience building and optimizing:
- ETL/ELT pipelines
- Data ingestion frameworks (batch & streaming)
- Strong understanding of:
- Medallion architecture (Bronze/Silver/Gold layers)
- Delta Lake and parquet-based storage
- Experience with CI/CD and DevOps practices (Azure DevOps, Git)
Data & Platform Expertise: - Data modeling experience (dimensional + semantic layer support)
- Experience integrating data for:
- Reporting (Power BI preferred)
- Downstream analytics and AI/ML use cases
- Familiarity with data governance and security (RBAC, HIPAA, data policies)
Nice to Have: Experience with: - Power BI semantic models / datasets
- Real-time analytics in Fabric
- Exposure to data catalog / lineage tools
|