|
Job Description
Role: AI-Assisted Data Engineer / Data Modernization Engineer (Contract) Duration: 6 months Location: Remote Rate: $90-$100/hour W2, dependent on skills and qualifications
Summary Contractor is responsible for executing a large-scale pilot data modernization initiative using AI-assisted development to transform decentralized campus reporting assets into governed, reusable data products in Microsoft Fabric. The position blends data engineering, automation, and applied AI to accelerate migration from legacy on-prem, Oracle database-dependent CAP environments to a modern, scalable data platform. The position works closely and collaboratively with the Data team’s architect, and extraction and engineering experts.
Key Responsibilities 1. CAP Inventory & Metadata Discovery - Develop automated scanning tools for:
- Access, Excel, Power BI, SQL Server, and other objects referring to REPL (on-site Oracle database) to be used on pilot schools’ CAP servers (1 college, 3 universities, system office) and for future use across 33 institutions.
- Identify data sources, Oracle dependencies, and usage patterns
- Capture and structure metadata for AI analysis of prioritizing data migration to Fabric
2. Data Engineering & Fabric Development - Develop data needed by campuses into:
- Bronze, Silver, and Gold data layers
- Design and build pipelines and transformations in Microsoft Fabric
- Optimize for scalability, performance, and reuse
3. AI-Assisted Development - Use AI tools to:
- Generate code for inventory and metadata gathering from CAP servers
- Automate analysis and prioritization of data moves to Fabric
- Translate usage patterns into enterprise data products
- Identify data gaps and recommend ingestion strategies
- Refactor CAP server assets to Fabric-based data sources
- Update queries, connections, and data models via AI-generated code
- Develop hybrid modernization approaches (Access front-end, SQL backend)
- Establish repeatable AI-assisted workflows
4. Validation & Testing - Perform reconciliation between REPL and Fabric outputs
- Ensure data accuracy, completeness, and performance
7. Documentation & Knowledge Transfer - Document tools, scripts, and methodologies
- Contribute to repeatable migration playbooks
- Support training and onboarding materials
Required Qualifications - 10+ years of experience in enterprise-level data engineering, analytics, or data architecture
- 5+ years of advanced SQL skills
- 5+ years of advanced Python
- 5+ ETL/ELT and data modeling
- 3+ years of extensive AI-assisted development tools (e.g. Copilot)
- 3+ years of extensive experience with PySpark required
- 3+ years building and optimizing large-scale data transformations in a Fabric lakehouse environment
- Experience with modern data platforms (Microsoft Fabric)
- Experience with ETL/ELT and data modeling
Expected Outcomes - Automated CAP inventory process
- Usage-based prioritization and analysis
- AI-assisted development of high-quality, reusable Fabric data products
- Successful migration of pilot campus assets
- Scalable framework for systemwide rollout
Reporting & Collaboration - Reports to: System Director, Architecture and Enablement
- Collaborates with:
|